2024-09-18T04:45:44.924 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-18T04:45:44.940 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-18T04:45:45.036 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909798 branch: reef description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nocache} email: ceph-qa@ceph.com first_in_suite: false job_id: '7909798' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: reef ceph: conf: client: rbd cache: false global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: lz4 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: reef sha1: a477b1894d5c5a613f23260ca0e6b96d79652ba7 owner: scheduled_teuthology@teuthology priority: 930 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 4623 sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 sleep_before_teardown: 0 subset: 80/128 suite: rbd suite_branch: reef suite_path: /home/teuthworker/src/git.ceph.com_ceph_a477b1894d5c5a613f23260ca0e6b96d79652ba7/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: a477b1894d5c5a613f23260ca0e6b96d79652ba7 targets: smithi002.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA5XRVOVzjCEemEYdvFHFdowKrx9SpYXnLgOxgAsdnDmTMKVQWTI9AaX+zqjW8CsHOORsP+cEn5izrMSzw9sG7Q= smithi099.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPrqtiZFc9dlncmXOvnjEjOflcccUdi8LsdQ0nquQ3+FHe1wyj2yml+y+5eHUvvsSCEjefzyQigqrkylK5I3DJk= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-17_22:16:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-18T04:45:45.036 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_a477b1894d5c5a613f23260ca0e6b96d79652ba7/qa; will attempt to use it 2024-09-18T04:45:45.037 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_a477b1894d5c5a613f23260ca0e6b96d79652ba7/qa/tasks 2024-09-18T04:45:45.037 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-18T04:45:45.039 INFO:teuthology.task.internal:Checking packages... 2024-09-18T04:45:45.059 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9' 2024-09-18T04:45:45.059 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-18T04:45:45.059 INFO:teuthology.packaging:ref: None 2024-09-18T04:45:45.060 INFO:teuthology.packaging:tag: None 2024-09-18T04:45:45.060 INFO:teuthology.packaging:branch: reef 2024-09-18T04:45:45.060 INFO:teuthology.packaging:sha1: 19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-18T04:45:45.060 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=reef 2024-09-18T04:45:45.351 INFO:teuthology.task.internal:Found packages for ceph version 18.2.4-855.g19e2a2be 2024-09-18T04:45:45.353 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-18T04:45:45.358 INFO:teuthology.task.internal:no buildpackages task found 2024-09-18T04:45:45.358 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-18T04:45:45.389 INFO:teuthology.task.internal:Saving configuration 2024-09-18T04:45:45.402 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-18T04:45:45.422 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-18T04:45:45.453 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi002.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909798', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-18 04:40:56.479262', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA5XRVOVzjCEemEYdvFHFdowKrx9SpYXnLgOxgAsdnDmTMKVQWTI9AaX+zqjW8CsHOORsP+cEn5izrMSzw9sG7Q='} 2024-09-18T04:45:45.477 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi099.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909798', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-18 04:40:56.480756', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPrqtiZFc9dlncmXOvnjEjOflcccUdi8LsdQ0nquQ3+FHe1wyj2yml+y+5eHUvvsSCEjefzyQigqrkylK5I3DJk='} 2024-09-18T04:45:45.477 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-18T04:45:45.483 INFO:teuthology.task.internal:roles: ubuntu@smithi002.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-09-18T04:45:45.483 INFO:teuthology.task.internal:roles: ubuntu@smithi099.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-09-18T04:45:45.484 INFO:teuthology.run_tasks:Running task console_log... 2024-09-18T04:45:45.574 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f870059f520>, signals=[15]) 2024-09-18T04:45:45.574 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-18T04:45:45.580 INFO:teuthology.task.internal:Opening connections... 2024-09-18T04:45:45.580 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi002.front.sepia.ceph.com 2024-09-18T04:45:45.581 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T04:45:45.661 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi099.front.sepia.ceph.com 2024-09-18T04:45:45.661 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T04:45:45.738 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-18T04:45:45.744 DEBUG:teuthology.orchestra.run.smithi002:> uname -m 2024-09-18T04:45:45.762 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64 2024-09-18T04:45:45.762 DEBUG:teuthology.orchestra.run.smithi002:> cat /etc/os-release 2024-09-18T04:45:45.821 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="CentOS Stream" 2024-09-18T04:45:45.821 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="9" 2024-09-18T04:45:45.821 INFO:teuthology.orchestra.run.smithi002.stdout:ID="centos" 2024-09-18T04:45:45.821 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="rhel fedora" 2024-09-18T04:45:45.821 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="9" 2024-09-18T04:45:45.821 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el9" 2024-09-18T04:45:45.821 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-18T04:45:45.821 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31" 2024-09-18T04:45:45.822 INFO:teuthology.orchestra.run.smithi002.stdout:LOGO="fedora-logo-icon" 2024-09-18T04:45:45.822 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-18T04:45:45.822 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://centos.org/" 2024-09-18T04:45:45.822 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-18T04:45:45.822 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-18T04:45:45.822 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-18T04:45:45.823 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2024-09-18T04:45:45.845 DEBUG:teuthology.orchestra.run.smithi099:> uname -m 2024-09-18T04:45:45.863 INFO:teuthology.orchestra.run.smithi099.stdout:x86_64 2024-09-18T04:45:45.864 DEBUG:teuthology.orchestra.run.smithi099:> cat /etc/os-release 2024-09-18T04:45:45.922 INFO:teuthology.orchestra.run.smithi099.stdout:NAME="CentOS Stream" 2024-09-18T04:45:45.922 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION="9" 2024-09-18T04:45:45.922 INFO:teuthology.orchestra.run.smithi099.stdout:ID="centos" 2024-09-18T04:45:45.922 INFO:teuthology.orchestra.run.smithi099.stdout:ID_LIKE="rhel fedora" 2024-09-18T04:45:45.923 INFO:teuthology.orchestra.run.smithi099.stdout:VERSION_ID="9" 2024-09-18T04:45:45.923 INFO:teuthology.orchestra.run.smithi099.stdout:PLATFORM_ID="platform:el9" 2024-09-18T04:45:45.923 INFO:teuthology.orchestra.run.smithi099.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-18T04:45:45.923 INFO:teuthology.orchestra.run.smithi099.stdout:ANSI_COLOR="0;31" 2024-09-18T04:45:45.923 INFO:teuthology.orchestra.run.smithi099.stdout:LOGO="fedora-logo-icon" 2024-09-18T04:45:45.923 INFO:teuthology.orchestra.run.smithi099.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-18T04:45:45.923 INFO:teuthology.orchestra.run.smithi099.stdout:HOME_URL="https://centos.org/" 2024-09-18T04:45:45.923 INFO:teuthology.orchestra.run.smithi099.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-18T04:45:45.923 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-18T04:45:45.923 INFO:teuthology.orchestra.run.smithi099.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-18T04:45:45.924 INFO:teuthology.lock.ops:Updating smithi099.front.sepia.ceph.com on lock server 2024-09-18T04:45:45.945 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-18T04:45:45.954 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-18T04:45:45.981 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-18T04:45:45.981 DEBUG:teuthology.orchestra.run.smithi002:> test '!' -e /home/ubuntu/cephtest 2024-09-18T04:45:45.984 DEBUG:teuthology.orchestra.run.smithi099:> test '!' -e /home/ubuntu/cephtest 2024-09-18T04:45:46.001 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-18T04:45:46.014 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-18T04:45:46.014 DEBUG:teuthology.orchestra.run.smithi002:> test -z $(ls -A /var/lib/ceph) 2024-09-18T04:45:46.040 DEBUG:teuthology.orchestra.run.smithi099:> test -z $(ls -A /var/lib/ceph) 2024-09-18T04:45:46.138 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-18T04:45:46.192 INFO:teuthology.run_tasks:Running task kernel... 2024-09-18T04:45:46.208 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-18T04:45:46.208 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-18T04:45:46.209 DEBUG:teuthology.orchestra.run.smithi002:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-18T04:45:46.209 DEBUG:teuthology.orchestra.run.smithi099:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-18T04:45:46.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T04:45:46.226 DEBUG:teuthology.orchestra.run.smithi002:> uname -r 2024-09-18T04:45:46.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T04:45:46.228 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2024-09-18T04:45:46.285 INFO:teuthology.orchestra.run.smithi002.stdout:5.14.0-503.el9.x86_64 2024-09-18T04:45:46.285 INFO:teuthology.task.kernel:Running kernel on smithi002: 5.14.0-503.el9.x86_64 2024-09-18T04:45:46.285 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum install -y kernel 2024-09-18T04:45:46.286 INFO:teuthology.orchestra.run.smithi099.stdout:5.14.0-503.el9.x86_64 2024-09-18T04:45:46.286 INFO:teuthology.task.kernel:Running kernel on smithi099: 5.14.0-503.el9.x86_64 2024-09-18T04:45:46.286 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum install -y kernel 2024-09-18T04:45:48.626 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.3 MB 00:00 2024-09-18T04:45:50.952 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - AppStream 17 MB/s | 20 MB 00:01 2024-09-18T04:45:52.338 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - BaseOS 1.9 MB/s | 8.3 MB 00:04 2024-09-18T04:45:54.698 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - AppStream 16 MB/s | 20 MB 00:01 2024-09-18T04:45:55.686 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - CRB 7.9 MB/s | 6.5 MB 00:00 2024-09-18T04:45:57.945 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 19 kB 00:01 2024-09-18T04:45:59.173 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - CRB 9.4 MB/s | 6.5 MB 00:00 2024-09-18T04:46:00.091 INFO:teuthology.orchestra.run.smithi002.stdout:Extra Packages for Enterprise Linux 11 MB/s | 23 MB 00:02 2024-09-18T04:46:01.448 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 19 kB 00:01 2024-09-18T04:46:03.177 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 14 MB/s | 23 MB 00:01 2024-09-18T04:46:05.621 INFO:teuthology.orchestra.run.smithi002.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-18T04:46:07.611 INFO:teuthology.orchestra.run.smithi002.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-18T04:46:07.611 INFO:teuthology.orchestra.run.smithi002.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-18T04:46:07.718 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-18T04:46:07.734 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:46:07.734 INFO:teuthology.orchestra.run.smithi002.stdout: Package Architecture Version Repository Size 2024-09-18T04:46:07.734 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:46:07.734 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-09-18T04:46:07.734 INFO:teuthology.orchestra.run.smithi002.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-18T04:46:07.734 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-09-18T04:46:07.734 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-18T04:46:07.735 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-18T04:46:07.735 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-18T04:46:07.735 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:46:07.735 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-09-18T04:46:07.735 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:46:07.735 INFO:teuthology.orchestra.run.smithi002.stdout:Install 4 Packages 2024-09-18T04:46:07.735 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:46:07.735 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 79 M 2024-09-18T04:46:07.736 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 126 M 2024-09-18T04:46:07.736 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-09-18T04:46:07.989 INFO:teuthology.orchestra.run.smithi002.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 293 kB/s | 36 kB 00:00 2024-09-18T04:46:08.799 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-09-18T04:46:10.681 INFO:teuthology.orchestra.run.smithi099.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-18T04:46:11.063 INFO:teuthology.orchestra.run.smithi099.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-18T04:46:11.064 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-18T04:46:11.064 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:46:11.065 INFO:teuthology.orchestra.run.smithi099.stdout: Package Architecture Version Repository Size 2024-09-18T04:46:11.065 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:46:11.065 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-09-18T04:46:11.065 INFO:teuthology.orchestra.run.smithi099.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-18T04:46:11.065 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2024-09-18T04:46:11.065 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-18T04:46:11.065 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-18T04:46:11.065 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-18T04:46:11.065 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:46:11.065 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-09-18T04:46:11.065 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:46:11.065 INFO:teuthology.orchestra.run.smithi099.stdout:Install 4 Packages 2024-09-18T04:46:11.065 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:46:11.066 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 79 M 2024-09-18T04:46:11.066 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 126 M 2024-09-18T04:46:11.066 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-09-18T04:46:11.066 INFO:teuthology.orchestra.run.smithi099.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 292 kB/s | 36 kB 00:00 2024-09-18T04:46:11.668 INFO:teuthology.orchestra.run.smithi099.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-09-18T04:46:12.143 INFO:teuthology.orchestra.run.smithi099.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 23 MB/s | 29 MB 00:01 2024-09-18T04:46:12.443 INFO:teuthology.orchestra.run.smithi099.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 21 MB/s | 35 MB 00:01 2024-09-18T04:46:12.444 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:46:12.444 INFO:teuthology.orchestra.run.smithi099.stdout:Total 47 MB/s | 79 MB 00:01 2024-09-18T04:46:13.120 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-09-18T04:46:13.464 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-09-18T04:46:13.465 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-09-18T04:46:14.434 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-09-18T04:46:14.434 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-09-18T04:46:14.738 INFO:teuthology.orchestra.run.smithi002.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 2.3 MB/s | 16 MB 00:06 2024-09-18T04:46:16.860 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-09-18T04:46:17.522 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-18T04:46:17.602 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T04:46:20.354 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T04:46:20.483 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T04:46:21.237 INFO:teuthology.orchestra.run.smithi002.stdout:(3/4): kernel-modules-core-5.14.0-508.el9.x86_6 2.2 MB/s | 29 MB 00:13 2024-09-18T04:46:23.938 INFO:teuthology.orchestra.run.smithi002.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 2.1 MB/s | 35 MB 00:16 2024-09-18T04:46:23.938 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:46:23.938 INFO:teuthology.orchestra.run.smithi002.stdout:Total 4.9 MB/s | 79 MB 00:16 2024-09-18T04:46:24.643 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-09-18T04:46:25.035 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-09-18T04:46:25.035 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-09-18T04:46:25.292 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T04:46:25.408 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-18T04:46:26.018 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-09-18T04:46:26.018 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-09-18T04:46:28.675 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-09-18T04:46:29.348 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-18T04:46:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T04:46:30.254 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T04:46:32.184 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T04:46:32.722 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T04:46:37.526 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T04:46:37.617 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-18T04:46:42.769 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T04:47:08.682 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T04:47:08.722 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-18T04:47:09.016 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-18T04:47:09.016 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-18T04:47:09.016 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T04:47:09.016 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T04:47:09.880 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T04:47:09.880 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:47:09.881 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-09-18T04:47:09.881 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-18T04:47:09.881 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-18T04:47:09.881 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-18T04:47:09.881 INFO:teuthology.orchestra.run.smithi099.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-18T04:47:09.881 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:47:09.882 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-18T04:47:10.239 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q kernel | sort -rV | head -n 1 2024-09-18T04:47:10.333 INFO:teuthology.orchestra.run.smithi099.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-18T04:47:10.334 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-18T04:47:10.334 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-18T04:47:10.334 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-18T04:47:10.334 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-18T04:47:10.334 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-18T04:47:10.334 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi099.front.sepia.ceph.com, path=None, version=distro) 2024-09-18T04:47:10.334 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q kernel | sort -rV | head -n 1 2024-09-18T04:47:10.395 INFO:teuthology.orchestra.run.smithi099.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-18T04:47:10.395 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-18T04:47:10.395 DEBUG:teuthology.orchestra.run.smithi099:> sudo rpm -qi grub2-tools 2024-09-18T04:47:10.436 INFO:teuthology.orchestra.run.smithi099.stdout:Name : grub2-tools 2024-09-18T04:47:10.436 INFO:teuthology.orchestra.run.smithi099.stdout:Epoch : 1 2024-09-18T04:47:10.436 INFO:teuthology.orchestra.run.smithi099.stdout:Version : 2.06 2024-09-18T04:47:10.436 INFO:teuthology.orchestra.run.smithi099.stdout:Release : 82.el9 2024-09-18T04:47:10.437 INFO:teuthology.orchestra.run.smithi099.stdout:Architecture: x86_64 2024-09-18T04:47:10.437 INFO:teuthology.orchestra.run.smithi099.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-18T04:47:10.438 INFO:teuthology.orchestra.run.smithi099.stdout:Group : Unspecified 2024-09-18T04:47:10.438 INFO:teuthology.orchestra.run.smithi099.stdout:Size : 8274670 2024-09-18T04:47:10.438 INFO:teuthology.orchestra.run.smithi099.stdout:License : GPLv3+ 2024-09-18T04:47:10.438 INFO:teuthology.orchestra.run.smithi099.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-18T04:47:10.438 INFO:teuthology.orchestra.run.smithi099.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-18T04:47:10.438 INFO:teuthology.orchestra.run.smithi099.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-18T04:47:10.438 INFO:teuthology.orchestra.run.smithi099.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-18T04:47:10.438 INFO:teuthology.orchestra.run.smithi099.stdout:Packager : builder@centos.org 2024-09-18T04:47:10.438 INFO:teuthology.orchestra.run.smithi099.stdout:Vendor : CentOS 2024-09-18T04:47:10.438 INFO:teuthology.orchestra.run.smithi099.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-18T04:47:10.439 INFO:teuthology.orchestra.run.smithi099.stdout:Summary : Support tools for GRUB. 2024-09-18T04:47:10.440 INFO:teuthology.orchestra.run.smithi099.stdout:Description : 2024-09-18T04:47:10.440 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:47:10.440 INFO:teuthology.orchestra.run.smithi099.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-18T04:47:10.440 INFO:teuthology.orchestra.run.smithi099.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-18T04:47:10.441 INFO:teuthology.orchestra.run.smithi099.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-18T04:47:10.441 INFO:teuthology.orchestra.run.smithi099.stdout:hardware devices. 2024-09-18T04:47:10.441 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:47:10.441 INFO:teuthology.orchestra.run.smithi099.stdout:This subpackage provides tools for support of all platforms. 2024-09-18T04:47:10.442 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-18T04:47:10.442 INFO:teuthology.task.kernel:Updating grub on smithi099 to boot 5.14.0-508.el9.x86_64 2024-09-18T04:47:10.442 DEBUG:teuthology.orchestra.run.smithi099:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-18T04:47:11.117 INFO:teuthology.orchestra.run.smithi099.stderr:Generating grub configuration file ... 2024-09-18T04:47:12.445 INFO:teuthology.orchestra.run.smithi099.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-18T04:47:12.465 INFO:teuthology.orchestra.run.smithi099.stderr:done 2024-09-18T04:47:12.467 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2024-09-18T04:47:12.483 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.fIMYX96LlQ 2024-09-18T04:47:12.483 DEBUG:teuthology.orchestra.run.smithi099:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.fIMYX96LlQ 2024-09-18T04:47:12.551 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.fIMYX96LlQ 2024-09-18T04:47:12.691 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.fIMYX96LlQ is 6KB 2024-09-18T04:47:12.740 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.fIMYX96LlQ 2024-09-18T04:47:12.757 DEBUG:teuthology.orchestra.run.smithi099:> sudo /bin/ls /boot/loader/entries || true 2024-09-18T04:47:12.817 INFO:teuthology.orchestra.run.smithi099.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-18T04:47:12.817 INFO:teuthology.orchestra.run.smithi099.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-18T04:47:12.817 INFO:teuthology.orchestra.run.smithi099.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-18T04:47:12.817 INFO:teuthology.orchestra.run.smithi099.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-18T04:47:12.818 DEBUG:teuthology.orchestra.run.smithi099:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-18T04:47:12.982 DEBUG:teuthology.orchestra.run.smithi099:> sudo shutdown -r now 2024-09-18T04:47:21.519 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T04:47:21.560 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-18T04:47:21.908 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-18T04:47:21.909 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-18T04:47:21.909 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-18T04:47:21.909 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-18T04:47:22.798 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-18T04:47:22.798 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:47:22.799 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-09-18T04:47:22.799 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-18T04:47:22.799 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-18T04:47:22.799 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-18T04:47:22.799 INFO:teuthology.orchestra.run.smithi002.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-18T04:47:22.799 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:47:22.799 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-18T04:47:23.132 DEBUG:teuthology.orchestra.run.smithi002:> rpm -q kernel | sort -rV | head -n 1 2024-09-18T04:47:23.210 INFO:teuthology.orchestra.run.smithi002.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-18T04:47:23.210 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-18T04:47:23.210 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-18T04:47:23.210 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-18T04:47:23.210 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-18T04:47:23.211 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-18T04:47:23.211 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi002.front.sepia.ceph.com, path=None, version=distro) 2024-09-18T04:47:23.211 DEBUG:teuthology.orchestra.run.smithi002:> rpm -q kernel | sort -rV | head -n 1 2024-09-18T04:47:23.270 INFO:teuthology.orchestra.run.smithi002.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-18T04:47:23.270 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-18T04:47:23.270 DEBUG:teuthology.orchestra.run.smithi002:> sudo rpm -qi grub2-tools 2024-09-18T04:47:23.302 INFO:teuthology.orchestra.run.smithi002.stdout:Name : grub2-tools 2024-09-18T04:47:23.303 INFO:teuthology.orchestra.run.smithi002.stdout:Epoch : 1 2024-09-18T04:47:23.303 INFO:teuthology.orchestra.run.smithi002.stdout:Version : 2.06 2024-09-18T04:47:23.303 INFO:teuthology.orchestra.run.smithi002.stdout:Release : 82.el9 2024-09-18T04:47:23.303 INFO:teuthology.orchestra.run.smithi002.stdout:Architecture: x86_64 2024-09-18T04:47:23.303 INFO:teuthology.orchestra.run.smithi002.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-18T04:47:23.303 INFO:teuthology.orchestra.run.smithi002.stdout:Group : Unspecified 2024-09-18T04:47:23.303 INFO:teuthology.orchestra.run.smithi002.stdout:Size : 8274670 2024-09-18T04:47:23.303 INFO:teuthology.orchestra.run.smithi002.stdout:License : GPLv3+ 2024-09-18T04:47:23.303 INFO:teuthology.orchestra.run.smithi002.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-18T04:47:23.304 INFO:teuthology.orchestra.run.smithi002.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-18T04:47:23.304 INFO:teuthology.orchestra.run.smithi002.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-18T04:47:23.304 INFO:teuthology.orchestra.run.smithi002.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-18T04:47:23.304 INFO:teuthology.orchestra.run.smithi002.stdout:Packager : builder@centos.org 2024-09-18T04:47:23.304 INFO:teuthology.orchestra.run.smithi002.stdout:Vendor : CentOS 2024-09-18T04:47:23.304 INFO:teuthology.orchestra.run.smithi002.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-18T04:47:23.304 INFO:teuthology.orchestra.run.smithi002.stdout:Summary : Support tools for GRUB. 2024-09-18T04:47:23.304 INFO:teuthology.orchestra.run.smithi002.stdout:Description : 2024-09-18T04:47:23.304 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:47:23.304 INFO:teuthology.orchestra.run.smithi002.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-18T04:47:23.304 INFO:teuthology.orchestra.run.smithi002.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-18T04:47:23.304 INFO:teuthology.orchestra.run.smithi002.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-18T04:47:23.304 INFO:teuthology.orchestra.run.smithi002.stdout:hardware devices. 2024-09-18T04:47:23.305 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:47:23.305 INFO:teuthology.orchestra.run.smithi002.stdout:This subpackage provides tools for support of all platforms. 2024-09-18T04:47:23.306 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-18T04:47:23.306 INFO:teuthology.task.kernel:Updating grub on smithi002 to boot 5.14.0-508.el9.x86_64 2024-09-18T04:47:23.306 DEBUG:teuthology.orchestra.run.smithi002:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-18T04:47:23.972 INFO:teuthology.orchestra.run.smithi002.stderr:Generating grub configuration file ... 2024-09-18T04:47:25.265 INFO:teuthology.orchestra.run.smithi002.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-18T04:47:25.297 INFO:teuthology.orchestra.run.smithi002.stderr:done 2024-09-18T04:47:25.299 DEBUG:teuthology.orchestra.run.smithi002:> mktemp 2024-09-18T04:47:25.314 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/tmp.Y2pzRubsjg 2024-09-18T04:47:25.314 DEBUG:teuthology.orchestra.run.smithi002:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Y2pzRubsjg 2024-09-18T04:47:25.386 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0666 /tmp/tmp.Y2pzRubsjg 2024-09-18T04:47:25.540 DEBUG:teuthology.orchestra.remote:smithi002:/tmp/tmp.Y2pzRubsjg is 6KB 2024-09-18T04:47:25.589 DEBUG:teuthology.orchestra.run.smithi002:> rm -fr /tmp/tmp.Y2pzRubsjg 2024-09-18T04:47:25.604 DEBUG:teuthology.orchestra.run.smithi002:> sudo /bin/ls /boot/loader/entries || true 2024-09-18T04:47:25.666 INFO:teuthology.orchestra.run.smithi002.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-18T04:47:25.666 INFO:teuthology.orchestra.run.smithi002.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-18T04:47:25.666 INFO:teuthology.orchestra.run.smithi002.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-18T04:47:25.666 INFO:teuthology.orchestra.run.smithi002.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-18T04:47:25.667 DEBUG:teuthology.orchestra.run.smithi002:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-18T04:47:25.815 DEBUG:teuthology.orchestra.run.smithi002:> sudo shutdown -r now 2024-09-18T04:47:43.000 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-18T04:47:43.001 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-18T04:47:43.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T04:47:55.831 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-18T04:47:55.832 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-09-18T04:47:55.833 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T04:48:01.341 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-09-18T04:48:10.350 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-18T04:48:10.351 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T04:48:13.405 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.99 2024-09-18T04:48:14.397 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.2 2024-09-18T04:48:23.405 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-09-18T04:48:23.405 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T04:48:25.408 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-18T04:48:25.409 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T04:48:26.461 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.2 2024-09-18T04:48:38.473 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-09-18T04:48:38.474 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T04:48:57.246 DEBUG:teuthology.orchestra.run.smithi099:> true 2024-09-18T04:48:57.597 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-18T04:48:57.597 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-18T04:48:57.597 DEBUG:teuthology.orchestra.run.smithi099:> uname -r 2024-09-18T04:48:57.611 INFO:teuthology.orchestra.run.smithi099.stdout:5.14.0-508.el9.x86_64 2024-09-18T04:48:57.611 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-18T04:48:57.611 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-18T04:48:57.611 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-18T04:48:58.613 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-18T04:48:58.613 DEBUG:teuthology.orchestra.run.smithi099:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-18T04:48:58.717 INFO:teuthology.orchestra.run.smithi099.stdout:ttyS1 2024-09-18T04:48:58.741 DEBUG:teuthology.parallel:result is None 2024-09-18T04:49:10.565 DEBUG:teuthology.orchestra.run.smithi002:> true 2024-09-18T04:49:11.236 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-09-18T04:49:11.236 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-18T04:49:11.236 DEBUG:teuthology.orchestra.run.smithi002:> uname -r 2024-09-18T04:49:11.251 INFO:teuthology.orchestra.run.smithi002.stdout:5.14.0-508.el9.x86_64 2024-09-18T04:49:11.251 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-18T04:49:11.252 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-18T04:49:11.252 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-18T04:49:12.252 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-18T04:49:12.252 DEBUG:teuthology.orchestra.run.smithi002:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-18T04:49:12.322 INFO:teuthology.orchestra.run.smithi002.stdout:ttyS1 2024-09-18T04:49:12.342 DEBUG:teuthology.parallel:result is None 2024-09-18T04:49:12.343 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-18T04:49:12.350 INFO:teuthology.task.internal:Creating test directory... 2024-09-18T04:49:12.350 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-18T04:49:12.352 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-18T04:49:12.372 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-18T04:49:12.391 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-18T04:49:12.432 INFO:teuthology.task.internal:Creating archive directory... 2024-09-18T04:49:12.432 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-18T04:49:12.435 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-18T04:49:12.515 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-18T04:49:12.523 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-18T04:49:12.523 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-18T04:49:12.526 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-18T04:49:12.557 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T04:49:12.565 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T04:49:12.569 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T04:49:12.577 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T04:49:12.580 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-18T04:49:12.588 INFO:teuthology.task.internal:Configuring sudo... 2024-09-18T04:49:12.588 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-18T04:49:12.621 DEBUG:teuthology.orchestra.run.smithi099:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-18T04:49:12.651 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-18T04:49:12.660 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-18T04:49:12.660 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-18T04:49:12.691 DEBUG:teuthology.orchestra.run.smithi099:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-18T04:49:12.709 DEBUG:teuthology.orchestra.run.smithi002:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T04:49:12.787 DEBUG:teuthology.orchestra.run.smithi002:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T04:49:12.849 DEBUG:teuthology.orchestra.run.smithi002:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T04:49:12.889 DEBUG:teuthology.orchestra.run.smithi002:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T04:49:12.939 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-18T04:49:12.939 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-18T04:49:13.010 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T04:49:13.050 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T04:49:13.106 DEBUG:teuthology.orchestra.run.smithi099:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T04:49:13.192 DEBUG:teuthology.orchestra.run.smithi099:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T04:49:13.235 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-18T04:49:13.235 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-18T04:49:13.303 DEBUG:teuthology.orchestra.run.smithi002:> sudo service rsyslog restart 2024-09-18T04:49:13.306 DEBUG:teuthology.orchestra.run.smithi099:> sudo service rsyslog restart 2024-09-18T04:49:13.350 INFO:teuthology.orchestra.run.smithi002.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-18T04:49:13.409 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-18T04:49:13.904 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-18T04:49:13.912 INFO:teuthology.task.internal:Starting timer... 2024-09-18T04:49:13.912 INFO:teuthology.run_tasks:Running task pcp... 2024-09-18T04:49:13.953 INFO:teuthology.run_tasks:Running task selinux... 2024-09-18T04:49:13.984 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-09-18T04:49:13.984 DEBUG:teuthology.orchestra.run.smithi002:> sudo service auditd rotate 2024-09-18T04:49:14.049 INFO:teuthology.orchestra.run.smithi002.stdout:Rotating logs: 2024-09-18T04:49:14.051 DEBUG:teuthology.orchestra.run.smithi099:> sudo service auditd rotate 2024-09-18T04:49:14.139 INFO:teuthology.orchestra.run.smithi099.stdout:Rotating logs: 2024-09-18T04:49:14.142 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-18T04:49:14.142 DEBUG:teuthology.orchestra.run.smithi002:> /usr/sbin/getenforce 2024-09-18T04:49:14.181 INFO:teuthology.orchestra.run.smithi002.stdout:Permissive 2024-09-18T04:49:14.181 DEBUG:teuthology.orchestra.run.smithi099:> /usr/sbin/getenforce 2024-09-18T04:49:14.207 INFO:teuthology.orchestra.run.smithi099.stdout:Permissive 2024-09-18T04:49:14.207 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi002.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi099.front.sepia.ceph.com': 'permissive'} 2024-09-18T04:49:14.208 DEBUG:teuthology.orchestra.run.smithi002:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-18T04:49:14.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T04:49:14.252 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-18T04:49:14.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T04:49:14.282 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-18T04:49:14.283 DEBUG:teuthology.orchestra.run.smithi002:> sudo /usr/sbin/setenforce permissive 2024-09-18T04:49:14.329 DEBUG:teuthology.orchestra.run.smithi099:> sudo /usr/sbin/setenforce permissive 2024-09-18T04:49:14.372 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-18T04:49:14.381 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-18T04:49:14.394 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-18T04:49:14.395 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi002.front.sepia.ceph.com,smithi099.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-18T04:53:04.173 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi002.front.sepia.ceph.com'), Remote(name='ubuntu@smithi099.front.sepia.ceph.com')] 2024-09-18T04:53:04.198 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-09-18T04:53:04.199 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T04:53:04.281 DEBUG:teuthology.orchestra.run.smithi002:> true 2024-09-18T04:53:04.367 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi002.front.sepia.ceph.com' 2024-09-18T04:53:04.368 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-18T04:53:04.368 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi099.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T04:53:04.445 DEBUG:teuthology.orchestra.run.smithi099:> true 2024-09-18T04:53:04.526 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi099.front.sepia.ceph.com' 2024-09-18T04:53:04.526 INFO:teuthology.run_tasks:Running task clock... 2024-09-18T04:53:04.536 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-18T04:53:04.536 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-18T04:53:04.537 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T04:53:04.543 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-18T04:53:04.543 DEBUG:teuthology.orchestra.run.smithi099:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T04:53:04.575 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-18T04:53:04.592 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-18T04:53:04.599 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-18T04:53:04.614 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-18T04:53:04.619 INFO:teuthology.orchestra.run.smithi002.stderr:sudo: ntpd: command not found 2024-09-18T04:53:04.632 INFO:teuthology.orchestra.run.smithi002.stdout:506 Cannot talk to daemon 2024-09-18T04:53:04.640 INFO:teuthology.orchestra.run.smithi099.stderr:sudo: ntpd: command not found 2024-09-18T04:53:04.648 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-18T04:53:04.652 INFO:teuthology.orchestra.run.smithi099.stdout:506 Cannot talk to daemon 2024-09-18T04:53:04.663 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-18T04:53:04.667 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-18T04:53:04.681 INFO:teuthology.orchestra.run.smithi099.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-18T04:53:04.722 INFO:teuthology.orchestra.run.smithi002.stderr:bash: line 1: ntpq: command not found 2024-09-18T04:53:04.725 INFO:teuthology.orchestra.run.smithi002.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-18T04:53:04.725 INFO:teuthology.orchestra.run.smithi002.stdout:=============================================================================== 2024-09-18T04:53:04.725 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T04:53:04.726 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T04:53:04.726 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T04:53:04.726 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T04:53:04.737 INFO:teuthology.orchestra.run.smithi099.stderr:bash: line 1: ntpq: command not found 2024-09-18T04:53:04.741 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-18T04:53:04.741 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2024-09-18T04:53:04.741 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T04:53:04.741 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T04:53:04.741 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T04:53:04.742 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T04:53:04.742 INFO:teuthology.run_tasks:Running task install... 2024-09-18T04:53:04.750 DEBUG:teuthology.task.install:project ceph 2024-09-18T04:53:04.750 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}} 2024-09-18T04:53:04.750 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'} 2024-09-18T04:53:04.750 INFO:teuthology.task.install:Using flavor: default 2024-09-18T04:53:04.755 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-18T04:53:04.755 INFO:teuthology.task.install:extra packages: [] 2024-09-18T04:53:04.756 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9', 'tag': None, 'wait_for_package': False} 2024-09-18T04:53:04.756 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-18T04:53:04.758 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9', 'tag': None, 'wait_for_package': False} 2024-09-18T04:53:04.758 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-18T04:53:04.906 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/ 2024-09-18T04:53:04.906 INFO:teuthology.task.install.rpm:Package version is 18.2.4-855.g19e2a2be 2024-09-18T04:53:04.910 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/ 2024-09-18T04:53:04.910 INFO:teuthology.task.install.rpm:Package version is 18.2.4-855.g19e2a2be 2024-09-18T04:53:05.018 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-18T04:53:05.018 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-18T04:53:05.018 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-18T04:53:05.025 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/reef/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-18T04:53:05.025 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-18T04:53:05.025 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-18T04:53:05.046 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-18T04:53:05.047 DEBUG:teuthology.orchestra.run.smithi002:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-18T04:53:05.050 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-18T04:53:05.051 DEBUG:teuthology.orchestra.run.smithi099:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/19e2a2beb6f41a3ab62650e38614eb29b59dcbb9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-18T04:53:05.115 DEBUG:teuthology.orchestra.run.smithi002:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-18T04:53:05.117 DEBUG:teuthology.orchestra.run.smithi099:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-18T04:53:05.192 DEBUG:teuthology.orchestra.run.smithi099:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-18T04:53:05.194 DEBUG:teuthology.orchestra.run.smithi002:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-18T04:53:05.257 INFO:teuthology.orchestra.run.smithi099.stdout:check_obsoletes = 1 2024-09-18T04:53:05.258 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum clean all 2024-09-18T04:53:05.259 INFO:teuthology.orchestra.run.smithi002.stdout:check_obsoletes = 1 2024-09-18T04:53:05.261 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum clean all 2024-09-18T04:53:05.493 INFO:teuthology.orchestra.run.smithi099.stdout:45 files removed 2024-09-18T04:53:05.497 INFO:teuthology.orchestra.run.smithi002.stdout:45 files removed 2024-09-18T04:53:05.520 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-radosgw 2024-09-18T04:53:05.524 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-radosgw 2024-09-18T04:53:06.072 INFO:teuthology.orchestra.run.smithi099.stdout:ceph packages for x86_64 288 kB/s | 77 kB 00:00 2024-09-18T04:53:06.096 INFO:teuthology.orchestra.run.smithi002.stdout:ceph packages for x86_64 286 kB/s | 77 kB 00:00 2024-09-18T04:53:06.347 INFO:teuthology.orchestra.run.smithi099.stdout:ceph noarch packages 49 kB/s | 11 kB 00:00 2024-09-18T04:53:06.382 INFO:teuthology.orchestra.run.smithi002.stdout:ceph noarch packages 48 kB/s | 11 kB 00:00 2024-09-18T04:53:06.615 INFO:teuthology.orchestra.run.smithi099.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-18T04:53:06.655 INFO:teuthology.orchestra.run.smithi002.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-18T04:53:08.605 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.3 MB 00:01 2024-09-18T04:53:08.946 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - BaseOS 3.6 MB/s | 8.3 MB 00:02 2024-09-18T04:53:15.830 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - AppStream 3.3 MB/s | 20 MB 00:06 2024-09-18T04:53:22.564 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 6.5 MB 00:02 2024-09-18T04:53:23.721 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - AppStream 1.5 MB/s | 20 MB 00:13 2024-09-18T04:53:24.240 INFO:teuthology.orchestra.run.smithi002.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 19 kB 00:00 2024-09-18T04:53:25.777 INFO:teuthology.orchestra.run.smithi002.stdout:Extra Packages for Enterprise Linux 16 MB/s | 23 MB 00:01 2024-09-18T04:53:28.338 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - CRB 7.6 MB/s | 6.5 MB 00:00 2024-09-18T04:53:30.155 INFO:teuthology.orchestra.run.smithi099.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-18T04:53:31.491 INFO:teuthology.orchestra.run.smithi002.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-18T04:53:31.684 INFO:teuthology.orchestra.run.smithi099.stdout:Extra Packages for Enterprise Linux 16 MB/s | 23 MB 00:01 2024-09-18T04:53:33.795 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-18T04:53:33.807 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:53:33.807 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2024-09-18T04:53:33.807 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:53:33.807 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-09-18T04:53:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-radosgw x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 7.7 M 2024-09-18T04:53:33.808 INFO:teuthology.orchestra.run.smithi002.stdout:Upgrading: 2024-09-18T04:53:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: librados2 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 3.3 M 2024-09-18T04:53:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: librbd1 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 3.0 M 2024-09-18T04:53:33.808 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-09-18T04:53:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-18T04:53:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-base x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 5.1 M 2024-09-18T04:53:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-common x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 18 M 2024-09-18T04:53:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-selinux x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 25 k 2024-09-18T04:53:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-18T04:53:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-18T04:53:33.808 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-18T04:53:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-18T04:53:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs2 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 712 k 2024-09-18T04:53:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-18T04:53:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-18T04:53:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-18T04:53:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: libradosstriper1 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 477 k 2024-09-18T04:53:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-18T04:53:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: librgw2 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 4.5 M 2024-09-18T04:53:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-18T04:53:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-18T04:53:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-18T04:53:33.809 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-argparse x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 45 k 2024-09-18T04:53:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-common x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 129 k 2024-09-18T04:53:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cephfs x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 162 k 2024-09-18T04:53:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-18T04:53:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rados x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 321 k 2024-09-18T04:53:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rbd x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 300 k 2024-09-18T04:53:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rgw x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 99 k 2024-09-18T04:53:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-18T04:53:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-18T04:53:33.810 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:53:33.810 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-09-18T04:53:33.810 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:53:33.811 INFO:teuthology.orchestra.run.smithi002.stdout:Install 28 Packages 2024-09-18T04:53:33.811 INFO:teuthology.orchestra.run.smithi002.stdout:Upgrade 2 Packages 2024-09-18T04:53:33.811 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:53:33.811 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 53 M 2024-09-18T04:53:33.811 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-09-18T04:53:34.542 INFO:teuthology.orchestra.run.smithi002.stdout:(1/30): ceph-base-18.2.4-855.g19e2a2be.el9.x86_ 9.5 MB/s | 5.1 MB 00:00 2024-09-18T04:53:34.651 INFO:teuthology.orchestra.run.smithi002.stdout:(2/30): ceph-radosgw-18.2.4-855.g19e2a2be.el9.x 12 MB/s | 7.7 MB 00:00 2024-09-18T04:53:34.676 INFO:teuthology.orchestra.run.smithi002.stdout:(3/30): ceph-selinux-18.2.4-855.g19e2a2be.el9.x 189 kB/s | 25 kB 00:00 2024-09-18T04:53:34.709 INFO:teuthology.orchestra.run.smithi002.stdout:(4/30): libcephfs2-18.2.4-855.g19e2a2be.el9.x86 12 MB/s | 712 kB 00:00 2024-09-18T04:53:34.735 INFO:teuthology.orchestra.run.smithi002.stdout:(5/30): libradosstriper1-18.2.4-855.g19e2a2be.e 8.0 MB/s | 477 kB 00:00 2024-09-18T04:53:34.794 INFO:teuthology.orchestra.run.smithi002.stdout:(6/30): python3-ceph-argparse-18.2.4-855.g19e2a 775 kB/s | 45 kB 00:00 2024-09-18T04:53:34.919 INFO:teuthology.orchestra.run.smithi002.stdout:(7/30): librgw2-18.2.4-855.g19e2a2be.el9.x86_64 21 MB/s | 4.5 MB 00:00 2024-09-18T04:53:34.935 INFO:teuthology.orchestra.run.smithi002.stdout:(8/30): python3-ceph-common-18.2.4-855.g19e2a2b 911 kB/s | 129 kB 00:00 2024-09-18T04:53:34.969 INFO:teuthology.orchestra.run.smithi002.stdout:(9/30): python3-cephfs-18.2.4-855.g19e2a2be.el9 3.2 MB/s | 162 kB 00:00 2024-09-18T04:53:34.994 INFO:teuthology.orchestra.run.smithi002.stdout:(10/30): python3-rados-18.2.4-855.g19e2a2be.el9 5.4 MB/s | 321 kB 00:00 2024-09-18T04:53:35.020 INFO:teuthology.orchestra.run.smithi002.stdout:(11/30): python3-rbd-18.2.4-855.g19e2a2be.el9.x 5.8 MB/s | 300 kB 00:00 2024-09-18T04:53:35.053 INFO:teuthology.orchestra.run.smithi002.stdout:(12/30): python3-rgw-18.2.4-855.g19e2a2be.el9.x 1.6 MB/s | 99 kB 00:00 2024-09-18T04:53:35.237 INFO:teuthology.orchestra.run.smithi002.stdout:(13/30): ceph-common-18.2.4-855.g19e2a2be.el9.x 15 MB/s | 18 MB 00:01 2024-09-18T04:53:35.371 INFO:teuthology.orchestra.run.smithi002.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 305 kB/s | 107 kB 00:00 2024-09-18T04:53:35.396 INFO:teuthology.orchestra.run.smithi002.stdout:(15/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 468 kB/s | 160 kB 00:00 2024-09-18T04:53:35.421 INFO:teuthology.orchestra.run.smithi002.stdout:(16/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 247 kB/s | 45 kB 00:00 2024-09-18T04:53:35.505 INFO:teuthology.orchestra.run.smithi002.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-18T04:53:35.538 INFO:teuthology.orchestra.run.smithi002.stdout:(18/30): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-09-18T04:53:35.563 INFO:teuthology.orchestra.run.smithi002.stdout:(19/30): python3-prettytable-0.7.2-27.el9.noarc 297 kB/s | 42 kB 00:00 2024-09-18T04:53:35.631 INFO:teuthology.orchestra.run.smithi002.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 882 kB/s | 111 kB 00:00 2024-09-18T04:53:35.664 INFO:teuthology.orchestra.run.smithi002.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 836 kB/s | 26 kB 00:00 2024-09-18T04:53:35.689 INFO:teuthology.orchestra.run.smithi002.stdout:(22/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.0 MB/s | 308 kB 00:00 2024-09-18T04:53:35.715 INFO:teuthology.orchestra.run.smithi002.stdout:(23/30): liboath-2.6.7-2.el9.x86_64.rpm 969 kB/s | 49 kB 00:00 2024-09-18T04:53:35.740 INFO:teuthology.orchestra.run.smithi002.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-18T04:53:35.799 INFO:teuthology.orchestra.run.smithi002.stdout:(25/30): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-18T04:53:35.832 INFO:teuthology.orchestra.run.smithi002.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-09-18T04:53:35.999 INFO:teuthology.orchestra.run.smithi002.stdout:(27/30): librados2-18.2.4-855.g19e2a2be.el9.x86 20 MB/s | 3.3 MB 00:00 2024-09-18T04:53:36.032 INFO:teuthology.orchestra.run.smithi002.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-09-18T04:53:36.124 INFO:teuthology.orchestra.run.smithi002.stdout:(29/30): librbd1-18.2.4-855.g19e2a2be.el9.x86_6 24 MB/s | 3.0 MB 00:00 2024-09-18T04:53:36.224 INFO:teuthology.orchestra.run.smithi002.stdout:(30/30): libarrow-9.0.0-11.el9.x86_64.rpm 6.9 MB/s | 4.4 MB 00:00 2024-09-18T04:53:36.229 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:53:36.229 INFO:teuthology.orchestra.run.smithi002.stdout:Total 22 MB/s | 53 MB 00:02 2024-09-18T04:53:36.307 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-09-18T04:53:36.347 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-09-18T04:53:36.348 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-09-18T04:53:36.987 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-09-18T04:53:36.987 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-09-18T04:53:37.065 INFO:teuthology.orchestra.run.smithi099.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-18T04:53:38.393 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-09-18T04:53:38.476 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-18T04:53:38.505 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-18T04:53:38.556 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-18T04:53:38.961 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-18T04:53:39.014 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 5/32 2024-09-18T04:53:39.107 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 5/32 2024-09-18T04:53:39.193 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 6/32 2024-09-18T04:53:39.224 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 7/32 2024-09-18T04:53:39.281 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-18T04:53:39.285 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:53:39.285 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2024-09-18T04:53:39.285 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:53:39.285 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-09-18T04:53:39.285 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 7.7 M 2024-09-18T04:53:39.285 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrading: 2024-09-18T04:53:39.285 INFO:teuthology.orchestra.run.smithi099.stdout: librados2 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 3.3 M 2024-09-18T04:53:39.285 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 3.0 M 2024-09-18T04:53:39.285 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2024-09-18T04:53:39.285 INFO:teuthology.orchestra.run.smithi099.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-18T04:53:39.286 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 5.1 M 2024-09-18T04:53:39.286 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 18 M 2024-09-18T04:53:39.286 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 25 k 2024-09-18T04:53:39.286 INFO:teuthology.orchestra.run.smithi099.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-18T04:53:39.286 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-18T04:53:39.286 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-18T04:53:39.286 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-18T04:53:39.286 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 712 k 2024-09-18T04:53:39.286 INFO:teuthology.orchestra.run.smithi099.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-18T04:53:39.286 INFO:teuthology.orchestra.run.smithi099.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-18T04:53:39.286 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-18T04:53:39.286 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 477 k 2024-09-18T04:53:39.286 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-18T04:53:39.287 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2 x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 4.5 M 2024-09-18T04:53:39.287 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-18T04:53:39.287 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-18T04:53:39.287 INFO:teuthology.orchestra.run.smithi099.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-18T04:53:39.287 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 45 k 2024-09-18T04:53:39.287 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 129 k 2024-09-18T04:53:39.287 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 162 k 2024-09-18T04:53:39.287 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-18T04:53:39.287 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 321 k 2024-09-18T04:53:39.287 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 300 k 2024-09-18T04:53:39.287 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 99 k 2024-09-18T04:53:39.287 INFO:teuthology.orchestra.run.smithi099.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-18T04:53:39.287 INFO:teuthology.orchestra.run.smithi099.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-18T04:53:39.288 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:53:39.288 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-09-18T04:53:39.288 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:53:39.288 INFO:teuthology.orchestra.run.smithi099.stdout:Install 28 Packages 2024-09-18T04:53:39.288 INFO:teuthology.orchestra.run.smithi099.stdout:Upgrade 2 Packages 2024-09-18T04:53:39.288 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:53:39.289 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 7/32 2024-09-18T04:53:39.289 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 53 M 2024-09-18T04:53:39.289 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-09-18T04:53:39.315 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-18T04:53:39.359 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ceph-argparse-2:18.2.4-855.g19e2a2be.el9.x 9/32 2024-09-18T04:53:39.431 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 10/32 2024-09-18T04:53:39.459 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 11/32 2024-09-18T04:53:39.509 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 11/32 2024-09-18T04:53:39.617 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-18T04:53:39.650 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-18T04:53:39.690 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-18T04:53:39.723 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-18T04:53:39.750 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-18T04:53:39.786 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-18T04:53:39.828 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-18T04:53:40.133 INFO:teuthology.orchestra.run.smithi099.stdout:(1/30): ceph-base-18.2.4-855.g19e2a2be.el9.x86_ 8.8 MB/s | 5.1 MB 00:00 2024-09-18T04:53:40.144 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-18T04:53:40.181 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 20/32 2024-09-18T04:53:40.241 INFO:teuthology.orchestra.run.smithi099.stdout:(2/30): ceph-radosgw-18.2.4-855.g19e2a2be.el9.x 11 MB/s | 7.7 MB 00:00 2024-09-18T04:53:40.266 INFO:teuthology.orchestra.run.smithi099.stdout:(3/30): ceph-selinux-18.2.4-855.g19e2a2be.el9.x 189 kB/s | 25 kB 00:00 2024-09-18T04:53:40.273 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 20/32 2024-09-18T04:53:40.304 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 21/32 2024-09-18T04:53:40.308 INFO:teuthology.orchestra.run.smithi099.stdout:(4/30): libcephfs2-18.2.4-855.g19e2a2be.el9.x86 10 MB/s | 712 kB 00:00 2024-09-18T04:53:40.334 INFO:teuthology.orchestra.run.smithi099.stdout:(5/30): libradosstriper1-18.2.4-855.g19e2a2be.e 7.0 MB/s | 477 kB 00:00 2024-09-18T04:53:40.339 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-18T04:53:40.392 INFO:teuthology.orchestra.run.smithi099.stdout:(6/30): python3-ceph-argparse-18.2.4-855.g19e2a 775 kB/s | 45 kB 00:00 2024-09-18T04:53:40.521 INFO:teuthology.orchestra.run.smithi099.stdout:(7/30): librgw2-18.2.4-855.g19e2a2be.el9.x86_64 21 MB/s | 4.5 MB 00:00 2024-09-18T04:53:40.542 INFO:teuthology.orchestra.run.smithi099.stdout:(8/30): python3-ceph-common-18.2.4-855.g19e2a2b 861 kB/s | 129 kB 00:00 2024-09-18T04:53:40.568 INFO:teuthology.orchestra.run.smithi099.stdout:(9/30): python3-cephfs-18.2.4-855.g19e2a2be.el9 3.2 MB/s | 162 kB 00:00 2024-09-18T04:53:40.594 INFO:teuthology.orchestra.run.smithi099.stdout:(10/30): python3-rados-18.2.4-855.g19e2a2be.el9 6.2 MB/s | 321 kB 00:00 2024-09-18T04:53:40.627 INFO:teuthology.orchestra.run.smithi099.stdout:(11/30): python3-rbd-18.2.4-855.g19e2a2be.el9.x 5.0 MB/s | 300 kB 00:00 2024-09-18T04:53:40.652 INFO:teuthology.orchestra.run.smithi099.stdout:(12/30): python3-rgw-18.2.4-855.g19e2a2be.el9.x 1.7 MB/s | 99 kB 00:00 2024-09-18T04:53:40.767 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-18T04:53:40.800 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 24/32 2024-09-18T04:53:40.845 INFO:teuthology.orchestra.run.smithi099.stdout:(13/30): ceph-common-18.2.4-855.g19e2a2be.el9.x 14 MB/s | 18 MB 00:01 2024-09-18T04:53:40.864 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 24/32 2024-09-18T04:53:40.944 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 25/32 2024-09-18T04:53:40.953 INFO:teuthology.orchestra.run.smithi099.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 328 kB/s | 107 kB 00:00 2024-09-18T04:53:40.979 INFO:teuthology.orchestra.run.smithi099.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 339 kB/s | 45 kB 00:00 2024-09-18T04:53:40.986 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ceph-common-2:18.2.4-855.g19e2a2be.el9.x86 26/32 2024-09-18T04:53:41.012 INFO:teuthology.orchestra.run.smithi099.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 446 kB/s | 160 kB 00:00 2024-09-18T04:53:41.054 INFO:teuthology.orchestra.run.smithi099.stdout:(17/30): librdkafka-1.6.1-102.el9.x86_64.rpm 6.4 MB/s | 662 kB 00:00 2024-09-18T04:53:41.071 INFO:teuthology.orchestra.run.smithi099.stdout:(18/30): python3-prettytable-0.7.2-27.el9.noarc 723 kB/s | 42 kB 00:00 2024-09-18T04:53:41.096 INFO:teuthology.orchestra.run.smithi099.stdout:(19/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-18T04:53:41.155 INFO:teuthology.orchestra.run.smithi099.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 1.1 MB/s | 111 kB 00:00 2024-09-18T04:53:41.180 INFO:teuthology.orchestra.run.smithi099.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.2 MB/s | 26 kB 00:00 2024-09-18T04:53:41.205 INFO:teuthology.orchestra.run.smithi099.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-18T04:53:41.247 INFO:teuthology.orchestra.run.smithi099.stdout:(23/30): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.7 MB/s | 308 kB 00:00 2024-09-18T04:53:41.272 INFO:teuthology.orchestra.run.smithi099.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-09-18T04:53:41.331 INFO:teuthology.orchestra.run.smithi099.stdout:(25/30): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-18T04:53:41.414 INFO:teuthology.orchestra.run.smithi099.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.9 MB/s | 839 kB 00:00 2024-09-18T04:53:41.573 INFO:teuthology.orchestra.run.smithi099.stdout:(27/30): librados2-18.2.4-855.g19e2a2be.el9.x86 21 MB/s | 3.3 MB 00:00 2024-09-18T04:53:41.615 INFO:teuthology.orchestra.run.smithi099.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 5.6 MB/s | 1.6 MB 00:00 2024-09-18T04:53:41.707 INFO:teuthology.orchestra.run.smithi099.stdout:(29/30): librbd1-18.2.4-855.g19e2a2be.el9.x86_6 23 MB/s | 3.0 MB 00:00 2024-09-18T04:53:41.782 INFO:teuthology.orchestra.run.smithi099.stdout:(30/30): libarrow-9.0.0-11.el9.x86_64.rpm 6.6 MB/s | 4.4 MB 00:00 2024-09-18T04:53:41.787 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:53:41.787 INFO:teuthology.orchestra.run.smithi099.stdout:Total 21 MB/s | 53 MB 00:02 2024-09-18T04:53:41.863 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-09-18T04:53:41.900 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-09-18T04:53:41.900 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-09-18T04:53:42.537 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-09-18T04:53:42.537 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-09-18T04:53:43.314 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 27/32 2024-09-18T04:53:43.349 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 27/32 2024-09-18T04:53:43.926 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 27/32 2024-09-18T04:53:43.927 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-09-18T04:53:43.969 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 28/32 2024-09-18T04:53:44.008 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-18T04:53:44.037 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-18T04:53:44.073 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 28/32 2024-09-18T04:53:44.074 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-18T04:53:44.074 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-18T04:53:44.074 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:53:44.086 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-18T04:53:44.104 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 29/32 2024-09-18T04:53:44.466 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-18T04:53:44.511 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 5/32 2024-09-18T04:53:44.603 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 5/32 2024-09-18T04:53:44.687 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 6/32 2024-09-18T04:53:44.721 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 7/32 2024-09-18T04:53:44.784 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 7/32 2024-09-18T04:53:44.828 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-18T04:53:44.872 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-argparse-2:18.2.4-855.g19e2a2be.el9.x 9/32 2024-09-18T04:53:44.948 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 10/32 2024-09-18T04:53:44.973 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 11/32 2024-09-18T04:53:45.022 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 11/32 2024-09-18T04:53:45.128 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-18T04:53:45.164 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-18T04:53:45.213 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-18T04:53:45.253 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/32 2024-09-18T04:53:45.281 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-18T04:53:45.317 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-18T04:53:45.365 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-18T04:53:45.669 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-18T04:53:45.720 INFO:teuthology.orchestra.run.smithi099.stdout: Upgrading : librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 20/32 2024-09-18T04:53:45.828 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 20/32 2024-09-18T04:53:45.859 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 21/32 2024-09-18T04:53:45.894 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-18T04:53:46.307 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-18T04:53:46.339 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 24/32 2024-09-18T04:53:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 24/32 2024-09-18T04:53:46.488 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 25/32 2024-09-18T04:53:46.517 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ceph-common-2:18.2.4-855.g19e2a2be.el9.x86 26/32 2024-09-18T04:53:48.606 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 27/32 2024-09-18T04:53:48.646 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 27/32 2024-09-18T04:53:49.176 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 27/32 2024-09-18T04:53:49.216 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 28/32 2024-09-18T04:53:49.296 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 28/32 2024-09-18T04:53:49.296 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-18T04:53:49.296 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-18T04:53:49.297 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:53:49.326 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 29/32 2024-09-18T04:53:54.608 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 29/32 2024-09-18T04:53:54.608 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /sys 2024-09-18T04:53:54.608 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /proc 2024-09-18T04:53:54.608 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /mnt 2024-09-18T04:53:54.608 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /var/tmp 2024-09-18T04:53:54.609 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /home 2024-09-18T04:53:54.609 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /root 2024-09-18T04:53:54.609 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /tmp 2024-09-18T04:53:54.609 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:53:55.394 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 30/32 2024-09-18T04:53:55.421 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 30/32 2024-09-18T04:53:55.421 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T04:53:55.422 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-18T04:53:55.422 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-18T04:53:55.422 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-18T04:53:55.422 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:53:55.423 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-18T04:53:55.514 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-18T04:53:55.514 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-18T04:53:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-18T04:53:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 1/32 2024-09-18T04:53:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 2/32 2024-09-18T04:53:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 3/32 2024-09-18T04:53:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 4/32 2024-09-18T04:53:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 5/32 2024-09-18T04:53:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 6/32 2024-09-18T04:53:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 7/32 2024-09-18T04:53:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ceph-argparse-2:18.2.4-855.g19e2a2be.el9.x 8/32 2024-09-18T04:53:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ceph-common-2:18.2.4-855.g19e2a2be.el9.x86 9/32 2024-09-18T04:53:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 10/32 2024-09-18T04:53:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 11/32 2024-09-18T04:53:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 12/32 2024-09-18T04:53:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 13/32 2024-09-18T04:53:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-18T04:53:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-18T04:53:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-18T04:53:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-18T04:53:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-18T04:53:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-18T04:53:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-18T04:53:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-18T04:53:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-18T04:53:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-18T04:53:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-18T04:53:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-18T04:53:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-18T04:53:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-18T04:53:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-18T04:53:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 29/32 2024-09-18T04:53:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-18T04:53:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 31/32 2024-09-18T04:53:57.513 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-18T04:53:57.513 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:53:57.513 INFO:teuthology.orchestra.run.smithi002.stdout:Upgraded: 2024-09-18T04:53:57.513 INFO:teuthology.orchestra.run.smithi002.stdout: librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:53:57.513 INFO:teuthology.orchestra.run.smithi002.stdout: librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:53:57.514 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-09-18T04:53:57.514 INFO:teuthology.orchestra.run.smithi002.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-18T04:53:57.514 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:53:57.514 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:53:57.514 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:53:57.514 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:53:57.514 INFO:teuthology.orchestra.run.smithi002.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-18T04:53:57.514 INFO:teuthology.orchestra.run.smithi002.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-18T04:53:57.515 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-18T04:53:57.515 INFO:teuthology.orchestra.run.smithi002.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-18T04:53:57.515 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:53:57.515 INFO:teuthology.orchestra.run.smithi002.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-18T04:53:57.515 INFO:teuthology.orchestra.run.smithi002.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-18T04:53:57.515 INFO:teuthology.orchestra.run.smithi002.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-18T04:53:57.515 INFO:teuthology.orchestra.run.smithi002.stdout: libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:53:57.515 INFO:teuthology.orchestra.run.smithi002.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-18T04:53:57.516 INFO:teuthology.orchestra.run.smithi002.stdout: librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:53:57.516 INFO:teuthology.orchestra.run.smithi002.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-18T04:53:57.516 INFO:teuthology.orchestra.run.smithi002.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-18T04:53:57.516 INFO:teuthology.orchestra.run.smithi002.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-18T04:53:57.516 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-argparse-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:53:57.516 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:53:57.516 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:53:57.517 INFO:teuthology.orchestra.run.smithi002.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-18T04:53:57.517 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:53:57.517 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:53:57.517 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:53:57.517 INFO:teuthology.orchestra.run.smithi002.stdout: re2-1:20211101-20.el9.x86_64 2024-09-18T04:53:57.517 INFO:teuthology.orchestra.run.smithi002.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-18T04:53:57.517 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:53:57.517 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-18T04:53:57.841 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-test 2024-09-18T04:53:58.442 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:00:27 ago on Wed 18 Sep 2024 04:53:31 AM UTC. 2024-09-18T04:53:58.592 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-18T04:53:58.592 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:53:58.593 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2024-09-18T04:53:58.593 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:53:58.593 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-09-18T04:53:58.593 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-test x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 40 M 2024-09-18T04:53:58.593 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-09-18T04:53:58.593 INFO:teuthology.orchestra.run.smithi002.stdout: libcephsqlite x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 166 k 2024-09-18T04:53:58.593 INFO:teuthology.orchestra.run.smithi002.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-18T04:53:58.593 INFO:teuthology.orchestra.run.smithi002.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-18T04:53:58.593 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:53:58.593 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-09-18T04:53:58.593 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:53:58.593 INFO:teuthology.orchestra.run.smithi002.stdout:Install 4 Packages 2024-09-18T04:53:58.593 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:53:58.594 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 40 M 2024-09-18T04:53:58.594 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 164 M 2024-09-18T04:53:58.594 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-09-18T04:53:58.902 INFO:teuthology.orchestra.run.smithi002.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.6 MB/s | 304 kB 00:00 2024-09-18T04:53:58.927 INFO:teuthology.orchestra.run.smithi002.stdout:(2/4): libcephsqlite-18.2.4-855.g19e2a2be.el9.x 799 kB/s | 166 kB 00:00 2024-09-18T04:53:58.952 INFO:teuthology.orchestra.run.smithi002.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-18T04:53:59.515 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 29/32 2024-09-18T04:53:59.515 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /sys 2024-09-18T04:53:59.515 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /proc 2024-09-18T04:53:59.515 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /mnt 2024-09-18T04:53:59.515 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /var/tmp 2024-09-18T04:53:59.515 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /home 2024-09-18T04:53:59.516 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /root 2024-09-18T04:53:59.516 INFO:teuthology.orchestra.run.smithi099.stdout:skipping the directory /tmp 2024-09-18T04:53:59.516 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:53:59.994 INFO:teuthology.orchestra.run.smithi002.stdout:(4/4): ceph-test-18.2.4-855.g19e2a2be.el9.x86_6 31 MB/s | 40 MB 00:01 2024-09-18T04:53:59.996 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:53:59.997 INFO:teuthology.orchestra.run.smithi002.stdout:Total 29 MB/s | 40 MB 00:01 2024-09-18T04:54:00.025 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-09-18T04:54:00.047 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-09-18T04:54:00.047 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-09-18T04:54:00.256 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 30/32 2024-09-18T04:54:00.279 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 30/32 2024-09-18T04:54:00.279 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T04:54:00.279 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-18T04:54:00.280 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-18T04:54:00.280 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-18T04:54:00.280 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:00.281 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-18T04:54:00.358 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-18T04:54:00.358 INFO:teuthology.orchestra.run.smithi099.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-18T04:54:00.393 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-09-18T04:54:00.393 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-09-18T04:54:00.893 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-09-18T04:54:01.011 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-18T04:54:01.037 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-18T04:54:01.063 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 3/4 2024-09-18T04:54:01.534 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-18T04:54:01.534 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 1/32 2024-09-18T04:54:01.534 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 2/32 2024-09-18T04:54:01.534 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 3/32 2024-09-18T04:54:01.534 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 4/32 2024-09-18T04:54:01.534 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 5/32 2024-09-18T04:54:01.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 6/32 2024-09-18T04:54:01.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 7/32 2024-09-18T04:54:01.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-argparse-2:18.2.4-855.g19e2a2be.el9.x 8/32 2024-09-18T04:54:01.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ceph-common-2:18.2.4-855.g19e2a2be.el9.x86 9/32 2024-09-18T04:54:01.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 10/32 2024-09-18T04:54:01.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 11/32 2024-09-18T04:54:01.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 12/32 2024-09-18T04:54:01.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 13/32 2024-09-18T04:54:01.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-18T04:54:01.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-18T04:54:01.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-18T04:54:01.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-18T04:54:01.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-18T04:54:01.535 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-18T04:54:01.536 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-18T04:54:01.536 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 21/32 2024-09-18T04:54:01.536 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-18T04:54:01.536 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-18T04:54:01.536 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-18T04:54:01.536 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-18T04:54:01.536 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-18T04:54:01.536 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-18T04:54:01.536 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-18T04:54:01.536 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 29/32 2024-09-18T04:54:01.536 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-18T04:54:01.536 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 31/32 2024-09-18T04:54:02.701 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-18T04:54:02.701 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:02.702 INFO:teuthology.orchestra.run.smithi099.stdout:Upgraded: 2024-09-18T04:54:02.702 INFO:teuthology.orchestra.run.smithi099.stdout: librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:02.702 INFO:teuthology.orchestra.run.smithi099.stdout: librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:02.702 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-09-18T04:54:02.702 INFO:teuthology.orchestra.run.smithi099.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-18T04:54:02.702 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:02.702 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:02.702 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-radosgw-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:02.702 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-selinux-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:02.702 INFO:teuthology.orchestra.run.smithi099.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-18T04:54:02.702 INFO:teuthology.orchestra.run.smithi099.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-18T04:54:02.702 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-18T04:54:02.702 INFO:teuthology.orchestra.run.smithi099.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-18T04:54:02.703 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:02.703 INFO:teuthology.orchestra.run.smithi099.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-18T04:54:02.703 INFO:teuthology.orchestra.run.smithi099.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-18T04:54:02.703 INFO:teuthology.orchestra.run.smithi099.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-18T04:54:02.703 INFO:teuthology.orchestra.run.smithi099.stdout: libradosstriper1-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:02.703 INFO:teuthology.orchestra.run.smithi099.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-18T04:54:02.703 INFO:teuthology.orchestra.run.smithi099.stdout: librgw2-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:02.703 INFO:teuthology.orchestra.run.smithi099.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-18T04:54:02.703 INFO:teuthology.orchestra.run.smithi099.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-18T04:54:02.703 INFO:teuthology.orchestra.run.smithi099.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-18T04:54:02.703 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-argparse-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:02.703 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ceph-common-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:02.703 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:02.704 INFO:teuthology.orchestra.run.smithi099.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-18T04:54:02.704 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:02.704 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:02.704 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:02.704 INFO:teuthology.orchestra.run.smithi099.stdout: re2-1:20211101-20.el9.x86_64 2024-09-18T04:54:02.704 INFO:teuthology.orchestra.run.smithi099.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-18T04:54:02.704 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:02.704 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-18T04:54:03.047 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-test 2024-09-18T04:54:03.625 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:00:26 ago on Wed 18 Sep 2024 04:53:37 AM UTC. 2024-09-18T04:54:03.759 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-18T04:54:03.760 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:03.760 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2024-09-18T04:54:03.760 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:03.760 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-09-18T04:54:03.760 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 40 M 2024-09-18T04:54:03.760 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2024-09-18T04:54:03.760 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 166 k 2024-09-18T04:54:03.760 INFO:teuthology.orchestra.run.smithi099.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-18T04:54:03.760 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-18T04:54:03.761 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:03.761 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-09-18T04:54:03.761 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:03.761 INFO:teuthology.orchestra.run.smithi099.stdout:Install 4 Packages 2024-09-18T04:54:03.761 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:03.762 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 40 M 2024-09-18T04:54:03.762 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 164 M 2024-09-18T04:54:03.762 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-09-18T04:54:04.256 INFO:teuthology.orchestra.run.smithi099.stdout:(1/4): libcephsqlite-18.2.4-855.g19e2a2be.el9.x 766 kB/s | 166 kB 00:00 2024-09-18T04:54:04.606 INFO:teuthology.orchestra.run.smithi099.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 182 kB/s | 64 kB 00:00 2024-09-18T04:54:04.640 INFO:teuthology.orchestra.run.smithi099.stdout:(3/4): socat-1.7.4.1-6.el9.x86_64.rpm 507 kB/s | 304 kB 00:00 2024-09-18T04:54:04.974 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 3/4 2024-09-18T04:54:05.055 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 4/4 2024-09-18T04:54:05.332 INFO:teuthology.orchestra.run.smithi099.stdout:(4/4): ceph-test-18.2.4-855.g19e2a2be.el9.x86_6 31 MB/s | 40 MB 00:01 2024-09-18T04:54:05.334 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:54:05.334 INFO:teuthology.orchestra.run.smithi099.stdout:Total 26 MB/s | 40 MB 00:01 2024-09-18T04:54:05.361 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-09-18T04:54:05.382 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-09-18T04:54:05.382 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-09-18T04:54:05.706 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-09-18T04:54:05.707 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-09-18T04:54:06.210 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-09-18T04:54:06.291 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-18T04:54:06.340 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-18T04:54:06.375 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 3/4 2024-09-18T04:54:07.048 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 4/4 2024-09-18T04:54:07.048 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 1/4 2024-09-18T04:54:07.048 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 2/4 2024-09-18T04:54:07.048 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-18T04:54:07.342 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-18T04:54:07.342 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:07.342 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-09-18T04:54:07.342 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:07.342 INFO:teuthology.orchestra.run.smithi002.stdout: libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:07.343 INFO:teuthology.orchestra.run.smithi002.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-18T04:54:07.343 INFO:teuthology.orchestra.run.smithi002.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-18T04:54:07.343 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:07.343 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-18T04:54:07.629 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph 2024-09-18T04:54:08.226 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:00:37 ago on Wed 18 Sep 2024 04:53:31 AM UTC. 2024-09-18T04:54:08.364 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-18T04:54:08.366 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:08.366 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2024-09-18T04:54:08.367 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:08.367 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-09-18T04:54:08.367 INFO:teuthology.orchestra.run.smithi002.stdout: ceph x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 6.5 k 2024-09-18T04:54:08.367 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-09-18T04:54:08.367 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mds x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 2.1 M 2024-09-18T04:54:08.367 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 1.5 M 2024-09-18T04:54:08.367 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-modules-core noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 246 k 2024-09-18T04:54:08.367 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mon x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 4.7 M 2024-09-18T04:54:08.367 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-osd x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 17 M 2024-09-18T04:54:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-18T04:54:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-18T04:54:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-18T04:54:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-18T04:54:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-18T04:54:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-18T04:54:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-18T04:54:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-18T04:54:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-18T04:54:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-18T04:54:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-18T04:54:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-18T04:54:08.368 INFO:teuthology.orchestra.run.smithi002.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-18T04:54:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-18T04:54:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-18T04:54:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-18T04:54:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-18T04:54:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-18T04:54:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-18T04:54:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-18T04:54:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-18T04:54:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-18T04:54:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-18T04:54:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-18T04:54:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-18T04:54:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-18T04:54:08.369 INFO:teuthology.orchestra.run.smithi002.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-18T04:54:08.370 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:08.370 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-09-18T04:54:08.370 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:08.370 INFO:teuthology.orchestra.run.smithi002.stdout:Install 33 Packages 2024-09-18T04:54:08.370 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:08.370 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 29 M 2024-09-18T04:54:08.370 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 104 M 2024-09-18T04:54:08.370 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-09-18T04:54:08.761 INFO:teuthology.orchestra.run.smithi002.stdout:(1/33): ceph-18.2.4-855.g19e2a2be.el9.x86_64.rp 55 kB/s | 6.5 kB 00:00 2024-09-18T04:54:08.969 INFO:teuthology.orchestra.run.smithi002.stdout:(2/33): ceph-mds-18.2.4-855.g19e2a2be.el9.x86_6 6.4 MB/s | 2.1 MB 00:00 2024-09-18T04:54:09.011 INFO:teuthology.orchestra.run.smithi002.stdout:(3/33): ceph-mgr-18.2.4-855.g19e2a2be.el9.x86_6 4.0 MB/s | 1.5 MB 00:00 2024-09-18T04:54:09.112 INFO:teuthology.orchestra.run.smithi002.stdout:(4/33): ceph-mgr-modules-core-18.2.4-855.g19e2a 2.4 MB/s | 246 kB 00:00 2024-09-18T04:54:09.179 INFO:teuthology.orchestra.run.smithi002.stdout:(5/33): ceph-mon-18.2.4-855.g19e2a2be.el9.x86_6 11 MB/s | 4.7 MB 00:00 2024-09-18T04:54:09.287 INFO:teuthology.orchestra.run.smithi002.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.4 MB/s | 253 kB 00:00 2024-09-18T04:54:09.329 INFO:teuthology.orchestra.run.smithi002.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-18T04:54:09.496 INFO:teuthology.orchestra.run.smithi002.stdout:(8/33): ceph-osd-18.2.4-855.g19e2a2be.el9.x86_6 31 MB/s | 17 MB 00:00 2024-09-18T04:54:09.530 INFO:teuthology.orchestra.run.smithi002.stdout:(9/33): python3-cryptography-36.0.1-4.el9.x86_6 3.5 MB/s | 1.2 MB 00:00 2024-09-18T04:54:09.555 INFO:teuthology.orchestra.run.smithi002.stdout:(10/33): python3-pycparser-2.20-6.el9.noarch.rp 600 kB/s | 135 kB 00:00 2024-09-18T04:54:09.580 INFO:teuthology.orchestra.run.smithi002.stdout:(11/33): python3-urllib3-1.26.5-6.el9.noarch.rp 4.2 MB/s | 215 kB 00:00 2024-09-18T04:54:09.631 INFO:teuthology.orchestra.run.smithi002.stdout:(12/33): python3-requests-2.25.1-8.el9.noarch.r 931 kB/s | 125 kB 00:00 2024-09-18T04:54:09.864 INFO:teuthology.orchestra.run.smithi002.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 122 kB/s | 35 kB 00:00 2024-09-18T04:54:09.998 INFO:teuthology.orchestra.run.smithi002.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 139 kB/s | 51 kB 00:00 2024-09-18T04:54:10.056 INFO:teuthology.orchestra.run.smithi002.stdout:(15/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 748 kB/s | 43 kB 00:00 2024-09-18T04:54:10.073 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 3/4 2024-09-18T04:54:10.082 INFO:teuthology.orchestra.run.smithi002.stdout:(16/33): python3-toml-0.10.2-6.el9.noarch.rpm 192 kB/s | 42 kB 00:00 2024-09-18T04:54:10.115 INFO:teuthology.orchestra.run.smithi002.stdout:(17/33): python3-mako-1.1.4-6.el9.noarch.rpm 307 kB/s | 172 kB 00:00 2024-09-18T04:54:10.140 INFO:teuthology.orchestra.run.smithi002.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-09-18T04:54:10.156 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 4/4 2024-09-18T04:54:10.174 INFO:teuthology.orchestra.run.smithi002.stdout:(19/33): python3-jaraco-classes-3.2.1-5.el9.noa 699 kB/s | 18 kB 00:00 2024-09-18T04:54:10.208 INFO:teuthology.orchestra.run.smithi002.stdout:(20/33): python3-cherrypy-18.6.1-2.el9.noarch.r 2.8 MB/s | 358 kB 00:00 2024-09-18T04:54:10.233 INFO:teuthology.orchestra.run.smithi002.stdout:(21/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 93 kB/s | 11 kB 00:00 2024-09-18T04:54:10.258 INFO:teuthology.orchestra.run.smithi002.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 283 kB/s | 23 kB 00:00 2024-09-18T04:54:10.283 INFO:teuthology.orchestra.run.smithi002.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 258 kB/s | 19 kB 00:00 2024-09-18T04:54:10.300 INFO:teuthology.orchestra.run.smithi002.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 293 kB/s | 20 kB 00:00 2024-09-18T04:54:10.317 INFO:teuthology.orchestra.run.smithi002.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 787 kB/s | 46 kB 00:00 2024-09-18T04:54:10.343 INFO:teuthology.orchestra.run.smithi002.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 1.3 MB/s | 79 kB 00:00 2024-09-18T04:54:10.360 INFO:teuthology.orchestra.run.smithi002.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 394 kB/s | 16 kB 00:00 2024-09-18T04:54:10.385 INFO:teuthology.orchestra.run.smithi002.stdout:(28/33): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-18T04:54:10.410 INFO:teuthology.orchestra.run.smithi002.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.3 MB/s | 90 kB 00:00 2024-09-18T04:54:10.435 INFO:teuthology.orchestra.run.smithi002.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-09-18T04:54:10.461 INFO:teuthology.orchestra.run.smithi002.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 803 kB/s | 20 kB 00:00 2024-09-18T04:54:10.486 INFO:teuthology.orchestra.run.smithi002.stdout:(32/33): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-18T04:54:10.511 INFO:teuthology.orchestra.run.smithi002.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-18T04:54:10.515 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:54:10.515 INFO:teuthology.orchestra.run.smithi002.stdout:Total 14 MB/s | 29 MB 00:02 2024-09-18T04:54:10.642 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-09-18T04:54:10.685 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-09-18T04:54:10.685 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-09-18T04:54:11.095 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-09-18T04:54:11.095 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-09-18T04:54:12.263 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 4/4 2024-09-18T04:54:12.263 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 1/4 2024-09-18T04:54:12.263 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 2/4 2024-09-18T04:54:12.263 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-18T04:54:12.368 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-09-18T04:54:12.439 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-18T04:54:12.472 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-18T04:54:12.513 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-18T04:54:12.562 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-18T04:54:12.562 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:12.562 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-09-18T04:54:12.562 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-test-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:12.562 INFO:teuthology.orchestra.run.smithi099.stdout: libcephsqlite-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:12.562 INFO:teuthology.orchestra.run.smithi099.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-18T04:54:12.562 INFO:teuthology.orchestra.run.smithi099.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-18T04:54:12.563 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:12.563 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-18T04:54:12.611 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-18T04:54:12.687 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-18T04:54:12.742 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-18T04:54:12.776 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-18T04:54:12.805 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-18T04:54:12.840 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-18T04:54:12.873 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-18T04:54:12.910 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph 2024-09-18T04:54:12.914 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-18T04:54:12.961 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-18T04:54:12.995 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-18T04:54:13.034 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-18T04:54:13.111 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-18T04:54:13.272 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-18T04:54:13.353 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-18T04:54:13.412 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-18T04:54:13.449 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-18T04:54:13.499 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:00:36 ago on Wed 18 Sep 2024 04:53:37 AM UTC. 2024-09-18T04:54:13.508 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-18T04:54:13.573 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-18T04:54:13.635 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-18T04:54:13.637 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:13.637 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2024-09-18T04:54:13.637 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:13.637 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-09-18T04:54:13.637 INFO:teuthology.orchestra.run.smithi099.stdout: ceph x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 6.5 k 2024-09-18T04:54:13.637 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2024-09-18T04:54:13.638 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 2.1 M 2024-09-18T04:54:13.638 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 1.5 M 2024-09-18T04:54:13.638 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 246 k 2024-09-18T04:54:13.638 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 4.7 M 2024-09-18T04:54:13.638 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 17 M 2024-09-18T04:54:13.638 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-18T04:54:13.638 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-18T04:54:13.638 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-18T04:54:13.638 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-18T04:54:13.638 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-18T04:54:13.638 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-18T04:54:13.638 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-18T04:54:13.638 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-18T04:54:13.639 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-18T04:54:13.639 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-18T04:54:13.639 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-18T04:54:13.639 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-18T04:54:13.639 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-18T04:54:13.639 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-18T04:54:13.639 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-18T04:54:13.639 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-18T04:54:13.639 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-18T04:54:13.639 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-18T04:54:13.639 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-18T04:54:13.639 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-18T04:54:13.639 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-18T04:54:13.640 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-18T04:54:13.640 INFO:teuthology.orchestra.run.smithi099.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-18T04:54:13.640 INFO:teuthology.orchestra.run.smithi099.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-18T04:54:13.640 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-18T04:54:13.640 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-18T04:54:13.640 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-18T04:54:13.640 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:13.640 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-09-18T04:54:13.640 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:13.640 INFO:teuthology.orchestra.run.smithi099.stdout:Install 33 Packages 2024-09-18T04:54:13.640 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:13.641 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 29 M 2024-09-18T04:54:13.641 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 104 M 2024-09-18T04:54:13.641 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-09-18T04:54:13.780 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-18T04:54:13.837 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-18T04:54:13.919 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-18T04:54:14.015 INFO:teuthology.orchestra.run.smithi099.stdout:(1/33): ceph-18.2.4-855.g19e2a2be.el9.x86_64.rp 51 kB/s | 6.5 kB 00:00 2024-09-18T04:54:14.017 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-18T04:54:14.052 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-18T04:54:14.178 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-18T04:54:14.198 INFO:teuthology.orchestra.run.smithi099.stdout:(2/33): ceph-mgr-18.2.4-855.g19e2a2be.el9.x86_6 4.7 MB/s | 1.5 MB 00:00 2024-09-18T04:54:14.257 INFO:teuthology.orchestra.run.smithi099.stdout:(3/33): ceph-mds-18.2.4-855.g19e2a2be.el9.x86_6 5.7 MB/s | 2.1 MB 00:00 2024-09-18T04:54:14.324 INFO:teuthology.orchestra.run.smithi099.stdout:(4/33): ceph-mgr-modules-core-18.2.4-855.g19e2a 3.6 MB/s | 246 kB 00:00 2024-09-18T04:54:14.349 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-855.g19e2a2be.el9.n 28/33 2024-09-18T04:54:14.379 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 29/33 2024-09-18T04:54:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 29/33 2024-09-18T04:54:14.406 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T04:54:14.406 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-18T04:54:14.406 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-18T04:54:14.406 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-18T04:54:14.406 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:14.491 INFO:teuthology.orchestra.run.smithi099.stdout:(5/33): ceph-mon-18.2.4-855.g19e2a2be.el9.x86_6 9.8 MB/s | 4.7 MB 00:00 2024-09-18T04:54:14.516 INFO:teuthology.orchestra.run.smithi099.stdout:(6/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-18T04:54:14.558 INFO:teuthology.orchestra.run.smithi099.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-18T04:54:14.600 INFO:teuthology.orchestra.run.smithi099.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 3.2 MB/s | 135 kB 00:00 2024-09-18T04:54:14.767 INFO:teuthology.orchestra.run.smithi099.stdout:(9/33): ceph-osd-18.2.4-855.g19e2a2be.el9.x86_6 29 MB/s | 17 MB 00:00 2024-09-18T04:54:14.792 INFO:teuthology.orchestra.run.smithi099.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 651 kB/s | 125 kB 00:00 2024-09-18T04:54:14.843 INFO:teuthology.orchestra.run.smithi099.stdout:(11/33): python3-cryptography-36.0.1-4.el9.x86_ 3.5 MB/s | 1.2 MB 00:00 2024-09-18T04:54:14.910 INFO:teuthology.orchestra.run.smithi099.stdout:(12/33): python3-markupsafe-1.1.1-12.el9.x86_64 522 kB/s | 35 kB 00:00 2024-09-18T04:54:14.935 INFO:teuthology.orchestra.run.smithi099.stdout:(13/33): python3-urllib3-1.26.5-6.el9.noarch.rp 1.3 MB/s | 215 kB 00:00 2024-09-18T04:54:14.960 INFO:teuthology.orchestra.run.smithi099.stdout:(14/33): python3-mako-1.1.4-6.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-09-18T04:54:14.985 INFO:teuthology.orchestra.run.smithi099.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-09-18T04:54:15.036 INFO:teuthology.orchestra.run.smithi099.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 578 kB/s | 43 kB 00:00 2024-09-18T04:54:15.064 INFO:teuthology.orchestra.run.smithi099.stdout:(17/33): python3-toml-0.10.2-6.el9.noarch.rpm 323 kB/s | 42 kB 00:00 2024-09-18T04:54:15.098 INFO:teuthology.orchestra.run.smithi099.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-09-18T04:54:15.123 INFO:teuthology.orchestra.run.smithi099.stdout:(19/33): python3-cherrypy-18.6.1-2.el9.noarch.r 4.1 MB/s | 358 kB 00:00 2024-09-18T04:54:15.149 INFO:teuthology.orchestra.run.smithi099.stdout:(20/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 127 kB/s | 11 kB 00:00 2024-09-18T04:54:15.174 INFO:teuthology.orchestra.run.smithi099.stdout:(21/33): python3-jaraco-classes-3.2.1-5.el9.noa 238 kB/s | 18 kB 00:00 2024-09-18T04:54:15.199 INFO:teuthology.orchestra.run.smithi099.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 307 kB/s | 23 kB 00:00 2024-09-18T04:54:15.224 INFO:teuthology.orchestra.run.smithi099.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 258 kB/s | 19 kB 00:00 2024-09-18T04:54:15.249 INFO:teuthology.orchestra.run.smithi099.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-09-18T04:54:15.275 INFO:teuthology.orchestra.run.smithi099.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 921 kB/s | 46 kB 00:00 2024-09-18T04:54:15.300 INFO:teuthology.orchestra.run.smithi099.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-18T04:54:15.325 INFO:teuthology.orchestra.run.smithi099.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 327 kB/s | 16 kB 00:00 2024-09-18T04:54:15.351 INFO:teuthology.orchestra.run.smithi099.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-18T04:54:15.376 INFO:teuthology.orchestra.run.smithi099.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 712 kB/s | 36 kB 00:00 2024-09-18T04:54:15.401 INFO:teuthology.orchestra.run.smithi099.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-18T04:54:15.435 INFO:teuthology.orchestra.run.smithi099.stdout:(31/33): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-18T04:54:15.460 INFO:teuthology.orchestra.run.smithi099.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 342 kB/s | 20 kB 00:00 2024-09-18T04:54:15.493 INFO:teuthology.orchestra.run.smithi099.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.6 MB/s | 427 kB 00:00 2024-09-18T04:54:15.498 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:54:15.498 INFO:teuthology.orchestra.run.smithi099.stdout:Total 16 MB/s | 29 MB 00:01 2024-09-18T04:54:15.615 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-09-18T04:54:15.653 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-09-18T04:54:15.653 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-09-18T04:54:15.914 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 30/33 2024-09-18T04:54:15.945 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 30/33 2024-09-18T04:54:15.945 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T04:54:15.945 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-18T04:54:15.945 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-18T04:54:15.945 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-18T04:54:15.946 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:16.052 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-09-18T04:54:16.053 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-09-18T04:54:16.422 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 31/33 2024-09-18T04:54:16.448 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 31/33 2024-09-18T04:54:16.448 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T04:54:16.448 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-18T04:54:16.449 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-18T04:54:16.449 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-18T04:54:16.449 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:16.657 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 32/33 2024-09-18T04:54:16.682 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 32/33 2024-09-18T04:54:16.682 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T04:54:16.682 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-18T04:54:16.682 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-18T04:54:16.682 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-18T04:54:16.682 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:16.757 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 33/33 2024-09-18T04:54:17.355 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-09-18T04:54:17.430 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-18T04:54:17.462 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-18T04:54:17.504 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-18T04:54:17.602 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-18T04:54:17.698 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-18T04:54:17.739 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-18T04:54:17.775 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-18T04:54:17.804 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-18T04:54:17.839 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-18T04:54:17.881 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-18T04:54:17.929 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-18T04:54:17.964 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-18T04:54:17.999 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-18T04:54:18.037 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-18T04:54:18.110 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-18T04:54:18.248 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-18T04:54:18.336 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-18T04:54:18.397 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-18T04:54:18.437 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-18T04:54:18.480 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 33/33 2024-09-18T04:54:18.480 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 1/33 2024-09-18T04:54:18.480 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 2/33 2024-09-18T04:54:18.480 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 3/33 2024-09-18T04:54:18.480 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 4/33 2024-09-18T04:54:18.480 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 5/33 2024-09-18T04:54:18.480 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-855.g19e2a2be.el9.n 6/33 2024-09-18T04:54:18.480 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-18T04:54:18.481 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-18T04:54:18.481 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-18T04:54:18.481 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-18T04:54:18.481 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-18T04:54:18.481 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-18T04:54:18.481 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-18T04:54:18.481 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-18T04:54:18.481 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-18T04:54:18.482 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-18T04:54:18.482 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-18T04:54:18.482 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-18T04:54:18.482 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-18T04:54:18.482 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-18T04:54:18.482 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-18T04:54:18.483 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-18T04:54:18.483 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-18T04:54:18.483 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-18T04:54:18.483 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-18T04:54:18.483 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-18T04:54:18.484 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-18T04:54:18.485 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-18T04:54:18.485 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-18T04:54:18.485 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-18T04:54:18.485 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-18T04:54:18.485 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-18T04:54:18.502 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-18T04:54:18.557 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-18T04:54:18.744 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-18T04:54:18.798 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-18T04:54:18.877 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-18T04:54:18.965 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-18T04:54:19.005 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-18T04:54:19.121 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-18T04:54:19.280 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-modules-core-2:18.2.4-855.g19e2a2be.el9.n 28/33 2024-09-18T04:54:19.307 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 29/33 2024-09-18T04:54:19.333 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 29/33 2024-09-18T04:54:19.333 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T04:54:19.333 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-18T04:54:19.333 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-18T04:54:19.333 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-18T04:54:19.334 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:19.518 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-18T04:54:19.518 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:19.518 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-09-18T04:54:19.518 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:19.518 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:19.518 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:19.518 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-modules-core-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-18T04:54:19.518 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:19.518 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:19.518 INFO:teuthology.orchestra.run.smithi002.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-18T04:54:19.519 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-18T04:54:19.519 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-18T04:54:19.519 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-18T04:54:19.519 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-18T04:54:19.519 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-18T04:54:19.519 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-18T04:54:19.519 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-18T04:54:19.519 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-18T04:54:19.519 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-18T04:54:19.519 INFO:teuthology.orchestra.run.smithi002.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-18T04:54:19.519 INFO:teuthology.orchestra.run.smithi002.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-18T04:54:19.519 INFO:teuthology.orchestra.run.smithi002.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-18T04:54:19.519 INFO:teuthology.orchestra.run.smithi002.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-18T04:54:19.520 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-18T04:54:19.520 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ply-3.11-14.el9.noarch 2024-09-18T04:54:19.520 INFO:teuthology.orchestra.run.smithi002.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-18T04:54:19.520 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-18T04:54:19.520 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-18T04:54:19.520 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-18T04:54:19.520 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-18T04:54:19.520 INFO:teuthology.orchestra.run.smithi002.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-18T04:54:19.520 INFO:teuthology.orchestra.run.smithi002.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-18T04:54:19.520 INFO:teuthology.orchestra.run.smithi002.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-18T04:54:19.520 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-18T04:54:19.520 INFO:teuthology.orchestra.run.smithi002.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-18T04:54:19.520 INFO:teuthology.orchestra.run.smithi002.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-18T04:54:19.521 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:19.521 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-18T04:54:19.889 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-base 2024-09-18T04:54:20.498 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:00:49 ago on Wed 18 Sep 2024 04:53:31 AM UTC. 2024-09-18T04:54:20.590 INFO:teuthology.orchestra.run.smithi002.stdout:Package ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-18T04:54:20.634 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-18T04:54:20.636 INFO:teuthology.orchestra.run.smithi002.stdout:Nothing to do. 2024-09-18T04:54:20.636 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-18T04:54:20.710 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install cephadm 2024-09-18T04:54:20.800 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 30/33 2024-09-18T04:54:20.830 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 30/33 2024-09-18T04:54:20.830 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T04:54:20.830 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-18T04:54:20.830 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-18T04:54:20.830 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-18T04:54:20.830 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:21.293 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 31/33 2024-09-18T04:54:21.310 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:00:50 ago on Wed 18 Sep 2024 04:53:31 AM UTC. 2024-09-18T04:54:21.318 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 31/33 2024-09-18T04:54:21.318 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T04:54:21.318 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-18T04:54:21.318 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-18T04:54:21.319 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-18T04:54:21.319 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:21.446 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-18T04:54:21.447 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:21.448 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2024-09-18T04:54:21.448 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:21.448 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-09-18T04:54:21.448 INFO:teuthology.orchestra.run.smithi002.stdout: cephadm noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 226 k 2024-09-18T04:54:21.448 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:21.448 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-09-18T04:54:21.448 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:21.448 INFO:teuthology.orchestra.run.smithi002.stdout:Install 1 Package 2024-09-18T04:54:21.448 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:21.448 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 226 k 2024-09-18T04:54:21.448 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 217 k 2024-09-18T04:54:21.448 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-09-18T04:54:21.518 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 32/33 2024-09-18T04:54:21.542 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 32/33 2024-09-18T04:54:21.543 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T04:54:21.543 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-18T04:54:21.543 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-18T04:54:21.543 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-18T04:54:21.543 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:21.607 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 33/33 2024-09-18T04:54:21.689 INFO:teuthology.orchestra.run.smithi002.stdout:cephadm-18.2.4-855.g19e2a2be.el9.noarch.rpm 938 kB/s | 226 kB 00:00 2024-09-18T04:54:21.690 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:54:21.690 INFO:teuthology.orchestra.run.smithi002.stdout:Total 933 kB/s | 226 kB 00:00 2024-09-18T04:54:21.690 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-09-18T04:54:21.696 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-09-18T04:54:21.696 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-09-18T04:54:21.698 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-09-18T04:54:21.699 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-09-18T04:54:21.772 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-09-18T04:54:22.211 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-18T04:54:22.296 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-18T04:54:22.665 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-18T04:54:22.914 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-18T04:54:22.914 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:22.914 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-09-18T04:54:22.914 INFO:teuthology.orchestra.run.smithi002.stdout: cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-18T04:54:22.915 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:22.915 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-18T04:54:23.109 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-immutable-object-cache 2024-09-18T04:54:23.304 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 33/33 2024-09-18T04:54:23.305 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 1/33 2024-09-18T04:54:23.305 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 2/33 2024-09-18T04:54:23.305 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 3/33 2024-09-18T04:54:23.305 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 4/33 2024-09-18T04:54:23.305 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 5/33 2024-09-18T04:54:23.305 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-modules-core-2:18.2.4-855.g19e2a2be.el9.n 6/33 2024-09-18T04:54:23.305 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-18T04:54:23.305 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-18T04:54:23.305 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-18T04:54:23.305 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-18T04:54:23.305 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-18T04:54:23.305 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-18T04:54:23.305 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-18T04:54:23.306 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-18T04:54:23.306 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-18T04:54:23.306 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-18T04:54:23.306 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-18T04:54:23.306 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-18T04:54:23.306 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-18T04:54:23.306 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-18T04:54:23.306 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-18T04:54:23.306 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-18T04:54:23.306 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-18T04:54:23.306 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-18T04:54:23.306 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-18T04:54:23.306 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-18T04:54:23.307 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-18T04:54:23.307 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-18T04:54:23.307 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-18T04:54:23.307 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-18T04:54:23.307 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-18T04:54:23.307 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-18T04:54:23.710 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:00:52 ago on Wed 18 Sep 2024 04:53:31 AM UTC. 2024-09-18T04:54:23.847 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-18T04:54:23.848 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:23.848 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repo Size 2024-09-18T04:54:23.848 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:23.848 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-09-18T04:54:23.848 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 143 k 2024-09-18T04:54:23.848 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:23.848 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-09-18T04:54:23.848 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:23.849 INFO:teuthology.orchestra.run.smithi002.stdout:Install 1 Package 2024-09-18T04:54:23.849 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:23.849 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 143 k 2024-09-18T04:54:23.849 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 395 k 2024-09-18T04:54:23.849 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-09-18T04:54:24.061 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-immutable-object-cache-18.2.4-855.g19e2a2b 674 kB/s | 143 kB 00:00 2024-09-18T04:54:24.061 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:54:24.062 INFO:teuthology.orchestra.run.smithi002.stdout:Total 669 kB/s | 143 kB 00:00 2024-09-18T04:54:24.062 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-09-18T04:54:24.070 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-09-18T04:54:24.070 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-09-18T04:54:24.131 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-09-18T04:54:24.131 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-09-18T04:54:24.289 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-09-18T04:54:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-18T04:54:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:24.329 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-09-18T04:54:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mds-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-modules-core-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-18T04:54:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mon-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-osd-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-18T04:54:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-18T04:54:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-18T04:54:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-18T04:54:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-18T04:54:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-18T04:54:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-18T04:54:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-18T04:54:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-18T04:54:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-18T04:54:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-18T04:54:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-18T04:54:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-18T04:54:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-18T04:54:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-18T04:54:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: python3-ply-3.11-14.el9.noarch 2024-09-18T04:54:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-18T04:54:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-18T04:54:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-18T04:54:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-18T04:54:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-18T04:54:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-18T04:54:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-18T04:54:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-18T04:54:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-18T04:54:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-18T04:54:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-18T04:54:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:24.331 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-18T04:54:24.337 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.e 1/1 2024-09-18T04:54:24.363 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.e 1/1 2024-09-18T04:54:24.363 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T04:54:24.363 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-18T04:54:24.363 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:24.726 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-base 2024-09-18T04:54:25.321 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:00:48 ago on Wed 18 Sep 2024 04:53:37 AM UTC. 2024-09-18T04:54:25.365 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.e 1/1 2024-09-18T04:54:25.365 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:25.365 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-09-18T04:54:25.365 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:25.365 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:25.365 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-18T04:54:25.409 INFO:teuthology.orchestra.run.smithi099.stdout:Package ceph-base-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-18T04:54:25.450 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-18T04:54:25.451 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2024-09-18T04:54:25.451 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-18T04:54:25.517 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install cephadm 2024-09-18T04:54:25.590 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-mgr 2024-09-18T04:54:26.087 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:00:49 ago on Wed 18 Sep 2024 04:53:37 AM UTC. 2024-09-18T04:54:26.197 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:00:55 ago on Wed 18 Sep 2024 04:53:31 AM UTC. 2024-09-18T04:54:26.216 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-18T04:54:26.217 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:26.217 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2024-09-18T04:54:26.217 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:26.217 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-09-18T04:54:26.217 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 226 k 2024-09-18T04:54:26.217 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:26.217 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-09-18T04:54:26.218 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:26.218 INFO:teuthology.orchestra.run.smithi099.stdout:Install 1 Package 2024-09-18T04:54:26.218 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:26.218 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 226 k 2024-09-18T04:54:26.218 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 217 k 2024-09-18T04:54:26.218 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-09-18T04:54:26.289 INFO:teuthology.orchestra.run.smithi002.stdout:Package ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-18T04:54:26.333 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-18T04:54:26.334 INFO:teuthology.orchestra.run.smithi002.stdout:Nothing to do. 2024-09-18T04:54:26.334 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-18T04:54:26.419 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-mgr-dashboard 2024-09-18T04:54:26.452 INFO:teuthology.orchestra.run.smithi099.stdout:cephadm-18.2.4-855.g19e2a2be.el9.noarch.rpm 966 kB/s | 226 kB 00:00 2024-09-18T04:54:26.453 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:54:26.453 INFO:teuthology.orchestra.run.smithi099.stdout:Total 957 kB/s | 226 kB 00:00 2024-09-18T04:54:26.454 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-09-18T04:54:26.458 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-09-18T04:54:26.459 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-09-18T04:54:26.461 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-09-18T04:54:26.461 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-09-18T04:54:26.526 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-09-18T04:54:26.989 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-18T04:54:27.027 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:00:56 ago on Wed 18 Sep 2024 04:53:31 AM UTC. 2024-09-18T04:54:27.122 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-18T04:54:27.164 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-18T04:54:27.164 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:27.164 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2024-09-18T04:54:27.164 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:27.164 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-09-18T04:54:27.165 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-dashboard noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 3.5 M 2024-09-18T04:54:27.165 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-09-18T04:54:27.165 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-grafana-dashboards noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 24 k 2024-09-18T04:54:27.165 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-prometheus-alerts noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 16 k 2024-09-18T04:54:27.165 INFO:teuthology.orchestra.run.smithi002.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-18T04:54:27.165 INFO:teuthology.orchestra.run.smithi002.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-18T04:54:27.165 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:27.165 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-09-18T04:54:27.165 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:27.165 INFO:teuthology.orchestra.run.smithi002.stdout:Install 5 Packages 2024-09-18T04:54:27.165 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:27.166 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 3.8 M 2024-09-18T04:54:27.166 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 87 M 2024-09-18T04:54:27.166 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-09-18T04:54:27.464 INFO:teuthology.orchestra.run.smithi002.stdout:(1/5): ceph-grafana-dashboards-18.2.4-855.g19e2 171 kB/s | 24 kB 00:00 2024-09-18T04:54:27.487 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-18T04:54:27.489 INFO:teuthology.orchestra.run.smithi002.stdout:(2/5): ceph-prometheus-alerts-18.2.4-855.g19e2a 98 kB/s | 16 kB 00:00 2024-09-18T04:54:27.549 INFO:teuthology.orchestra.run.smithi002.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 367 kB/s | 31 kB 00:00 2024-09-18T04:54:27.590 INFO:teuthology.orchestra.run.smithi002.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 1.8 MB/s | 188 kB 00:00 2024-09-18T04:54:27.707 INFO:teuthology.orchestra.run.smithi002.stdout:(5/5): ceph-mgr-dashboard-18.2.4-855.g19e2a2be. 9.2 MB/s | 3.5 MB 00:00 2024-09-18T04:54:27.710 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:54:27.710 INFO:teuthology.orchestra.run.smithi002.stdout:Total 6.9 MB/s | 3.8 MB 00:00 2024-09-18T04:54:27.711 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-09-18T04:54:27.718 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-09-18T04:54:27.718 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-09-18T04:54:27.732 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/1 2024-09-18T04:54:27.733 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:27.733 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-09-18T04:54:27.733 INFO:teuthology.orchestra.run.smithi099.stdout: cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-18T04:54:27.733 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:27.733 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-18T04:54:27.810 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-09-18T04:54:27.811 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-09-18T04:54:27.923 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-immutable-object-cache 2024-09-18T04:54:28.074 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-09-18T04:54:28.148 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-09-18T04:54:28.179 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-09-18T04:54:28.208 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-855.g19e2a2be.el9.no 3/5 2024-09-18T04:54:28.503 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:00:51 ago on Wed 18 Sep 2024 04:53:37 AM UTC. 2024-09-18T04:54:28.638 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-18T04:54:28.638 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repo Size 2024-09-18T04:54:28.639 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:28.639 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-09-18T04:54:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 143 k 2024-09-18T04:54:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:28.639 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-09-18T04:54:28.639 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:28.639 INFO:teuthology.orchestra.run.smithi099.stdout:Install 1 Package 2024-09-18T04:54:28.639 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:28.639 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 143 k 2024-09-18T04:54:28.639 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 395 k 2024-09-18T04:54:28.639 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-09-18T04:54:28.849 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-immutable-object-cache-18.2.4-855.g19e2a2b 682 kB/s | 143 kB 00:00 2024-09-18T04:54:28.849 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:54:28.850 INFO:teuthology.orchestra.run.smithi099.stdout:Total 678 kB/s | 143 kB 00:00 2024-09-18T04:54:28.850 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-09-18T04:54:28.857 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-09-18T04:54:28.857 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-09-18T04:54:28.913 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-09-18T04:54:28.913 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-09-18T04:54:29.064 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-09-18T04:54:29.124 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.e 1/1 2024-09-18T04:54:29.149 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.e 1/1 2024-09-18T04:54:29.149 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T04:54:29.149 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-18T04:54:29.149 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:29.327 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-855.g19e2a2be.el9.n 4/5 2024-09-18T04:54:29.361 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 5/5 2024-09-18T04:54:30.091 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.e 1/1 2024-09-18T04:54:30.091 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:30.091 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-09-18T04:54:30.091 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-immutable-object-cache-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:30.091 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:30.091 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-18T04:54:30.335 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-mgr 2024-09-18T04:54:30.405 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 5/5 2024-09-18T04:54:30.405 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-855.g19e2a2be.el9.n 1/5 2024-09-18T04:54:30.405 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 2/5 2024-09-18T04:54:30.405 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-855.g19e2a2be.el9.no 3/5 2024-09-18T04:54:30.405 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-09-18T04:54:30.663 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-09-18T04:54:30.663 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:30.663 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-09-18T04:54:30.663 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-grafana-dashboards-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-18T04:54:30.663 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-18T04:54:30.663 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-prometheus-alerts-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-18T04:54:30.663 INFO:teuthology.orchestra.run.smithi002.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-18T04:54:30.664 INFO:teuthology.orchestra.run.smithi002.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-18T04:54:30.664 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:30.664 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-18T04:54:30.872 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-18T04:54:30.931 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:00:53 ago on Wed 18 Sep 2024 04:53:37 AM UTC. 2024-09-18T04:54:31.020 INFO:teuthology.orchestra.run.smithi099.stdout:Package ceph-mgr-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-18T04:54:31.062 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-18T04:54:31.063 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2024-09-18T04:54:31.063 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-18T04:54:31.135 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-mgr-dashboard 2024-09-18T04:54:31.473 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:00 ago on Wed 18 Sep 2024 04:53:31 AM UTC. 2024-09-18T04:54:31.611 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-18T04:54:31.613 INFO:teuthology.orchestra.run.smithi002.stdout:====================================================================================== 2024-09-18T04:54:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2024-09-18T04:54:31.613 INFO:teuthology.orchestra.run.smithi002.stdout:====================================================================================== 2024-09-18T04:54:31.613 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-09-18T04:54:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 7.4 M 2024-09-18T04:54:31.613 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-09-18T04:54:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-18T04:54:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-18T04:54:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-18T04:54:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-18T04:54:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-18T04:54:31.613 INFO:teuthology.orchestra.run.smithi002.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-18T04:54:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-18T04:54:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-18T04:54:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-18T04:54:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-18T04:54:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-18T04:54:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:31.614 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-09-18T04:54:31.614 INFO:teuthology.orchestra.run.smithi002.stdout:====================================================================================== 2024-09-18T04:54:31.614 INFO:teuthology.orchestra.run.smithi002.stdout:Install 12 Packages 2024-09-18T04:54:31.614 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:31.615 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 38 M 2024-09-18T04:54:31.615 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 212 M 2024-09-18T04:54:31.615 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-09-18T04:54:31.709 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:00:54 ago on Wed 18 Sep 2024 04:53:37 AM UTC. 2024-09-18T04:54:31.838 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-18T04:54:31.839 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:31.839 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2024-09-18T04:54:31.839 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:31.839 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-09-18T04:54:31.840 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 3.5 M 2024-09-18T04:54:31.840 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2024-09-18T04:54:31.840 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 24 k 2024-09-18T04:54:31.840 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 16 k 2024-09-18T04:54:31.840 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-18T04:54:31.840 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-18T04:54:31.840 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:31.840 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-09-18T04:54:31.840 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:31.840 INFO:teuthology.orchestra.run.smithi099.stdout:Install 5 Packages 2024-09-18T04:54:31.840 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:31.841 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 3.8 M 2024-09-18T04:54:31.841 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 87 M 2024-09-18T04:54:31.841 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-09-18T04:54:32.061 INFO:teuthology.orchestra.run.smithi099.stdout:(1/5): ceph-grafana-dashboards-18.2.4-855.g19e2 129 kB/s | 24 kB 00:00 2024-09-18T04:54:32.086 INFO:teuthology.orchestra.run.smithi099.stdout:(2/5): ceph-prometheus-alerts-18.2.4-855.g19e2a 77 kB/s | 16 kB 00:00 2024-09-18T04:54:32.128 INFO:teuthology.orchestra.run.smithi099.stdout:(3/5): python3-repoze-lru-0.7-16.el9.noarch.rpm 460 kB/s | 31 kB 00:00 2024-09-18T04:54:32.132 INFO:teuthology.orchestra.run.smithi002.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.4-85 17 MB/s | 7.4 MB 00:00 2024-09-18T04:54:32.178 INFO:teuthology.orchestra.run.smithi099.stdout:(4/5): python3-routes-2.5.1-5.el9.noarch.rpm 2.0 MB/s | 188 kB 00:00 2024-09-18T04:54:32.233 INFO:teuthology.orchestra.run.smithi002.stdout:(2/12): libquadmath-11.5.0-2.el9.x86_64.rpm 348 kB/s | 191 kB 00:00 2024-09-18T04:54:32.258 INFO:teuthology.orchestra.run.smithi002.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 261 kB/s | 33 kB 00:00 2024-09-18T04:54:32.303 INFO:teuthology.orchestra.run.smithi099.stdout:(5/5): ceph-mgr-dashboard-18.2.4-855.g19e2a2be. 8.2 MB/s | 3.5 MB 00:00 2024-09-18T04:54:32.306 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:54:32.307 INFO:teuthology.orchestra.run.smithi099.stdout:Total 8.1 MB/s | 3.8 MB 00:00 2024-09-18T04:54:32.307 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-09-18T04:54:32.314 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-09-18T04:54:32.314 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-09-18T04:54:32.341 INFO:teuthology.orchestra.run.smithi002.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 218 kB/s | 18 kB 00:00 2024-09-18T04:54:32.397 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-09-18T04:54:32.397 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-09-18T04:54:32.417 INFO:teuthology.orchestra.run.smithi002.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 514 kB/s | 38 kB 00:00 2024-09-18T04:54:32.500 INFO:teuthology.orchestra.run.smithi002.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 11 MB/s | 3.0 MB 00:00 2024-09-18T04:54:32.534 INFO:teuthology.orchestra.run.smithi002.stdout:(7/12): libgfortran-11.5.0-2.el9.x86_64.rpm 942 kB/s | 801 kB 00:00 2024-09-18T04:54:32.559 INFO:teuthology.orchestra.run.smithi002.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-09-18T04:54:32.601 INFO:teuthology.orchestra.run.smithi002.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 5.9 MB/s | 250 kB 00:00 2024-09-18T04:54:32.657 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-09-18T04:54:32.702 INFO:teuthology.orchestra.run.smithi002.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 17 MB/s | 4.9 MB 00:00 2024-09-18T04:54:32.743 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 1/5 2024-09-18T04:54:32.774 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 2/5 2024-09-18T04:54:32.803 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-prometheus-alerts-2:18.2.4-855.g19e2a2be.el9.no 3/5 2024-09-18T04:54:32.977 INFO:teuthology.orchestra.run.smithi002.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2024-09-18T04:54:33.044 INFO:teuthology.orchestra.run.smithi002.stdout:(12/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 10 MB/s | 5.1 MB 00:00 2024-09-18T04:54:33.044 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:54:33.044 INFO:teuthology.orchestra.run.smithi002.stdout:Total 27 MB/s | 38 MB 00:01 2024-09-18T04:54:33.414 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-09-18T04:54:33.445 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-09-18T04:54:33.445 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-09-18T04:54:33.840 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-grafana-dashboards-2:18.2.4-855.g19e2a2be.el9.n 4/5 2024-09-18T04:54:33.890 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 5/5 2024-09-18T04:54:33.975 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-09-18T04:54:33.975 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-09-18T04:54:34.761 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-09-18T04:54:34.842 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-18T04:54:34.886 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-18T04:54:34.991 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-18T04:54:35.039 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 5/5 2024-09-18T04:54:35.039 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-grafana-dashboards-2:18.2.4-855.g19e2a2be.el9.n 1/5 2024-09-18T04:54:35.039 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 2/5 2024-09-18T04:54:35.039 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-prometheus-alerts-2:18.2.4-855.g19e2a2be.el9.no 3/5 2024-09-18T04:54:35.039 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 4/5 2024-09-18T04:54:35.135 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-18T04:54:35.301 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 5/5 2024-09-18T04:54:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:35.302 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-09-18T04:54:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-grafana-dashboards-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-18T04:54:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-dashboard-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-18T04:54:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-prometheus-alerts-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-18T04:54:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-18T04:54:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-18T04:54:35.302 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:35.302 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-18T04:54:35.327 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-18T04:54:35.358 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-18T04:54:35.437 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-18T04:54:35.503 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-18T04:54:36.071 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:00:59 ago on Wed 18 Sep 2024 04:53:37 AM UTC. 2024-09-18T04:54:36.127 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-18T04:54:36.202 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-18T04:54:36.203 INFO:teuthology.orchestra.run.smithi099.stdout:====================================================================================== 2024-09-18T04:54:36.204 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2024-09-18T04:54:36.204 INFO:teuthology.orchestra.run.smithi099.stdout:====================================================================================== 2024-09-18T04:54:36.204 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-09-18T04:54:36.204 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 7.4 M 2024-09-18T04:54:36.204 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2024-09-18T04:54:36.204 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-18T04:54:36.204 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-18T04:54:36.204 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-18T04:54:36.204 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-18T04:54:36.204 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-18T04:54:36.204 INFO:teuthology.orchestra.run.smithi099.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-18T04:54:36.205 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-18T04:54:36.205 INFO:teuthology.orchestra.run.smithi099.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-18T04:54:36.205 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-18T04:54:36.205 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-18T04:54:36.205 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-18T04:54:36.205 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:36.205 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-09-18T04:54:36.205 INFO:teuthology.orchestra.run.smithi099.stdout:====================================================================================== 2024-09-18T04:54:36.205 INFO:teuthology.orchestra.run.smithi099.stdout:Install 12 Packages 2024-09-18T04:54:36.205 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:36.206 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 38 M 2024-09-18T04:54:36.206 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 212 M 2024-09-18T04:54:36.206 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-09-18T04:54:36.242 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-18T04:54:36.435 INFO:teuthology.orchestra.run.smithi099.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 191 kB 00:00 2024-09-18T04:54:36.477 INFO:teuthology.orchestra.run.smithi099.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.5 MB/s | 801 kB 00:00 2024-09-18T04:54:36.536 INFO:teuthology.orchestra.run.smithi099.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 326 kB/s | 33 kB 00:00 2024-09-18T04:54:36.569 INFO:teuthology.orchestra.run.smithi099.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 549 kB/s | 18 kB 00:00 2024-09-18T04:54:36.620 INFO:teuthology.orchestra.run.smithi099.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 767 kB/s | 38 kB 00:00 2024-09-18T04:54:36.713 INFO:teuthology.orchestra.run.smithi099.stdout:(6/12): ceph-mgr-diskprediction-local-18.2.4-85 16 MB/s | 7.4 MB 00:00 2024-09-18T04:54:36.778 INFO:teuthology.orchestra.run.smithi099.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 9.9 MB/s | 3.0 MB 00:00 2024-09-18T04:54:36.904 INFO:teuthology.orchestra.run.smithi099.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 41 MB/s | 5.1 MB 00:00 2024-09-18T04:54:36.979 INFO:teuthology.orchestra.run.smithi099.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 14 MB/s | 4.9 MB 00:00 2024-09-18T04:54:37.004 INFO:teuthology.orchestra.run.smithi099.stdout:(10/12): python3-devel-3.9.19-8.el9.x86_64.rpm 837 kB/s | 245 kB 00:00 2024-09-18T04:54:37.029 INFO:teuthology.orchestra.run.smithi099.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 2.0 MB/s | 250 kB 00:00 2024-09-18T04:54:37.273 INFO:teuthology.orchestra.run.smithi099.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 56 MB/s | 16 MB 00:00 2024-09-18T04:54:37.277 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:54:37.277 INFO:teuthology.orchestra.run.smithi099.stdout:Total 36 MB/s | 38 MB 00:01 2024-09-18T04:54:37.561 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-18T04:54:37.625 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-09-18T04:54:37.650 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-09-18T04:54:37.651 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-09-18T04:54:38.164 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-09-18T04:54:38.164 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-09-18T04:54:38.643 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-18T04:54:38.674 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2 12/12 2024-09-18T04:54:38.917 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-09-18T04:54:38.979 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-18T04:54:39.028 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-18T04:54:39.144 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-18T04:54:39.221 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-18T04:54:39.404 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-18T04:54:39.436 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-18T04:54:39.513 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-18T04:54:40.167 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-18T04:54:40.282 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-18T04:54:41.035 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2 12/12 2024-09-18T04:54:41.035 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2 1/12 2024-09-18T04:54:41.035 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-18T04:54:41.035 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-18T04:54:41.035 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-18T04:54:41.035 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-18T04:54:41.035 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-18T04:54:41.035 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-18T04:54:41.035 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-18T04:54:41.035 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-18T04:54:41.036 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-18T04:54:41.036 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-18T04:54:41.415 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-18T04:54:41.415 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:41.415 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-09-18T04:54:41.415 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-18T04:54:41.415 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-18T04:54:41.415 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-18T04:54:41.415 INFO:teuthology.orchestra.run.smithi002.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-18T04:54:41.415 INFO:teuthology.orchestra.run.smithi002.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-18T04:54:41.415 INFO:teuthology.orchestra.run.smithi002.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-18T04:54:41.415 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-18T04:54:41.415 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-18T04:54:41.416 INFO:teuthology.orchestra.run.smithi002.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-18T04:54:41.416 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-18T04:54:41.416 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-18T04:54:41.416 INFO:teuthology.orchestra.run.smithi002.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-18T04:54:41.416 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:41.416 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-18T04:54:41.516 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-18T04:54:41.790 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-mgr-rook 2024-09-18T04:54:42.401 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:11 ago on Wed 18 Sep 2024 04:53:31 AM UTC. 2024-09-18T04:54:42.537 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-18T04:54:42.538 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:42.539 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2024-09-18T04:54:42.539 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:42.539 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-09-18T04:54:42.539 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-rook noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 49 k 2024-09-18T04:54:42.539 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-09-18T04:54:42.539 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-18T04:54:42.539 INFO:teuthology.orchestra.run.smithi002.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-18T04:54:42.539 INFO:teuthology.orchestra.run.smithi002.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-18T04:54:42.539 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-18T04:54:42.539 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-18T04:54:42.539 INFO:teuthology.orchestra.run.smithi002.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-18T04:54:42.539 INFO:teuthology.orchestra.run.smithi002.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-18T04:54:42.539 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-18T04:54:42.539 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-18T04:54:42.540 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-18T04:54:42.540 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-18T04:54:42.540 INFO:teuthology.orchestra.run.smithi002.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-18T04:54:42.540 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:42.540 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-09-18T04:54:42.540 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:42.540 INFO:teuthology.orchestra.run.smithi002.stdout:Install 13 Packages 2024-09-18T04:54:42.540 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:42.541 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 2.2 M 2024-09-18T04:54:42.541 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 26 M 2024-09-18T04:54:42.541 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-09-18T04:54:42.556 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-18T04:54:42.576 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2 12/12 2024-09-18T04:54:42.723 INFO:teuthology.orchestra.run.smithi002.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 211 kB/s | 26 kB 00:00 2024-09-18T04:54:42.748 INFO:teuthology.orchestra.run.smithi002.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 131 kB/s | 19 kB 00:00 2024-09-18T04:54:42.773 INFO:teuthology.orchestra.run.smithi002.stdout:(3/13): ceph-mgr-rook-18.2.4-855.g19e2a2be.el9. 287 kB/s | 49 kB 00:00 2024-09-18T04:54:42.865 INFO:teuthology.orchestra.run.smithi002.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-18T04:54:42.890 INFO:teuthology.orchestra.run.smithi002.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.5 MB/s | 222 kB 00:00 2024-09-18T04:54:42.924 INFO:teuthology.orchestra.run.smithi002.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 917 kB/s | 54 kB 00:00 2024-09-18T04:54:42.958 INFO:teuthology.orchestra.run.smithi002.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 480 kB/s | 32 kB 00:00 2024-09-18T04:54:42.991 INFO:teuthology.orchestra.run.smithi002.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 217 kB/s | 14 kB 00:00 2024-09-18T04:54:43.017 INFO:teuthology.orchestra.run.smithi002.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.1 MB/s | 279 kB 00:00 2024-09-18T04:54:43.058 INFO:teuthology.orchestra.run.smithi002.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.2 MB/s | 220 kB 00:00 2024-09-18T04:54:43.092 INFO:teuthology.orchestra.run.smithi002.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 787 kB/s | 59 kB 00:00 2024-09-18T04:54:43.117 INFO:teuthology.orchestra.run.smithi002.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.5 MB/s | 90 kB 00:00 2024-09-18T04:54:43.226 INFO:teuthology.orchestra.run.smithi002.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 4.4 MB/s | 1.0 MB 00:00 2024-09-18T04:54:43.229 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:54:43.229 INFO:teuthology.orchestra.run.smithi002.stdout:Total 3.2 MB/s | 2.2 MB 00:00 2024-09-18T04:54:43.297 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-09-18T04:54:43.312 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-09-18T04:54:43.313 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-09-18T04:54:43.408 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-09-18T04:54:43.409 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-09-18T04:54:43.929 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-09-18T04:54:44.111 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-18T04:54:44.172 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-18T04:54:44.227 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-18T04:54:44.262 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-18T04:54:44.304 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-18T04:54:44.430 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-18T04:54:44.547 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-18T04:54:44.598 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-18T04:54:45.224 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2 12/12 2024-09-18T04:54:45.224 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2 1/12 2024-09-18T04:54:45.224 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-18T04:54:45.224 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-18T04:54:45.224 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-18T04:54:45.224 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-18T04:54:45.225 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-18T04:54:45.225 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-18T04:54:45.225 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-18T04:54:45.225 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-18T04:54:45.225 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-18T04:54:45.225 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-18T04:54:45.384 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-18T04:54:45.444 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-18T04:54:45.477 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-18T04:54:45.543 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-18T04:54:45.572 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 13/13 2024-09-18T04:54:45.615 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-18T04:54:45.616 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:45.616 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-09-18T04:54:45.616 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-diskprediction-local-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-18T04:54:45.616 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-18T04:54:45.616 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-18T04:54:45.616 INFO:teuthology.orchestra.run.smithi099.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-18T04:54:45.616 INFO:teuthology.orchestra.run.smithi099.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-18T04:54:45.616 INFO:teuthology.orchestra.run.smithi099.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-18T04:54:45.616 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-18T04:54:45.616 INFO:teuthology.orchestra.run.smithi099.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-18T04:54:45.616 INFO:teuthology.orchestra.run.smithi099.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-18T04:54:45.616 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-18T04:54:45.617 INFO:teuthology.orchestra.run.smithi099.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-18T04:54:45.617 INFO:teuthology.orchestra.run.smithi099.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-18T04:54:45.617 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:45.617 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-18T04:54:45.993 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-mgr-rook 2024-09-18T04:54:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 13/13 2024-09-18T04:54:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 1/13 2024-09-18T04:54:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-18T04:54:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-18T04:54:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-18T04:54:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-18T04:54:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-18T04:54:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-18T04:54:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-18T04:54:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-18T04:54:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-18T04:54:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-18T04:54:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-18T04:54:46.582 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:09 ago on Wed 18 Sep 2024 04:53:37 AM UTC. 2024-09-18T04:54:46.713 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-18T04:54:46.714 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:46.714 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2024-09-18T04:54:46.714 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:46.714 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-09-18T04:54:46.715 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 49 k 2024-09-18T04:54:46.715 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2024-09-18T04:54:46.715 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-18T04:54:46.715 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-18T04:54:46.715 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-18T04:54:46.715 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-18T04:54:46.715 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-18T04:54:46.715 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-18T04:54:46.715 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-18T04:54:46.715 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-18T04:54:46.715 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-18T04:54:46.715 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-18T04:54:46.715 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-18T04:54:46.716 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-18T04:54:46.716 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:46.716 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-09-18T04:54:46.716 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:46.716 INFO:teuthology.orchestra.run.smithi099.stdout:Install 13 Packages 2024-09-18T04:54:46.716 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:46.717 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 2.2 M 2024-09-18T04:54:46.717 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 26 M 2024-09-18T04:54:46.717 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-09-18T04:54:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-18T04:54:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:46.863 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-09-18T04:54:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-18T04:54:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-18T04:54:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-18T04:54:46.863 INFO:teuthology.orchestra.run.smithi002.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-18T04:54:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-18T04:54:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-18T04:54:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-18T04:54:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-18T04:54:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-18T04:54:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-18T04:54:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-18T04:54:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-18T04:54:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-18T04:54:46.864 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:46.864 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-18T04:54:46.950 INFO:teuthology.orchestra.run.smithi099.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 161 kB/s | 26 kB 00:00 2024-09-18T04:54:46.976 INFO:teuthology.orchestra.run.smithi099.stdout:(2/13): ceph-mgr-rook-18.2.4-855.g19e2a2be.el9. 267 kB/s | 49 kB 00:00 2024-09-18T04:54:47.001 INFO:teuthology.orchestra.run.smithi099.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 91 kB/s | 19 kB 00:00 2024-09-18T04:54:47.104 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-mgr-cephadm 2024-09-18T04:54:47.226 INFO:teuthology.orchestra.run.smithi099.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 706 kB/s | 159 kB 00:00 2024-09-18T04:54:47.251 INFO:teuthology.orchestra.run.smithi099.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 739 kB/s | 222 kB 00:00 2024-09-18T04:54:47.293 INFO:teuthology.orchestra.run.smithi099.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 773 kB/s | 32 kB 00:00 2024-09-18T04:54:47.319 INFO:teuthology.orchestra.run.smithi099.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 581 kB/s | 54 kB 00:00 2024-09-18T04:54:47.345 INFO:teuthology.orchestra.run.smithi099.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 280 kB/s | 14 kB 00:00 2024-09-18T04:54:47.428 INFO:teuthology.orchestra.run.smithi099.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 654 kB/s | 279 kB 00:00 2024-09-18T04:54:47.462 INFO:teuthology.orchestra.run.smithi099.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 1.5 MB/s | 220 kB 00:00 2024-09-18T04:54:47.504 INFO:teuthology.orchestra.run.smithi099.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 787 kB/s | 59 kB 00:00 2024-09-18T04:54:47.549 INFO:teuthology.orchestra.run.smithi099.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.2 MB/s | 90 kB 00:00 2024-09-18T04:54:47.579 INFO:teuthology.orchestra.run.smithi099.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 4.5 MB/s | 1.0 MB 00:00 2024-09-18T04:54:47.581 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:54:47.581 INFO:teuthology.orchestra.run.smithi099.stdout:Total 2.6 MB/s | 2.2 MB 00:00 2024-09-18T04:54:47.648 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-09-18T04:54:47.663 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-09-18T04:54:47.663 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-09-18T04:54:47.713 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:16 ago on Wed 18 Sep 2024 04:53:31 AM UTC. 2024-09-18T04:54:47.753 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-09-18T04:54:47.753 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-09-18T04:54:47.852 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-18T04:54:47.854 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:47.854 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2024-09-18T04:54:47.854 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:47.854 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-09-18T04:54:47.854 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-cephadm noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 139 k 2024-09-18T04:54:47.854 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-09-18T04:54:47.854 INFO:teuthology.orchestra.run.smithi002.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-18T04:54:47.855 INFO:teuthology.orchestra.run.smithi002.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-18T04:54:47.855 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-18T04:54:47.855 INFO:teuthology.orchestra.run.smithi002.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-18T04:54:47.855 INFO:teuthology.orchestra.run.smithi002.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-18T04:54:47.855 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:47.855 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-09-18T04:54:47.855 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:47.855 INFO:teuthology.orchestra.run.smithi002.stdout:Install 6 Packages 2024-09-18T04:54:47.855 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:47.856 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 7.0 M 2024-09-18T04:54:47.856 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 33 M 2024-09-18T04:54:47.856 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-09-18T04:54:48.243 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-09-18T04:54:48.406 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-18T04:54:48.428 INFO:teuthology.orchestra.run.smithi002.stdout:(1/6): ceph-mgr-cephadm-18.2.4-855.g19e2a2be.el 664 kB/s | 139 kB 00:00 2024-09-18T04:54:48.462 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-18T04:54:48.520 INFO:teuthology.orchestra.run.smithi002.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 5.8 MB/s | 548 kB 00:00 2024-09-18T04:54:48.526 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-18T04:54:48.545 INFO:teuthology.orchestra.run.smithi002.stdout:(3/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 760 kB/s | 248 kB 00:00 2024-09-18T04:54:48.555 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-18T04:54:48.570 INFO:teuthology.orchestra.run.smithi002.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-18T04:54:48.597 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-18T04:54:48.629 INFO:teuthology.orchestra.run.smithi002.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 894 kB/s | 74 kB 00:00 2024-09-18T04:54:48.714 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-18T04:54:48.823 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-18T04:54:48.873 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-18T04:54:49.471 INFO:teuthology.orchestra.run.smithi002.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.8 MB/s | 6.0 MB 00:01 2024-09-18T04:54:49.473 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:54:49.474 INFO:teuthology.orchestra.run.smithi002.stdout:Total 4.3 MB/s | 7.0 MB 00:01 2024-09-18T04:54:49.551 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-09-18T04:54:49.563 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-09-18T04:54:49.563 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-09-18T04:54:49.582 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-18T04:54:49.637 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-18T04:54:49.669 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-09-18T04:54:49.671 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-09-18T04:54:49.679 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-18T04:54:49.727 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-18T04:54:49.749 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 13/13 2024-09-18T04:54:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-09-18T04:54:50.160 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-18T04:54:50.215 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-18T04:54:50.602 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 13/13 2024-09-18T04:54:50.602 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 1/13 2024-09-18T04:54:50.602 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-18T04:54:50.603 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-18T04:54:50.603 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-18T04:54:50.603 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-18T04:54:50.603 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-18T04:54:50.603 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-18T04:54:50.603 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-18T04:54:50.603 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-18T04:54:50.603 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-18T04:54:50.603 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-18T04:54:50.603 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-18T04:54:50.769 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-18T04:54:50.853 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-18T04:54:50.923 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-18T04:54:50.951 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 6/6 2024-09-18T04:54:50.954 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-18T04:54:50.954 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:50.954 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-09-18T04:54:50.954 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-rook-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-18T04:54:50.954 INFO:teuthology.orchestra.run.smithi099.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-18T04:54:50.954 INFO:teuthology.orchestra.run.smithi099.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-18T04:54:50.954 INFO:teuthology.orchestra.run.smithi099.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-18T04:54:50.954 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-18T04:54:50.954 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-18T04:54:50.954 INFO:teuthology.orchestra.run.smithi099.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-18T04:54:50.954 INFO:teuthology.orchestra.run.smithi099.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-18T04:54:50.954 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-18T04:54:50.954 INFO:teuthology.orchestra.run.smithi099.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-18T04:54:50.955 INFO:teuthology.orchestra.run.smithi099.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-18T04:54:50.955 INFO:teuthology.orchestra.run.smithi099.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-18T04:54:50.955 INFO:teuthology.orchestra.run.smithi099.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-18T04:54:50.955 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:50.955 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-18T04:54:51.209 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-mgr-cephadm 2024-09-18T04:54:51.771 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 6/6 2024-09-18T04:54:51.771 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/6 2024-09-18T04:54:51.771 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-18T04:54:51.771 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-18T04:54:51.771 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-18T04:54:51.771 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-18T04:54:51.788 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:14 ago on Wed 18 Sep 2024 04:53:37 AM UTC. 2024-09-18T04:54:51.920 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-18T04:54:51.920 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:51.921 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2024-09-18T04:54:51.921 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:51.921 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-09-18T04:54:51.921 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 139 k 2024-09-18T04:54:51.921 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2024-09-18T04:54:51.921 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-18T04:54:51.921 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-18T04:54:51.921 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-18T04:54:51.921 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-18T04:54:51.921 INFO:teuthology.orchestra.run.smithi099.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-18T04:54:51.921 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:51.921 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-09-18T04:54:51.921 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:51.922 INFO:teuthology.orchestra.run.smithi099.stdout:Install 6 Packages 2024-09-18T04:54:51.922 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:51.922 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 7.0 M 2024-09-18T04:54:51.922 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 33 M 2024-09-18T04:54:51.922 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-09-18T04:54:52.053 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-18T04:54:52.053 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:52.053 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-09-18T04:54:52.053 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-18T04:54:52.053 INFO:teuthology.orchestra.run.smithi002.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-18T04:54:52.053 INFO:teuthology.orchestra.run.smithi002.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-18T04:54:52.054 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-18T04:54:52.054 INFO:teuthology.orchestra.run.smithi002.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-18T04:54:52.054 INFO:teuthology.orchestra.run.smithi002.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-18T04:54:52.054 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:52.054 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-18T04:54:52.286 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-fuse 2024-09-18T04:54:52.447 INFO:teuthology.orchestra.run.smithi099.stdout:(1/6): ceph-mgr-cephadm-18.2.4-855.g19e2a2be.el 650 kB/s | 139 kB 00:00 2024-09-18T04:54:52.472 INFO:teuthology.orchestra.run.smithi099.stdout:(2/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.0 MB/s | 248 kB 00:00 2024-09-18T04:54:52.539 INFO:teuthology.orchestra.run.smithi099.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 867 kB/s | 58 kB 00:00 2024-09-18T04:54:52.573 INFO:teuthology.orchestra.run.smithi099.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.2 MB/s | 74 kB 00:00 2024-09-18T04:54:52.606 INFO:teuthology.orchestra.run.smithi099.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 3.4 MB/s | 548 kB 00:00 2024-09-18T04:54:52.898 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:21 ago on Wed 18 Sep 2024 04:53:31 AM UTC. 2024-09-18T04:54:53.015 INFO:teuthology.orchestra.run.smithi099.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 7.6 MB/s | 6.0 MB 00:00 2024-09-18T04:54:53.017 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:54:53.017 INFO:teuthology.orchestra.run.smithi099.stdout:Total 6.4 MB/s | 7.0 MB 00:01 2024-09-18T04:54:53.036 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-18T04:54:53.037 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2024-09-18T04:54:53.037 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:53.037 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-09-18T04:54:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-fuse x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 852 k 2024-09-18T04:54:53.037 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-09-18T04:54:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-18T04:54:53.037 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:53.037 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-09-18T04:54:53.037 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:53.038 INFO:teuthology.orchestra.run.smithi002.stdout:Install 2 Packages 2024-09-18T04:54:53.038 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:53.038 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 932 k 2024-09-18T04:54:53.038 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 2.7 M 2024-09-18T04:54:53.038 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-09-18T04:54:53.087 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-09-18T04:54:53.099 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-09-18T04:54:53.099 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-09-18T04:54:53.197 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-09-18T04:54:53.197 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-09-18T04:54:53.281 INFO:teuthology.orchestra.run.smithi002.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 663 kB/s | 80 kB 00:00 2024-09-18T04:54:53.447 INFO:teuthology.orchestra.run.smithi002.stdout:(2/2): ceph-fuse-18.2.4-855.g19e2a2be.el9.x86_6 2.9 MB/s | 852 kB 00:00 2024-09-18T04:54:53.448 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:54:53.448 INFO:teuthology.orchestra.run.smithi002.stdout:Total 2.2 MB/s | 932 kB 00:00 2024-09-18T04:54:53.461 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-09-18T04:54:53.484 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-09-18T04:54:53.484 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-09-18T04:54:53.499 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-09-18T04:54:53.568 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-09-18T04:54:53.569 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-09-18T04:54:53.653 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-18T04:54:53.699 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-18T04:54:53.761 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-09-18T04:54:53.909 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-18T04:54:53.991 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 2/2 2024-09-18T04:54:54.218 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-18T04:54:54.310 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-18T04:54:54.372 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-18T04:54:54.403 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 6/6 2024-09-18T04:54:54.653 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 2/2 2024-09-18T04:54:54.653 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/2 2024-09-18T04:54:54.919 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-18T04:54:54.919 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:54.919 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-09-18T04:54:54.919 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-18T04:54:54.919 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:54.919 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-18T04:54:55.203 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-volume 2024-09-18T04:54:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 6/6 2024-09-18T04:54:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 1/6 2024-09-18T04:54:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-18T04:54:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-18T04:54:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-18T04:54:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-18T04:54:55.474 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-18T04:54:55.474 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:55.474 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-09-18T04:54:55.474 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-mgr-cephadm-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-18T04:54:55.474 INFO:teuthology.orchestra.run.smithi099.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-18T04:54:55.475 INFO:teuthology.orchestra.run.smithi099.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-18T04:54:55.475 INFO:teuthology.orchestra.run.smithi099.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-18T04:54:55.475 INFO:teuthology.orchestra.run.smithi099.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-18T04:54:55.475 INFO:teuthology.orchestra.run.smithi099.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-18T04:54:55.475 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:55.475 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-18T04:54:55.714 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-fuse 2024-09-18T04:54:55.812 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:24 ago on Wed 18 Sep 2024 04:53:31 AM UTC. 2024-09-18T04:54:55.951 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-18T04:54:55.952 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:55.952 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2024-09-18T04:54:55.952 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:55.952 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-09-18T04:54:55.952 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-volume noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 264 k 2024-09-18T04:54:55.952 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2024-09-18T04:54:55.952 INFO:teuthology.orchestra.run.smithi002.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-18T04:54:55.952 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:55.952 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-09-18T04:54:55.952 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:55.952 INFO:teuthology.orchestra.run.smithi002.stdout:Install 2 Packages 2024-09-18T04:54:55.953 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:55.953 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 342 k 2024-09-18T04:54:55.953 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 1.5 M 2024-09-18T04:54:55.953 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-09-18T04:54:56.152 INFO:teuthology.orchestra.run.smithi002.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 510 kB/s | 77 kB 00:00 2024-09-18T04:54:56.277 INFO:teuthology.orchestra.run.smithi002.stdout:(2/2): ceph-volume-18.2.4-855.g19e2a2be.el9.noa 956 kB/s | 264 kB 00:00 2024-09-18T04:54:56.277 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:54:56.278 INFO:teuthology.orchestra.run.smithi002.stdout:Total 1.0 MB/s | 342 kB 00:00 2024-09-18T04:54:56.290 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-09-18T04:54:56.295 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:19 ago on Wed 18 Sep 2024 04:53:37 AM UTC. 2024-09-18T04:54:56.300 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-09-18T04:54:56.300 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-09-18T04:54:56.338 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-09-18T04:54:56.338 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-09-18T04:54:56.428 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-18T04:54:56.429 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:56.429 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2024-09-18T04:54:56.429 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:56.429 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-09-18T04:54:56.429 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 852 k 2024-09-18T04:54:56.429 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2024-09-18T04:54:56.429 INFO:teuthology.orchestra.run.smithi099.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-18T04:54:56.429 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:56.429 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-09-18T04:54:56.429 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:56.429 INFO:teuthology.orchestra.run.smithi099.stdout:Install 2 Packages 2024-09-18T04:54:56.429 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:56.430 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 932 k 2024-09-18T04:54:56.430 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 2.7 M 2024-09-18T04:54:56.430 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-09-18T04:54:56.495 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-09-18T04:54:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-09-18T04:54:56.716 INFO:teuthology.orchestra.run.smithi099.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 524 kB/s | 80 kB 00:00 2024-09-18T04:54:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 2/2 2024-09-18T04:54:56.738 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 2/2 2024-09-18T04:54:56.738 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T04:54:56.738 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-18T04:54:56.739 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:56.841 INFO:teuthology.orchestra.run.smithi099.stdout:(2/2): ceph-fuse-18.2.4-855.g19e2a2be.el9.x86_6 3.0 MB/s | 852 kB 00:00 2024-09-18T04:54:56.842 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:54:56.842 INFO:teuthology.orchestra.run.smithi099.stdout:Total 2.2 MB/s | 932 kB 00:00 2024-09-18T04:54:56.853 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-09-18T04:54:56.875 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-09-18T04:54:56.875 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-09-18T04:54:56.954 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-09-18T04:54:56.954 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-09-18T04:54:57.141 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-09-18T04:54:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-18T04:54:57.348 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 2/2 2024-09-18T04:54:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 1/2 2024-09-18T04:54:57.740 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-09-18T04:54:57.741 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:57.741 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-09-18T04:54:57.741 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-18T04:54:57.741 INFO:teuthology.orchestra.run.smithi002.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-18T04:54:57.741 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:57.741 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-18T04:54:58.018 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 2/2 2024-09-18T04:54:58.018 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/2 2024-09-18T04:54:58.035 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install librados-devel 2024-09-18T04:54:58.308 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-18T04:54:58.309 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:58.309 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-09-18T04:54:58.309 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-18T04:54:58.309 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:58.309 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-18T04:54:58.594 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install ceph-volume 2024-09-18T04:54:58.652 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:27 ago on Wed 18 Sep 2024 04:53:31 AM UTC. 2024-09-18T04:54:58.790 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-18T04:54:58.791 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:58.791 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repo Size 2024-09-18T04:54:58.791 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:58.791 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-09-18T04:54:58.791 INFO:teuthology.orchestra.run.smithi002.stdout: librados-devel x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 126 k 2024-09-18T04:54:58.791 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:58.791 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-09-18T04:54:58.791 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:54:58.791 INFO:teuthology.orchestra.run.smithi002.stdout:Install 1 Package 2024-09-18T04:54:58.792 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:58.792 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 126 k 2024-09-18T04:54:58.792 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 456 k 2024-09-18T04:54:58.792 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-09-18T04:54:58.999 INFO:teuthology.orchestra.run.smithi002.stdout:librados-devel-18.2.4-855.g19e2a2be.el9.x86_64. 610 kB/s | 126 kB 00:00 2024-09-18T04:54:59.000 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:54:59.000 INFO:teuthology.orchestra.run.smithi002.stdout:Total 606 kB/s | 126 kB 00:00 2024-09-18T04:54:59.000 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-09-18T04:54:59.006 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-09-18T04:54:59.006 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-09-18T04:54:59.073 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-09-18T04:54:59.073 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-09-18T04:54:59.192 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:22 ago on Wed 18 Sep 2024 04:53:37 AM UTC. 2024-09-18T04:54:59.234 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-09-18T04:54:59.330 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-18T04:54:59.330 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:59.330 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repository Size 2024-09-18T04:54:59.330 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:59.330 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-09-18T04:54:59.330 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume noarch 2:18.2.4-855.g19e2a2be.el9 ceph-noarch 264 k 2024-09-18T04:54:59.330 INFO:teuthology.orchestra.run.smithi099.stdout:Installing dependencies: 2024-09-18T04:54:59.330 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-18T04:54:59.330 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:59.330 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-09-18T04:54:59.331 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:54:59.331 INFO:teuthology.orchestra.run.smithi099.stdout:Install 2 Packages 2024-09-18T04:54:59.331 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:54:59.331 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 342 k 2024-09-18T04:54:59.331 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 1.5 M 2024-09-18T04:54:59.331 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-09-18T04:54:59.333 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:54:59.733 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:54:59.796 INFO:teuthology.orchestra.run.smithi099.stdout:(1/2): ceph-volume-18.2.4-855.g19e2a2be.el9.noa 1.2 MB/s | 264 kB 00:00 2024-09-18T04:54:59.921 INFO:teuthology.orchestra.run.smithi099.stdout:(2/2): python3-packaging-20.9-5.el9.noarch.rpm 229 kB/s | 77 kB 00:00 2024-09-18T04:54:59.921 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:54:59.922 INFO:teuthology.orchestra.run.smithi099.stdout:Total 578 kB/s | 342 kB 00:00 2024-09-18T04:54:59.934 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-09-18T04:54:59.943 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-09-18T04:54:59.943 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-09-18T04:54:59.979 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-09-18T04:54:59.979 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-09-18T04:54:59.993 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:54:59.993 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:59.993 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-09-18T04:54:59.993 INFO:teuthology.orchestra.run.smithi002.stdout: librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:54:59.993 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:54:59.993 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-18T04:55:00.136 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-09-18T04:55:00.218 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install libcephfs2 2024-09-18T04:55:00.329 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-09-18T04:55:00.373 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 2/2 2024-09-18T04:55:00.390 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 2/2 2024-09-18T04:55:00.390 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T04:55:00.390 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-18T04:55:00.391 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:00.830 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:29 ago on Wed 18 Sep 2024 04:53:31 AM UTC. 2024-09-18T04:55:00.924 INFO:teuthology.orchestra.run.smithi002.stdout:Package libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-18T04:55:00.969 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-18T04:55:00.970 INFO:teuthology.orchestra.run.smithi002.stdout:Nothing to do. 2024-09-18T04:55:00.971 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-18T04:55:01.047 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install libcephfs-devel 2024-09-18T04:55:01.127 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 1/2 2024-09-18T04:55:01.468 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-09-18T04:55:01.468 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:01.468 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-09-18T04:55:01.468 INFO:teuthology.orchestra.run.smithi099.stdout: ceph-volume-2:18.2.4-855.g19e2a2be.el9.noarch 2024-09-18T04:55:01.468 INFO:teuthology.orchestra.run.smithi099.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-18T04:55:01.468 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:01.468 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-18T04:55:01.650 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:30 ago on Wed 18 Sep 2024 04:53:31 AM UTC. 2024-09-18T04:55:01.760 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install librados-devel 2024-09-18T04:55:01.789 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-18T04:55:01.790 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:55:01.790 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repo Size 2024-09-18T04:55:01.790 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:55:01.790 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-09-18T04:55:01.790 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs-devel x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 31 k 2024-09-18T04:55:01.791 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:01.791 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-09-18T04:55:01.791 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:55:01.791 INFO:teuthology.orchestra.run.smithi002.stdout:Install 1 Package 2024-09-18T04:55:01.791 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:01.791 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 31 k 2024-09-18T04:55:01.791 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 139 k 2024-09-18T04:55:01.791 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-09-18T04:55:01.936 INFO:teuthology.orchestra.run.smithi002.stdout:libcephfs-devel-18.2.4-855.g19e2a2be.el9.x86_64 215 kB/s | 31 kB 00:00 2024-09-18T04:55:01.937 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:55:01.937 INFO:teuthology.orchestra.run.smithi002.stdout:Total 212 kB/s | 31 kB 00:00 2024-09-18T04:55:01.937 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-09-18T04:55:01.940 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-09-18T04:55:01.941 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-09-18T04:55:01.953 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-09-18T04:55:01.953 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-09-18T04:55:02.045 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-09-18T04:55:02.161 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:02.357 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:25 ago on Wed 18 Sep 2024 04:53:37 AM UTC. 2024-09-18T04:55:02.489 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-18T04:55:02.490 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:55:02.490 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repo Size 2024-09-18T04:55:02.490 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:55:02.490 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-09-18T04:55:02.490 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 126 k 2024-09-18T04:55:02.490 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:02.491 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-09-18T04:55:02.491 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:55:02.491 INFO:teuthology.orchestra.run.smithi099.stdout:Install 1 Package 2024-09-18T04:55:02.491 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:02.491 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 126 k 2024-09-18T04:55:02.491 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 456 k 2024-09-18T04:55:02.492 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-09-18T04:55:02.519 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:02.701 INFO:teuthology.orchestra.run.smithi099.stdout:librados-devel-18.2.4-855.g19e2a2be.el9.x86_64. 602 kB/s | 126 kB 00:00 2024-09-18T04:55:02.701 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:55:02.702 INFO:teuthology.orchestra.run.smithi099.stdout:Total 597 kB/s | 126 kB 00:00 2024-09-18T04:55:02.702 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-09-18T04:55:02.707 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-09-18T04:55:02.707 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-09-18T04:55:02.750 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:02.750 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:02.751 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-09-18T04:55:02.751 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:55:02.751 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:02.751 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-18T04:55:02.770 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-09-18T04:55:02.770 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-09-18T04:55:02.928 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install librados2 2024-09-18T04:55:02.931 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-09-18T04:55:03.040 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:03.503 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:03.533 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:32 ago on Wed 18 Sep 2024 04:53:31 AM UTC. 2024-09-18T04:55:03.625 INFO:teuthology.orchestra.run.smithi002.stdout:Package librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-18T04:55:03.671 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-18T04:55:03.672 INFO:teuthology.orchestra.run.smithi002.stdout:Nothing to do. 2024-09-18T04:55:03.672 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-18T04:55:03.746 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install librbd1 2024-09-18T04:55:03.791 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:03.791 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:03.791 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-09-18T04:55:03.791 INFO:teuthology.orchestra.run.smithi099.stdout: librados-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:55:03.791 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:03.791 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-18T04:55:04.043 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install libcephfs2 2024-09-18T04:55:04.350 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:33 ago on Wed 18 Sep 2024 04:53:31 AM UTC. 2024-09-18T04:55:04.443 INFO:teuthology.orchestra.run.smithi002.stdout:Package librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-18T04:55:04.488 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-18T04:55:04.489 INFO:teuthology.orchestra.run.smithi002.stdout:Nothing to do. 2024-09-18T04:55:04.489 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-18T04:55:04.551 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install python3-rados 2024-09-18T04:55:04.640 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:27 ago on Wed 18 Sep 2024 04:53:37 AM UTC. 2024-09-18T04:55:04.730 INFO:teuthology.orchestra.run.smithi099.stdout:Package libcephfs2-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-18T04:55:04.774 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-18T04:55:04.776 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2024-09-18T04:55:04.776 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-18T04:55:04.854 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install libcephfs-devel 2024-09-18T04:55:05.152 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:34 ago on Wed 18 Sep 2024 04:53:31 AM UTC. 2024-09-18T04:55:05.244 INFO:teuthology.orchestra.run.smithi002.stdout:Package python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-18T04:55:05.289 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-18T04:55:05.291 INFO:teuthology.orchestra.run.smithi002.stdout:Nothing to do. 2024-09-18T04:55:05.291 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-18T04:55:05.359 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install python3-rgw 2024-09-18T04:55:05.430 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:28 ago on Wed 18 Sep 2024 04:53:37 AM UTC. 2024-09-18T04:55:05.560 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-18T04:55:05.561 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:55:05.561 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repo Size 2024-09-18T04:55:05.561 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:55:05.562 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-09-18T04:55:05.562 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 31 k 2024-09-18T04:55:05.562 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:05.562 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-09-18T04:55:05.563 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:55:05.563 INFO:teuthology.orchestra.run.smithi099.stdout:Install 1 Package 2024-09-18T04:55:05.563 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:05.563 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 31 k 2024-09-18T04:55:05.563 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 139 k 2024-09-18T04:55:05.563 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-09-18T04:55:05.712 INFO:teuthology.orchestra.run.smithi099.stdout:libcephfs-devel-18.2.4-855.g19e2a2be.el9.x86_64 206 kB/s | 31 kB 00:00 2024-09-18T04:55:05.713 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:55:05.713 INFO:teuthology.orchestra.run.smithi099.stdout:Total 204 kB/s | 31 kB 00:00 2024-09-18T04:55:05.713 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-09-18T04:55:05.717 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-09-18T04:55:05.717 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-09-18T04:55:05.733 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-09-18T04:55:05.733 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-09-18T04:55:05.813 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-09-18T04:55:05.934 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:05.971 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:34 ago on Wed 18 Sep 2024 04:53:31 AM UTC. 2024-09-18T04:55:06.063 INFO:teuthology.orchestra.run.smithi002.stdout:Package python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-18T04:55:06.108 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-18T04:55:06.109 INFO:teuthology.orchestra.run.smithi002.stdout:Nothing to do. 2024-09-18T04:55:06.109 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-18T04:55:06.179 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install python3-cephfs 2024-09-18T04:55:06.304 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:06.538 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:06.538 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:06.538 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-09-18T04:55:06.538 INFO:teuthology.orchestra.run.smithi099.stdout: libcephfs-devel-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:55:06.538 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:06.538 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-18T04:55:06.711 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install librados2 2024-09-18T04:55:06.779 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:35 ago on Wed 18 Sep 2024 04:53:31 AM UTC. 2024-09-18T04:55:06.872 INFO:teuthology.orchestra.run.smithi002.stdout:Package python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-18T04:55:06.917 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-18T04:55:06.918 INFO:teuthology.orchestra.run.smithi002.stdout:Nothing to do. 2024-09-18T04:55:06.918 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-18T04:55:06.996 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install python3-rbd 2024-09-18T04:55:07.284 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:30 ago on Wed 18 Sep 2024 04:53:37 AM UTC. 2024-09-18T04:55:07.373 INFO:teuthology.orchestra.run.smithi099.stdout:Package librados2-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-18T04:55:07.416 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-18T04:55:07.417 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2024-09-18T04:55:07.417 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-18T04:55:07.481 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install librbd1 2024-09-18T04:55:07.599 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:36 ago on Wed 18 Sep 2024 04:53:31 AM UTC. 2024-09-18T04:55:07.692 INFO:teuthology.orchestra.run.smithi002.stdout:Package python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-18T04:55:07.737 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-18T04:55:07.739 INFO:teuthology.orchestra.run.smithi002.stdout:Nothing to do. 2024-09-18T04:55:07.739 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-18T04:55:07.813 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install rbd-fuse 2024-09-18T04:55:08.063 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:31 ago on Wed 18 Sep 2024 04:53:37 AM UTC. 2024-09-18T04:55:08.152 INFO:teuthology.orchestra.run.smithi099.stdout:Package librbd1-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-18T04:55:08.195 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-18T04:55:08.196 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2024-09-18T04:55:08.196 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-18T04:55:08.268 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install python3-rados 2024-09-18T04:55:08.422 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:37 ago on Wed 18 Sep 2024 04:53:31 AM UTC. 2024-09-18T04:55:08.560 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-18T04:55:08.561 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:55:08.561 INFO:teuthology.orchestra.run.smithi002.stdout: Package Architecture Version Repository Size 2024-09-18T04:55:08.561 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:55:08.561 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-09-18T04:55:08.561 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-fuse x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 87 k 2024-09-18T04:55:08.561 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:08.561 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-09-18T04:55:08.562 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:55:08.562 INFO:teuthology.orchestra.run.smithi002.stdout:Install 1 Package 2024-09-18T04:55:08.562 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:08.562 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 87 k 2024-09-18T04:55:08.562 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 227 k 2024-09-18T04:55:08.562 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-09-18T04:55:08.748 INFO:teuthology.orchestra.run.smithi002.stdout:rbd-fuse-18.2.4-855.g19e2a2be.el9.x86_64.rpm 467 kB/s | 87 kB 00:00 2024-09-18T04:55:08.748 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:55:08.748 INFO:teuthology.orchestra.run.smithi002.stdout:Total 464 kB/s | 87 kB 00:00 2024-09-18T04:55:08.748 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-09-18T04:55:08.756 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-09-18T04:55:08.756 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-09-18T04:55:08.821 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-09-18T04:55:08.822 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-09-18T04:55:08.847 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:31 ago on Wed 18 Sep 2024 04:53:37 AM UTC. 2024-09-18T04:55:08.936 INFO:teuthology.orchestra.run.smithi099.stdout:Package python3-rados-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-18T04:55:08.978 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-18T04:55:08.979 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-09-18T04:55:08.979 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2024-09-18T04:55:08.979 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-18T04:55:09.050 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install python3-rgw 2024-09-18T04:55:09.089 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:09.487 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:09.624 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:32 ago on Wed 18 Sep 2024 04:53:37 AM UTC. 2024-09-18T04:55:09.712 INFO:teuthology.orchestra.run.smithi099.stdout:Package python3-rgw-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-18T04:55:09.747 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:09.747 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:09.747 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-09-18T04:55:09.747 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:55:09.747 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:09.747 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-18T04:55:09.754 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-18T04:55:09.755 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2024-09-18T04:55:09.755 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-18T04:55:09.835 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install python3-cephfs 2024-09-18T04:55:09.991 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install rbd-mirror 2024-09-18T04:55:10.410 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:33 ago on Wed 18 Sep 2024 04:53:37 AM UTC. 2024-09-18T04:55:10.498 INFO:teuthology.orchestra.run.smithi099.stdout:Package python3-cephfs-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-18T04:55:10.541 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-18T04:55:10.542 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2024-09-18T04:55:10.542 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-18T04:55:10.598 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:39 ago on Wed 18 Sep 2024 04:53:31 AM UTC. 2024-09-18T04:55:10.608 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install python3-rbd 2024-09-18T04:55:10.737 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-18T04:55:10.737 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:55:10.738 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repo Size 2024-09-18T04:55:10.738 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:55:10.738 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-09-18T04:55:10.738 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-mirror x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 3.0 M 2024-09-18T04:55:10.738 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:10.738 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-09-18T04:55:10.738 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:55:10.738 INFO:teuthology.orchestra.run.smithi002.stdout:Install 1 Package 2024-09-18T04:55:10.738 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:10.738 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 3.0 M 2024-09-18T04:55:10.738 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 12 M 2024-09-18T04:55:10.738 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-09-18T04:55:11.059 INFO:teuthology.orchestra.run.smithi002.stdout:rbd-mirror-18.2.4-855.g19e2a2be.el9.x86_64.rpm 9.3 MB/s | 3.0 MB 00:00 2024-09-18T04:55:11.059 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:55:11.060 INFO:teuthology.orchestra.run.smithi002.stdout:Total 9.3 MB/s | 3.0 MB 00:00 2024-09-18T04:55:11.060 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-09-18T04:55:11.070 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-09-18T04:55:11.070 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-09-18T04:55:11.154 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-09-18T04:55:11.154 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-09-18T04:55:11.186 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:34 ago on Wed 18 Sep 2024 04:53:37 AM UTC. 2024-09-18T04:55:11.274 INFO:teuthology.orchestra.run.smithi099.stdout:Package python3-rbd-2:18.2.4-855.g19e2a2be.el9.x86_64 is already installed. 2024-09-18T04:55:11.316 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-18T04:55:11.317 INFO:teuthology.orchestra.run.smithi099.stdout:Nothing to do. 2024-09-18T04:55:11.317 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-18T04:55:11.395 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install rbd-fuse 2024-09-18T04:55:11.592 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-09-18T04:55:11.643 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:11.670 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:11.670 INFO:teuthology.orchestra.run.smithi002.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T04:55:11.670 INFO:teuthology.orchestra.run.smithi002.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-18T04:55:11.670 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-18T04:55:11.670 INFO:teuthology.orchestra.run.smithi002.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-18T04:55:11.670 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:11.975 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:34 ago on Wed 18 Sep 2024 04:53:37 AM UTC. 2024-09-18T04:55:12.109 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-18T04:55:12.109 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:55:12.109 INFO:teuthology.orchestra.run.smithi099.stdout: Package Architecture Version Repository Size 2024-09-18T04:55:12.109 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:55:12.109 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-09-18T04:55:12.109 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 87 k 2024-09-18T04:55:12.109 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:12.109 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-09-18T04:55:12.109 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:55:12.109 INFO:teuthology.orchestra.run.smithi099.stdout:Install 1 Package 2024-09-18T04:55:12.110 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:12.110 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 87 k 2024-09-18T04:55:12.110 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 227 k 2024-09-18T04:55:12.110 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-09-18T04:55:12.290 INFO:teuthology.orchestra.run.smithi099.stdout:rbd-fuse-18.2.4-855.g19e2a2be.el9.x86_64.rpm 477 kB/s | 87 kB 00:00 2024-09-18T04:55:12.291 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:55:12.291 INFO:teuthology.orchestra.run.smithi099.stdout:Total 473 kB/s | 87 kB 00:00 2024-09-18T04:55:12.291 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-09-18T04:55:12.298 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-09-18T04:55:12.298 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-09-18T04:55:12.360 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-09-18T04:55:12.361 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-09-18T04:55:12.513 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-09-18T04:55:12.628 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:12.788 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:12.789 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:12.789 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-09-18T04:55:12.789 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:55:12.789 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:12.789 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-18T04:55:13.031 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:13.033 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install rbd-nbd 2024-09-18T04:55:13.297 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:13.297 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:13.297 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-09-18T04:55:13.297 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-fuse-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:55:13.297 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:13.297 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-18T04:55:13.558 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install rbd-mirror 2024-09-18T04:55:13.652 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:01:42 ago on Wed 18 Sep 2024 04:53:31 AM UTC. 2024-09-18T04:55:13.789 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2024-09-18T04:55:13.790 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:55:13.790 INFO:teuthology.orchestra.run.smithi002.stdout: Package Architecture Version Repository Size 2024-09-18T04:55:13.790 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:55:13.790 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2024-09-18T04:55:13.790 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-nbd x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 172 k 2024-09-18T04:55:13.790 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:13.790 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2024-09-18T04:55:13.790 INFO:teuthology.orchestra.run.smithi002.stdout:================================================================================ 2024-09-18T04:55:13.790 INFO:teuthology.orchestra.run.smithi002.stdout:Install 1 Package 2024-09-18T04:55:13.790 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:13.790 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 172 k 2024-09-18T04:55:13.790 INFO:teuthology.orchestra.run.smithi002.stdout:Installed size: 494 k 2024-09-18T04:55:13.791 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2024-09-18T04:55:13.996 INFO:teuthology.orchestra.run.smithi002.stdout:rbd-nbd-18.2.4-855.g19e2a2be.el9.x86_64.rpm 838 kB/s | 172 kB 00:00 2024-09-18T04:55:13.997 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:55:13.997 INFO:teuthology.orchestra.run.smithi002.stdout:Total 832 kB/s | 172 kB 00:00 2024-09-18T04:55:13.997 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2024-09-18T04:55:14.005 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2024-09-18T04:55:14.006 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2024-09-18T04:55:14.072 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2024-09-18T04:55:14.072 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2024-09-18T04:55:14.141 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:37 ago on Wed 18 Sep 2024 04:53:37 AM UTC. 2024-09-18T04:55:14.235 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2024-09-18T04:55:14.275 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-18T04:55:14.276 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:55:14.276 INFO:teuthology.orchestra.run.smithi099.stdout: Package Arch Version Repo Size 2024-09-18T04:55:14.276 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:55:14.276 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-09-18T04:55:14.276 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 3.0 M 2024-09-18T04:55:14.276 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:14.276 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-09-18T04:55:14.276 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:55:14.276 INFO:teuthology.orchestra.run.smithi099.stdout:Install 1 Package 2024-09-18T04:55:14.276 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:14.277 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 3.0 M 2024-09-18T04:55:14.277 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 12 M 2024-09-18T04:55:14.277 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-09-18T04:55:14.329 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:14.602 INFO:teuthology.orchestra.run.smithi099.stdout:rbd-mirror-18.2.4-855.g19e2a2be.el9.x86_64.rpm 9.2 MB/s | 3.0 MB 00:00 2024-09-18T04:55:14.602 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:55:14.603 INFO:teuthology.orchestra.run.smithi099.stdout:Total 9.2 MB/s | 3.0 MB 00:00 2024-09-18T04:55:14.603 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-09-18T04:55:14.612 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-09-18T04:55:14.612 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-09-18T04:55:14.691 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-09-18T04:55:14.692 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-09-18T04:55:14.723 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:14.999 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:14.999 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:14.999 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2024-09-18T04:55:14.999 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:55:14.999 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:14.999 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2024-09-18T04:55:15.110 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-09-18T04:55:15.170 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:15.194 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:15.194 INFO:teuthology.orchestra.run.smithi099.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-18T04:55:15.195 INFO:teuthology.orchestra.run.smithi099.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-18T04:55:15.195 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-18T04:55:15.195 INFO:teuthology.orchestra.run.smithi099.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-18T04:55:15.195 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:15.235 DEBUG:teuthology.parallel:result is None 2024-09-18T04:55:16.264 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:16.264 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:16.265 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-09-18T04:55:16.265 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-mirror-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:55:16.265 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:16.265 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-18T04:55:16.549 DEBUG:teuthology.orchestra.run.smithi099:> sudo yum -y install rbd-nbd 2024-09-18T04:55:17.143 INFO:teuthology.orchestra.run.smithi099.stdout:Last metadata expiration check: 0:01:40 ago on Wed 18 Sep 2024 04:53:37 AM UTC. 2024-09-18T04:55:17.274 INFO:teuthology.orchestra.run.smithi099.stdout:Dependencies resolved. 2024-09-18T04:55:17.275 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:55:17.275 INFO:teuthology.orchestra.run.smithi099.stdout: Package Architecture Version Repository Size 2024-09-18T04:55:17.275 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:55:17.275 INFO:teuthology.orchestra.run.smithi099.stdout:Installing: 2024-09-18T04:55:17.275 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd x86_64 2:18.2.4-855.g19e2a2be.el9 ceph 172 k 2024-09-18T04:55:17.275 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:17.275 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction Summary 2024-09-18T04:55:17.275 INFO:teuthology.orchestra.run.smithi099.stdout:================================================================================ 2024-09-18T04:55:17.276 INFO:teuthology.orchestra.run.smithi099.stdout:Install 1 Package 2024-09-18T04:55:17.276 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:17.276 INFO:teuthology.orchestra.run.smithi099.stdout:Total download size: 172 k 2024-09-18T04:55:17.276 INFO:teuthology.orchestra.run.smithi099.stdout:Installed size: 494 k 2024-09-18T04:55:17.276 INFO:teuthology.orchestra.run.smithi099.stdout:Downloading Packages: 2024-09-18T04:55:17.497 INFO:teuthology.orchestra.run.smithi099.stdout:rbd-nbd-18.2.4-855.g19e2a2be.el9.x86_64.rpm 778 kB/s | 172 kB 00:00 2024-09-18T04:55:17.498 INFO:teuthology.orchestra.run.smithi099.stdout:-------------------------------------------------------------------------------- 2024-09-18T04:55:17.498 INFO:teuthology.orchestra.run.smithi099.stdout:Total 772 kB/s | 172 kB 00:00 2024-09-18T04:55:17.498 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction check 2024-09-18T04:55:17.506 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction check succeeded. 2024-09-18T04:55:17.506 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction test 2024-09-18T04:55:17.569 INFO:teuthology.orchestra.run.smithi099.stdout:Transaction test succeeded. 2024-09-18T04:55:17.569 INFO:teuthology.orchestra.run.smithi099.stdout:Running transaction 2024-09-18T04:55:17.724 INFO:teuthology.orchestra.run.smithi099.stdout: Preparing : 1/1 2024-09-18T04:55:17.827 INFO:teuthology.orchestra.run.smithi099.stdout: Installing : rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:18.253 INFO:teuthology.orchestra.run.smithi099.stdout: Running scriptlet: rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:18.522 INFO:teuthology.orchestra.run.smithi099.stdout: Verifying : rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 1/1 2024-09-18T04:55:18.522 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:18.522 INFO:teuthology.orchestra.run.smithi099.stdout:Installed: 2024-09-18T04:55:18.522 INFO:teuthology.orchestra.run.smithi099.stdout: rbd-nbd-2:18.2.4-855.g19e2a2be.el9.x86_64 2024-09-18T04:55:18.522 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:18.522 INFO:teuthology.orchestra.run.smithi099.stdout:Complete! 2024-09-18T04:55:18.776 DEBUG:teuthology.parallel:result is None 2024-09-18T04:55:18.777 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-18T04:55:18.936 DEBUG:teuthology.orchestra.run.smithi002:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-18T04:55:18.963 INFO:teuthology.orchestra.run.smithi002.stdout:18.2.4-855.g19e2a2be.el9 2024-09-18T04:55:18.963 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-855.g19e2a2be.el9 2024-09-18T04:55:18.963 INFO:teuthology.task.install:The correct ceph version 18.2.4-855.g19e2a2be is installed. 2024-09-18T04:55:18.965 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=19e2a2beb6f41a3ab62650e38614eb29b59dcbb9 2024-09-18T04:55:19.108 DEBUG:teuthology.orchestra.run.smithi099:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-18T04:55:19.129 INFO:teuthology.orchestra.run.smithi099.stdout:18.2.4-855.g19e2a2be.el9 2024-09-18T04:55:19.130 INFO:teuthology.packaging:The installed version of ceph is 18.2.4-855.g19e2a2be.el9 2024-09-18T04:55:19.130 INFO:teuthology.task.install:The correct ceph version 18.2.4-855.g19e2a2be is installed. 2024-09-18T04:55:19.132 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-18T04:55:19.132 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-18T04:55:19.132 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-18T04:55:19.165 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-18T04:55:19.165 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-18T04:55:19.195 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-18T04:55:19.195 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-18T04:55:19.196 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/daemon-helper 2024-09-18T04:55:19.235 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-18T04:55:19.304 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-18T04:55:19.304 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/daemon-helper 2024-09-18T04:55:19.329 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-18T04:55:19.410 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-18T04:55:19.410 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-18T04:55:19.410 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-18T04:55:19.439 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-18T04:55:19.508 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-18T04:55:19.508 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-18T04:55:19.533 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-18T04:55:19.598 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-18T04:55:19.598 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-18T04:55:19.598 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/stdin-killer 2024-09-18T04:55:19.627 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-18T04:55:19.697 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-18T04:55:19.697 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/usr/bin/stdin-killer 2024-09-18T04:55:19.721 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-18T04:55:19.786 INFO:teuthology.run_tasks:Running task ceph... 2024-09-18T04:55:19.871 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-18T04:55:19.872 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 777 /var/log/ceph 2024-09-18T04:55:19.874 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 777 /var/log/ceph 2024-09-18T04:55:19.901 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-18T04:55:19.901 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-18T04:55:19.942 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-18T04:55:19.967 INFO:tasks.ceph:Creating extra log directories... 2024-09-18T04:55:19.968 DEBUG:teuthology.orchestra.run.smithi002:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-18T04:55:20.010 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-18T04:55:20.143 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-18T04:55:20.144 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': False}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-18T04:55:20.144 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909798', 'branch': 'reef', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nocache}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7909798', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'rbd cache': False}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': 'a477b1894d5c5a613f23260ca0e6b96d79652ba7'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 930, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 4623, 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9', 'sleep_before_teardown': 0, 'subset': '80/128', 'suite': 'rbd', 'suite_branch': 'reef', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_a477b1894d5c5a613f23260ca0e6b96d79652ba7/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'a477b1894d5c5a613f23260ca0e6b96d79652ba7', 'targets': {'smithi002.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBA5XRVOVzjCEemEYdvFHFdowKrx9SpYXnLgOxgAsdnDmTMKVQWTI9AaX+zqjW8CsHOORsP+cEn5izrMSzw9sG7Q=', 'smithi099.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPrqtiZFc9dlncmXOvnjEjOflcccUdi8LsdQ0nquQ3+FHe1wyj2yml+y+5eHUvvsSCEjefzyQigqrkylK5I3DJk='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-17_22:16:03', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-18T04:55:20.144 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-18T04:55:20.147 DEBUG:teuthology.orchestra.run.smithi099:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-18T04:55:20.162 DEBUG:teuthology.orchestra.run.smithi002:> sudo install -d -m0777 -- /var/run/ceph 2024-09-18T04:55:20.205 DEBUG:teuthology.orchestra.run.smithi099:> sudo install -d -m0777 -- /var/run/ceph 2024-09-18T04:55:20.230 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-18T04:55:20.230 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/scratch_devs of=/dev/stdout 2024-09-18T04:55:20.287 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-18T04:55:20.288 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_1 2024-09-18T04:55:20.345 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-18T04:55:20.345 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T04:55:20.345 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-18T04:55:20.345 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T04:55:20.345 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T04:55:20.346 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-09-18 04:55:11.910670080 +0000 2024-09-18T04:55:20.346 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-09-18 04:52:44.070495892 +0000 2024-09-18T04:55:20.346 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-09-18 04:52:44.070495892 +0000 2024-09-18T04:55:20.346 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-09-18 04:52:44.070495892 +0000 2024-09-18T04:55:20.346 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-18T04:55:20.410 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-09-18T04:55:20.410 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-09-18T04:55:20.410 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000108027 s, 4.7 MB/s 2024-09-18T04:55:20.412 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-18T04:55:20.470 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_2 2024-09-18T04:55:20.526 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-18T04:55:20.527 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T04:55:20.527 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-18T04:55:20.527 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T04:55:20.527 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T04:55:20.527 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-09-18 04:55:11.911670057 +0000 2024-09-18T04:55:20.527 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-09-18 04:52:44.398488256 +0000 2024-09-18T04:55:20.527 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-09-18 04:52:44.398488256 +0000 2024-09-18T04:55:20.527 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-09-18 04:52:44.398488256 +0000 2024-09-18T04:55:20.528 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-18T04:55:20.592 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-09-18T04:55:20.592 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-09-18T04:55:20.592 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 8.9242e-05 s, 5.7 MB/s 2024-09-18T04:55:20.593 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-18T04:55:20.650 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_3 2024-09-18T04:55:20.706 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-18T04:55:20.707 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T04:55:20.707 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-18T04:55:20.707 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T04:55:20.707 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T04:55:20.708 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-09-18 04:55:11.911670057 +0000 2024-09-18T04:55:20.708 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-09-18 04:52:44.731480503 +0000 2024-09-18T04:55:20.708 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-09-18 04:52:44.731480503 +0000 2024-09-18T04:55:20.708 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-09-18 04:52:44.731480503 +0000 2024-09-18T04:55:20.709 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-18T04:55:20.772 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-09-18T04:55:20.772 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-09-18T04:55:20.772 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000103903 s, 4.9 MB/s 2024-09-18T04:55:20.773 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-18T04:55:20.830 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_4 2024-09-18T04:55:20.886 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-18T04:55:20.886 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T04:55:20.886 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-18T04:55:20.886 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T04:55:20.887 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T04:55:20.887 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2024-09-18 04:55:11.912670034 +0000 2024-09-18T04:55:20.887 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2024-09-18 04:52:45.055472960 +0000 2024-09-18T04:55:20.887 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2024-09-18 04:52:45.055472960 +0000 2024-09-18T04:55:20.887 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: 2024-09-18 04:52:45.055472960 +0000 2024-09-18T04:55:20.887 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-18T04:55:20.950 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2024-09-18T04:55:20.950 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2024-09-18T04:55:20.950 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000100935 s, 5.1 MB/s 2024-09-18T04:55:20.951 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-18T04:55:21.007 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T04:55:21.008 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-18T04:55:21.008 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/scratch_devs of=/dev/stdout 2024-09-18T04:55:21.024 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-18T04:55:21.024 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_1 2024-09-18T04:55:21.079 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-18T04:55:21.079 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T04:55:21.079 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-18T04:55:21.080 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T04:55:21.080 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T04:55:21.080 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-18 04:55:15.439924234 +0000 2024-09-18T04:55:21.080 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-18 04:52:42.546253992 +0000 2024-09-18T04:55:21.080 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-18 04:52:42.546253992 +0000 2024-09-18T04:55:21.080 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-18 04:52:42.546253992 +0000 2024-09-18T04:55:21.080 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-18T04:55:21.143 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-18T04:55:21.143 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-18T04:55:21.143 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000207276 s, 2.5 MB/s 2024-09-18T04:55:21.144 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-18T04:55:21.200 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_2 2024-09-18T04:55:21.255 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-18T04:55:21.256 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T04:55:21.256 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-18T04:55:21.256 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T04:55:21.256 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T04:55:21.256 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-18 04:55:15.439924234 +0000 2024-09-18T04:55:21.256 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-18 04:52:42.877246184 +0000 2024-09-18T04:55:21.256 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-18 04:52:42.877246184 +0000 2024-09-18T04:55:21.256 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-18 04:52:42.877246184 +0000 2024-09-18T04:55:21.256 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-18T04:55:21.319 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-18T04:55:21.319 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-18T04:55:21.319 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000193987 s, 2.6 MB/s 2024-09-18T04:55:21.320 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-18T04:55:21.376 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_3 2024-09-18T04:55:21.432 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-18T04:55:21.432 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T04:55:21.432 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-18T04:55:21.432 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T04:55:21.432 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T04:55:21.432 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-18 04:55:15.440924211 +0000 2024-09-18T04:55:21.433 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-18 04:52:43.206238424 +0000 2024-09-18T04:55:21.433 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-18 04:52:43.206238424 +0000 2024-09-18T04:55:21.433 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-18 04:52:43.206238424 +0000 2024-09-18T04:55:21.433 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-18T04:55:21.495 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-18T04:55:21.495 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-18T04:55:21.495 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000160533 s, 3.2 MB/s 2024-09-18T04:55:21.496 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-18T04:55:21.552 DEBUG:teuthology.orchestra.run.smithi099:> stat /dev/vg_nvme/lv_4 2024-09-18T04:55:21.608 INFO:teuthology.orchestra.run.smithi099.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-18T04:55:21.608 INFO:teuthology.orchestra.run.smithi099.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T04:55:21.608 INFO:teuthology.orchestra.run.smithi099.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-18T04:55:21.609 INFO:teuthology.orchestra.run.smithi099.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T04:55:21.609 INFO:teuthology.orchestra.run.smithi099.stdout:Context: system_u:object_r:device_t:s0 2024-09-18T04:55:21.609 INFO:teuthology.orchestra.run.smithi099.stdout:Access: 2024-09-18 04:55:15.440924211 +0000 2024-09-18T04:55:21.609 INFO:teuthology.orchestra.run.smithi099.stdout:Modify: 2024-09-18 04:52:43.535230664 +0000 2024-09-18T04:55:21.609 INFO:teuthology.orchestra.run.smithi099.stdout:Change: 2024-09-18 04:52:43.535230664 +0000 2024-09-18T04:55:21.609 INFO:teuthology.orchestra.run.smithi099.stdout: Birth: 2024-09-18 04:52:43.535230664 +0000 2024-09-18T04:55:21.610 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-18T04:55:21.670 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records in 2024-09-18T04:55:21.671 INFO:teuthology.orchestra.run.smithi099.stderr:1+0 records out 2024-09-18T04:55:21.671 INFO:teuthology.orchestra.run.smithi099.stderr:512 bytes copied, 0.000205847 s, 2.5 MB/s 2024-09-18T04:55:21.672 DEBUG:teuthology.orchestra.run.smithi099:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-18T04:55:21.728 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T04:55:21.728 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi002.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi099.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-18T04:55:21.728 INFO:tasks.ceph:Generating config... 2024-09-18T04:55:21.729 INFO:tasks.ceph:[client] rbd cache = False 2024-09-18T04:55:21.730 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-09-18T04:55:21.730 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-18T04:55:21.730 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-09-18T04:55:21.730 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-18T04:55:21.730 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-18T04:55:21.730 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-18T04:55:21.730 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-18T04:55:21.730 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-18T04:55:21.730 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-18T04:55:21.730 INFO:tasks.ceph:[osd] bluestore compression algorithm = lz4 2024-09-18T04:55:21.730 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-18T04:55:21.730 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-18T04:55:21.730 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-18T04:55:21.731 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-18T04:55:21.731 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-18T04:55:21.731 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-18T04:55:21.731 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-18T04:55:21.731 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-18T04:55:21.731 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-18T04:55:21.731 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-18T04:55:21.731 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-18T04:55:21.731 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-18T04:55:21.731 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-18T04:55:21.731 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-18T04:55:21.731 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-18T04:55:21.731 INFO:tasks.ceph:Setting up mon.a... 2024-09-18T04:55:21.732 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-18T04:55:21.789 INFO:teuthology.orchestra.run.smithi002.stdout:creating /etc/ceph/ceph.keyring 2024-09-18T04:55:21.792 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-18T04:55:21.839 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T04:55:21.866 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.2'), ('mon.c', '[v2:172.21.15.2:3301,v1:172.21.15.2:6790]'), ('mon.b', '172.21.15.99')] 2024-09-18T04:55:21.866 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.2,[v2:172.21.15.2:3301,v1:172.21.15.2:6790],172.21.15.99', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': False}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-09-18T04:55:21.867 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-18T04:55:21.867 DEBUG:teuthology.orchestra.run.smithi002:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-18T04:55:21.922 DEBUG:teuthology.orchestra.run.smithi002:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.2 --addv c '[v2:172.21.15.2:3301,v1:172.21.15.2:6790]' --add b 172.21.15.99 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-18T04:55:22.010 INFO:teuthology.orchestra.run.smithi002.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-18T04:55:22.011 INFO:teuthology.orchestra.run.smithi002.stdout:monmaptool: generated fsid c0761d0c-69d5-4b64-b8b9-c7d098b2d06d 2024-09-18T04:55:22.011 INFO:teuthology.orchestra.run.smithi002.stdout:setting min_mon_release = pacific 2024-09-18T04:55:22.011 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 0 2024-09-18T04:55:22.011 INFO:teuthology.orchestra.run.smithi002.stdout:fsid c0761d0c-69d5-4b64-b8b9-c7d098b2d06d 2024-09-18T04:55:22.011 INFO:teuthology.orchestra.run.smithi002.stdout:last_changed 2024-09-18T04:55:22.009275+0000 2024-09-18T04:55:22.011 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-09-18T04:55:22.009275+0000 2024-09-18T04:55:22.011 INFO:teuthology.orchestra.run.smithi002.stdout:min_mon_release 16 (pacific) 2024-09-18T04:55:22.011 INFO:teuthology.orchestra.run.smithi002.stdout:election_strategy: 1 2024-09-18T04:55:22.011 INFO:teuthology.orchestra.run.smithi002.stdout:0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2024-09-18T04:55:22.011 INFO:teuthology.orchestra.run.smithi002.stdout:1: [v2:172.21.15.99:3300/0,v1:172.21.15.99:6789/0] mon.b 2024-09-18T04:55:22.011 INFO:teuthology.orchestra.run.smithi002.stdout:2: [v2:172.21.15.2:3301/0,v1:172.21.15.2:6790/0] mon.c 2024-09-18T04:55:22.011 INFO:teuthology.orchestra.run.smithi002.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-18T04:55:22.013 DEBUG:teuthology.orchestra.run.smithi002:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-18T04:55:22.028 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c0761d0c-69d5-4b64-b8b9-c7d098b2d06d... 2024-09-18T04:55:22.029 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-18T04:55:22.070 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-18T04:55:22.112 INFO:teuthology.orchestra.run.smithi002.stdout:[global] 2024-09-18T04:55:22.113 INFO:teuthology.orchestra.run.smithi002.stdout: chdir = "" 2024-09-18T04:55:22.113 INFO:teuthology.orchestra.run.smithi002.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-18T04:55:22.113 INFO:teuthology.orchestra.run.smithi002.stdout: auth supported = cephx 2024-09-18T04:55:22.113 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.113 INFO:teuthology.orchestra.run.smithi002.stdout: filestore xattr use omap = true 2024-09-18T04:55:22.113 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.113 INFO:teuthology.orchestra.run.smithi002.stdout: mon clock drift allowed = 1.000 2024-09-18T04:55:22.113 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.113 INFO:teuthology.orchestra.run.smithi002.stdout: osd crush chooseleaf type = 0 2024-09-18T04:55:22.113 INFO:teuthology.orchestra.run.smithi002.stdout: auth debug = true 2024-09-18T04:55:22.113 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.113 INFO:teuthology.orchestra.run.smithi002.stdout: ms die on old message = true 2024-09-18T04:55:22.113 INFO:teuthology.orchestra.run.smithi002.stdout: ms die on bug = true 2024-09-18T04:55:22.113 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.114 INFO:teuthology.orchestra.run.smithi002.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-18T04:55:22.114 INFO:teuthology.orchestra.run.smithi002.stdout: mon pg warn max object skew = 0 2024-09-18T04:55:22.114 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.114 INFO:teuthology.orchestra.run.smithi002.stdout: # disable pg_autoscaler by default for new pools 2024-09-18T04:55:22.114 INFO:teuthology.orchestra.run.smithi002.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-18T04:55:22.114 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.114 INFO:teuthology.orchestra.run.smithi002.stdout: osd pool default size = 2 2024-09-18T04:55:22.114 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.114 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd allow primary affinity = true 2024-09-18T04:55:22.114 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd allow pg remap = true 2024-09-18T04:55:22.114 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on legacy crush tunables = false 2024-09-18T04:55:22.114 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on crush straw calc version zero = false 2024-09-18T04:55:22.114 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on no sortbitwise = false 2024-09-18T04:55:22.114 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on osd down out interval zero = false 2024-09-18T04:55:22.115 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on too few osds = false 2024-09-18T04:55:22.115 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-18T04:55:22.115 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-18T04:55:22.115 INFO:teuthology.orchestra.run.smithi002.stdout: mon_allow_pool_size_one = true 2024-09-18T04:55:22.115 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.115 INFO:teuthology.orchestra.run.smithi002.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-18T04:55:22.115 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.115 INFO:teuthology.orchestra.run.smithi002.stdout: osd default data pool replay window = 5 2024-09-18T04:55:22.115 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.115 INFO:teuthology.orchestra.run.smithi002.stdout: mon allow pool delete = true 2024-09-18T04:55:22.115 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.115 INFO:teuthology.orchestra.run.smithi002.stdout: mon cluster log file level = debug 2024-09-18T04:55:22.115 INFO:teuthology.orchestra.run.smithi002.stdout: debug asserts on shutdown = true 2024-09-18T04:55:22.116 INFO:teuthology.orchestra.run.smithi002.stdout: mon health detail to clog = false 2024-09-18T04:55:22.116 INFO:teuthology.orchestra.run.smithi002.stdout: mon host = "172.21.15.2,[v2:172.21.15.2:3301,v1:172.21.15.2:6790],172.21.15.99" 2024-09-18T04:55:22.116 INFO:teuthology.orchestra.run.smithi002.stdout: mon client directed command retry = 5 2024-09-18T04:55:22.116 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on pool no app = False 2024-09-18T04:55:22.116 INFO:teuthology.orchestra.run.smithi002.stdout: ms inject socket failures = 5000 2024-09-18T04:55:22.116 INFO:teuthology.orchestra.run.smithi002.stdout: fsid = c0761d0c-69d5-4b64-b8b9-c7d098b2d06d 2024-09-18T04:55:22.116 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.116 INFO:teuthology.orchestra.run.smithi002.stdout:[osd] 2024-09-18T04:55:22.116 INFO:teuthology.orchestra.run.smithi002.stdout: osd journal size = 100 2024-09-18T04:55:22.116 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.116 INFO:teuthology.orchestra.run.smithi002.stdout: osd scrub load threshold = 5.0 2024-09-18T04:55:22.116 INFO:teuthology.orchestra.run.smithi002.stdout: osd scrub max interval = 600 2024-09-18T04:55:22.116 INFO:teuthology.orchestra.run.smithi002.stdout: osd mclock profile = high_recovery_ops 2024-09-18T04:55:22.117 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.117 INFO:teuthology.orchestra.run.smithi002.stdout: osd recover clone overlap = true 2024-09-18T04:55:22.117 INFO:teuthology.orchestra.run.smithi002.stdout: osd recovery max chunk = 1048576 2024-09-18T04:55:22.117 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.117 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug shutdown = true 2024-09-18T04:55:22.117 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug op order = true 2024-09-18T04:55:22.117 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug verify stray on activate = true 2024-09-18T04:55:22.117 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.117 INFO:teuthology.orchestra.run.smithi002.stdout: osd open classes on start = true 2024-09-18T04:55:22.117 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug pg log writeout = true 2024-09-18T04:55:22.117 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.117 INFO:teuthology.orchestra.run.smithi002.stdout: osd deep scrub update digest min age = 30 2024-09-18T04:55:22.117 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.117 INFO:teuthology.orchestra.run.smithi002.stdout: osd map max advance = 10 2024-09-18T04:55:22.117 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.118 INFO:teuthology.orchestra.run.smithi002.stdout: journal zero on create = true 2024-09-18T04:55:22.118 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.118 INFO:teuthology.orchestra.run.smithi002.stdout: filestore ondisk finisher threads = 3 2024-09-18T04:55:22.118 INFO:teuthology.orchestra.run.smithi002.stdout: filestore apply finisher threads = 3 2024-09-18T04:55:22.119 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.120 INFO:teuthology.orchestra.run.smithi002.stdout: bdev debug aio = true 2024-09-18T04:55:22.120 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug misdirected ops = true 2024-09-18T04:55:22.120 INFO:teuthology.orchestra.run.smithi002.stdout: bluestore block size = 96636764160 2024-09-18T04:55:22.120 INFO:teuthology.orchestra.run.smithi002.stdout: bluestore compression algorithm = lz4 2024-09-18T04:55:22.120 INFO:teuthology.orchestra.run.smithi002.stdout: bluestore compression mode = aggressive 2024-09-18T04:55:22.120 INFO:teuthology.orchestra.run.smithi002.stdout: bluestore fsck on mount = True 2024-09-18T04:55:22.120 INFO:teuthology.orchestra.run.smithi002.stdout: debug bluefs = 1/20 2024-09-18T04:55:22.120 INFO:teuthology.orchestra.run.smithi002.stdout: debug bluestore = 1/20 2024-09-18T04:55:22.120 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2024-09-18T04:55:22.120 INFO:teuthology.orchestra.run.smithi002.stdout: debug osd = 20 2024-09-18T04:55:22.120 INFO:teuthology.orchestra.run.smithi002.stdout: debug rocksdb = 4/10 2024-09-18T04:55:22.120 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd backfillfull_ratio = 0.85 2024-09-18T04:55:22.120 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd full ratio = 0.9 2024-09-18T04:55:22.120 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd nearfull ratio = 0.8 2024-09-18T04:55:22.121 INFO:teuthology.orchestra.run.smithi002.stdout: osd blocked scrub grace period = 3600 2024-09-18T04:55:22.121 INFO:teuthology.orchestra.run.smithi002.stdout: osd failsafe full ratio = 0.95 2024-09-18T04:55:22.121 INFO:teuthology.orchestra.run.smithi002.stdout: osd max markdown count = 1000 2024-09-18T04:55:22.121 INFO:teuthology.orchestra.run.smithi002.stdout: osd objectstore = bluestore 2024-09-18T04:55:22.121 INFO:teuthology.orchestra.run.smithi002.stdout: osd shutdown pgref assert = True 2024-09-18T04:55:22.121 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.121 INFO:teuthology.orchestra.run.smithi002.stdout:[mgr] 2024-09-18T04:55:22.121 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2024-09-18T04:55:22.121 INFO:teuthology.orchestra.run.smithi002.stdout: debug mgr = 20 2024-09-18T04:55:22.121 INFO:teuthology.orchestra.run.smithi002.stdout: debug mon = 20 2024-09-18T04:55:22.121 INFO:teuthology.orchestra.run.smithi002.stdout: debug auth = 20 2024-09-18T04:55:22.121 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min pgs per osd = 4 2024-09-18T04:55:22.121 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min bytes per osd = 10 2024-09-18T04:55:22.121 INFO:teuthology.orchestra.run.smithi002.stdout: mgr/telemetry/nag = false 2024-09-18T04:55:22.122 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.122 INFO:teuthology.orchestra.run.smithi002.stdout:[mon] 2024-09-18T04:55:22.122 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2024-09-18T04:55:22.122 INFO:teuthology.orchestra.run.smithi002.stdout: debug mon = 20 2024-09-18T04:55:22.122 INFO:teuthology.orchestra.run.smithi002.stdout: debug paxos = 20 2024-09-18T04:55:22.122 INFO:teuthology.orchestra.run.smithi002.stdout: debug auth = 20 2024-09-18T04:55:22.122 INFO:teuthology.orchestra.run.smithi002.stdout: mon data avail warn = 5 2024-09-18T04:55:22.122 INFO:teuthology.orchestra.run.smithi002.stdout: mon mgr mkfs grace = 240 2024-09-18T04:55:22.122 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min pgs per osd = 4 2024-09-18T04:55:22.122 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd reporter subtree level = osd 2024-09-18T04:55:22.122 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd prime pg temp = true 2024-09-18T04:55:22.122 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min bytes per osd = 10 2024-09-18T04:55:22.122 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.122 INFO:teuthology.orchestra.run.smithi002.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-18T04:55:22.122 INFO:teuthology.orchestra.run.smithi002.stdout: auth mon ticket ttl = 660 # 11m 2024-09-18T04:55:22.123 INFO:teuthology.orchestra.run.smithi002.stdout: auth service ticket ttl = 240 # 4m 2024-09-18T04:55:22.123 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.123 INFO:teuthology.orchestra.run.smithi002.stdout: # don't complain about insecure global_id in the test suite 2024-09-18T04:55:22.123 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-18T04:55:22.123 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-18T04:55:22.123 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.123 INFO:teuthology.orchestra.run.smithi002.stdout: # 1m isn't quite enough 2024-09-18T04:55:22.123 INFO:teuthology.orchestra.run.smithi002.stdout: mon_down_mkfs_grace = 2m 2024-09-18T04:55:22.123 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.123 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_filestore_osds = false 2024-09-18T04:55:22.123 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:55:22.123 INFO:teuthology.orchestra.run.smithi002.stdout:[client] 2024-09-18T04:55:22.123 INFO:teuthology.orchestra.run.smithi002.stdout: rgw cache enabled = true 2024-09-18T04:55:22.123 INFO:teuthology.orchestra.run.smithi002.stdout: rgw enable ops log = true 2024-09-18T04:55:22.124 INFO:teuthology.orchestra.run.smithi002.stdout: rgw enable usage log = true 2024-09-18T04:55:22.124 INFO:teuthology.orchestra.run.smithi002.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-18T04:55:22.124 INFO:teuthology.orchestra.run.smithi002.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-18T04:55:22.124 INFO:teuthology.orchestra.run.smithi002.stdout: rbd cache = False 2024-09-18T04:55:22.124 INFO:teuthology.orchestra.run.smithi002.stdout:[mon.a] 2024-09-18T04:55:22.124 INFO:teuthology.orchestra.run.smithi002.stdout:[mon.c] 2024-09-18T04:55:22.124 INFO:teuthology.orchestra.run.smithi002.stdout:[mon.b] 2024-09-18T04:55:22.124 INFO:teuthology.orchestra.run.smithi099.stdout:[global] 2024-09-18T04:55:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: chdir = "" 2024-09-18T04:55:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-18T04:55:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: auth supported = cephx 2024-09-18T04:55:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: filestore xattr use omap = true 2024-09-18T04:55:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: mon clock drift allowed = 1.000 2024-09-18T04:55:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: osd crush chooseleaf type = 0 2024-09-18T04:55:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: auth debug = true 2024-09-18T04:55:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.125 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on old message = true 2024-09-18T04:55:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: ms die on bug = true 2024-09-18T04:55:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-18T04:55:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: mon pg warn max object skew = 0 2024-09-18T04:55:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: # disable pg_autoscaler by default for new pools 2024-09-18T04:55:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-18T04:55:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: osd pool default size = 2 2024-09-18T04:55:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd allow primary affinity = true 2024-09-18T04:55:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd allow pg remap = true 2024-09-18T04:55:22.126 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on legacy crush tunables = false 2024-09-18T04:55:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on crush straw calc version zero = false 2024-09-18T04:55:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on no sortbitwise = false 2024-09-18T04:55:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on osd down out interval zero = false 2024-09-18T04:55:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on too few osds = false 2024-09-18T04:55:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-18T04:55:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-18T04:55:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: mon_allow_pool_size_one = true 2024-09-18T04:55:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-18T04:55:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: osd default data pool replay window = 5 2024-09-18T04:55:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: mon allow pool delete = true 2024-09-18T04:55:22.127 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: mon cluster log file level = debug 2024-09-18T04:55:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: debug asserts on shutdown = true 2024-09-18T04:55:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: mon health detail to clog = false 2024-09-18T04:55:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: mon host = "172.21.15.2,[v2:172.21.15.2:3301,v1:172.21.15.2:6790],172.21.15.99" 2024-09-18T04:55:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: mon client directed command retry = 5 2024-09-18T04:55:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: mon warn on pool no app = False 2024-09-18T04:55:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: ms inject socket failures = 5000 2024-09-18T04:55:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: fsid = c0761d0c-69d5-4b64-b8b9-c7d098b2d06d 2024-09-18T04:55:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.128 INFO:teuthology.orchestra.run.smithi099.stdout:[osd] 2024-09-18T04:55:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: osd journal size = 100 2024-09-18T04:55:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.128 INFO:teuthology.orchestra.run.smithi099.stdout: osd scrub load threshold = 5.0 2024-09-18T04:55:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: osd scrub max interval = 600 2024-09-18T04:55:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: osd mclock profile = high_recovery_ops 2024-09-18T04:55:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: osd recover clone overlap = true 2024-09-18T04:55:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: osd recovery max chunk = 1048576 2024-09-18T04:55:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug shutdown = true 2024-09-18T04:55:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug op order = true 2024-09-18T04:55:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug verify stray on activate = true 2024-09-18T04:55:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: osd open classes on start = true 2024-09-18T04:55:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug pg log writeout = true 2024-09-18T04:55:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.129 INFO:teuthology.orchestra.run.smithi099.stdout: osd deep scrub update digest min age = 30 2024-09-18T04:55:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: osd map max advance = 10 2024-09-18T04:55:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: journal zero on create = true 2024-09-18T04:55:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: filestore ondisk finisher threads = 3 2024-09-18T04:55:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: filestore apply finisher threads = 3 2024-09-18T04:55:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: bdev debug aio = true 2024-09-18T04:55:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: osd debug misdirected ops = true 2024-09-18T04:55:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore block size = 96636764160 2024-09-18T04:55:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore compression algorithm = lz4 2024-09-18T04:55:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore compression mode = aggressive 2024-09-18T04:55:22.130 INFO:teuthology.orchestra.run.smithi099.stdout: bluestore fsck on mount = True 2024-09-18T04:55:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: debug bluefs = 1/20 2024-09-18T04:55:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: debug bluestore = 1/20 2024-09-18T04:55:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2024-09-18T04:55:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: debug osd = 20 2024-09-18T04:55:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: debug rocksdb = 4/10 2024-09-18T04:55:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd backfillfull_ratio = 0.85 2024-09-18T04:55:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd full ratio = 0.9 2024-09-18T04:55:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd nearfull ratio = 0.8 2024-09-18T04:55:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: osd blocked scrub grace period = 3600 2024-09-18T04:55:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: osd failsafe full ratio = 0.95 2024-09-18T04:55:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: osd max markdown count = 1000 2024-09-18T04:55:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: osd objectstore = bluestore 2024-09-18T04:55:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: osd shutdown pgref assert = True 2024-09-18T04:55:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.131 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr] 2024-09-18T04:55:22.131 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2024-09-18T04:55:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: debug mgr = 20 2024-09-18T04:55:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: debug mon = 20 2024-09-18T04:55:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: debug auth = 20 2024-09-18T04:55:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min pgs per osd = 4 2024-09-18T04:55:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min bytes per osd = 10 2024-09-18T04:55:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: mgr/telemetry/nag = false 2024-09-18T04:55:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.132 INFO:teuthology.orchestra.run.smithi099.stdout:[mon] 2024-09-18T04:55:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: debug ms = 1 2024-09-18T04:55:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: debug mon = 20 2024-09-18T04:55:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: debug paxos = 20 2024-09-18T04:55:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: debug auth = 20 2024-09-18T04:55:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: mon data avail warn = 5 2024-09-18T04:55:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: mon mgr mkfs grace = 240 2024-09-18T04:55:22.132 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min pgs per osd = 4 2024-09-18T04:55:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd reporter subtree level = osd 2024-09-18T04:55:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: mon osd prime pg temp = true 2024-09-18T04:55:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: mon reweight min bytes per osd = 10 2024-09-18T04:55:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-18T04:55:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: auth mon ticket ttl = 660 # 11m 2024-09-18T04:55:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: auth service ticket ttl = 240 # 4m 2024-09-18T04:55:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: # don't complain about insecure global_id in the test suite 2024-09-18T04:55:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-18T04:55:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-18T04:55:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.133 INFO:teuthology.orchestra.run.smithi099.stdout: # 1m isn't quite enough 2024-09-18T04:55:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: mon_down_mkfs_grace = 2m 2024-09-18T04:55:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: mon_warn_on_filestore_osds = false 2024-09-18T04:55:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: 2024-09-18T04:55:22.134 INFO:teuthology.orchestra.run.smithi099.stdout:[client] 2024-09-18T04:55:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: rgw cache enabled = true 2024-09-18T04:55:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: rgw enable ops log = true 2024-09-18T04:55:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: rgw enable usage log = true 2024-09-18T04:55:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-18T04:55:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-18T04:55:22.134 INFO:teuthology.orchestra.run.smithi099.stdout: rbd cache = False 2024-09-18T04:55:22.134 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.a] 2024-09-18T04:55:22.134 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.c] 2024-09-18T04:55:22.134 INFO:teuthology.orchestra.run.smithi099.stdout:[mon.b] 2024-09-18T04:55:22.136 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-18T04:55:22.136 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-18T04:55:22.230 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-18T04:55:22.231 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-18T04:55:22.231 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-18T04:55:22.246 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-18T04:55:22.246 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-18T04:55:22.301 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi002.front.sepia.ceph.com 2024-09-18T04:55:22.302 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-18T04:55:22.302 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-18T04:55:22.302 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T04:55:22.378 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-18T04:55:22.378 DEBUG:teuthology.orchestra.run.smithi002:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-18T04:55:22.434 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi099.front.sepia.ceph.com 2024-09-18T04:55:22.434 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-18T04:55:22.434 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-18T04:55:22.434 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T04:55:22.469 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-18T04:55:22.469 DEBUG:teuthology.orchestra.run.smithi099:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-18T04:55:22.524 INFO:tasks.ceph:Setting up mon nodes... 2024-09-18T04:55:22.525 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-18T04:55:22.525 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-18T04:55:22.577 INFO:teuthology.orchestra.run.smithi002.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-18T04:55:22.580 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-18T04:55:22.635 INFO:teuthology.orchestra.run.smithi099.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-18T04:55:22.639 INFO:tasks.ceph:Setting up mds nodes... 2024-09-18T04:55:22.639 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-18T04:55:22.639 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-18T04:55:22.681 INFO:teuthology.orchestra.run.smithi002.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-18T04:55:22.694 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-18T04:55:22.734 INFO:teuthology.orchestra.run.smithi099.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-18T04:55:22.747 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-18T04:55:22.748 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi002.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi099.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-18T04:55:22.748 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-18T04:55:22.774 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T04:55:22.774 INFO:tasks.ceph:role: osd.0 2024-09-18T04:55:22.774 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi002.front.sepia.ceph.com 2024-09-18T04:55:22.774 DEBUG:teuthology.orchestra.run.smithi002:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-18T04:55:22.839 INFO:teuthology.orchestra.run.smithi002.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T04:55:22.839 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T04:55:22.839 INFO:teuthology.orchestra.run.smithi002.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T04:55:22.839 INFO:teuthology.orchestra.run.smithi002.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T04:55:22.839 INFO:teuthology.orchestra.run.smithi002.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T04:55:22.839 INFO:teuthology.orchestra.run.smithi002.stdout: = sunit=0 swidth=0 blks 2024-09-18T04:55:22.839 INFO:teuthology.orchestra.run.smithi002.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T04:55:22.840 INFO:teuthology.orchestra.run.smithi002.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T04:55:22.840 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T04:55:22.840 INFO:teuthology.orchestra.run.smithi002.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T04:55:22.873 INFO:teuthology.orchestra.run.smithi002.stdout:Discarding blocks...Done. 2024-09-18T04:55:22.875 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi002.front.sepia.ceph.com -o noatime 2024-09-18T04:55:22.875 DEBUG:teuthology.orchestra.run.smithi002:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-18T04:55:22.945 DEBUG:teuthology.orchestra.run.smithi002:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-18T04:55:23.016 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-18T04:55:23.082 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T04:55:23.082 INFO:tasks.ceph:role: osd.1 2024-09-18T04:55:23.082 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi002.front.sepia.ceph.com 2024-09-18T04:55:23.083 DEBUG:teuthology.orchestra.run.smithi002:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-18T04:55:23.147 INFO:teuthology.orchestra.run.smithi002.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T04:55:23.147 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T04:55:23.147 INFO:teuthology.orchestra.run.smithi002.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T04:55:23.147 INFO:teuthology.orchestra.run.smithi002.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T04:55:23.147 INFO:teuthology.orchestra.run.smithi002.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T04:55:23.148 INFO:teuthology.orchestra.run.smithi002.stdout: = sunit=0 swidth=0 blks 2024-09-18T04:55:23.148 INFO:teuthology.orchestra.run.smithi002.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T04:55:23.148 INFO:teuthology.orchestra.run.smithi002.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T04:55:23.148 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T04:55:23.148 INFO:teuthology.orchestra.run.smithi002.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T04:55:23.180 INFO:teuthology.orchestra.run.smithi002.stdout:Discarding blocks...Done. 2024-09-18T04:55:23.182 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi002.front.sepia.ceph.com -o noatime 2024-09-18T04:55:23.182 DEBUG:teuthology.orchestra.run.smithi002:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-18T04:55:23.252 DEBUG:teuthology.orchestra.run.smithi002:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-18T04:55:23.323 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-18T04:55:23.389 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T04:55:23.389 INFO:tasks.ceph:role: osd.2 2024-09-18T04:55:23.389 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi002.front.sepia.ceph.com 2024-09-18T04:55:23.389 DEBUG:teuthology.orchestra.run.smithi002:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-18T04:55:23.454 INFO:teuthology.orchestra.run.smithi002.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T04:55:23.454 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T04:55:23.454 INFO:teuthology.orchestra.run.smithi002.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T04:55:23.454 INFO:teuthology.orchestra.run.smithi002.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T04:55:23.454 INFO:teuthology.orchestra.run.smithi002.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T04:55:23.454 INFO:teuthology.orchestra.run.smithi002.stdout: = sunit=0 swidth=0 blks 2024-09-18T04:55:23.454 INFO:teuthology.orchestra.run.smithi002.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T04:55:23.455 INFO:teuthology.orchestra.run.smithi002.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T04:55:23.455 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T04:55:23.455 INFO:teuthology.orchestra.run.smithi002.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T04:55:23.486 INFO:teuthology.orchestra.run.smithi002.stdout:Discarding blocks...Done. 2024-09-18T04:55:23.488 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi002.front.sepia.ceph.com -o noatime 2024-09-18T04:55:23.488 DEBUG:teuthology.orchestra.run.smithi002:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-18T04:55:23.558 DEBUG:teuthology.orchestra.run.smithi002:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-18T04:55:23.628 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-18T04:55:23.693 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T04:55:23.694 INFO:tasks.ceph:role: osd.3 2024-09-18T04:55:23.694 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi002.front.sepia.ceph.com 2024-09-18T04:55:23.694 DEBUG:teuthology.orchestra.run.smithi002:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-18T04:55:23.759 INFO:teuthology.orchestra.run.smithi002.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T04:55:23.759 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T04:55:23.759 INFO:teuthology.orchestra.run.smithi002.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T04:55:23.759 INFO:teuthology.orchestra.run.smithi002.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T04:55:23.759 INFO:teuthology.orchestra.run.smithi002.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T04:55:23.759 INFO:teuthology.orchestra.run.smithi002.stdout: = sunit=0 swidth=0 blks 2024-09-18T04:55:23.759 INFO:teuthology.orchestra.run.smithi002.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T04:55:23.759 INFO:teuthology.orchestra.run.smithi002.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T04:55:23.760 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T04:55:23.760 INFO:teuthology.orchestra.run.smithi002.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T04:55:23.792 INFO:teuthology.orchestra.run.smithi002.stdout:Discarding blocks...Done. 2024-09-18T04:55:23.793 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi002.front.sepia.ceph.com -o noatime 2024-09-18T04:55:23.794 DEBUG:teuthology.orchestra.run.smithi002:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-18T04:55:23.868 DEBUG:teuthology.orchestra.run.smithi002:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-18T04:55:23.938 DEBUG:teuthology.orchestra.run.smithi002:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T04:55:24.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-18T04:55:24.031+0000 7f4377415640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-18T04:55:24.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-18T04:55:24.032+0000 7f4377415640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-18T04:55:24.034 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-18T04:55:24.032+0000 7f4377415640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-18T04:55:24.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-18T04:55:24.032+0000 7f4377415640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-18T04:55:24.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-18T04:55:24.032+0000 7f4377415640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-18T04:55:24.035 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-18T04:55:24.032+0000 7f4377415640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-18T04:55:26.891 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-18T04:55:26.922 DEBUG:teuthology.orchestra.run.smithi002:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T04:55:27.016 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-18T04:55:27.012+0000 7f8665864640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-18T04:55:27.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-18T04:55:27.012+0000 7f8665864640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-18T04:55:27.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-18T04:55:27.012+0000 7f8665864640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-18T04:55:27.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-18T04:55:27.012+0000 7f8665864640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-18T04:55:27.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-18T04:55:27.012+0000 7f8665864640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-18T04:55:27.017 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-18T04:55:27.013+0000 7f8665864640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-18T04:55:29.870 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-18T04:55:29.900 DEBUG:teuthology.orchestra.run.smithi002:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T04:55:29.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-18T04:55:29.989+0000 7f33d5bfd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-18T04:55:29.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-18T04:55:29.989+0000 7f33d5bfd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-18T04:55:29.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-18T04:55:29.989+0000 7f33d5bfd640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-18T04:55:29.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-18T04:55:29.989+0000 7f33d5bfd640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-18T04:55:29.992 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-18T04:55:29.989+0000 7f33d5bfd640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-18T04:55:29.993 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-18T04:55:29.990+0000 7f33d5bfd640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-18T04:55:32.838 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-18T04:55:32.869 DEBUG:teuthology.orchestra.run.smithi002:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T04:55:32.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-18T04:55:32.959+0000 7f0a5481e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-18T04:55:32.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-18T04:55:32.959+0000 7f0a5481e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-18T04:55:32.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-18T04:55:32.959+0000 7f0a5481e640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-18T04:55:32.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-18T04:55:32.959+0000 7f0a5481e640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-18T04:55:32.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-18T04:55:32.959+0000 7f0a5481e640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-18T04:55:32.962 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-18T04:55:32.959+0000 7f0a5481e640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-18T04:55:35.814 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-18T04:55:35.845 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-18T04:55:35.875 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T04:55:35.875 INFO:tasks.ceph:role: osd.4 2024-09-18T04:55:35.875 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-18T04:55:35.875 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-18T04:55:35.951 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T04:55:35.952 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T04:55:35.952 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T04:55:35.952 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T04:55:35.952 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T04:55:35.952 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2024-09-18T04:55:35.952 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T04:55:35.952 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T04:55:35.952 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T04:55:35.952 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T04:55:36.471 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2024-09-18T04:55:36.474 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2024-09-18T04:55:36.474 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-18T04:55:36.589 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-18T04:55:36.623 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-18T04:55:36.690 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T04:55:36.690 INFO:tasks.ceph:role: osd.5 2024-09-18T04:55:36.690 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-18T04:55:36.690 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-18T04:55:36.763 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T04:55:36.764 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T04:55:36.764 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T04:55:36.764 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T04:55:36.764 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T04:55:36.764 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2024-09-18T04:55:36.764 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T04:55:36.764 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T04:55:36.764 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T04:55:36.764 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T04:55:37.225 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2024-09-18T04:55:37.227 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2024-09-18T04:55:37.227 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-18T04:55:37.310 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-18T04:55:37.380 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-18T04:55:37.446 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T04:55:37.446 INFO:tasks.ceph:role: osd.6 2024-09-18T04:55:37.446 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-18T04:55:37.446 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-18T04:55:37.519 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T04:55:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T04:55:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T04:55:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T04:55:37.519 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T04:55:37.519 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2024-09-18T04:55:37.519 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T04:55:37.519 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T04:55:37.520 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T04:55:37.520 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T04:55:38.057 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2024-09-18T04:55:38.059 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2024-09-18T04:55:38.059 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-18T04:55:38.205 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-18T04:55:38.237 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-18T04:55:38.302 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-18T04:55:38.302 INFO:tasks.ceph:role: osd.7 2024-09-18T04:55:38.302 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-18T04:55:38.302 DEBUG:teuthology.orchestra.run.smithi099:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-18T04:55:38.374 INFO:teuthology.orchestra.run.smithi099.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T04:55:38.374 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T04:55:38.374 INFO:teuthology.orchestra.run.smithi099.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T04:55:38.374 INFO:teuthology.orchestra.run.smithi099.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-18T04:55:38.374 INFO:teuthology.orchestra.run.smithi099.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T04:55:38.375 INFO:teuthology.orchestra.run.smithi099.stdout: = sunit=0 swidth=0 blks 2024-09-18T04:55:38.375 INFO:teuthology.orchestra.run.smithi099.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T04:55:38.375 INFO:teuthology.orchestra.run.smithi099.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-18T04:55:38.375 INFO:teuthology.orchestra.run.smithi099.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T04:55:38.375 INFO:teuthology.orchestra.run.smithi099.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T04:55:38.887 INFO:teuthology.orchestra.run.smithi099.stdout:Discarding blocks...Done. 2024-09-18T04:55:38.889 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi099.front.sepia.ceph.com -o noatime 2024-09-18T04:55:38.889 DEBUG:teuthology.orchestra.run.smithi099:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-18T04:55:38.991 DEBUG:teuthology.orchestra.run.smithi099:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-18T04:55:39.021 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T04:55:39.113 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-18T04:55:39.110+0000 7fe5a7a0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-18T04:55:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-18T04:55:39.111+0000 7fe5a7a0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-18T04:55:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-18T04:55:39.111+0000 7fe5a7a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-18T04:55:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-18T04:55:39.111+0000 7fe5a7a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-18T04:55:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-18T04:55:39.111+0000 7fe5a7a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-18T04:55:39.114 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-18T04:55:39.111+0000 7fe5a7a0f640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-18T04:55:41.967 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-18T04:55:41.997 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T04:55:42.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-18T04:55:42.084+0000 7fc26552c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-18T04:55:42.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-18T04:55:42.085+0000 7fc26552c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-18T04:55:42.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-18T04:55:42.085+0000 7fc26552c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-18T04:55:42.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-18T04:55:42.085+0000 7fc26552c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-18T04:55:42.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-18T04:55:42.085+0000 7fc26552c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-18T04:55:42.088 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-18T04:55:42.085+0000 7fc26552c640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-18T04:55:44.945 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-18T04:55:44.973 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T04:55:45.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-18T04:55:45.061+0000 7f48a5614640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-18T04:55:45.064 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-18T04:55:45.061+0000 7f48a5614640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-18T04:55:45.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-18T04:55:45.061+0000 7f48a5614640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-18T04:55:45.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-18T04:55:45.061+0000 7f48a5614640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-18T04:55:45.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-18T04:55:45.061+0000 7f48a5614640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-18T04:55:45.065 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-18T04:55:45.061+0000 7f48a5614640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-18T04:55:47.919 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-18T04:55:47.948 DEBUG:teuthology.orchestra.run.smithi099:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T04:55:48.041 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-18T04:55:48.037+0000 7f87d2b89640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-18T04:55:48.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-18T04:55:48.037+0000 7f87d2b89640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-18T04:55:48.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-18T04:55:48.037+0000 7f87d2b89640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-18T04:55:48.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-18T04:55:48.037+0000 7f87d2b89640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-18T04:55:48.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-18T04:55:48.037+0000 7f87d2b89640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-18T04:55:48.042 INFO:teuthology.orchestra.run.smithi099.stderr:2024-09-18T04:55:48.038+0000 7f87d2b89640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-18T04:55:50.898 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-18T04:55:50.928 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-18T04:55:50.929 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-18T04:55:50.929 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-18T04:55:50.961 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-18T04:55:50.961 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-18T04:55:51.033 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-18T04:55:51.034 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-18T04:55:51.109 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-18T04:55:51.110 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-18T04:55:51.181 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-18T04:55:51.181 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-18T04:55:51.251 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-18T04:55:51.252 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-18T04:55:51.281 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-18T04:55:51.281 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-18T04:55:51.351 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-18T04:55:51.351 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-18T04:55:51.420 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-18T04:55:51.420 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-18T04:55:51.488 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-18T04:55:51.489 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-18T04:55:51.555 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-18T04:55:51.555 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-18T04:55:51.575 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-18T04:55:51.575 DEBUG:teuthology.orchestra.run.smithi099:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-18T04:55:51.612 INFO:tasks.ceph:Adding keys to all mons... 2024-09-18T04:55:51.613 DEBUG:teuthology.orchestra.run.smithi002:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-18T04:55:51.618 DEBUG:teuthology.orchestra.run.smithi099:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-18T04:55:51.657 INFO:teuthology.orchestra.run.smithi002.stdout:[mgr.y] 2024-09-18T04:55:51.657 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQA6Xepm1cFWIhAA4eJ1NtEJ3Rq5oXw8cEOOow== 2024-09-18T04:55:51.657 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.0] 2024-09-18T04:55:51.657 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQA8XepmwQjyARAAGvys3VMkhh3WCsGBsSO2gw== 2024-09-18T04:55:51.657 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.1] 2024-09-18T04:55:51.657 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQA/XepmSsPLABAARlVS0hx0xR0MEaL7+kHK+Q== 2024-09-18T04:55:51.657 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.2] 2024-09-18T04:55:51.657 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQBBXepm0bwIOxAAL9dfzrFOowddUfibI0ijVw== 2024-09-18T04:55:51.657 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.3] 2024-09-18T04:55:51.657 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQBEXepmAzg6ORAATiDyv2WFACFARUmco1JL3g== 2024-09-18T04:55:51.657 INFO:teuthology.orchestra.run.smithi002.stdout:[mgr.x] 2024-09-18T04:55:51.657 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQA6Xepmx8/OJRAA61l6FiBl6A0NRY0IDuTlCA== 2024-09-18T04:55:51.658 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.4] 2024-09-18T04:55:51.658 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQBLXepmdUysBhAAxqJp5shyrNSMvkc7m6X9rQ== 2024-09-18T04:55:51.658 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.5] 2024-09-18T04:55:51.658 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQBOXepmT1wdBRAAi8JsT358d3xVXlZ1Z5Y48g== 2024-09-18T04:55:51.658 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.6] 2024-09-18T04:55:51.658 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQBRXepmL9qzAxAAUvIq0067Mkshwbq7Hs3d2A== 2024-09-18T04:55:51.658 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.7] 2024-09-18T04:55:51.658 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQBUXepmqjNIAhAAB0AN79hn/J7KgL1aZaTbPg== 2024-09-18T04:55:51.658 INFO:teuthology.orchestra.run.smithi002.stdout:[client.0] 2024-09-18T04:55:51.658 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQA6Xepm6NqFKBAA73Urkv47HTLx8Iv3N4ftjw== 2024-09-18T04:55:51.658 INFO:teuthology.orchestra.run.smithi002.stdout:[client.1] 2024-09-18T04:55:51.658 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQA6XepmfdG3KxAAJicEm/5HzF73EWXBrwyyOA== 2024-09-18T04:55:51.678 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr.y] 2024-09-18T04:55:51.678 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQA6Xepm1cFWIhAA4eJ1NtEJ3Rq5oXw8cEOOow== 2024-09-18T04:55:51.678 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.0] 2024-09-18T04:55:51.678 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQA8XepmwQjyARAAGvys3VMkhh3WCsGBsSO2gw== 2024-09-18T04:55:51.678 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.1] 2024-09-18T04:55:51.678 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQA/XepmSsPLABAARlVS0hx0xR0MEaL7+kHK+Q== 2024-09-18T04:55:51.679 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.2] 2024-09-18T04:55:51.679 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBBXepm0bwIOxAAL9dfzrFOowddUfibI0ijVw== 2024-09-18T04:55:51.679 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.3] 2024-09-18T04:55:51.679 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBEXepmAzg6ORAATiDyv2WFACFARUmco1JL3g== 2024-09-18T04:55:51.679 INFO:teuthology.orchestra.run.smithi099.stdout:[mgr.x] 2024-09-18T04:55:51.679 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQA6Xepmx8/OJRAA61l6FiBl6A0NRY0IDuTlCA== 2024-09-18T04:55:51.679 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.4] 2024-09-18T04:55:51.679 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBLXepmdUysBhAAxqJp5shyrNSMvkc7m6X9rQ== 2024-09-18T04:55:51.679 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.5] 2024-09-18T04:55:51.679 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBOXepmT1wdBRAAi8JsT358d3xVXlZ1Z5Y48g== 2024-09-18T04:55:51.679 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.6] 2024-09-18T04:55:51.679 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBRXepmL9qzAxAAUvIq0067Mkshwbq7Hs3d2A== 2024-09-18T04:55:51.679 INFO:teuthology.orchestra.run.smithi099.stdout:[osd.7] 2024-09-18T04:55:51.679 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQBUXepmqjNIAhAAB0AN79hn/J7KgL1aZaTbPg== 2024-09-18T04:55:51.680 INFO:teuthology.orchestra.run.smithi099.stdout:[client.0] 2024-09-18T04:55:51.680 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQA6Xepm6NqFKBAA73Urkv47HTLx8Iv3N4ftjw== 2024-09-18T04:55:51.680 INFO:teuthology.orchestra.run.smithi099.stdout:[client.1] 2024-09-18T04:55:51.680 INFO:teuthology.orchestra.run.smithi099.stdout: key = AQA6XepmfdG3KxAAJicEm/5HzF73EWXBrwyyOA== 2024-09-18T04:55:51.681 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T04:55:51.702 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T04:55:51.773 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T04:55:51.776 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T04:55:51.829 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T04:55:51.832 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T04:55:51.883 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T04:55:51.885 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T04:55:51.936 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T04:55:51.938 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T04:55:51.988 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T04:55:51.991 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T04:55:52.040 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T04:55:52.043 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T04:55:52.093 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T04:55:52.095 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T04:55:52.148 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T04:55:52.150 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T04:55:52.201 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T04:55:52.203 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T04:55:52.253 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T04:55:52.257 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T04:55:52.311 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T04:55:52.313 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T04:55:52.366 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-18T04:55:52.366 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-18T04:55:52.392 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T04:55:52.490 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-18T04:55:52.515 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-18T04:55:52.582 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T04:55:52.678 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-18T04:55:52.703 DEBUG:teuthology.orchestra.run.smithi099:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-18T04:55:52.729 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T04:55:52.825 DEBUG:teuthology.orchestra.run.smithi099:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-18T04:55:52.849 DEBUG:teuthology.orchestra.run.smithi002:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-18T04:55:52.852 DEBUG:teuthology.orchestra.run.smithi099:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-18T04:55:52.904 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-18T04:55:52.905 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-18T04:55:52.905 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-18T04:55:52.907 INFO:tasks.ceph.mon.a:Started 2024-09-18T04:55:52.907 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-18T04:55:52.908 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-18T04:55:52.909 INFO:tasks.ceph.mon.c:Started 2024-09-18T04:55:52.910 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-18T04:55:52.910 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-18T04:55:52.946 INFO:tasks.ceph.mon.b:Started 2024-09-18T04:55:52.947 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-18T04:55:52.947 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-18T04:55:52.947 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-18T04:55:52.949 INFO:tasks.ceph.mgr.y:Started 2024-09-18T04:55:52.952 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-18T04:55:52.952 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-18T04:55:52.954 INFO:tasks.ceph.mgr.x:Started 2024-09-18T04:55:52.954 INFO:tasks.ceph:Setting crush tunables to default 2024-09-18T04:55:52.955 DEBUG:teuthology.orchestra.run.smithi002:> sudo ceph --cluster ceph osd crush tunables default 2024-09-18T04:55:58.319 INFO:teuthology.orchestra.run.smithi002.stderr:adjusted tunables profile to default 2024-09-18T04:55:58.331 INFO:tasks.ceph:check_enable_crimson: False 2024-09-18T04:55:58.331 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-18T04:55:58.332 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-18T04:55:58.332 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-18T04:55:58.347 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:55:58.345+0000 7ff1ed42f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-18T04:55:58.359 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-18T04:55:58.359 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-18T04:55:58.389 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:55:58.386+0000 7ff1ed42f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-18T04:55:58.425 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-18T04:55:58.425 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-18T04:55:58.490 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2024-09-18T04:55:58.491 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-18T04:55:58.527 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:55:58.524+0000 7ff1ed42f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-18T04:55:58.557 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-18T04:55:58.557 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-18T04:55:58.579 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:55:58.577+0000 7ff1ed42f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-18T04:55:58.583 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-18T04:55:58.583 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-18T04:55:58.652 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-18T04:55:58.652 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-18T04:55:58.717 DEBUG:teuthology.orchestra.run.smithi099:> set -ex 2024-09-18T04:55:58.717 DEBUG:teuthology.orchestra.run.smithi099:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-18T04:55:58.746 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:55:58.743+0000 7ff1ed42f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-18T04:55:58.770 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new c43f8024-3a3e-424e-aa54-398310ceb70d 0 2024-09-18T04:55:58.861 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:55:58.859+0000 7ff1ed42f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-18T04:55:58.987 INFO:teuthology.orchestra.run.smithi099.stdout:0 2024-09-18T04:55:58.998 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new c664b970-145c-4b6f-8852-2f6dcf1fa30d 1 2024-09-18T04:55:59.140 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-18T04:55:59.138+0000 7efc8a923040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-18T04:55:59.178 INFO:teuthology.orchestra.run.smithi099.stdout:1 2024-09-18T04:55:59.183 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-18T04:55:59.181+0000 7efc8a923040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-18T04:55:59.189 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new 0609ea8c-d975-490e-8844-04a3e4512efd 2 2024-09-18T04:55:59.194 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:55:59.192+0000 7ff1ed42f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-18T04:55:59.250 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:55:59.247+0000 7ff1ed42f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-18T04:55:59.333 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-18T04:55:59.331+0000 7efc8a923040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-18T04:55:59.368 INFO:teuthology.orchestra.run.smithi099.stdout:2 2024-09-18T04:55:59.379 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new c6bd5ac8-ab89-4fa2-b843-3b166d1b238b 3 2024-09-18T04:55:59.386 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-18T04:55:59.384+0000 7efc8a923040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-18T04:55:59.404 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:55:59.402+0000 7ff1ed42f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-18T04:55:59.553 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-18T04:55:59.550+0000 7efc8a923040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-18T04:55:59.583 INFO:teuthology.orchestra.run.smithi099.stdout:3 2024-09-18T04:55:59.594 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new 89d6a371-4f4e-465e-a023-d5083cce6e6c 4 2024-09-18T04:55:59.668 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-18T04:55:59.665+0000 7efc8a923040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-18T04:55:59.771 INFO:teuthology.orchestra.run.smithi099.stdout:4 2024-09-18T04:55:59.782 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new da99686b-d979-4753-b4aa-a3b26e9a0497 5 2024-09-18T04:55:59.959 INFO:teuthology.orchestra.run.smithi099.stdout:5 2024-09-18T04:55:59.970 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new ceec6aa6-603d-4706-bfea-3c07dbdaed3a 6 2024-09-18T04:56:00.008 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-18T04:56:00.006+0000 7efc8a923040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-18T04:56:00.063 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-18T04:56:00.061+0000 7efc8a923040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-18T04:56:00.149 INFO:teuthology.orchestra.run.smithi099.stdout:6 2024-09-18T04:56:00.160 DEBUG:teuthology.orchestra.run.smithi099:> sudo ceph --cluster ceph osd new 6779dfad-f60b-41f4-b666-a99db6dc1814 7 2024-09-18T04:56:00.220 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-18T04:56:00.217+0000 7efc8a923040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-18T04:56:00.338 INFO:teuthology.orchestra.run.smithi099.stdout:7 2024-09-18T04:56:00.348 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-18T04:56:00.349 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-18T04:56:00.351 INFO:tasks.ceph.osd.0:Started 2024-09-18T04:56:00.351 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-18T04:56:00.352 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-18T04:56:00.354 INFO:tasks.ceph.osd.1:Started 2024-09-18T04:56:00.354 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-18T04:56:00.354 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-18T04:56:00.356 INFO:tasks.ceph.osd.2:Started 2024-09-18T04:56:00.357 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-18T04:56:00.357 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-18T04:56:00.359 INFO:tasks.ceph.osd.3:Started 2024-09-18T04:56:00.359 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-18T04:56:00.360 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-18T04:56:00.362 INFO:tasks.ceph.osd.4:Started 2024-09-18T04:56:00.362 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-18T04:56:00.362 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-18T04:56:00.364 INFO:tasks.ceph.osd.5:Started 2024-09-18T04:56:00.364 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-18T04:56:00.365 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-18T04:56:00.366 INFO:tasks.ceph.osd.6:Started 2024-09-18T04:56:00.367 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-18T04:56:00.367 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-18T04:56:00.369 INFO:tasks.ceph.osd.7:Started 2024-09-18T04:56:00.370 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T04:56:00.390 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:56:00.387+0000 7ff1ed42f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-18T04:56:00.455 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:56:00.452+0000 7ff1ed42f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-18T04:56:00.572 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:56:00.573 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":10,"fsid":"c0761d0c-69d5-4b64-b8b9-c7d098b2d06d","created":"2024-09-18T04:55:58.216538+0000","modified":"2024-09-18T04:56:00.333800+0000","last_up_change":"0.000000","last_in_change":"2024-09-18T04:56:00.333800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c43f8024-3a3e-424e-aa54-398310ceb70d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c664b970-145c-4b6f-8852-2f6dcf1fa30d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0609ea8c-d975-490e-8844-04a3e4512efd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c6bd5ac8-ab89-4fa2-b843-3b166d1b238b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"89d6a371-4f4e-465e-a023-d5083cce6e6c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"da99686b-d979-4753-b4aa-a3b26e9a0497","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ceec6aa6-603d-4706-bfea-3c07dbdaed3a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6779dfad-f60b-41f4-b666-a99db6dc1814","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T04:56:00.584 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-18T04:56:00.584 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-18T04:56:00.820 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:56:00.818+0000 7ff1ed42f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-18T04:56:00.903 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:56:00.901+0000 7ff1ed42f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-18T04:56:00.970 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:00.967+0000 7f576d812640 -1 Falling back to public interface 2024-09-18T04:56:00.974 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:00.971+0000 7f3a1b80b640 -1 Falling back to public interface 2024-09-18T04:56:00.977 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:00.974+0000 7fb4b1685640 -1 Falling back to public interface 2024-09-18T04:56:00.977 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:00.975+0000 7f2a6aa9c640 -1 Falling back to public interface 2024-09-18T04:56:00.979 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:00.976+0000 7fae98725640 -1 Falling back to public interface 2024-09-18T04:56:00.981 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:00.979+0000 7f36fec9c640 -1 Falling back to public interface 2024-09-18T04:56:00.985 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:00.983+0000 7f3c28fbd640 -1 Falling back to public interface 2024-09-18T04:56:00.991 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:00.988+0000 7f861eda8640 -1 Falling back to public interface 2024-09-18T04:56:01.013 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:56:01.010+0000 7ff1ed42f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-18T04:56:01.058 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:56:01.055+0000 7ff1ed42f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-18T04:56:01.167 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:56:01.164+0000 7ff1ed42f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-18T04:56:01.218 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:56:01.215+0000 7ff1ed42f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-18T04:56:01.220 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-18T04:56:01.218+0000 7efc8a923040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-18T04:56:01.264 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:56:01.261+0000 7ff1ed42f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-18T04:56:01.275 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-18T04:56:01.272+0000 7efc8a923040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-18T04:56:01.373 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:56:01.371+0000 7ff1ed42f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-18T04:56:01.390 DEBUG:teuthology.orchestra.run.smithi002:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-18T04:56:01.422 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:56:01.419+0000 7ff1ed42f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-18T04:56:01.472 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:56:01.470+0000 7ff1ed42f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-18T04:56:01.541 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:56:01.538+0000 7ff1ed42f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-18T04:56:01.553 INFO:teuthology.misc.health.smithi002.stdout: 2024-09-18T04:56:01.553 INFO:teuthology.misc.health.smithi002.stdout:{"epoch":10,"fsid":"c0761d0c-69d5-4b64-b8b9-c7d098b2d06d","created":"2024-09-18T04:55:58.216538+0000","modified":"2024-09-18T04:56:00.333800+0000","last_up_change":"0.000000","last_in_change":"2024-09-18T04:56:00.333800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c43f8024-3a3e-424e-aa54-398310ceb70d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c664b970-145c-4b6f-8852-2f6dcf1fa30d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"0609ea8c-d975-490e-8844-04a3e4512efd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"c6bd5ac8-ab89-4fa2-b843-3b166d1b238b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"89d6a371-4f4e-465e-a023-d5083cce6e6c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"da99686b-d979-4753-b4aa-a3b26e9a0497","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ceec6aa6-603d-4706-bfea-3c07dbdaed3a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"6779dfad-f60b-41f4-b666-a99db6dc1814","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T04:56:01.563 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-18T04:56:01.585 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:56:01.582+0000 7ff1ed42f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-18T04:56:01.628 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:56:01.625+0000 7ff1ed42f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-18T04:56:01.662 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-18T04:56:01.660+0000 7efc8a923040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-18T04:56:01.750 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-18T04:56:01.748+0000 7efc8a923040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-18T04:56:01.860 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-18T04:56:01.858+0000 7efc8a923040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-18T04:56:01.908 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-18T04:56:01.905+0000 7efc8a923040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-18T04:56:02.023 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-18T04:56:02.021+0000 7efc8a923040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-18T04:56:02.078 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-18T04:56:02.076+0000 7efc8a923040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-18T04:56:02.125 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-18T04:56:02.122+0000 7efc8a923040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-18T04:56:02.240 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-18T04:56:02.237+0000 7efc8a923040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-18T04:56:02.262 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:56:02.259+0000 7ff1ed42f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-18T04:56:02.295 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-18T04:56:02.292+0000 7efc8a923040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-18T04:56:02.349 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-18T04:56:02.346+0000 7efc8a923040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-18T04:56:02.419 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-18T04:56:02.416+0000 7efc8a923040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-18T04:56:02.465 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-18T04:56:02.462+0000 7efc8a923040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-18T04:56:02.510 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-18T04:56:02.508+0000 7efc8a923040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-18T04:56:03.178 INFO:tasks.ceph.mgr.y.smithi002.stderr:2024-09-18T04:56:03.175+0000 7efc8a923040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-18T04:56:03.277 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:56:03.274+0000 7ff18d579640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-18T04:56:03.277 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:56:03.274+0000 7ff18d579640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-18T04:56:04.508 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:04.505+0000 7f3c28fbd640 -1 osd.5 0 log_to_monitors true 2024-09-18T04:56:04.508 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:04.506+0000 7fae98725640 -1 osd.7 0 log_to_monitors true 2024-09-18T04:56:04.509 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:04.507+0000 7f36fec9c640 -1 osd.0 0 log_to_monitors true 2024-09-18T04:56:04.510 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:04.507+0000 7fb4b1685640 -1 osd.6 0 log_to_monitors true 2024-09-18T04:56:04.512 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:04.509+0000 7f2a6aa9c640 -1 osd.3 0 log_to_monitors true 2024-09-18T04:56:04.513 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:04.510+0000 7f861eda8640 -1 osd.4 0 log_to_monitors true 2024-09-18T04:56:04.522 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:04.520+0000 7f3a1b80b640 -1 osd.1 0 log_to_monitors true 2024-09-18T04:56:04.523 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:04.521+0000 7f576d812640 -1 osd.2 0 log_to_monitors true 2024-09-18T04:56:07.357 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:07.354+0000 7f861ad35640 -1 osd.4 0 waiting for initial osdmap 2024-09-18T04:56:07.365 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:07.363+0000 7fb4ad612640 -1 osd.6 0 waiting for initial osdmap 2024-09-18T04:56:07.369 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:07.367+0000 7f8612b56640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T04:56:07.379 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:07.377+0000 7fb4a5433640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T04:56:07.387 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:07.385+0000 7f3c24f4a640 -1 osd.5 0 waiting for initial osdmap 2024-09-18T04:56:07.397 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:07.395+0000 7f3c1cd6b640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T04:56:07.443 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:07.440+0000 7f2a66a29640 -1 osd.3 0 waiting for initial osdmap 2024-09-18T04:56:07.456 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:07.454+0000 7f2a5e84a640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T04:56:07.457 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:07.454+0000 7f36fac29640 -1 osd.0 0 waiting for initial osdmap 2024-09-18T04:56:07.470 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:07.467+0000 7fae946b2640 -1 osd.7 0 waiting for initial osdmap 2024-09-18T04:56:07.476 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:07.473+0000 7f36f2a4a640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T04:56:07.483 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:07.481+0000 7fae8c4d3640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T04:56:07.488 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:07.485+0000 7f57691de640 -1 osd.2 0 waiting for initial osdmap 2024-09-18T04:56:07.498 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:07.496+0000 7f3a1721a640 -1 osd.1 0 waiting for initial osdmap 2024-09-18T04:56:07.501 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:07.499+0000 7f5760fff640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T04:56:07.514 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:07.512+0000 7f3a0f03b640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T04:56:08.371 DEBUG:teuthology.orchestra.run.smithi002:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-18T04:56:08.662 INFO:teuthology.misc.health.smithi002.stdout: 2024-09-18T04:56:08.662 INFO:teuthology.misc.health.smithi002.stdout:{"epoch":13,"fsid":"c0761d0c-69d5-4b64-b8b9-c7d098b2d06d","created":"2024-09-18T04:55:58.216538+0000","modified":"2024-09-18T04:56:08.369924+0000","last_up_change":"2024-09-18T04:56:08.369924+0000","last_in_change":"2024-09-18T04:56:00.333800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c43f8024-3a3e-424e-aa54-398310ceb70d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4106704527},{"type":"v1","addr":"172.21.15.2:6825","nonce":4106704527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":4106704527},{"type":"v1","addr":"172.21.15.2:6827","nonce":4106704527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":4106704527},{"type":"v1","addr":"172.21.15.2:6831","nonce":4106704527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":4106704527},{"type":"v1","addr":"172.21.15.2:6829","nonce":4106704527}]},"public_addr":"172.21.15.2:6825/4106704527","cluster_addr":"172.21.15.2:6827/4106704527","heartbeat_back_addr":"172.21.15.2:6831/4106704527","heartbeat_front_addr":"172.21.15.2:6829/4106704527","state":["exists","up"]},{"osd":1,"uuid":"c664b970-145c-4b6f-8852-2f6dcf1fa30d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1781959259},{"type":"v1","addr":"172.21.15.2:6809","nonce":1781959259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1781959259},{"type":"v1","addr":"172.21.15.2:6811","nonce":1781959259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1781959259},{"type":"v1","addr":"172.21.15.2:6815","nonce":1781959259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1781959259},{"type":"v1","addr":"172.21.15.2:6813","nonce":1781959259}]},"public_addr":"172.21.15.2:6809/1781959259","cluster_addr":"172.21.15.2:6811/1781959259","heartbeat_back_addr":"172.21.15.2:6815/1781959259","heartbeat_front_addr":"172.21.15.2:6813/1781959259","state":["exists","up"]},{"osd":2,"uuid":"0609ea8c-d975-490e-8844-04a3e4512efd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":1153570802},{"type":"v1","addr":"172.21.15.2:6801","nonce":1153570802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1153570802},{"type":"v1","addr":"172.21.15.2:6803","nonce":1153570802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1153570802},{"type":"v1","addr":"172.21.15.2:6807","nonce":1153570802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1153570802},{"type":"v1","addr":"172.21.15.2:6805","nonce":1153570802}]},"public_addr":"172.21.15.2:6801/1153570802","cluster_addr":"172.21.15.2:6803/1153570802","heartbeat_back_addr":"172.21.15.2:6807/1153570802","heartbeat_front_addr":"172.21.15.2:6805/1153570802","state":["exists","up"]},{"osd":3,"uuid":"c6bd5ac8-ab89-4fa2-b843-3b166d1b238b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":2008321375},{"type":"v1","addr":"172.21.15.2:6817","nonce":2008321375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2008321375},{"type":"v1","addr":"172.21.15.2:6819","nonce":2008321375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2008321375},{"type":"v1","addr":"172.21.15.2:6823","nonce":2008321375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2008321375},{"type":"v1","addr":"172.21.15.2:6821","nonce":2008321375}]},"public_addr":"172.21.15.2:6817/2008321375","cluster_addr":"172.21.15.2:6819/2008321375","heartbeat_back_addr":"172.21.15.2:6823/2008321375","heartbeat_front_addr":"172.21.15.2:6821/2008321375","state":["exists","up"]},{"osd":4,"uuid":"89d6a371-4f4e-465e-a023-d5083cce6e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2939896598},{"type":"v1","addr":"172.21.15.99:6825","nonce":2939896598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":2939896598},{"type":"v1","addr":"172.21.15.99:6827","nonce":2939896598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":2939896598},{"type":"v1","addr":"172.21.15.99:6831","nonce":2939896598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":2939896598},{"type":"v1","addr":"172.21.15.99:6829","nonce":2939896598}]},"public_addr":"172.21.15.99:6825/2939896598","cluster_addr":"172.21.15.99:6827/2939896598","heartbeat_back_addr":"172.21.15.99:6831/2939896598","heartbeat_front_addr":"172.21.15.99:6829/2939896598","state":["exists","up"]},{"osd":5,"uuid":"da99686b-d979-4753-b4aa-a3b26e9a0497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":3440879618},{"type":"v1","addr":"172.21.15.99:6817","nonce":3440879618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":3440879618},{"type":"v1","addr":"172.21.15.99:6819","nonce":3440879618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":3440879618},{"type":"v1","addr":"172.21.15.99:6823","nonce":3440879618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":3440879618},{"type":"v1","addr":"172.21.15.99:6821","nonce":3440879618}]},"public_addr":"172.21.15.99:6817/3440879618","cluster_addr":"172.21.15.99:6819/3440879618","heartbeat_back_addr":"172.21.15.99:6823/3440879618","heartbeat_front_addr":"172.21.15.99:6821/3440879618","state":["exists","up"]},{"osd":6,"uuid":"ceec6aa6-603d-4706-bfea-3c07dbdaed3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":2493830504},{"type":"v1","addr":"172.21.15.99:6801","nonce":2493830504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":2493830504},{"type":"v1","addr":"172.21.15.99:6803","nonce":2493830504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":2493830504},{"type":"v1","addr":"172.21.15.99:6807","nonce":2493830504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":2493830504},{"type":"v1","addr":"172.21.15.99:6805","nonce":2493830504}]},"public_addr":"172.21.15.99:6801/2493830504","cluster_addr":"172.21.15.99:6803/2493830504","heartbeat_back_addr":"172.21.15.99:6807/2493830504","heartbeat_front_addr":"172.21.15.99:6805/2493830504","state":["exists","up"]},{"osd":7,"uuid":"6779dfad-f60b-41f4-b666-a99db6dc1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":91038858},{"type":"v1","addr":"172.21.15.99:6809","nonce":91038858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":91038858},{"type":"v1","addr":"172.21.15.99:6811","nonce":91038858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":91038858},{"type":"v1","addr":"172.21.15.99:6815","nonce":91038858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":91038858},{"type":"v1","addr":"172.21.15.99:6813","nonce":91038858}]},"public_addr":"172.21.15.99:6809/91038858","cluster_addr":"172.21.15.99:6811/91038858","heartbeat_back_addr":"172.21.15.99:6815/91038858","heartbeat_front_addr":"172.21.15.99:6813/91038858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T04:56:08.673 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-18T04:56:08.673 INFO:tasks.ceph:Creating RBD pool 2024-09-18T04:56:08.673 DEBUG:teuthology.orchestra.run.smithi002:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-18T04:56:09.376 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'rbd' created 2024-09-18T04:56:09.390 DEBUG:teuthology.orchestra.run.smithi002:> rbd --cluster ceph pool init rbd 2024-09-18T04:56:12.405 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-18T04:56:12.406 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-18T04:56:12.406 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-18T04:56:12.406 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-18T04:56:12.406 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-18T04:56:12.749 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:56:12.759 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":4,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6832","nonce":2787988735},{"type":"v1","addr":"172.21.15.99:6833","nonce":2787988735}]},"active_addr":"172.21.15.99:6833/2787988735","active_change":"2024-09-18T04:56:02.262184+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4119,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":3155863658}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":2278895132}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.99:0","nonce":3052035987}]}]} 2024-09-18T04:56:12.763 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-18T04:56:12.763 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-18T04:56:12.763 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T04:56:13.045 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:56:13.045 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":17,"fsid":"c0761d0c-69d5-4b64-b8b9-c7d098b2d06d","created":"2024-09-18T04:55:58.216538+0000","modified":"2024-09-18T04:56:12.386465+0000","last_up_change":"2024-09-18T04:56:08.369924+0000","last_in_change":"2024-09-18T04:56:00.333800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-18T04:56:08.983344+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-18T04:56:10.313988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c43f8024-3a3e-424e-aa54-398310ceb70d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4106704527},{"type":"v1","addr":"172.21.15.2:6825","nonce":4106704527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":4106704527},{"type":"v1","addr":"172.21.15.2:6827","nonce":4106704527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":4106704527},{"type":"v1","addr":"172.21.15.2:6831","nonce":4106704527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":4106704527},{"type":"v1","addr":"172.21.15.2:6829","nonce":4106704527}]},"public_addr":"172.21.15.2:6825/4106704527","cluster_addr":"172.21.15.2:6827/4106704527","heartbeat_back_addr":"172.21.15.2:6831/4106704527","heartbeat_front_addr":"172.21.15.2:6829/4106704527","state":["exists","up"]},{"osd":1,"uuid":"c664b970-145c-4b6f-8852-2f6dcf1fa30d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1781959259},{"type":"v1","addr":"172.21.15.2:6809","nonce":1781959259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1781959259},{"type":"v1","addr":"172.21.15.2:6811","nonce":1781959259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1781959259},{"type":"v1","addr":"172.21.15.2:6815","nonce":1781959259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1781959259},{"type":"v1","addr":"172.21.15.2:6813","nonce":1781959259}]},"public_addr":"172.21.15.2:6809/1781959259","cluster_addr":"172.21.15.2:6811/1781959259","heartbeat_back_addr":"172.21.15.2:6815/1781959259","heartbeat_front_addr":"172.21.15.2:6813/1781959259","state":["exists","up"]},{"osd":2,"uuid":"0609ea8c-d975-490e-8844-04a3e4512efd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":1153570802},{"type":"v1","addr":"172.21.15.2:6801","nonce":1153570802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1153570802},{"type":"v1","addr":"172.21.15.2:6803","nonce":1153570802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1153570802},{"type":"v1","addr":"172.21.15.2:6807","nonce":1153570802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1153570802},{"type":"v1","addr":"172.21.15.2:6805","nonce":1153570802}]},"public_addr":"172.21.15.2:6801/1153570802","cluster_addr":"172.21.15.2:6803/1153570802","heartbeat_back_addr":"172.21.15.2:6807/1153570802","heartbeat_front_addr":"172.21.15.2:6805/1153570802","state":["exists","up"]},{"osd":3,"uuid":"c6bd5ac8-ab89-4fa2-b843-3b166d1b238b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":2008321375},{"type":"v1","addr":"172.21.15.2:6817","nonce":2008321375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2008321375},{"type":"v1","addr":"172.21.15.2:6819","nonce":2008321375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2008321375},{"type":"v1","addr":"172.21.15.2:6823","nonce":2008321375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2008321375},{"type":"v1","addr":"172.21.15.2:6821","nonce":2008321375}]},"public_addr":"172.21.15.2:6817/2008321375","cluster_addr":"172.21.15.2:6819/2008321375","heartbeat_back_addr":"172.21.15.2:6823/2008321375","heartbeat_front_addr":"172.21.15.2:6821/2008321375","state":["exists","up"]},{"osd":4,"uuid":"89d6a371-4f4e-465e-a023-d5083cce6e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2939896598},{"type":"v1","addr":"172.21.15.99:6825","nonce":2939896598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":2939896598},{"type":"v1","addr":"172.21.15.99:6827","nonce":2939896598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":2939896598},{"type":"v1","addr":"172.21.15.99:6831","nonce":2939896598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":2939896598},{"type":"v1","addr":"172.21.15.99:6829","nonce":2939896598}]},"public_addr":"172.21.15.99:6825/2939896598","cluster_addr":"172.21.15.99:6827/2939896598","heartbeat_back_addr":"172.21.15.99:6831/2939896598","heartbeat_front_addr":"172.21.15.99:6829/2939896598","state":["exists","up"]},{"osd":5,"uuid":"da99686b-d979-4753-b4aa-a3b26e9a0497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":3440879618},{"type":"v1","addr":"172.21.15.99:6817","nonce":3440879618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":3440879618},{"type":"v1","addr":"172.21.15.99:6819","nonce":3440879618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":3440879618},{"type":"v1","addr":"172.21.15.99:6823","nonce":3440879618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":3440879618},{"type":"v1","addr":"172.21.15.99:6821","nonce":3440879618}]},"public_addr":"172.21.15.99:6817/3440879618","cluster_addr":"172.21.15.99:6819/3440879618","heartbeat_back_addr":"172.21.15.99:6823/3440879618","heartbeat_front_addr":"172.21.15.99:6821/3440879618","state":["exists","up"]},{"osd":6,"uuid":"ceec6aa6-603d-4706-bfea-3c07dbdaed3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":2493830504},{"type":"v1","addr":"172.21.15.99:6801","nonce":2493830504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":2493830504},{"type":"v1","addr":"172.21.15.99:6803","nonce":2493830504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":2493830504},{"type":"v1","addr":"172.21.15.99:6807","nonce":2493830504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":2493830504},{"type":"v1","addr":"172.21.15.99:6805","nonce":2493830504}]},"public_addr":"172.21.15.99:6801/2493830504","cluster_addr":"172.21.15.99:6803/2493830504","heartbeat_back_addr":"172.21.15.99:6807/2493830504","heartbeat_front_addr":"172.21.15.99:6805/2493830504","state":["exists","up"]},{"osd":7,"uuid":"6779dfad-f60b-41f4-b666-a99db6dc1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":91038858},{"type":"v1","addr":"172.21.15.99:6809","nonce":91038858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":91038858},{"type":"v1","addr":"172.21.15.99:6811","nonce":91038858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":91038858},{"type":"v1","addr":"172.21.15.99:6815","nonce":91038858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":91038858},{"type":"v1","addr":"172.21.15.99:6813","nonce":91038858}]},"public_addr":"172.21.15.99:6809/91038858","cluster_addr":"172.21.15.99:6811/91038858","heartbeat_back_addr":"172.21.15.99:6815/91038858","heartbeat_front_addr":"172.21.15.99:6813/91038858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.468213+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.538610+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.500881+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.474179+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.483737+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.460693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.475970+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.519287+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T04:56:13.058 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-18T04:56:13.058 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T04:56:13.346 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:56:13.346 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":17,"fsid":"c0761d0c-69d5-4b64-b8b9-c7d098b2d06d","created":"2024-09-18T04:55:58.216538+0000","modified":"2024-09-18T04:56:12.386465+0000","last_up_change":"2024-09-18T04:56:08.369924+0000","last_in_change":"2024-09-18T04:56:00.333800+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-18T04:56:08.983344+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-18T04:56:10.313988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c43f8024-3a3e-424e-aa54-398310ceb70d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4106704527},{"type":"v1","addr":"172.21.15.2:6825","nonce":4106704527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":4106704527},{"type":"v1","addr":"172.21.15.2:6827","nonce":4106704527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":4106704527},{"type":"v1","addr":"172.21.15.2:6831","nonce":4106704527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":4106704527},{"type":"v1","addr":"172.21.15.2:6829","nonce":4106704527}]},"public_addr":"172.21.15.2:6825/4106704527","cluster_addr":"172.21.15.2:6827/4106704527","heartbeat_back_addr":"172.21.15.2:6831/4106704527","heartbeat_front_addr":"172.21.15.2:6829/4106704527","state":["exists","up"]},{"osd":1,"uuid":"c664b970-145c-4b6f-8852-2f6dcf1fa30d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1781959259},{"type":"v1","addr":"172.21.15.2:6809","nonce":1781959259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1781959259},{"type":"v1","addr":"172.21.15.2:6811","nonce":1781959259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1781959259},{"type":"v1","addr":"172.21.15.2:6815","nonce":1781959259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1781959259},{"type":"v1","addr":"172.21.15.2:6813","nonce":1781959259}]},"public_addr":"172.21.15.2:6809/1781959259","cluster_addr":"172.21.15.2:6811/1781959259","heartbeat_back_addr":"172.21.15.2:6815/1781959259","heartbeat_front_addr":"172.21.15.2:6813/1781959259","state":["exists","up"]},{"osd":2,"uuid":"0609ea8c-d975-490e-8844-04a3e4512efd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":1153570802},{"type":"v1","addr":"172.21.15.2:6801","nonce":1153570802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1153570802},{"type":"v1","addr":"172.21.15.2:6803","nonce":1153570802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1153570802},{"type":"v1","addr":"172.21.15.2:6807","nonce":1153570802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1153570802},{"type":"v1","addr":"172.21.15.2:6805","nonce":1153570802}]},"public_addr":"172.21.15.2:6801/1153570802","cluster_addr":"172.21.15.2:6803/1153570802","heartbeat_back_addr":"172.21.15.2:6807/1153570802","heartbeat_front_addr":"172.21.15.2:6805/1153570802","state":["exists","up"]},{"osd":3,"uuid":"c6bd5ac8-ab89-4fa2-b843-3b166d1b238b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":2008321375},{"type":"v1","addr":"172.21.15.2:6817","nonce":2008321375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2008321375},{"type":"v1","addr":"172.21.15.2:6819","nonce":2008321375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2008321375},{"type":"v1","addr":"172.21.15.2:6823","nonce":2008321375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2008321375},{"type":"v1","addr":"172.21.15.2:6821","nonce":2008321375}]},"public_addr":"172.21.15.2:6817/2008321375","cluster_addr":"172.21.15.2:6819/2008321375","heartbeat_back_addr":"172.21.15.2:6823/2008321375","heartbeat_front_addr":"172.21.15.2:6821/2008321375","state":["exists","up"]},{"osd":4,"uuid":"89d6a371-4f4e-465e-a023-d5083cce6e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2939896598},{"type":"v1","addr":"172.21.15.99:6825","nonce":2939896598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":2939896598},{"type":"v1","addr":"172.21.15.99:6827","nonce":2939896598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":2939896598},{"type":"v1","addr":"172.21.15.99:6831","nonce":2939896598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":2939896598},{"type":"v1","addr":"172.21.15.99:6829","nonce":2939896598}]},"public_addr":"172.21.15.99:6825/2939896598","cluster_addr":"172.21.15.99:6827/2939896598","heartbeat_back_addr":"172.21.15.99:6831/2939896598","heartbeat_front_addr":"172.21.15.99:6829/2939896598","state":["exists","up"]},{"osd":5,"uuid":"da99686b-d979-4753-b4aa-a3b26e9a0497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":3440879618},{"type":"v1","addr":"172.21.15.99:6817","nonce":3440879618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":3440879618},{"type":"v1","addr":"172.21.15.99:6819","nonce":3440879618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":3440879618},{"type":"v1","addr":"172.21.15.99:6823","nonce":3440879618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":3440879618},{"type":"v1","addr":"172.21.15.99:6821","nonce":3440879618}]},"public_addr":"172.21.15.99:6817/3440879618","cluster_addr":"172.21.15.99:6819/3440879618","heartbeat_back_addr":"172.21.15.99:6823/3440879618","heartbeat_front_addr":"172.21.15.99:6821/3440879618","state":["exists","up"]},{"osd":6,"uuid":"ceec6aa6-603d-4706-bfea-3c07dbdaed3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":2493830504},{"type":"v1","addr":"172.21.15.99:6801","nonce":2493830504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":2493830504},{"type":"v1","addr":"172.21.15.99:6803","nonce":2493830504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":2493830504},{"type":"v1","addr":"172.21.15.99:6807","nonce":2493830504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":2493830504},{"type":"v1","addr":"172.21.15.99:6805","nonce":2493830504}]},"public_addr":"172.21.15.99:6801/2493830504","cluster_addr":"172.21.15.99:6803/2493830504","heartbeat_back_addr":"172.21.15.99:6807/2493830504","heartbeat_front_addr":"172.21.15.99:6805/2493830504","state":["exists","up"]},{"osd":7,"uuid":"6779dfad-f60b-41f4-b666-a99db6dc1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":91038858},{"type":"v1","addr":"172.21.15.99:6809","nonce":91038858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":91038858},{"type":"v1","addr":"172.21.15.99:6811","nonce":91038858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":91038858},{"type":"v1","addr":"172.21.15.99:6815","nonce":91038858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":91038858},{"type":"v1","addr":"172.21.15.99:6813","nonce":91038858}]},"public_addr":"172.21.15.99:6809/91038858","cluster_addr":"172.21.15.99:6811/91038858","heartbeat_back_addr":"172.21.15.99:6815/91038858","heartbeat_front_addr":"172.21.15.99:6813/91038858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.468213+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.538610+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.500881+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.474179+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.483737+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.460693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.475970+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.519287+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T04:56:13.359 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-18T04:56:13.360 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-18T04:56:13.360 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-18T04:56:13.360 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-18T04:56:13.361 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-18T04:56:13.361 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-18T04:56:13.362 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-18T04:56:13.362 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-18T04:56:13.598 INFO:teuthology.orchestra.run.smithi002.stdout:55834574852 2024-09-18T04:56:13.598 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T04:56:13.613 INFO:teuthology.orchestra.run.smithi002.stdout:55834574852 2024-09-18T04:56:13.613 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T04:56:13.614 INFO:teuthology.orchestra.run.smithi002.stdout:55834574852 2024-09-18T04:56:13.614 INFO:teuthology.orchestra.run.smithi002.stdout:55834574851 2024-09-18T04:56:13.615 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T04:56:13.615 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T04:56:13.616 INFO:teuthology.orchestra.run.smithi002.stdout:55834574851 2024-09-18T04:56:13.617 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T04:56:13.618 INFO:teuthology.orchestra.run.smithi002.stdout:55834574851 2024-09-18T04:56:13.618 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T04:56:13.619 INFO:teuthology.orchestra.run.smithi002.stdout:55834574851 2024-09-18T04:56:13.620 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T04:56:13.621 INFO:teuthology.orchestra.run.smithi002.stdout:55834574851 2024-09-18T04:56:13.621 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T04:56:14.142 INFO:teuthology.orchestra.run.smithi002.stdout:55834574851 2024-09-18T04:56:14.162 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.7 2024-09-18T04:56:14.174 INFO:teuthology.orchestra.run.smithi002.stdout:55834574851 2024-09-18T04:56:14.192 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.0 2024-09-18T04:56:14.215 INFO:teuthology.orchestra.run.smithi002.stdout:55834574850 2024-09-18T04:56:14.215 INFO:teuthology.orchestra.run.smithi002.stdout:55834574850 2024-09-18T04:56:14.229 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.4 2024-09-18T04:56:14.230 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.3 2024-09-18T04:56:14.241 INFO:teuthology.orchestra.run.smithi002.stdout:55834574851 2024-09-18T04:56:14.252 INFO:teuthology.orchestra.run.smithi002.stdout:55834574850 2024-09-18T04:56:14.260 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.1 2024-09-18T04:56:14.261 INFO:teuthology.orchestra.run.smithi002.stdout:55834574850 2024-09-18T04:56:14.266 INFO:teuthology.orchestra.run.smithi002.stdout:55834574850 2024-09-18T04:56:14.267 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.2 2024-09-18T04:56:14.275 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.5 2024-09-18T04:56:14.280 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.6 2024-09-18T04:56:15.163 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T04:56:15.192 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T04:56:15.230 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T04:56:15.231 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T04:56:15.261 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T04:56:15.269 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T04:56:15.276 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T04:56:15.281 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T04:56:15.682 INFO:teuthology.orchestra.run.smithi002.stdout:55834574852 2024-09-18T04:56:15.701 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.7 2024-09-18T04:56:15.701 DEBUG:teuthology.parallel:result is None 2024-09-18T04:56:15.769 INFO:teuthology.orchestra.run.smithi002.stdout:55834574852 2024-09-18T04:56:15.780 INFO:teuthology.orchestra.run.smithi002.stdout:55834574851 2024-09-18T04:56:15.785 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.0 2024-09-18T04:56:15.786 DEBUG:teuthology.parallel:result is None 2024-09-18T04:56:15.794 INFO:teuthology.orchestra.run.smithi002.stdout:55834574851 2024-09-18T04:56:15.796 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.6 2024-09-18T04:56:15.796 DEBUG:teuthology.parallel:result is None 2024-09-18T04:56:15.807 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.3 2024-09-18T04:56:15.807 DEBUG:teuthology.parallel:result is None 2024-09-18T04:56:15.842 INFO:teuthology.orchestra.run.smithi002.stdout:55834574851 2024-09-18T04:56:15.859 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.4 2024-09-18T04:56:15.860 DEBUG:teuthology.parallel:result is None 2024-09-18T04:56:15.870 INFO:teuthology.orchestra.run.smithi002.stdout:55834574852 2024-09-18T04:56:15.883 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.1 2024-09-18T04:56:15.883 DEBUG:teuthology.parallel:result is None 2024-09-18T04:56:15.889 INFO:teuthology.orchestra.run.smithi002.stdout:55834574851 2024-09-18T04:56:15.904 INFO:teuthology.orchestra.run.smithi002.stdout:55834574851 2024-09-18T04:56:15.906 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.2 2024-09-18T04:56:15.906 DEBUG:teuthology.parallel:result is None 2024-09-18T04:56:15.917 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.5 2024-09-18T04:56:15.917 DEBUG:teuthology.parallel:result is None 2024-09-18T04:56:15.917 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T04:56:15.918 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T04:56:16.237 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:56:16.237 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-18T04:56:16.249 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-09-18T04:56:14.269293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":17,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234048,"kb_used_data":1424,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754740672,"statfs":{"total":773094113280,"available":772854448128,"internally_reserved":0,"allocated":1458176,"data_stored":1605342,"data_compressed":17342,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12742,"internal_metadata":238014010},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.390222+0000","last_change":"2024-09-18T04:56:12.390222+0000","last_active":"2024-09-18T04:56:12.390222+0000","last_peered":"2024-09-18T04:56:12.390222+0000","last_clean":"2024-09-18T04:56:12.390222+0000","last_became_active":"2024-09-18T04:56:10.401272+0000","last_became_peered":"2024-09-18T04:56:10.401272+0000","last_unstale":"2024-09-18T04:56:12.390222+0000","last_undegraded":"2024-09-18T04:56:12.390222+0000","last_fullsized":"2024-09-18T04:56:12.390222+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:33:50.692096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063436799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.389971+0000","last_change":"2024-09-18T04:56:12.389971+0000","last_active":"2024-09-18T04:56:12.389971+0000","last_peered":"2024-09-18T04:56:12.389971+0000","last_clean":"2024-09-18T04:56:12.389971+0000","last_became_active":"2024-09-18T04:56:10.412595+0000","last_became_peered":"2024-09-18T04:56:10.412595+0000","last_unstale":"2024-09-18T04:56:12.389971+0000","last_undegraded":"2024-09-18T04:56:12.389971+0000","last_fullsized":"2024-09-18T04:56:12.389971+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:45.638420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038576799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.412298+0000","last_change":"2024-09-18T04:56:12.412298+0000","last_active":"2024-09-18T04:56:12.412298+0000","last_peered":"2024-09-18T04:56:12.412298+0000","last_clean":"2024-09-18T04:56:12.412298+0000","last_became_active":"2024-09-18T04:56:10.399718+0000","last_became_peered":"2024-09-18T04:56:10.399718+0000","last_unstale":"2024-09-18T04:56:12.412298+0000","last_undegraded":"2024-09-18T04:56:12.412298+0000","last_fullsized":"2024-09-18T04:56:12.412298+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:41:31.956685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024059099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.413007+0000","last_change":"2024-09-18T04:56:12.413007+0000","last_active":"2024-09-18T04:56:12.413007+0000","last_peered":"2024-09-18T04:56:12.413007+0000","last_clean":"2024-09-18T04:56:12.413007+0000","last_became_active":"2024-09-18T04:56:10.399076+0000","last_became_peered":"2024-09-18T04:56:10.399076+0000","last_unstale":"2024-09-18T04:56:12.413007+0000","last_undegraded":"2024-09-18T04:56:12.413007+0000","last_fullsized":"2024-09-18T04:56:12.413007+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:54:29.826665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045144100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"17'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.677049+0000","last_change":"2024-09-18T04:56:11.428051+0000","last_active":"2024-09-18T04:56:12.677049+0000","last_peered":"2024-09-18T04:56:12.677049+0000","last_clean":"2024-09-18T04:56:12.677049+0000","last_became_active":"2024-09-18T04:56:11.427170+0000","last_became_peered":"2024-09-18T04:56:11.427170+0000","last_unstale":"2024-09-18T04:56:12.677049+0000","last_undegraded":"2024-09-18T04:56:12.677049+0000","last_fullsized":"2024-09-18T04:56:12.677049+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_clean_scrub_stamp":"2024-09-18T04:56:10.373785+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:16:08.102627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.413125+0000","last_change":"2024-09-18T04:56:12.413125+0000","last_active":"2024-09-18T04:56:12.413125+0000","last_peered":"2024-09-18T04:56:12.413125+0000","last_clean":"2024-09-18T04:56:12.413125+0000","last_became_active":"2024-09-18T04:56:10.401641+0000","last_became_peered":"2024-09-18T04:56:10.401641+0000","last_unstale":"2024-09-18T04:56:12.413125+0000","last_undegraded":"2024-09-18T04:56:12.413125+0000","last_fullsized":"2024-09-18T04:56:12.413125+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:19:46.375505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000622431,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.399396+0000","last_change":"2024-09-18T04:56:12.398631+0000","last_active":"2024-09-18T04:56:12.399396+0000","last_peered":"2024-09-18T04:56:12.399396+0000","last_clean":"2024-09-18T04:56:12.399396+0000","last_became_active":"2024-09-18T04:56:10.412614+0000","last_became_peered":"2024-09-18T04:56:10.412614+0000","last_unstale":"2024-09-18T04:56:12.399396+0000","last_undegraded":"2024-09-18T04:56:12.399396+0000","last_fullsized":"2024-09-18T04:56:12.399396+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:44:13.401320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075126299999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.390044+0000","last_change":"2024-09-18T04:56:12.390044+0000","last_active":"2024-09-18T04:56:12.390044+0000","last_peered":"2024-09-18T04:56:12.390044+0000","last_clean":"2024-09-18T04:56:12.390044+0000","last_became_active":"2024-09-18T04:56:10.413042+0000","last_became_peered":"2024-09-18T04:56:10.413042+0000","last_unstale":"2024-09-18T04:56:12.390044+0000","last_undegraded":"2024-09-18T04:56:12.390044+0000","last_fullsized":"2024-09-18T04:56:12.390044+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:46:11.074243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.398299+0000","last_change":"2024-09-18T04:56:12.398299+0000","last_active":"2024-09-18T04:56:12.398299+0000","last_peered":"2024-09-18T04:56:12.398299+0000","last_clean":"2024-09-18T04:56:12.398299+0000","last_became_active":"2024-09-18T04:56:10.413140+0000","last_became_peered":"2024-09-18T04:56:10.413140+0000","last_unstale":"2024-09-18T04:56:12.398299+0000","last_undegraded":"2024-09-18T04:56:12.398299+0000","last_fullsized":"2024-09-18T04:56:12.398299+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:37:11.125495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041592799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17398,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":643698,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":163840,"data_stored":52351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2769999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31364,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340476,"statfs":{"total":96636764160,"available":96604647424,"internally_reserved":0,"allocated":225280,"data_stored":643679,"data_compressed":8643,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":52370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T04:56:16.250 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T04:56:16.535 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:56:16.536 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-18T04:56:16.547 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-09-18T04:56:16.269801+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234028,"kb_used_data":1424,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754740692,"statfs":{"total":773094113280,"available":772854468608,"internally_reserved":0,"allocated":1458176,"data_stored":1605342,"data_compressed":17398,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12742,"internal_metadata":238014010},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000508"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.390222+0000","last_change":"2024-09-18T04:56:12.390222+0000","last_active":"2024-09-18T04:56:12.390222+0000","last_peered":"2024-09-18T04:56:12.390222+0000","last_clean":"2024-09-18T04:56:12.390222+0000","last_became_active":"2024-09-18T04:56:10.401272+0000","last_became_peered":"2024-09-18T04:56:10.401272+0000","last_unstale":"2024-09-18T04:56:12.390222+0000","last_undegraded":"2024-09-18T04:56:12.390222+0000","last_fullsized":"2024-09-18T04:56:12.390222+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:33:50.692096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063436799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.389971+0000","last_change":"2024-09-18T04:56:12.389971+0000","last_active":"2024-09-18T04:56:12.389971+0000","last_peered":"2024-09-18T04:56:12.389971+0000","last_clean":"2024-09-18T04:56:12.389971+0000","last_became_active":"2024-09-18T04:56:10.412595+0000","last_became_peered":"2024-09-18T04:56:10.412595+0000","last_unstale":"2024-09-18T04:56:12.389971+0000","last_undegraded":"2024-09-18T04:56:12.389971+0000","last_fullsized":"2024-09-18T04:56:12.389971+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:45.638420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038576799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.412298+0000","last_change":"2024-09-18T04:56:12.412298+0000","last_active":"2024-09-18T04:56:12.412298+0000","last_peered":"2024-09-18T04:56:12.412298+0000","last_clean":"2024-09-18T04:56:12.412298+0000","last_became_active":"2024-09-18T04:56:10.399718+0000","last_became_peered":"2024-09-18T04:56:10.399718+0000","last_unstale":"2024-09-18T04:56:12.412298+0000","last_undegraded":"2024-09-18T04:56:12.412298+0000","last_fullsized":"2024-09-18T04:56:12.412298+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:41:31.956685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024059099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.413007+0000","last_change":"2024-09-18T04:56:12.413007+0000","last_active":"2024-09-18T04:56:12.413007+0000","last_peered":"2024-09-18T04:56:12.413007+0000","last_clean":"2024-09-18T04:56:12.413007+0000","last_became_active":"2024-09-18T04:56:10.399076+0000","last_became_peered":"2024-09-18T04:56:10.399076+0000","last_unstale":"2024-09-18T04:56:12.413007+0000","last_undegraded":"2024-09-18T04:56:12.413007+0000","last_fullsized":"2024-09-18T04:56:12.413007+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:54:29.826665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045144100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"17'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.677049+0000","last_change":"2024-09-18T04:56:11.428051+0000","last_active":"2024-09-18T04:56:12.677049+0000","last_peered":"2024-09-18T04:56:12.677049+0000","last_clean":"2024-09-18T04:56:12.677049+0000","last_became_active":"2024-09-18T04:56:11.427170+0000","last_became_peered":"2024-09-18T04:56:11.427170+0000","last_unstale":"2024-09-18T04:56:12.677049+0000","last_undegraded":"2024-09-18T04:56:12.677049+0000","last_fullsized":"2024-09-18T04:56:12.677049+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_clean_scrub_stamp":"2024-09-18T04:56:10.373785+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:16:08.102627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.413125+0000","last_change":"2024-09-18T04:56:12.413125+0000","last_active":"2024-09-18T04:56:12.413125+0000","last_peered":"2024-09-18T04:56:12.413125+0000","last_clean":"2024-09-18T04:56:12.413125+0000","last_became_active":"2024-09-18T04:56:10.401641+0000","last_became_peered":"2024-09-18T04:56:10.401641+0000","last_unstale":"2024-09-18T04:56:12.413125+0000","last_undegraded":"2024-09-18T04:56:12.413125+0000","last_fullsized":"2024-09-18T04:56:12.413125+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:19:46.375505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000622431,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.399396+0000","last_change":"2024-09-18T04:56:12.398631+0000","last_active":"2024-09-18T04:56:12.399396+0000","last_peered":"2024-09-18T04:56:12.399396+0000","last_clean":"2024-09-18T04:56:12.399396+0000","last_became_active":"2024-09-18T04:56:10.412614+0000","last_became_peered":"2024-09-18T04:56:10.412614+0000","last_unstale":"2024-09-18T04:56:12.399396+0000","last_undegraded":"2024-09-18T04:56:12.399396+0000","last_fullsized":"2024-09-18T04:56:12.399396+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:44:13.401320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075126299999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.390044+0000","last_change":"2024-09-18T04:56:12.390044+0000","last_active":"2024-09-18T04:56:12.390044+0000","last_peered":"2024-09-18T04:56:12.390044+0000","last_clean":"2024-09-18T04:56:12.390044+0000","last_became_active":"2024-09-18T04:56:10.413042+0000","last_became_peered":"2024-09-18T04:56:10.413042+0000","last_unstale":"2024-09-18T04:56:12.390044+0000","last_undegraded":"2024-09-18T04:56:12.390044+0000","last_fullsized":"2024-09-18T04:56:12.390044+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:46:11.074243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.398299+0000","last_change":"2024-09-18T04:56:12.398299+0000","last_active":"2024-09-18T04:56:12.398299+0000","last_peered":"2024-09-18T04:56:12.398299+0000","last_clean":"2024-09-18T04:56:12.398299+0000","last_became_active":"2024-09-18T04:56:10.413140+0000","last_became_peered":"2024-09-18T04:56:10.413140+0000","last_unstale":"2024-09-18T04:56:12.398299+0000","last_undegraded":"2024-09-18T04:56:12.398299+0000","last_fullsized":"2024-09-18T04:56:12.398299+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:37:11.125495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041592799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17398,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":643698,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":163840,"data_stored":52351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":225280,"data_stored":643679,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":52370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T04:56:16.549 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T04:56:16.549 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-18T04:56:16.549 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-18T04:56:16.549 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-18T04:56:16.874 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:56:16.874 INFO:teuthology.orchestra.run.smithi002.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-18T04:56:16.886 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-18T04:56:16.887 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-18T04:56:17.809 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-18T04:56:17.809 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'reef'}, 'ceph': {'conf': {'client': {'rbd cache': False}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from'], 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '19e2a2beb6f41a3ab62650e38614eb29b59dcbb9'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'reef', 'sha1': 'a477b1894d5c5a613f23260ca0e6b96d79652ba7'}} 2024-09-18T04:56:17.809 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-09-18T04:56:17.810 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-18T04:56:18.128 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-18T04:56:18.129 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T04:56:18.129 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T04:56:18.414 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:56:18.414 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-18T04:56:18.425 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-09-18T04:56:18.270183+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234028,"kb_used_data":1424,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754740692,"statfs":{"total":773094113280,"available":772854468608,"internally_reserved":0,"allocated":1458176,"data_stored":1605342,"data_compressed":17398,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12742,"internal_metadata":238014010},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000890"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.390222+0000","last_change":"2024-09-18T04:56:12.390222+0000","last_active":"2024-09-18T04:56:12.390222+0000","last_peered":"2024-09-18T04:56:12.390222+0000","last_clean":"2024-09-18T04:56:12.390222+0000","last_became_active":"2024-09-18T04:56:10.401272+0000","last_became_peered":"2024-09-18T04:56:10.401272+0000","last_unstale":"2024-09-18T04:56:12.390222+0000","last_undegraded":"2024-09-18T04:56:12.390222+0000","last_fullsized":"2024-09-18T04:56:12.390222+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:33:50.692096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063436799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.389971+0000","last_change":"2024-09-18T04:56:12.389971+0000","last_active":"2024-09-18T04:56:12.389971+0000","last_peered":"2024-09-18T04:56:12.389971+0000","last_clean":"2024-09-18T04:56:12.389971+0000","last_became_active":"2024-09-18T04:56:10.412595+0000","last_became_peered":"2024-09-18T04:56:10.412595+0000","last_unstale":"2024-09-18T04:56:12.389971+0000","last_undegraded":"2024-09-18T04:56:12.389971+0000","last_fullsized":"2024-09-18T04:56:12.389971+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:45.638420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038576799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.412298+0000","last_change":"2024-09-18T04:56:12.412298+0000","last_active":"2024-09-18T04:56:12.412298+0000","last_peered":"2024-09-18T04:56:12.412298+0000","last_clean":"2024-09-18T04:56:12.412298+0000","last_became_active":"2024-09-18T04:56:10.399718+0000","last_became_peered":"2024-09-18T04:56:10.399718+0000","last_unstale":"2024-09-18T04:56:12.412298+0000","last_undegraded":"2024-09-18T04:56:12.412298+0000","last_fullsized":"2024-09-18T04:56:12.412298+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:41:31.956685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024059099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.413007+0000","last_change":"2024-09-18T04:56:12.413007+0000","last_active":"2024-09-18T04:56:12.413007+0000","last_peered":"2024-09-18T04:56:12.413007+0000","last_clean":"2024-09-18T04:56:12.413007+0000","last_became_active":"2024-09-18T04:56:10.399076+0000","last_became_peered":"2024-09-18T04:56:10.399076+0000","last_unstale":"2024-09-18T04:56:12.413007+0000","last_undegraded":"2024-09-18T04:56:12.413007+0000","last_fullsized":"2024-09-18T04:56:12.413007+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:54:29.826665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045144100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"17'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.677049+0000","last_change":"2024-09-18T04:56:11.428051+0000","last_active":"2024-09-18T04:56:12.677049+0000","last_peered":"2024-09-18T04:56:12.677049+0000","last_clean":"2024-09-18T04:56:12.677049+0000","last_became_active":"2024-09-18T04:56:11.427170+0000","last_became_peered":"2024-09-18T04:56:11.427170+0000","last_unstale":"2024-09-18T04:56:12.677049+0000","last_undegraded":"2024-09-18T04:56:12.677049+0000","last_fullsized":"2024-09-18T04:56:12.677049+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_clean_scrub_stamp":"2024-09-18T04:56:10.373785+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:16:08.102627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.413125+0000","last_change":"2024-09-18T04:56:12.413125+0000","last_active":"2024-09-18T04:56:12.413125+0000","last_peered":"2024-09-18T04:56:12.413125+0000","last_clean":"2024-09-18T04:56:12.413125+0000","last_became_active":"2024-09-18T04:56:10.401641+0000","last_became_peered":"2024-09-18T04:56:10.401641+0000","last_unstale":"2024-09-18T04:56:12.413125+0000","last_undegraded":"2024-09-18T04:56:12.413125+0000","last_fullsized":"2024-09-18T04:56:12.413125+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:19:46.375505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000622431,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.399396+0000","last_change":"2024-09-18T04:56:12.398631+0000","last_active":"2024-09-18T04:56:12.399396+0000","last_peered":"2024-09-18T04:56:12.399396+0000","last_clean":"2024-09-18T04:56:12.399396+0000","last_became_active":"2024-09-18T04:56:10.412614+0000","last_became_peered":"2024-09-18T04:56:10.412614+0000","last_unstale":"2024-09-18T04:56:12.399396+0000","last_undegraded":"2024-09-18T04:56:12.399396+0000","last_fullsized":"2024-09-18T04:56:12.399396+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:44:13.401320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075126299999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.390044+0000","last_change":"2024-09-18T04:56:12.390044+0000","last_active":"2024-09-18T04:56:12.390044+0000","last_peered":"2024-09-18T04:56:12.390044+0000","last_clean":"2024-09-18T04:56:12.390044+0000","last_became_active":"2024-09-18T04:56:10.413042+0000","last_became_peered":"2024-09-18T04:56:10.413042+0000","last_unstale":"2024-09-18T04:56:12.390044+0000","last_undegraded":"2024-09-18T04:56:12.390044+0000","last_fullsized":"2024-09-18T04:56:12.390044+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:46:11.074243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.398299+0000","last_change":"2024-09-18T04:56:12.398299+0000","last_active":"2024-09-18T04:56:12.398299+0000","last_peered":"2024-09-18T04:56:12.398299+0000","last_clean":"2024-09-18T04:56:12.398299+0000","last_became_active":"2024-09-18T04:56:10.413140+0000","last_became_peered":"2024-09-18T04:56:10.413140+0000","last_unstale":"2024-09-18T04:56:12.398299+0000","last_undegraded":"2024-09-18T04:56:12.398299+0000","last_fullsized":"2024-09-18T04:56:12.398299+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:37:11.125495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041592799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17398,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":643698,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":163840,"data_stored":52351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":225280,"data_stored":643679,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":52370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T04:56:18.427 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T04:56:18.710 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:56:18.711 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-18T04:56:18.722 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-09-18T04:56:18.270183+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":161,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120,"ondisk_log_size":120,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234028,"kb_used_data":1424,"kb_used_omap":12,"kb_used_meta":232435,"kb_avail":754740692,"statfs":{"total":773094113280,"available":772854468608,"internally_reserved":0,"allocated":1458176,"data_stored":1605342,"data_compressed":17398,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":12742,"internal_metadata":238014010},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000890"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.390222+0000","last_change":"2024-09-18T04:56:12.390222+0000","last_active":"2024-09-18T04:56:12.390222+0000","last_peered":"2024-09-18T04:56:12.390222+0000","last_clean":"2024-09-18T04:56:12.390222+0000","last_became_active":"2024-09-18T04:56:10.401272+0000","last_became_peered":"2024-09-18T04:56:10.401272+0000","last_unstale":"2024-09-18T04:56:12.390222+0000","last_undegraded":"2024-09-18T04:56:12.390222+0000","last_fullsized":"2024-09-18T04:56:12.390222+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:33:50.692096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063436799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.389971+0000","last_change":"2024-09-18T04:56:12.389971+0000","last_active":"2024-09-18T04:56:12.389971+0000","last_peered":"2024-09-18T04:56:12.389971+0000","last_clean":"2024-09-18T04:56:12.389971+0000","last_became_active":"2024-09-18T04:56:10.412595+0000","last_became_peered":"2024-09-18T04:56:10.412595+0000","last_unstale":"2024-09-18T04:56:12.389971+0000","last_undegraded":"2024-09-18T04:56:12.389971+0000","last_fullsized":"2024-09-18T04:56:12.389971+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:45.638420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038576799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.412298+0000","last_change":"2024-09-18T04:56:12.412298+0000","last_active":"2024-09-18T04:56:12.412298+0000","last_peered":"2024-09-18T04:56:12.412298+0000","last_clean":"2024-09-18T04:56:12.412298+0000","last_became_active":"2024-09-18T04:56:10.399718+0000","last_became_peered":"2024-09-18T04:56:10.399718+0000","last_unstale":"2024-09-18T04:56:12.412298+0000","last_undegraded":"2024-09-18T04:56:12.412298+0000","last_fullsized":"2024-09-18T04:56:12.412298+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:41:31.956685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024059099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.413007+0000","last_change":"2024-09-18T04:56:12.413007+0000","last_active":"2024-09-18T04:56:12.413007+0000","last_peered":"2024-09-18T04:56:12.413007+0000","last_clean":"2024-09-18T04:56:12.413007+0000","last_became_active":"2024-09-18T04:56:10.399076+0000","last_became_peered":"2024-09-18T04:56:10.399076+0000","last_unstale":"2024-09-18T04:56:12.413007+0000","last_undegraded":"2024-09-18T04:56:12.413007+0000","last_fullsized":"2024-09-18T04:56:12.413007+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:54:29.826665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045144100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"17'117","reported_seq":160,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.677049+0000","last_change":"2024-09-18T04:56:11.428051+0000","last_active":"2024-09-18T04:56:12.677049+0000","last_peered":"2024-09-18T04:56:12.677049+0000","last_clean":"2024-09-18T04:56:12.677049+0000","last_became_active":"2024-09-18T04:56:11.427170+0000","last_became_peered":"2024-09-18T04:56:11.427170+0000","last_unstale":"2024-09-18T04:56:12.677049+0000","last_undegraded":"2024-09-18T04:56:12.677049+0000","last_fullsized":"2024-09-18T04:56:12.677049+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_clean_scrub_stamp":"2024-09-18T04:56:10.373785+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:16:08.102627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.413125+0000","last_change":"2024-09-18T04:56:12.413125+0000","last_active":"2024-09-18T04:56:12.413125+0000","last_peered":"2024-09-18T04:56:12.413125+0000","last_clean":"2024-09-18T04:56:12.413125+0000","last_became_active":"2024-09-18T04:56:10.401641+0000","last_became_peered":"2024-09-18T04:56:10.401641+0000","last_unstale":"2024-09-18T04:56:12.413125+0000","last_undegraded":"2024-09-18T04:56:12.413125+0000","last_fullsized":"2024-09-18T04:56:12.413125+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:19:46.375505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000622431,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.399396+0000","last_change":"2024-09-18T04:56:12.398631+0000","last_active":"2024-09-18T04:56:12.399396+0000","last_peered":"2024-09-18T04:56:12.399396+0000","last_clean":"2024-09-18T04:56:12.399396+0000","last_became_active":"2024-09-18T04:56:10.412614+0000","last_became_peered":"2024-09-18T04:56:10.412614+0000","last_unstale":"2024-09-18T04:56:12.399396+0000","last_undegraded":"2024-09-18T04:56:12.399396+0000","last_fullsized":"2024-09-18T04:56:12.399396+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:44:13.401320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075126299999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.390044+0000","last_change":"2024-09-18T04:56:12.390044+0000","last_active":"2024-09-18T04:56:12.390044+0000","last_peered":"2024-09-18T04:56:12.390044+0000","last_clean":"2024-09-18T04:56:12.390044+0000","last_became_active":"2024-09-18T04:56:10.413042+0000","last_became_peered":"2024-09-18T04:56:10.413042+0000","last_unstale":"2024-09-18T04:56:12.390044+0000","last_undegraded":"2024-09-18T04:56:12.390044+0000","last_fullsized":"2024-09-18T04:56:12.390044+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:46:11.074243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-09-18T04:56:12.398299+0000","last_change":"2024-09-18T04:56:12.398299+0000","last_active":"2024-09-18T04:56:12.398299+0000","last_peered":"2024-09-18T04:56:12.398299+0000","last_clean":"2024-09-18T04:56:12.398299+0000","last_became_active":"2024-09-18T04:56:10.413140+0000","last_became_peered":"2024-09-18T04:56:10.413140+0000","last_unstale":"2024-09-18T04:56:12.398299+0000","last_undegraded":"2024-09-18T04:56:12.398299+0000","last_fullsized":"2024-09-18T04:56:12.398299+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:37:11.125495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041592799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17398,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":643698,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":163840,"data_stored":52351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45600000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27192,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344648,"statfs":{"total":96636764160,"available":96608919552,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.313}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64700000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":53311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31344,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340496,"statfs":{"total":96636764160,"available":96604667904,"internally_reserved":0,"allocated":225280,"data_stored":643679,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98299999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31288,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340552,"statfs":{"total":96636764160,"available":96604725248,"internally_reserved":0,"allocated":167936,"data_stored":52370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T04:56:18.723 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T04:56:18.723 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-18T04:56:19.007 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 18 2024-09-18T04:56:19.008 INFO:teuthology.orchestra.run.smithi002.stdout:fsid c0761d0c-69d5-4b64-b8b9-c7d098b2d06d 2024-09-18T04:56:19.008 INFO:teuthology.orchestra.run.smithi002.stdout:created 2024-09-18T04:55:58.216538+0000 2024-09-18T04:56:19.008 INFO:teuthology.orchestra.run.smithi002.stdout:modified 2024-09-18T04:56:17.994118+0000 2024-09-18T04:56:19.008 INFO:teuthology.orchestra.run.smithi002.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-18T04:56:19.008 INFO:teuthology.orchestra.run.smithi002.stdout:crush_version 4 2024-09-18T04:56:19.009 INFO:teuthology.orchestra.run.smithi002.stdout:full_ratio 0.95 2024-09-18T04:56:19.009 INFO:teuthology.orchestra.run.smithi002.stdout:backfillfull_ratio 0.9 2024-09-18T04:56:19.009 INFO:teuthology.orchestra.run.smithi002.stdout:nearfull_ratio 0.85 2024-09-18T04:56:19.009 INFO:teuthology.orchestra.run.smithi002.stdout:require_min_compat_client luminous 2024-09-18T04:56:19.009 INFO:teuthology.orchestra.run.smithi002.stdout:min_compat_client jewel 2024-09-18T04:56:19.009 INFO:teuthology.orchestra.run.smithi002.stdout:require_osd_release reef 2024-09-18T04:56:19.009 INFO:teuthology.orchestra.run.smithi002.stdout:stretch_mode_enabled false 2024-09-18T04:56:19.010 INFO:teuthology.orchestra.run.smithi002.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 17 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 2.00 2024-09-18T04:56:19.010 INFO:teuthology.orchestra.run.smithi002.stdout:pool 2 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-09-18T04:56:19.010 INFO:teuthology.orchestra.run.smithi002.stdout:max_osd 8 2024-09-18T04:56:19.010 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6824/4106704527,v1:172.21.15.2:6825/4106704527] [v2:172.21.15.2:6826/4106704527,v1:172.21.15.2:6827/4106704527] exists,up c43f8024-3a3e-424e-aa54-398310ceb70d 2024-09-18T04:56:19.010 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6808/1781959259,v1:172.21.15.2:6809/1781959259] [v2:172.21.15.2:6810/1781959259,v1:172.21.15.2:6811/1781959259] exists,up c664b970-145c-4b6f-8852-2f6dcf1fa30d 2024-09-18T04:56:19.010 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6800/1153570802,v1:172.21.15.2:6801/1153570802] [v2:172.21.15.2:6802/1153570802,v1:172.21.15.2:6803/1153570802] exists,up 0609ea8c-d975-490e-8844-04a3e4512efd 2024-09-18T04:56:19.010 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6816/2008321375,v1:172.21.15.2:6817/2008321375] [v2:172.21.15.2:6818/2008321375,v1:172.21.15.2:6819/2008321375] exists,up c6bd5ac8-ab89-4fa2-b843-3b166d1b238b 2024-09-18T04:56:19.011 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6824/2939896598,v1:172.21.15.99:6825/2939896598] [v2:172.21.15.99:6826/2939896598,v1:172.21.15.99:6827/2939896598] exists,up 89d6a371-4f4e-465e-a023-d5083cce6e6c 2024-09-18T04:56:19.011 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6816/3440879618,v1:172.21.15.99:6817/3440879618] [v2:172.21.15.99:6818/3440879618,v1:172.21.15.99:6819/3440879618] exists,up da99686b-d979-4753-b4aa-a3b26e9a0497 2024-09-18T04:56:19.011 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6800/2493830504,v1:172.21.15.99:6801/2493830504] [v2:172.21.15.99:6802/2493830504,v1:172.21.15.99:6803/2493830504] exists,up ceec6aa6-603d-4706-bfea-3c07dbdaed3a 2024-09-18T04:56:19.011 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6808/91038858,v1:172.21.15.99:6809/91038858] [v2:172.21.15.99:6810/91038858,v1:172.21.15.99:6811/91038858] exists,up 6779dfad-f60b-41f4-b666-a99db6dc1814 2024-09-18T04:56:19.021 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6824/4106704527,v1:172.21.15.2:6825/4106704527] [v2:172.21.15.2:6826/4106704527,v1:172.21.15.2:6827/4106704527] exists,up c43f8024-3a3e-424e-aa54-398310ceb70d', 'osd.1 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6808/1781959259,v1:172.21.15.2:6809/1781959259] [v2:172.21.15.2:6810/1781959259,v1:172.21.15.2:6811/1781959259] exists,up c664b970-145c-4b6f-8852-2f6dcf1fa30d', 'osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6800/1153570802,v1:172.21.15.2:6801/1153570802] [v2:172.21.15.2:6802/1153570802,v1:172.21.15.2:6803/1153570802] exists,up 0609ea8c-d975-490e-8844-04a3e4512efd', 'osd.3 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.2:6816/2008321375,v1:172.21.15.2:6817/2008321375] [v2:172.21.15.2:6818/2008321375,v1:172.21.15.2:6819/2008321375] exists,up c6bd5ac8-ab89-4fa2-b843-3b166d1b238b', 'osd.4 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6824/2939896598,v1:172.21.15.99:6825/2939896598] [v2:172.21.15.99:6826/2939896598,v1:172.21.15.99:6827/2939896598] exists,up 89d6a371-4f4e-465e-a023-d5083cce6e6c', 'osd.5 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6816/3440879618,v1:172.21.15.99:6817/3440879618] [v2:172.21.15.99:6818/3440879618,v1:172.21.15.99:6819/3440879618] exists,up da99686b-d979-4753-b4aa-a3b26e9a0497', 'osd.6 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6800/2493830504,v1:172.21.15.99:6801/2493830504] [v2:172.21.15.99:6802/2493830504,v1:172.21.15.99:6803/2493830504] exists,up ceec6aa6-603d-4706-bfea-3c07dbdaed3a', 'osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.99:6808/91038858,v1:172.21.15.99:6809/91038858] [v2:172.21.15.99:6810/91038858,v1:172.21.15.99:6811/91038858] exists,up 6779dfad-f60b-41f4-b666-a99db6dc1814'] 2024-09-18T04:56:19.021 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-09-18T04:56:19.125 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:19.125 INFO:teuthology.orchestra.run.smithi002.stdout: "name": "mon.a", 2024-09-18T04:56:19.125 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster": "ceph", 2024-09-18T04:56:19.126 INFO:teuthology.orchestra.run.smithi002.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-18T04:56:19.126 INFO:teuthology.orchestra.run.smithi002.stdout: "admin_socket_mode": "", 2024-09-18T04:56:19.126 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-09-18T04:56:19.126 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_client_required": "cephx, none", 2024-09-18T04:56:19.126 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_cluster_required": "cephx", 2024-09-18T04:56:19.126 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_debug": "true", 2024-09-18T04:56:19.126 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-18T04:56:19.128 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-18T04:56:19.128 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_service_required": "cephx", 2024-09-18T04:56:19.128 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-18T04:56:19.128 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_supported": "cephx", 2024-09-18T04:56:19.128 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_aio": "true", 2024-09-18T04:56:19.128 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-18T04:56:19.128 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_aio_poll_ms": "250", 2024-09-18T04:56:19.128 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_aio_reap_max": "16", 2024-09-18T04:56:19.128 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_async_discard": "false", 2024-09-18T04:56:19.128 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_block_size": "4096", 2024-09-18T04:56:19.128 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_debug_aio": "false", 2024-09-18T04:56:19.128 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-18T04:56:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-18T04:56:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_debug_inflight_ios": "false", 2024-09-18T04:56:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_enable_discard": "false", 2024-09-18T04:56:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_flock_retry": "3", 2024-09-18T04:56:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-18T04:56:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_inject_crash": "0", 2024-09-18T04:56:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-18T04:56:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_ioring": "false", 2024-09-18T04:56:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_ioring_hipri": "false", 2024-09-18T04:56:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-18T04:56:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-18T04:56:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-18T04:56:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-18T04:56:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: "bdev_type": "", 2024-09-18T04:56:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_alloc_size": "1048576", 2024-09-18T04:56:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_allocator": "hybrid", 2024-09-18T04:56:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_buffered_io": "true", 2024-09-18T04:56:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_check_for_zeros": "false", 2024-09-18T04:56:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-18T04:56:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-18T04:56:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_compact_log_sync": "false", 2024-09-18T04:56:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-18T04:56:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-18T04:56:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-18T04:56:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-18T04:56:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_max_log_runway": "4194304", 2024-09-18T04:56:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_max_prefetch": "1048576", 2024-09-18T04:56:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_min_flush_size": "524288", 2024-09-18T04:56:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_min_log_runway": "1048576", 2024-09-18T04:56:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_replay_recovery": "false", 2024-09-18T04:56:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-18T04:56:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-18T04:56:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: "bluefs_sync_write": "false", 2024-09-18T04:56:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-18T04:56:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-18T04:56:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-18T04:56:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_allocation_from_file": "true", 2024-09-18T04:56:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_allocator": "hybrid", 2024-09-18T04:56:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-18T04:56:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-18T04:56:19.132 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-18T04:56:19.132 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-18T04:56:19.132 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-18T04:56:19.132 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-18T04:56:19.132 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-18T04:56:19.132 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_create": "true", 2024-09-18T04:56:19.132 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_db_create": "false", 2024-09-18T04:56:19.132 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_db_path": "", 2024-09-18T04:56:19.132 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_db_size": "0", 2024-09-18T04:56:19.132 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_path": "", 2024-09-18T04:56:19.132 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_preallocate_file": "false", 2024-09-18T04:56:19.132 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_size": "107374182400", 2024-09-18T04:56:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_wal_create": "false", 2024-09-18T04:56:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_wal_path": "", 2024-09-18T04:56:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_block_wal_size": "100663296", 2024-09-18T04:56:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_bluefs": "true", 2024-09-18T04:56:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-18T04:56:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-18T04:56:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-18T04:56:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-18T04:56:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-18T04:56:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-18T04:56:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-18T04:56:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-18T04:56:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_autotune": "true", 2024-09-18T04:56:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-18T04:56:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-18T04:56:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-18T04:56:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-18T04:56:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_size": "0", 2024-09-18T04:56:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-18T04:56:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-18T04:56:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-18T04:56:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-18T04:56:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cache_type": "2q", 2024-09-18T04:56:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-18T04:56:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_clone_cow": "true", 2024-09-18T04:56:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-18T04:56:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-18T04:56:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-18T04:56:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-18T04:56:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-18T04:56:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-18T04:56:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-18T04:56:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_mode": "none", 2024-09-18T04:56:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-18T04:56:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_csum_type": "crc32c", 2024-09-18T04:56:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-18T04:56:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_freelist": "false", 2024-09-18T04:56:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-18T04:56:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-18T04:56:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_inject_bug21040": "false", 2024-09-18T04:56:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-18T04:56:19.136 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-18T04:56:19.136 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-18T04:56:19.136 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-18T04:56:19.136 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-18T04:56:19.136 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-18T04:56:19.136 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-18T04:56:19.136 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_prefill": "0.000000", 2024-09-18T04:56:19.136 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-18T04:56:19.136 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-18T04:56:19.136 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-18T04:56:19.136 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_small_allocations": "0", 2024-09-18T04:56:19.136 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-18T04:56:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_default_buffered_read": "true", 2024-09-18T04:56:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_default_buffered_write": "false", 2024-09-18T04:56:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-18T04:56:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-18T04:56:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-18T04:56:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-18T04:56:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-18T04:56:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-18T04:56:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-18T04:56:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-18T04:56:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fail_eio": "false", 2024-09-18T04:56:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-18T04:56:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-18T04:56:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-18T04:56:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-18T04:56:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-18T04:56:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-18T04:56:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_on_mount": "false", 2024-09-18T04:56:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-18T04:56:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_on_umount": "false", 2024-09-18T04:56:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-18T04:56:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-18T04:56:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-18T04:56:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-18T04:56:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-18T04:56:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-18T04:56:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-18T04:56:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-18T04:56:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_ignore_data_csum": "false", 2024-09-18T04:56:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-18T04:56:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-18T04:56:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-18T04:56:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-18T04:56:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_log_op_age": "5.000000", 2024-09-18T04:56:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_max_alloc_size": "0", 2024-09-18T04:56:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_max_blob_size": "0", 2024-09-18T04:56:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-18T04:56:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-18T04:56:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-18T04:56:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_max_deferred_txc": "32", 2024-09-18T04:56:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_min_alloc_size": "0", 2024-09-18T04:56:19.140 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-18T04:56:19.140 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-18T04:56:19.140 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_nid_prealloc": "1024", 2024-09-18T04:56:19.140 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-18T04:56:19.140 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-18T04:56:19.140 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-18T04:56:19.140 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-18T04:56:19.140 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_retry_disk_reads": "3", 2024-09-18T04:56:19.140 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_rocksdb_cf": "true", 2024-09-18T04:56:19.140 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-18T04:56:19.140 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-18T04:56:19.140 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-18T04:56:19.141 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-18T04:56:19.141 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-18T04:56:19.141 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-18T04:56:19.141 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_spdk_mem": "512", 2024-09-18T04:56:19.141 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-18T04:56:19.141 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-18T04:56:19.141 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-18T04:56:19.141 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-18T04:56:19.141 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-18T04:56:19.141 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-18T04:56:19.141 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-18T04:56:19.141 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_tracing": "false", 2024-09-18T04:56:19.141 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-18T04:56:19.141 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-18T04:56:19.142 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-18T04:56:19.142 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-18T04:56:19.142 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-18T04:56:19.142 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-18T04:56:19.142 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-18T04:56:19.142 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-18T04:56:19.142 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-18T04:56:19.142 INFO:teuthology.orchestra.run.smithi002.stdout: "bluestore_zero_block_detection": "false", 2024-09-18T04:56:19.142 INFO:teuthology.orchestra.run.smithi002.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-18T04:56:19.142 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-18T04:56:19.142 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-18T04:56:19.142 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-18T04:56:19.142 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-18T04:56:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-18T04:56:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-18T04:56:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-18T04:56:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-18T04:56:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-18T04:56:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-18T04:56:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-18T04:56:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-18T04:56:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-18T04:56:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: "cephx_cluster_require_signatures": "false", 2024-09-18T04:56:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: "cephx_cluster_require_version": "2", 2024-09-18T04:56:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: "cephx_require_signatures": "false", 2024-09-18T04:56:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: "cephx_require_version": "2", 2024-09-18T04:56:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: "cephx_service_require_signatures": "false", 2024-09-18T04:56:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: "cephx_service_require_version": "2", 2024-09-18T04:56:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: "cephx_sign_messages": "true", 2024-09-18T04:56:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: "chdir": "", 2024-09-18T04:56:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: "client_acl_type": "", 2024-09-18T04:56:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: "client_asio_thread_count": "2", 2024-09-18T04:56:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: "client_cache_mid": "0.750000", 2024-09-18T04:56:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: "client_cache_size": "16384", 2024-09-18T04:56:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: "client_caps_release_delay": "5", 2024-09-18T04:56:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: "client_check_pool_perm": "true", 2024-09-18T04:56:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-18T04:56:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: "client_debug_force_sync_read": "false", 2024-09-18T04:56:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: "client_debug_getattr_caps": "false", 2024-09-18T04:56:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: "client_debug_inject_features": "", 2024-09-18T04:56:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: "client_debug_inject_tick_delay": "0", 2024-09-18T04:56:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-18T04:56:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: "client_die_on_failed_remount": "false", 2024-09-18T04:56:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: "client_dirsize_rbytes": "true", 2024-09-18T04:56:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: "client_force_lazyio": "false", 2024-09-18T04:56:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: "client_fs": "", 2024-09-18T04:56:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-18T04:56:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: "client_inject_release_failure": "false", 2024-09-18T04:56:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: "client_max_inline_size": "4096", 2024-09-18T04:56:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-18T04:56:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: "client_mds_namespace": "", 2024-09-18T04:56:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: "client_metadata": "", 2024-09-18T04:56:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: "client_mount_gid": "-1", 2024-09-18T04:56:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client_mount_timeout": "300", 2024-09-18T04:56:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client_mount_uid": "-1", 2024-09-18T04:56:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client_mountpoint": "/", 2024-09-18T04:56:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client_notify_timeout": "10", 2024-09-18T04:56:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client_oc": "true", 2024-09-18T04:56:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client_oc_max_dirty": "104857600", 2024-09-18T04:56:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-18T04:56:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client_oc_max_objects": "1000", 2024-09-18T04:56:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client_oc_size": "209715200", 2024-09-18T04:56:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client_oc_target_dirty": "8388608", 2024-09-18T04:56:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client_permissions": "true", 2024-09-18T04:56:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client_quota": "true", 2024-09-18T04:56:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client_quota_df": "true", 2024-09-18T04:56:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client_readahead_max_bytes": "0", 2024-09-18T04:56:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: "client_readahead_max_periods": "4", 2024-09-18T04:56:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: "client_readahead_min": "131072", 2024-09-18T04:56:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: "client_reconnect_stale": "false", 2024-09-18T04:56:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: "client_shutdown_timeout": "30", 2024-09-18T04:56:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: "client_snapdir": ".snap", 2024-09-18T04:56:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: "client_tick_interval": "1", 2024-09-18T04:56:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: "client_trace": "", 2024-09-18T04:56:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: "client_try_dentry_invalidate": "false", 2024-09-18T04:56:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: "client_use_faked_inos": "false", 2024-09-18T04:56:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: "client_use_random_mds": "false", 2024-09-18T04:56:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: "clog_to_graylog": "false", 2024-09-18T04:56:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-18T04:56:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: "clog_to_graylog_port": "12201", 2024-09-18T04:56:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: "clog_to_monitors": "default=true", 2024-09-18T04:56:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: "clog_to_syslog": "false", 2024-09-18T04:56:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-18T04:56:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: "clog_to_syslog_level": "info", 2024-09-18T04:56:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "-", 2024-09-18T04:56:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_network": "", 2024-09-18T04:56:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_network_interface": "", 2024-09-18T04:56:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: "compressor_zlib_isal": "false", 2024-09-18T04:56:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: "compressor_zlib_level": "5", 2024-09-18T04:56:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: "compressor_zlib_winsize": "-15", 2024-09-18T04:56:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: "compressor_zstd_level": "1", 2024-09-18T04:56:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-09-18T04:56:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-18T04:56:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: "crimson_alien_op_num_threads": "6", 2024-09-18T04:56:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-18T04:56:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: "crimson_osd_obc_lru_size": "10", 2024-09-18T04:56:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-18T04:56:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: "crimson_seastar_smp": "1", 2024-09-18T04:56:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_location": "", 2024-09-18T04:56:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_location_hook": "", 2024-09-18T04:56:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_location_hook_timeout": "10", 2024-09-18T04:56:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: "daemonize": "false", 2024-09-18T04:56:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: "daos_pool": "tank", 2024-09-18T04:56:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-18T04:56:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-18T04:56:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-18T04:56:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_alienstore": "0/5", 2024-09-18T04:56:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_allow_any_pool_priority": "false", 2024-09-18T04:56:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_asok": "1/5", 2024-09-18T04:56:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_asok_assert_abort": "false", 2024-09-18T04:56:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_asserts_on_shutdown": "true", 2024-09-18T04:56:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_auth": "20/20", 2024-09-18T04:56:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_bdev": "1/3", 2024-09-18T04:56:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_bluefs": "1/5", 2024-09-18T04:56:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_bluestore": "1/5", 2024-09-18T04:56:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_buffer": "0/1", 2024-09-18T04:56:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_ceph_exporter": "1/5", 2024-09-18T04:56:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_cephfs_mirror": "0/5", 2024-09-18T04:56:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_cephsqlite": "0/5", 2024-09-18T04:56:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_client": "0/5", 2024-09-18T04:56:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_compressor": "1/5", 2024-09-18T04:56:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_context": "0/1", 2024-09-18T04:56:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_crush": "1/1", 2024-09-18T04:56:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_crypto": "1/5", 2024-09-18T04:56:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_cyanstore": "0/5", 2024-09-18T04:56:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_deliberately_leak_memory": "false", 2024-09-18T04:56:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_disable_randomized_ping": "false", 2024-09-18T04:56:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_dpdk": "1/5", 2024-09-18T04:56:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_eventtrace": "1/5", 2024-09-18T04:56:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_filer": "0/1", 2024-09-18T04:56:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_filestore": "1/3", 2024-09-18T04:56:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_finisher": "1/1", 2024-09-18T04:56:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_fuse": "1/5", 2024-09-18T04:56:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_heartbeat_testing_span": "0", 2024-09-18T04:56:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_heartbeatmap": "1/5", 2024-09-18T04:56:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-18T04:56:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_javaclient": "1/5", 2024-09-18T04:56:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_journal": "1/3", 2024-09-18T04:56:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_journaler": "0/5", 2024-09-18T04:56:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_kstore": "1/5", 2024-09-18T04:56:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_leveldb": "4/5", 2024-09-18T04:56:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_lockdep": "0/1", 2024-09-18T04:56:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mclock": "1/5", 2024-09-18T04:56:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mds": "1/5", 2024-09-18T04:56:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mds_balancer": "1/5", 2024-09-18T04:56:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mds_locker": "1/5", 2024-09-18T04:56:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mds_log": "1/5", 2024-09-18T04:56:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mds_log_expire": "1/5", 2024-09-18T04:56:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mds_migrator": "1/5", 2024-09-18T04:56:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_memstore": "1/5", 2024-09-18T04:56:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mgr": "2/5", 2024-09-18T04:56:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mgrc": "1/5", 2024-09-18T04:56:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_mon": "20/20", 2024-09-18T04:56:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_monc": "0/10", 2024-09-18T04:56:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_ms": "1/1", 2024-09-18T04:56:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_none": "0/5", 2024-09-18T04:56:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_objclass": "0/5", 2024-09-18T04:56:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_objectcacher": "0/5", 2024-09-18T04:56:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_objecter": "0/1", 2024-09-18T04:56:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_optracker": "0/5", 2024-09-18T04:56:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_osd": "1/5", 2024-09-18T04:56:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_paxos": "20/20", 2024-09-18T04:56:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_perfcounter": "1/5", 2024-09-18T04:56:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_prioritycache": "1/5", 2024-09-18T04:56:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rados": "0/5", 2024-09-18T04:56:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rbd": "0/5", 2024-09-18T04:56:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rbd_mirror": "0/5", 2024-09-18T04:56:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rbd_pwl": "0/5", 2024-09-18T04:56:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rbd_replay": "0/5", 2024-09-18T04:56:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_refs": "0/0", 2024-09-18T04:56:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_reserver": "1/1", 2024-09-18T04:56:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rgw": "1/5", 2024-09-18T04:56:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rgw_access": "1/5", 2024-09-18T04:56:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rgw_datacache": "1/5", 2024-09-18T04:56:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rgw_dbstore": "1/5", 2024-09-18T04:56:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rgw_flight": "1/5", 2024-09-18T04:56:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rgw_sync": "1/5", 2024-09-18T04:56:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_rocksdb": "4/5", 2024-09-18T04:56:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore": "0/5", 2024-09-18T04:56:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_backref": "0/5", 2024-09-18T04:56:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_cache": "0/5", 2024-09-18T04:56:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_cleaner": "0/5", 2024-09-18T04:56:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_device": "0/5", 2024-09-18T04:56:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_epm": "0/5", 2024-09-18T04:56:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-18T04:56:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_journal": "0/5", 2024-09-18T04:56:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_lba": "0/5", 2024-09-18T04:56:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_odata": "0/5", 2024-09-18T04:56:19.156 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_omap": "0/5", 2024-09-18T04:56:19.156 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_onode": "0/5", 2024-09-18T04:56:19.156 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_t": "0/5", 2024-09-18T04:56:19.156 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_seastore_tm": "0/5", 2024-09-18T04:56:19.156 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_striper": "0/1", 2024-09-18T04:56:19.156 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_test": "0/5", 2024-09-18T04:56:19.156 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_throttle": "1/1", 2024-09-18T04:56:19.156 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_timer": "0/1", 2024-09-18T04:56:19.156 INFO:teuthology.orchestra.run.smithi002.stdout: "debug_tp": "0/5", 2024-09-18T04:56:19.156 INFO:teuthology.orchestra.run.smithi002.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-18T04:56:19.156 INFO:teuthology.orchestra.run.smithi002.stdout: "device_failure_prediction_mode": "none", 2024-09-18T04:56:19.156 INFO:teuthology.orchestra.run.smithi002.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-18T04:56:19.157 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-18T04:56:19.157 INFO:teuthology.orchestra.run.smithi002.stdout: "err_to_graylog": "false", 2024-09-18T04:56:19.157 INFO:teuthology.orchestra.run.smithi002.stdout: "err_to_journald": "false", 2024-09-18T04:56:19.157 INFO:teuthology.orchestra.run.smithi002.stdout: "err_to_stderr": "true", 2024-09-18T04:56:19.157 INFO:teuthology.orchestra.run.smithi002.stdout: "err_to_syslog": "false", 2024-09-18T04:56:19.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event_tracing": "false", 2024-09-18T04:56:19.157 INFO:teuthology.orchestra.run.smithi002.stdout: "exporter_addr": "0.0.0.0", 2024-09-18T04:56:19.157 INFO:teuthology.orchestra.run.smithi002.stdout: "exporter_http_port": "9926", 2024-09-18T04:56:19.157 INFO:teuthology.orchestra.run.smithi002.stdout: "exporter_prio_limit": "5", 2024-09-18T04:56:19.157 INFO:teuthology.orchestra.run.smithi002.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-18T04:56:19.157 INFO:teuthology.orchestra.run.smithi002.stdout: "exporter_sort_metrics": "true", 2024-09-18T04:56:19.157 INFO:teuthology.orchestra.run.smithi002.stdout: "exporter_stats_period": "5", 2024-09-18T04:56:19.157 INFO:teuthology.orchestra.run.smithi002.stdout: "fake_statfs_for_testing": "0", 2024-09-18T04:56:19.157 INFO:teuthology.orchestra.run.smithi002.stdout: "fatal_signal_handlers": "true", 2024-09-18T04:56:19.158 INFO:teuthology.orchestra.run.smithi002.stdout: "filer_max_purge_ops": "10", 2024-09-18T04:56:19.158 INFO:teuthology.orchestra.run.smithi002.stdout: "filer_max_truncate_ops": "128", 2024-09-18T04:56:19.158 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_apply_finisher_threads": "1", 2024-09-18T04:56:19.158 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_blackhole": "false", 2024-09-18T04:56:19.158 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_btrfs_clone_range": "true", 2024-09-18T04:56:19.158 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_btrfs_snap": "true", 2024-09-18T04:56:19.158 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_caller_concurrency": "10", 2024-09-18T04:56:19.158 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_collect_device_partition_information": "true", 2024-09-18T04:56:19.158 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_commit_timeout": "600.000000", 2024-09-18T04:56:19.158 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_debug_inject_read_err": "false", 2024-09-18T04:56:19.158 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_debug_omap_check": "false", 2024-09-18T04:56:19.158 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-18T04:56:19.158 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_debug_verify_split": "false", 2024-09-18T04:56:19.159 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_dump_file": "", 2024-09-18T04:56:19.159 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-18T04:56:19.159 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-18T04:56:19.159 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_fadvise": "true", 2024-09-18T04:56:19.159 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_fail_eio": "true", 2024-09-18T04:56:19.159 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_fd_cache_shards": "16", 2024-09-18T04:56:19.159 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_fd_cache_size": "128", 2024-09-18T04:56:19.159 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_fiemap": "false", 2024-09-18T04:56:19.159 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_fiemap_threshold": "4096", 2024-09-18T04:56:19.159 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-18T04:56:19.159 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-18T04:56:19.159 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_inject_stall": "0", 2024-09-18T04:56:19.159 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_journal_parallel": "false", 2024-09-18T04:56:19.159 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_journal_trailing": "false", 2024-09-18T04:56:19.160 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_journal_writeahead": "false", 2024-09-18T04:56:19.160 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_kill_at": "0", 2024-09-18T04:56:19.160 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-18T04:56:19.160 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-18T04:56:19.160 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-18T04:56:19.160 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-18T04:56:19.160 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-18T04:56:19.160 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_inline_xattrs": "0", 2024-09-18T04:56:19.160 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-18T04:56:19.160 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-18T04:56:19.160 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-18T04:56:19.160 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-18T04:56:19.160 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_xattr_value_size": "0", 2024-09-18T04:56:19.161 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-18T04:56:19.161 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-18T04:56:19.161 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-18T04:56:19.161 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_merge_threshold": "-10", 2024-09-18T04:56:19.161 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-18T04:56:19.161 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_odsync_write": "false", 2024-09-18T04:56:19.161 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_omap_backend": "rocksdb", 2024-09-18T04:56:19.161 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_omap_backend_path": "", 2024-09-18T04:56:19.161 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-18T04:56:19.161 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-18T04:56:19.161 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-18T04:56:19.161 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_op_thread_timeout": "60", 2024-09-18T04:56:19.161 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_op_threads": "2", 2024-09-18T04:56:19.162 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_punch_hole": "false", 2024-09-18T04:56:19.162 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-18T04:56:19.162 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-18T04:56:19.162 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-18T04:56:19.162 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-18T04:56:19.162 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-18T04:56:19.162 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-18T04:56:19.162 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-18T04:56:19.162 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-18T04:56:19.162 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-18T04:56:19.162 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_queue_max_ops": "50", 2024-09-18T04:56:19.162 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-18T04:56:19.162 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_seek_data_hole": "false", 2024-09-18T04:56:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_sloppy_crc": "false", 2024-09-18T04:56:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-18T04:56:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_splice": "false", 2024-09-18T04:56:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_split_multiple": "2", 2024-09-18T04:56:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_split_rand_factor": "20", 2024-09-18T04:56:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_update_to": "1000", 2024-09-18T04:56:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-18T04:56:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-18T04:56:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-18T04:56:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-18T04:56:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-18T04:56:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-18T04:56:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_enable": "true", 2024-09-18T04:56:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-18T04:56:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-18T04:56:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-18T04:56:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-18T04:56:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-18T04:56:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-18T04:56:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_xfs_extsize": "false", 2024-09-18T04:56:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: "filestore_zfs_snap": "false", 2024-09-18T04:56:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: "fio_dir": "/tmp/fio", 2024-09-18T04:56:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: "fsid": "c0761d0c-69d5-4b64-b8b9-c7d098b2d06d", 2024-09-18T04:56:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_allow_other": "true", 2024-09-18T04:56:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_atomic_o_trunc": "true", 2024-09-18T04:56:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_big_writes": "true", 2024-09-18T04:56:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_debug": "false", 2024-09-18T04:56:19.165 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_default_permissions": "false", 2024-09-18T04:56:19.165 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_disable_pagecache": "false", 2024-09-18T04:56:19.165 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_max_write": "0", 2024-09-18T04:56:19.165 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_multithreaded": "true", 2024-09-18T04:56:19.165 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_require_active_mds": "true", 2024-09-18T04:56:19.165 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_set_user_groups": "true", 2024-09-18T04:56:19.165 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_splice_move": "true", 2024-09-18T04:56:19.165 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_splice_read": "true", 2024-09-18T04:56:19.165 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_splice_write": "true", 2024-09-18T04:56:19.165 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-18T04:56:19.165 INFO:teuthology.orchestra.run.smithi002.stdout: "fuse_use_invalidate_cb": "true", 2024-09-18T04:56:19.165 INFO:teuthology.orchestra.run.smithi002.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-18T04:56:19.165 INFO:teuthology.orchestra.run.smithi002.stdout: "gss_target_name": "ceph", 2024-09-18T04:56:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_file": "", 2024-09-18T04:56:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_inject_failure": "0", 2024-09-18T04:56:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_interval": "5", 2024-09-18T04:56:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: "host": "smithi002", 2024-09-18T04:56:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-18T04:56:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-18T04:56:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-18T04:56:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-18T04:56:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-18T04:56:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-18T04:56:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-18T04:56:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-18T04:56:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-18T04:56:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-18T04:56:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-18T04:56:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-18T04:56:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-18T04:56:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: "inject_early_sigterm": "false", 2024-09-18T04:56:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: "jaeger_agent_port": "6799", 2024-09-18T04:56:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: "jaeger_tracing_enable": "false", 2024-09-18T04:56:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_aio": "true", 2024-09-18T04:56:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_align_min_size": "65536", 2024-09-18T04:56:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_block_align": "true", 2024-09-18T04:56:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_block_size": "4096", 2024-09-18T04:56:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_dio": "true", 2024-09-18T04:56:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_discard": "false", 2024-09-18T04:56:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_force_aio": "false", 2024-09-18T04:56:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_ignore_corruption": "false", 2024-09-18T04:56:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_max_write_bytes": "10485760", 2024-09-18T04:56:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_max_write_entries": "100", 2024-09-18T04:56:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_replay_from": "0", 2024-09-18T04:56:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-18T04:56:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-18T04:56:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-18T04:56:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-18T04:56:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_write_header_frequency": "0", 2024-09-18T04:56:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: "journal_zero_on_create": "false", 2024-09-18T04:56:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: "journaler_prefetch_periods": "10", 2024-09-18T04:56:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: "journaler_prezero_periods": "5", 2024-09-18T04:56:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: "journaler_write_head_interval": "15", 2024-09-18T04:56:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "", 2024-09-18T04:56:19.169 INFO:teuthology.orchestra.run.smithi002.stdout: "keyfile": "", 2024-09-18T04:56:19.169 INFO:teuthology.orchestra.run.smithi002.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-18T04:56:19.169 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_backend": "rocksdb", 2024-09-18T04:56:19.169 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_default_stripe_size": "65536", 2024-09-18T04:56:19.169 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_fsck_on_mount": "false", 2024-09-18T04:56:19.169 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-18T04:56:19.169 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_max_bytes": "67108864", 2024-09-18T04:56:19.169 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_max_ops": "512", 2024-09-18T04:56:19.169 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_nid_prealloc": "1024", 2024-09-18T04:56:19.169 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_onode_map_size": "1024", 2024-09-18T04:56:19.169 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-18T04:56:19.170 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_sync_submit_transaction": "false", 2024-09-18T04:56:19.170 INFO:teuthology.orchestra.run.smithi002.stdout: "kstore_sync_transaction": "false", 2024-09-18T04:56:19.170 INFO:teuthology.orchestra.run.smithi002.stdout: "leveldb_block_size": "65536", 2024-09-18T04:56:19.170 INFO:teuthology.orchestra.run.smithi002.stdout: "leveldb_bloom_size": "0", 2024-09-18T04:56:19.170 INFO:teuthology.orchestra.run.smithi002.stdout: "leveldb_cache_size": "536870912", 2024-09-18T04:56:19.170 INFO:teuthology.orchestra.run.smithi002.stdout: "leveldb_compact_on_mount": "false", 2024-09-18T04:56:19.170 INFO:teuthology.orchestra.run.smithi002.stdout: "leveldb_compression": "false", 2024-09-18T04:56:19.170 INFO:teuthology.orchestra.run.smithi002.stdout: "leveldb_log": "", 2024-09-18T04:56:19.170 INFO:teuthology.orchestra.run.smithi002.stdout: "leveldb_log_to_ceph_log": "true", 2024-09-18T04:56:19.170 INFO:teuthology.orchestra.run.smithi002.stdout: "leveldb_max_open_files": "0", 2024-09-18T04:56:19.170 INFO:teuthology.orchestra.run.smithi002.stdout: "leveldb_paranoid": "false", 2024-09-18T04:56:19.170 INFO:teuthology.orchestra.run.smithi002.stdout: "leveldb_write_buffer_size": "33554432", 2024-09-18T04:56:19.171 INFO:teuthology.orchestra.run.smithi002.stdout: "librados_thread_count": "2", 2024-09-18T04:56:19.171 INFO:teuthology.orchestra.run.smithi002.stdout: "lockdep": "false", 2024-09-18T04:56:19.171 INFO:teuthology.orchestra.run.smithi002.stdout: "lockdep_force_backtrace": "false", 2024-09-18T04:56:19.172 INFO:teuthology.orchestra.run.smithi002.stdout: "log_coarse_timestamps": "true", 2024-09-18T04:56:19.172 INFO:teuthology.orchestra.run.smithi002.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-18T04:56:19.172 INFO:teuthology.orchestra.run.smithi002.stdout: "log_flush_on_exit": "false", 2024-09-18T04:56:19.172 INFO:teuthology.orchestra.run.smithi002.stdout: "log_graylog_host": "127.0.0.1", 2024-09-18T04:56:19.172 INFO:teuthology.orchestra.run.smithi002.stdout: "log_graylog_port": "12201", 2024-09-18T04:56:19.172 INFO:teuthology.orchestra.run.smithi002.stdout: "log_max_new": "1000", 2024-09-18T04:56:19.172 INFO:teuthology.orchestra.run.smithi002.stdout: "log_max_recent": "10000", 2024-09-18T04:56:19.172 INFO:teuthology.orchestra.run.smithi002.stdout: "log_stderr_prefix": "", 2024-09-18T04:56:19.172 INFO:teuthology.orchestra.run.smithi002.stdout: "log_stop_at_utilization": "0.970000", 2024-09-18T04:56:19.172 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_file": "true", 2024-09-18T04:56:19.172 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_graylog": "false", 2024-09-18T04:56:19.172 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_journald": "false", 2024-09-18T04:56:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_stderr": "false", 2024-09-18T04:56:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: "log_to_syslog": "false", 2024-09-18T04:56:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: "max_rotating_auth_attempts": "10", 2024-09-18T04:56:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-18T04:56:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_action_on_write_error": "1", 2024-09-18T04:56:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_alternate_name_max": "8192", 2024-09-18T04:56:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_asio_thread_count": "2", 2024-09-18T04:56:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_export_pin": "true", 2024-09-18T04:56:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_fragment_dirs": "true", 2024-09-18T04:56:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-18T04:56:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_fragment_interval": "5", 2024-09-18T04:56:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-18T04:56:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-18T04:56:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_interval": "10", 2024-09-18T04:56:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_max": "-1", 2024-09-18T04:56:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_max_until": "-1", 2024-09-18T04:56:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_merge_size": "50", 2024-09-18T04:56:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_midchunk": "0.300000", 2024-09-18T04:56:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-18T04:56:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_min_start": "0.200000", 2024-09-18T04:56:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_minchunk": "0.001000", 2024-09-18T04:56:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_mode": "0", 2024-09-18T04:56:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_need_max": "1.200000", 2024-09-18T04:56:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_need_min": "0.800000", 2024-09-18T04:56:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-18T04:56:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-18T04:56:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_split_bits": "3", 2024-09-18T04:56:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-18T04:56:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_split_size": "10000", 2024-09-18T04:56:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-18T04:56:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_target_decay": "10.000000", 2024-09-18T04:56:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-18T04:56:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_beacon_grace": "15.000000", 2024-09-18T04:56:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_beacon_interval": "4.000000", 2024-09-18T04:56:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-18T04:56:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-18T04:56:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_mid": "0.700000", 2024-09-18T04:56:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_release_free_interval": "10", 2024-09-18T04:56:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_reservation": "0.050000", 2024-09-18T04:56:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-18T04:56:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_trim_interval": "1", 2024-09-18T04:56:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cache_trim_threshold": "262144", 2024-09-18T04:56:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-18T04:56:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-18T04:56:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-18T04:56:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_client_prealloc_inos": "1000", 2024-09-18T04:56:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-18T04:56:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_connect_bootstrapping": "false", 2024-09-18T04:56:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_damage_table_max_entries": "10000", 2024-09-18T04:56:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-18T04:56:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_debug_auth_pins": "false", 2024-09-18T04:56:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_debug_frag": "false", 2024-09-18T04:56:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_debug_scatterstat": "false", 2024-09-18T04:56:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_debug_subtrees": "false", 2024-09-18T04:56:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_decay_halflife": "5.000000", 2024-09-18T04:56:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_default_dir_hash": "2", 2024-09-18T04:56:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_defer_session_stale": "true", 2024-09-18T04:56:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_deny_all_reconnect": "false", 2024-09-18T04:56:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dir_keys_per_op": "16384", 2024-09-18T04:56:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dir_max_commit_size": "10", 2024-09-18T04:56:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dir_max_entries": "0", 2024-09-18T04:56:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dir_prefetch": "true", 2024-09-18T04:56:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-18T04:56:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-18T04:56:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dump_cache_on_map": "false", 2024-09-18T04:56:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-18T04:56:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-18T04:56:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_early_reply": "true", 2024-09-18T04:56:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_enable_op_tracker": "true", 2024-09-18T04:56:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_enforce_unique_name": "true", 2024-09-18T04:56:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-18T04:56:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-18T04:56:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_export_ephemeral_random": "true", 2024-09-18T04:56:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-18T04:56:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-18T04:56:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-18T04:56:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-18T04:56:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-18T04:56:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-18T04:56:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-18T04:56:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-18T04:56:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_health_summarize_threshold": "10", 2024-09-18T04:56:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-18T04:56:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-18T04:56:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_inject_health_dummy": "false", 2024-09-18T04:56:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-18T04:56:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_inject_migrator_session_race": "false", 2024-09-18T04:56:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-18T04:56:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-18T04:56:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-18T04:56:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_join_fs": "", 2024-09-18T04:56:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_journal_format": "1", 2024-09-18T04:56:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-18T04:56:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_create_at": "0", 2024-09-18T04:56:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_export_at": "0", 2024-09-18T04:56:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_import_at": "0", 2024-09-18T04:56:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_journal_at": "0", 2024-09-18T04:56:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_journal_expire_at": "0", 2024-09-18T04:56:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_journal_replay_at": "0", 2024-09-18T04:56:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_link_at": "0", 2024-09-18T04:56:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_mdstable_at": "0", 2024-09-18T04:56:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_openc_at": "0", 2024-09-18T04:56:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_kill_rename_at": "0", 2024-09-18T04:56:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_events_per_segment": "1024", 2024-09-18T04:56:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_max_events": "-1", 2024-09-18T04:56:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_max_segments": "128", 2024-09-18T04:56:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_pause": "false", 2024-09-18T04:56:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_segment_size": "0", 2024-09-18T04:56:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-18T04:56:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_log_warn_factor": "2.000000", 2024-09-18T04:56:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_caps_per_client": "1048576", 2024-09-18T04:56:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_completed_flushes": "100000", 2024-09-18T04:56:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_completed_requests": "100000", 2024-09-18T04:56:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_export_size": "20971520", 2024-09-18T04:56:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_file_recover": "32", 2024-09-18T04:56:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_purge_files": "64", 2024-09-18T04:56:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_purge_ops": "8192", 2024-09-18T04:56:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-18T04:56:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-18T04:56:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_max_snaps_per_dir": "100", 2024-09-18T04:56:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_metrics_update_interval": "2", 2024-09-18T04:56:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_min_caps_per_client": "100", 2024-09-18T04:56:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_min_caps_working_set": "10000", 2024-09-18T04:56:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-18T04:56:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_numa_node": "-1", 2024-09-18T04:56:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-18T04:56:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_op_complaint_time": "30.000000", 2024-09-18T04:56:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_op_history_duration": "600", 2024-09-18T04:56:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_op_history_size": "20", 2024-09-18T04:56:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_op_history_slow_op_size": "20", 2024-09-18T04:56:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-18T04:56:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_op_log_threshold": "5", 2024-09-18T04:56:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_ping_grace": "15", 2024-09-18T04:56:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_ping_interval": "5", 2024-09-18T04:56:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-18T04:56:19.183 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-18T04:56:19.183 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_recall_max_caps": "30000", 2024-09-18T04:56:19.183 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-18T04:56:19.183 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-18T04:56:19.183 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-18T04:56:19.183 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_recall_warning_threshold": "262144", 2024-09-18T04:56:19.183 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-18T04:56:19.183 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_replay_interval": "1.000000", 2024-09-18T04:56:19.183 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-18T04:56:19.183 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-18T04:56:19.183 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_root_ino_gid": "0", 2024-09-18T04:56:19.183 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_root_ino_uid": "0", 2024-09-18T04:56:19.183 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-18T04:56:19.184 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-18T04:56:19.184 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-18T04:56:19.184 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-18T04:56:19.184 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-18T04:56:19.184 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-18T04:56:19.184 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-18T04:56:19.184 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-18T04:56:19.184 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-18T04:56:19.184 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-18T04:56:19.184 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_shutdown_check": "0", 2024-09-18T04:56:19.184 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_skip_ino": "0", 2024-09-18T04:56:19.184 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-18T04:56:19.184 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_snap_max_uid": "4294967294", 2024-09-18T04:56:19.184 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_snap_min_uid": "0", 2024-09-18T04:56:19.185 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_snap_rstat": "false", 2024-09-18T04:56:19.185 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_standby_replay_damaged": "false", 2024-09-18T04:56:19.185 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_symlink_recovery": "true", 2024-09-18T04:56:19.185 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-18T04:56:19.185 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_thrash_exports": "0", 2024-09-18T04:56:19.185 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_thrash_fragments": "0", 2024-09-18T04:56:19.185 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_tick_interval": "5.000000", 2024-09-18T04:56:19.185 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_valgrind_exit": "false", 2024-09-18T04:56:19.185 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_verify_backtrace": "1", 2024-09-18T04:56:19.185 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_verify_scatter": "false", 2024-09-18T04:56:19.185 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-18T04:56:19.185 INFO:teuthology.orchestra.run.smithi002.stdout: "mds_wipe_sessions": "false", 2024-09-18T04:56:19.185 INFO:teuthology.orchestra.run.smithi002.stdout: "mempool_debug": "false", 2024-09-18T04:56:19.186 INFO:teuthology.orchestra.run.smithi002.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-18T04:56:19.186 INFO:teuthology.orchestra.run.smithi002.stdout: "memstore_device_bytes": "1073741824", 2024-09-18T04:56:19.186 INFO:teuthology.orchestra.run.smithi002.stdout: "memstore_page_set": "false", 2024-09-18T04:56:19.186 INFO:teuthology.orchestra.run.smithi002.stdout: "memstore_page_size": "65536", 2024-09-18T04:56:19.186 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_client_bytes": "134217728", 2024-09-18T04:56:19.186 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_client_messages": "512", 2024-09-18T04:56:19.186 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-18T04:56:19.186 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-18T04:56:19.186 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-18T04:56:19.186 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-18T04:56:19.186 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_disabled_modules": "", 2024-09-18T04:56:19.186 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-18T04:56:19.186 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_max_pg_num_change": "128", 2024-09-18T04:56:19.186 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_mds_bytes": "134217728", 2024-09-18T04:56:19.187 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_mds_messages": "128", 2024-09-18T04:56:19.187 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-18T04:56:19.187 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_mon_bytes": "134217728", 2024-09-18T04:56:19.187 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_mon_messages": "128", 2024-09-18T04:56:19.187 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_osd_bytes": "536870912", 2024-09-18T04:56:19.187 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_osd_messages": "8192", 2024-09-18T04:56:19.187 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_pool": "true", 2024-09-18T04:56:19.187 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-18T04:56:19.187 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_standby_modules": "true", 2024-09-18T04:56:19.187 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_stats_period": "5", 2024-09-18T04:56:19.187 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_stats_threshold": "5", 2024-09-18T04:56:19.187 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_tick_period": "2", 2024-09-18T04:56:19.187 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-18T04:56:19.188 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-18T04:56:19.188 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_allow_pool_delete": "true", 2024-09-18T04:56:19.188 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_allow_pool_size_one": "true", 2024-09-18T04:56:19.188 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_auth_validate_all_caps": "true", 2024-09-18T04:56:19.188 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-18T04:56:19.188 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-18T04:56:19.188 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_bytes": "104857600", 2024-09-18T04:56:19.188 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_directed_command_retry": "5", 2024-09-18T04:56:19.188 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-18T04:56:19.188 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-18T04:56:19.188 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-18T04:56:19.188 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-18T04:56:19.188 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_hunt_parallel": "3", 2024-09-18T04:56:19.189 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_log_interval": "1.000000", 2024-09-18T04:56:19.189 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-18T04:56:19.189 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_ping_interval": "10.000000", 2024-09-18T04:56:19.189 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-18T04:56:19.189 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-18T04:56:19.189 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-18T04:56:19.189 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-18T04:56:19.189 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_file_level": "debug", 2024-09-18T04:56:19.189 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_file": "true", 2024-09-18T04:56:19.189 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-18T04:56:19.190 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-18T04:56:19.190 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-18T04:56:19.190 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_journald": "false", 2024-09-18T04:56:19.190 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-18T04:56:19.190 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-18T04:56:19.190 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-18T04:56:19.190 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-18T04:56:19.190 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_compact_on_bootstrap": "false", 2024-09-18T04:56:19.190 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_compact_on_start": "false", 2024-09-18T04:56:19.190 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_compact_on_trim": "true", 2024-09-18T04:56:19.190 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-18T04:56:19.190 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-18T04:56:19.190 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-18T04:56:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_cpu_threads": "4", 2024-09-18T04:56:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_crush_min_required_version": "hammer", 2024-09-18T04:56:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_daemon_bytes": "419430400", 2024-09-18T04:56:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-18T04:56:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_data_avail_crit": "5", 2024-09-18T04:56:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_data_avail_warn": "5", 2024-09-18T04:56:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_data_size_warn": "16106127360", 2024-09-18T04:56:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-18T04:56:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-18T04:56:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_dump_json": "false", 2024-09-18T04:56:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-18T04:56:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_dump_transactions": "false", 2024-09-18T04:56:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_extra_checks": "false", 2024-09-18T04:56:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-18T04:56:19.192 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-18T04:56:19.192 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_no_require_quincy": "false", 2024-09-18T04:56:19.192 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_no_require_reef": "false", 2024-09-18T04:56:19.192 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-18T04:56:19.192 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-18T04:56:19.192 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-18T04:56:19.192 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_down_mkfs_grace": "120", 2024-09-18T04:56:19.192 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_election_default_strategy": "1", 2024-09-18T04:56:19.192 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_election_timeout": "5.000000", 2024-09-18T04:56:19.192 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-18T04:56:19.192 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_elector_ping_divisor": "2", 2024-09-18T04:56:19.192 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-18T04:56:19.192 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_enable_op_tracker": "true", 2024-09-18T04:56:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_fake_pool_delete": "false", 2024-09-18T04:56:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_force_quorum_join": "false", 2024-09-18T04:56:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_globalid_prealloc": "10000", 2024-09-18T04:56:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_health_detail_to_clog": "false", 2024-09-18T04:56:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_health_log_update_period": "5", 2024-09-18T04:56:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_health_max_detail": "50", 2024-09-18T04:56:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_health_to_clog": "true", 2024-09-18T04:56:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_health_to_clog_interval": "600", 2024-09-18T04:56:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-18T04:56:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_host": "172.21.15.2,[v2:172.21.15.2:3301,v1:172.21.15.2:6790],172.21.15.99", 2024-09-18T04:56:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_host_override": "", 2024-09-18T04:56:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_initial_members": "", 2024-09-18T04:56:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-18T04:56:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-18T04:56:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-18T04:56:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-18T04:56:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-18T04:56:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_lease": "5.000000", 2024-09-18T04:56:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-18T04:56:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-18T04:56:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_log_full_interval": "50", 2024-09-18T04:56:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_log_max": "10000", 2024-09-18T04:56:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_log_max_summary": "50", 2024-09-18T04:56:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-18T04:56:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_max_log_epochs": "500", 2024-09-18T04:56:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-18T04:56:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-18T04:56:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_max_osd": "10000", 2024-09-18T04:56:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_max_pg_per_osd": "10000", 2024-09-18T04:56:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_max_pool_pg_num": "65536", 2024-09-18T04:56:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-18T04:56:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-18T04:56:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mds_force_trim_to": "0", 2024-09-18T04:56:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mds_skip_sanity": "false", 2024-09-18T04:56:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_memory_autotune": "true", 2024-09-18T04:56:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_memory_target": "2147483648", 2024-09-18T04:56:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mgr_beacon_grace": "30", 2024-09-18T04:56:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-18T04:56:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mgr_digest_period": "5", 2024-09-18T04:56:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mgr_inactive_grace": "60", 2024-09-18T04:56:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-18T04:56:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-18T04:56:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_min_osdmap_epochs": "500", 2024-09-18T04:56:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_op_complaint_time": "30", 2024-09-18T04:56:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_op_history_duration": "600", 2024-09-18T04:56:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_op_history_size": "20", 2024-09-18T04:56:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_op_history_slow_op_size": "20", 2024-09-18T04:56:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-18T04:56:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_op_log_threshold": "5", 2024-09-18T04:56:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-18T04:56:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-18T04:56:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-18T04:56:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_auto_mark_in": "false", 2024-09-18T04:56:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-18T04:56:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-18T04:56:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-18T04:56:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_cache_size": "500", 2024-09-18T04:56:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-18T04:56:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-18T04:56:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-18T04:56:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_down_out_interval": "600", 2024-09-18T04:56:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-18T04:56:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-18T04:56:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_force_trim_to": "0", 2024-09-18T04:56:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-18T04:56:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-18T04:56:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-18T04:56:19.198 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-18T04:56:19.198 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-18T04:56:19.198 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-18T04:56:19.198 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-18T04:56:19.198 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-18T04:56:19.198 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_min_down_reporters": "2", 2024-09-18T04:56:19.198 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-18T04:56:19.198 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-18T04:56:19.199 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-18T04:56:19.199 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-18T04:56:19.199 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-18T04:56:19.199 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-18T04:56:19.199 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_report_timeout": "900", 2024-09-18T04:56:19.199 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-18T04:56:19.199 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-18T04:56:19.199 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-18T04:56:19.199 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-18T04:56:19.199 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-18T04:56:19.199 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osdmap_full_prune_interval": "10", 2024-09-18T04:56:19.199 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osdmap_full_prune_min": "10000", 2024-09-18T04:56:19.200 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-09-18T04:56:19.200 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-18T04:56:19.200 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_pg_stuck_threshold": "60", 2024-09-18T04:56:19.200 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-18T04:56:19.200 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-18T04:56:19.200 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-18T04:56:19.200 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-18T04:56:19.200 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-18T04:56:19.200 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-18T04:56:19.200 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_probe_timeout": "2.000000", 2024-09-18T04:56:19.200 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_reweight_max_change": "0.050000", 2024-09-18T04:56:19.200 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_reweight_max_osds": "4", 2024-09-18T04:56:19.200 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-18T04:56:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-18T04:56:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-18T04:56:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-18T04:56:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-18T04:56:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_scrub_interval": "86400", 2024-09-18T04:56:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_scrub_max_keys": "100", 2024-09-18T04:56:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_scrub_timeout": "300", 2024-09-18T04:56:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_session_timeout": "300", 2024-09-18T04:56:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_smart_report_timeout": "5", 2024-09-18T04:56:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_stat_smooth_intervals": "6", 2024-09-18T04:56:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-18T04:56:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_stretch_pool_min_size": "2", 2024-09-18T04:56:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_stretch_pool_size": "4", 2024-09-18T04:56:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-18T04:56:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-18T04:56:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_sync_debug": "false", 2024-09-18T04:56:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-18T04:56:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-18T04:56:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_sync_provider_kill_at": "0", 2024-09-18T04:56:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_sync_requester_kill_at": "0", 2024-09-18T04:56:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_sync_timeout": "60.000000", 2024-09-18T04:56:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_target_pg_per_osd": "100", 2024-09-18T04:56:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_tick_interval": "5", 2024-09-18T04:56:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_timecheck_interval": "300.000000", 2024-09-18T04:56:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-18T04:56:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_use_min_delay_socket": "false", 2024-09-18T04:56:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_older_version_delay": "604800", 2024-09-18T04:56:19.203 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-18T04:56:19.203 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-18T04:56:19.203 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-18T04:56:19.203 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_filestore_osds": "false", 2024-09-18T04:56:19.203 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-18T04:56:19.203 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-18T04:56:19.203 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-18T04:56:19.203 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_misplaced": "false", 2024-09-18T04:56:19.203 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-18T04:56:19.203 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_older_version": "true", 2024-09-18T04:56:19.203 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-18T04:56:19.203 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_pool_no_app": "false", 2024-09-18T04:56:19.203 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-09-18T04:56:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-18T04:56:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-18T04:56:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-18T04:56:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-18T04:56:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-18T04:56:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-18T04:56:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: "monmap": "", 2024-09-18T04:56:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-18T04:56:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-18T04:56:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-18T04:56:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-18T04:56:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-18T04:56:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-18T04:56:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: "motr_tracing_enabled": "false", 2024-09-18T04:56:19.205 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_op_threads": "3", 2024-09-18T04:56:19.205 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-18T04:56:19.205 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_cm": "false", 2024-09-18T04:56:19.205 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_device_name": "", 2024-09-18T04:56:19.205 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_dscp": "96", 2024-09-18T04:56:19.205 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-18T04:56:19.205 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-18T04:56:19.205 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_local_gid": "", 2024-09-18T04:56:19.205 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-18T04:56:19.205 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_port_num": "1", 2024-09-18T04:56:19.205 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-18T04:56:19.205 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-18T04:56:19.205 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-18T04:56:19.206 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-18T04:56:19.206 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_sl": "3", 2024-09-18T04:56:19.206 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_support_srq": "true", 2024-09-18T04:56:19.206 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_rdma_type": "ib", 2024-09-18T04:56:19.206 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_async_reap_threshold": "5", 2024-09-18T04:56:19.206 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_before_connect": "false", 2024-09-18T04:56:19.206 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_ipv4": "true", 2024-09-18T04:56:19.206 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_ipv6": "false", 2024-09-18T04:56:19.206 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_msgr1": "true", 2024-09-18T04:56:19.206 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_msgr2": "true", 2024-09-18T04:56:19.206 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_port_max": "7568", 2024-09-18T04:56:19.206 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_port_min": "6800", 2024-09-18T04:56:19.206 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-18T04:56:19.206 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_retry_count": "3", 2024-09-18T04:56:19.207 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_bind_retry_delay": "5", 2024-09-18T04:56:19.207 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_blackhole_client": "false", 2024-09-18T04:56:19.207 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_blackhole_mds": "false", 2024-09-18T04:56:19.207 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_blackhole_mgr": "false", 2024-09-18T04:56:19.207 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_blackhole_mon": "false", 2024-09-18T04:56:19.207 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_blackhole_osd": "false", 2024-09-18T04:56:19.207 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_client_mode": "crc secure", 2024-09-18T04:56:19.207 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_cluster_mode": "crc secure", 2024-09-18T04:56:19.207 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_cluster_type": "", 2024-09-18T04:56:19.207 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_compress_secure": "false", 2024-09-18T04:56:19.207 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_connection_idle_timeout": "900", 2024-09-18T04:56:19.207 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_connection_ready_timeout": "10", 2024-09-18T04:56:19.207 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_crc_data": "true", 2024-09-18T04:56:19.208 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_crc_header": "true", 2024-09-18T04:56:19.209 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_die_on_bad_msg": "false", 2024-09-18T04:56:19.209 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_die_on_bug": "true", 2024-09-18T04:56:19.209 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_die_on_old_message": "true", 2024-09-18T04:56:19.209 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_die_on_skipped_message": "false", 2024-09-18T04:56:19.209 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-18T04:56:19.209 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-18T04:56:19.209 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_coremask": "0xF", 2024-09-18T04:56:19.209 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-18T04:56:19.209 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-18T04:56:19.209 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_enable_tso": "true", 2024-09-18T04:56:19.209 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-18T04:56:19.209 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-18T04:56:19.209 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_hugepages": "", 2024-09-18T04:56:19.210 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-18T04:56:19.210 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-18T04:56:19.210 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_lro": "true", 2024-09-18T04:56:19.210 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_memory_channel": "4", 2024-09-18T04:56:19.210 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-18T04:56:19.210 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_pmd": "", 2024-09-18T04:56:19.210 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_port_id": "0", 2024-09-18T04:56:19.210 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-18T04:56:19.211 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-18T04:56:19.211 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_dump_on_send": "false", 2024-09-18T04:56:19.211 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_initial_backoff": "0.200000", 2024-09-18T04:56:19.211 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_inject_delay_max": "1.000000", 2024-09-18T04:56:19.211 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-18T04:56:19.211 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_inject_delay_type": "", 2024-09-18T04:56:19.211 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-18T04:56:19.211 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_inject_network_congestion": "0", 2024-09-18T04:56:19.211 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_inject_socket_failures": "5000", 2024-09-18T04:56:19.211 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_learn_addr_from_peer": "true", 2024-09-18T04:56:19.211 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_max_accept_failures": "4", 2024-09-18T04:56:19.212 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_max_backoff": "15.000000", 2024-09-18T04:56:19.212 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_mon_client_mode": "secure crc", 2024-09-18T04:56:19.212 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-18T04:56:19.212 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_mon_service_mode": "secure crc", 2024-09-18T04:56:19.212 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_osd_compress_min_size": "1024", 2024-09-18T04:56:19.212 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_osd_compress_mode": "none", 2024-09-18T04:56:19.212 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-18T04:56:19.212 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-18T04:56:19.212 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_pq_min_cost": "65536", 2024-09-18T04:56:19.212 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_public_type": "", 2024-09-18T04:56:19.212 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_service_mode": "crc secure", 2024-09-18T04:56:19.212 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_tcp_listen_backlog": "512", 2024-09-18T04:56:19.212 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_tcp_nodelay": "true", 2024-09-18T04:56:19.213 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-18T04:56:19.213 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_tcp_rcvbuf": "0", 2024-09-18T04:56:19.213 INFO:teuthology.orchestra.run.smithi002.stdout: "ms_type": "async+posix", 2024-09-18T04:56:19.213 INFO:teuthology.orchestra.run.smithi002.stdout: "no_config_file": "false", 2024-09-18T04:56:19.213 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter_completion_locks_per_session": "32", 2024-09-18T04:56:19.213 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-18T04:56:19.213 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-18T04:56:19.213 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter_inflight_ops": "1024", 2024-09-18T04:56:19.213 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-18T04:56:19.213 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-18T04:56:19.213 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter_tick_interval": "5.000000", 2024-09-18T04:56:19.213 INFO:teuthology.orchestra.run.smithi002.stdout: "objecter_timeout": "10.000000", 2024-09-18T04:56:19.213 INFO:teuthology.orchestra.run.smithi002.stdout: "objectstore_blackhole": "false", 2024-09-18T04:56:19.214 INFO:teuthology.orchestra.run.smithi002.stdout: "openssl_engine_opts": "", 2024-09-18T04:56:19.214 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_agent_delay_time": "5.000000", 2024-09-18T04:56:19.214 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_agent_hist_halflife": "1000", 2024-09-18T04:56:19.214 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_agent_max_low_ops": "2", 2024-09-18T04:56:19.214 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_agent_max_ops": "4", 2024-09-18T04:56:19.214 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-18T04:56:19.214 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-18T04:56:19.214 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_agent_slop": "0.020000", 2024-09-18T04:56:19.214 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-18T04:56:19.214 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-18T04:56:19.214 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_asio_thread_count": "2", 2024-09-18T04:56:19.214 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_async_recovery_min_cost": "100", 2024-09-18T04:56:19.214 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-18T04:56:19.214 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-18T04:56:19.215 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_backfill_scan_max": "512", 2024-09-18T04:56:19.215 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_backfill_scan_min": "64", 2024-09-18T04:56:19.215 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_backoff_on_degraded": "false", 2024-09-18T04:56:19.215 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_backoff_on_peering": "false", 2024-09-18T04:56:19.215 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_backoff_on_unfound": "true", 2024-09-18T04:56:19.215 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_beacon_report_interval": "300", 2024-09-18T04:56:19.215 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bench_duration": "30", 2024-09-18T04:56:19.215 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-18T04:56:19.215 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bench_max_block_size": "67108864", 2024-09-18T04:56:19.215 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-18T04:56:19.215 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_blkin_trace_all": "false", 2024-09-18T04:56:19.215 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-18T04:56:19.215 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-18T04:56:19.215 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-18T04:56:19.216 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-18T04:56:19.216 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_check_for_log_corruption": "false", 2024-09-18T04:56:19.216 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-18T04:56:19.216 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-18T04:56:19.216 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-18T04:56:19.216 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-18T04:56:19.216 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_class_update_on_start": "true", 2024-09-18T04:56:19.216 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_client_message_cap": "256", 2024-09-18T04:56:19.216 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_client_message_size_cap": "524288000", 2024-09-18T04:56:19.216 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_client_op_priority": "63", 2024-09-18T04:56:19.216 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_client_watch_timeout": "30", 2024-09-18T04:56:19.216 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_command_max_records": "256", 2024-09-18T04:56:19.217 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_compact_on_start": "false", 2024-09-18T04:56:19.217 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-18T04:56:19.217 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_crush_chooseleaf_type": "0", 2024-09-18T04:56:19.217 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-18T04:56:19.217 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_crush_update_on_start": "true", 2024-09-18T04:56:19.217 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_crush_update_weight_set": "true", 2024-09-18T04:56:19.217 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-18T04:56:19.218 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-18T04:56:19.218 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-18T04:56:19.218 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-18T04:56:19.218 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-18T04:56:19.218 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_feed_pullee": "-1", 2024-09-18T04:56:19.218 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-18T04:56:19.218 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-18T04:56:19.218 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-18T04:56:19.218 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_misdirected_ops": "false", 2024-09-18T04:56:19.218 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_no_acting_change": "false", 2024-09-18T04:56:19.218 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_no_purge_strays": "false", 2024-09-18T04:56:19.218 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_op_order": "false", 2024-09-18T04:56:19.218 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-18T04:56:19.218 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-18T04:56:19.219 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-18T04:56:19.219 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-18T04:56:19.219 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_shutdown": "false", 2024-09-18T04:56:19.219 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-18T04:56:19.219 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-18T04:56:19.219 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-18T04:56:19.219 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-18T04:56:19.219 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_verify_snaps": "false", 2024-09-18T04:56:19.219 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-18T04:56:19.219 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-18T04:56:19.219 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_deep_scrub_keys": "1024", 2024-09-18T04:56:19.219 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-18T04:56:19.219 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-18T04:56:19.220 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-18T04:56:19.220 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_deep_scrub_stride": "524288", 2024-09-18T04:56:19.220 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-18T04:56:19.220 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_default_data_pool_replay_window": "5", 2024-09-18T04:56:19.220 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_default_notify_timeout": "30", 2024-09-18T04:56:19.220 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_delete_sleep": "0.000000", 2024-09-18T04:56:19.220 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-18T04:56:19.220 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-18T04:56:19.220 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-18T04:56:19.220 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_discard_disconnected_ops": "true", 2024-09-18T04:56:19.220 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_enable_op_tracker": "true", 2024-09-18T04:56:19.220 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-18T04:56:19.220 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-18T04:56:19.221 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-18T04:56:19.221 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-18T04:56:19.221 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_fast_info": "true", 2024-09-18T04:56:19.221 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_fast_shutdown": "true", 2024-09-18T04:56:19.221 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-18T04:56:19.221 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-18T04:56:19.221 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-18T04:56:19.221 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-18T04:56:19.221 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-18T04:56:19.221 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_function_tracing": "false", 2024-09-18T04:56:19.221 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_heartbeat_grace": "20", 2024-09-18T04:56:19.221 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_heartbeat_interval": "6", 2024-09-18T04:56:19.221 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-18T04:56:19.222 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_heartbeat_min_peers": "10", 2024-09-18T04:56:19.222 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_heartbeat_min_size": "2000", 2024-09-18T04:56:19.222 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_heartbeat_stale": "600", 2024-09-18T04:56:19.222 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-18T04:56:19.222 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_hit_set_max_size": "100000", 2024-09-18T04:56:19.222 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_hit_set_min_size": "1000", 2024-09-18T04:56:19.222 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-18T04:56:19.222 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-18T04:56:19.222 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-18T04:56:19.222 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-18T04:56:19.222 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-18T04:56:19.222 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-18T04:56:19.222 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_journal_size": "5120", 2024-09-18T04:56:19.222 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_kill_backfill_at": "0", 2024-09-18T04:56:19.223 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-18T04:56:19.223 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_map_cache_size": "50", 2024-09-18T04:56:19.223 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_map_dedup": "true", 2024-09-18T04:56:19.223 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_map_message_max": "40", 2024-09-18T04:56:19.223 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-18T04:56:19.223 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_map_share_max_epochs": "40", 2024-09-18T04:56:19.223 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_attr_name_len": "100", 2024-09-18T04:56:19.223 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_attr_size": "0", 2024-09-18T04:56:19.223 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_backfills": "1", 2024-09-18T04:56:19.223 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_markdown_count": "5", 2024-09-18T04:56:19.223 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_markdown_period": "600", 2024-09-18T04:56:19.223 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_object_name_len": "2048", 2024-09-18T04:56:19.223 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_object_namespace_len": "256", 2024-09-18T04:56:19.224 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_object_size": "134217728", 2024-09-18T04:56:19.224 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-18T04:56:19.224 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-18T04:56:19.224 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_pg_blocked_by": "16", 2024-09-18T04:56:19.224 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_pg_log_entries": "10000", 2024-09-18T04:56:19.224 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-18T04:56:19.224 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_pgls": "1024", 2024-09-18T04:56:19.224 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_push_cost": "8388608", 2024-09-18T04:56:19.224 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_push_objects": "10", 2024-09-18T04:56:19.224 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_scrubs": "3", 2024-09-18T04:56:19.224 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-18T04:56:19.224 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_trimming_pgs": "2", 2024-09-18T04:56:19.224 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_write_op_reply_len": "64", 2024-09-18T04:56:19.224 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_max_write_size": "90", 2024-09-18T04:56:19.225 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-18T04:56:19.225 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-18T04:56:19.225 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-18T04:56:19.225 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-18T04:56:19.225 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-18T04:56:19.225 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-18T04:56:19.225 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-18T04:56:19.225 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-18T04:56:19.225 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_profile": "balanced", 2024-09-18T04:56:19.225 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-18T04:56:19.225 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-18T04:56:19.225 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-18T04:56:19.225 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-18T04:56:19.226 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-18T04:56:19.226 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-18T04:56:19.226 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-18T04:56:19.226 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-18T04:56:19.226 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-18T04:56:19.226 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-18T04:56:19.226 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-18T04:56:19.226 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_memory_base": "805306368", 2024-09-18T04:56:19.226 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_memory_cache_min": "134217728", 2024-09-18T04:56:19.226 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-18T04:56:19.226 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-18T04:56:19.226 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_memory_target": "4294967296", 2024-09-18T04:56:19.226 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_memory_target_autotune": "false", 2024-09-18T04:56:19.226 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-18T04:56:19.227 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_min_pg_log_entries": "250", 2024-09-18T04:56:19.227 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_min_recovery_priority": "0", 2024-09-18T04:56:19.227 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-18T04:56:19.227 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-18T04:56:19.227 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mon_report_interval": "5", 2024-09-18T04:56:19.227 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-18T04:56:19.227 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-18T04:56:19.227 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_num_cache_shards": "32", 2024-09-18T04:56:19.227 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_num_op_tracker_shard": "32", 2024-09-18T04:56:19.227 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_numa_auto_affinity": "true", 2024-09-18T04:56:19.227 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_numa_node": "-1", 2024-09-18T04:56:19.227 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_numa_prefer_iface": "true", 2024-09-18T04:56:19.227 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-18T04:56:19.228 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_objecter_finishers": "1", 2024-09-18T04:56:19.228 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_objectstore": "bluestore", 2024-09-18T04:56:19.228 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_objectstore_fuse": "false", 2024-09-18T04:56:19.228 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_objectstore_tracing": "false", 2024-09-18T04:56:19.228 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_complaint_time": "30.000000", 2024-09-18T04:56:19.228 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_history_duration": "600", 2024-09-18T04:56:19.228 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_history_size": "20", 2024-09-18T04:56:19.228 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_history_slow_op_size": "20", 2024-09-18T04:56:19.228 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-18T04:56:19.228 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_log_threshold": "5", 2024-09-18T04:56:19.228 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_num_shards": "0", 2024-09-18T04:56:19.228 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_num_shards_hdd": "5", 2024-09-18T04:56:19.228 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_num_shards_ssd": "8", 2024-09-18T04:56:19.229 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-18T04:56:19.229 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-18T04:56:19.229 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-18T04:56:19.229 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-18T04:56:19.229 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_pq_min_cost": "65536", 2024-09-18T04:56:19.229 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-18T04:56:19.229 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_queue_cut_off": "high", 2024-09-18T04:56:19.229 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-18T04:56:19.229 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_op_thread_timeout": "15", 2024-09-18T04:56:19.229 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_open_classes_on_start": "true", 2024-09-18T04:56:19.229 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_os_flags": "0", 2024-09-18T04:56:19.229 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_peering_op_priority": "255", 2024-09-18T04:56:19.229 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_delete_cost": "1048576", 2024-09-18T04:56:19.229 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_delete_priority": "5", 2024-09-18T04:56:19.230 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-18T04:56:19.230 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-18T04:56:19.230 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-18T04:56:19.230 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_log_trim_max": "10000", 2024-09-18T04:56:19.230 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_log_trim_min": "100", 2024-09-18T04:56:19.230 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-18T04:56:19.230 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-18T04:56:19.230 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pg_stat_report_interval_max": "500", 2024-09-18T04:56:19.230 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-18T04:56:19.230 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-18T04:56:19.230 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-18T04:56:19.230 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-18T04:56:19.230 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-18T04:56:19.231 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-18T04:56:19.231 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_crimson": "false", 2024-09-18T04:56:19.231 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-18T04:56:19.231 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-18T04:56:19.231 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-18T04:56:19.231 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-18T04:56:19.231 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-18T04:56:19.231 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-18T04:56:19.231 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-18T04:56:19.231 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-18T04:56:19.231 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_flags": "0", 2024-09-18T04:56:19.231 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-18T04:56:19.231 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_min_size": "0", 2024-09-18T04:56:19.231 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-18T04:56:19.232 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_pg_num": "32", 2024-09-18T04:56:19.232 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_pgp_num": "0", 2024-09-18T04:56:19.232 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-18T04:56:19.232 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_size": "2", 2024-09-18T04:56:19.232 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_default_type": "replicated", 2024-09-18T04:56:19.232 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-18T04:56:19.232 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-18T04:56:19.232 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_push_per_object_cost": "1000", 2024-09-18T04:56:19.232 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-18T04:56:19.232 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recover_clone_overlap": "true", 2024-09-18T04:56:19.232 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-18T04:56:19.232 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_cost": "20971520", 2024-09-18T04:56:19.232 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-18T04:56:19.232 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_max_active": "0", 2024-09-18T04:56:19.233 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-18T04:56:19.233 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-18T04:56:19.233 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-18T04:56:19.233 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-18T04:56:19.233 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_max_single_start": "1", 2024-09-18T04:56:19.233 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_op_priority": "3", 2024-09-18T04:56:19.233 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-18T04:56:19.233 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_priority": "5", 2024-09-18T04:56:19.233 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-18T04:56:19.233 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_sleep": "0.000000", 2024-09-18T04:56:19.233 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-18T04:56:19.233 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-18T04:56:19.233 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-18T04:56:19.234 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_repair_during_recovery": "false", 2024-09-18T04:56:19.234 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_requested_scrub_priority": "120", 2024-09-18T04:56:19.234 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-18T04:56:19.234 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-18T04:56:19.234 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_auto_repair": "false", 2024-09-18T04:56:19.234 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-18T04:56:19.234 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-18T04:56:19.234 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_begin_hour": "0", 2024-09-18T04:56:19.234 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_begin_week_day": "0", 2024-09-18T04:56:19.234 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_chunk_max": "25", 2024-09-18T04:56:19.234 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_chunk_min": "5", 2024-09-18T04:56:19.234 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_cost": "52428800", 2024-09-18T04:56:19.234 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_during_recovery": "false", 2024-09-18T04:56:19.234 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_end_hour": "0", 2024-09-18T04:56:19.235 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_end_week_day": "0", 2024-09-18T04:56:19.235 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_event_cost": "4096", 2024-09-18T04:56:19.235 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-18T04:56:19.235 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-18T04:56:19.235 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_invalid_stats": "true", 2024-09-18T04:56:19.235 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-18T04:56:19.235 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-18T04:56:19.235 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_max_preemptions": "5", 2024-09-18T04:56:19.235 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-18T04:56:19.235 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_priority": "5", 2024-09-18T04:56:19.235 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_reservation_timeout": "5000", 2024-09-18T04:56:19.235 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_sleep": "0.000000", 2024-09-18T04:56:19.235 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_scrub_slow_reservation_response": "2200", 2024-09-18T04:56:19.236 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-18T04:56:19.236 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-18T04:56:19.236 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-18T04:56:19.236 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_skip_data_digest": "false", 2024-09-18T04:56:19.236 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_smart_report_timeout": "5", 2024-09-18T04:56:19.236 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_snap_trim_cost": "1048576", 2024-09-18T04:56:19.236 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_snap_trim_priority": "5", 2024-09-18T04:56:19.236 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-18T04:56:19.236 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-18T04:56:19.236 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-18T04:56:19.236 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-18T04:56:19.236 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-18T04:56:19.236 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-18T04:56:19.237 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-18T04:56:19.237 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_target_transaction_size": "30", 2024-09-18T04:56:19.237 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-18T04:56:19.237 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-18T04:56:19.237 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-18T04:56:19.237 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-18T04:56:19.237 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-18T04:56:19.237 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-18T04:56:19.237 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-18T04:56:19.237 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-18T04:56:19.237 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-18T04:56:19.237 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-18T04:56:19.237 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_tracing": "false", 2024-09-18T04:56:19.237 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_use_stale_snap": "false", 2024-09-18T04:56:19.238 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-18T04:56:19.238 INFO:teuthology.orchestra.run.smithi002.stdout: "osdc_blkin_trace_all": "false", 2024-09-18T04:56:19.238 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_kill_at": "0", 2024-09-18T04:56:19.238 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_max_join_drift": "10", 2024-09-18T04:56:19.238 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_min": "500", 2024-09-18T04:56:19.238 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_min_wait": "0.050000", 2024-09-18T04:56:19.238 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_propose_interval": "1.000000", 2024-09-18T04:56:19.238 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_service_trim_max": "500", 2024-09-18T04:56:19.238 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-18T04:56:19.238 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_service_trim_min": "250", 2024-09-18T04:56:19.238 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_stash_full_interval": "25", 2024-09-18T04:56:19.238 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_trim_max": "500", 2024-09-18T04:56:19.238 INFO:teuthology.orchestra.run.smithi002.stdout: "paxos_trim_min": "250", 2024-09-18T04:56:19.239 INFO:teuthology.orchestra.run.smithi002.stdout: "perf": "true", 2024-09-18T04:56:19.239 INFO:teuthology.orchestra.run.smithi002.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-09-18T04:56:19.239 INFO:teuthology.orchestra.run.smithi002.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-18T04:56:19.239 INFO:teuthology.orchestra.run.smithi002.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-18T04:56:19.239 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "-", 2024-09-18T04:56:19.239 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrv": "", 2024-09-18T04:56:19.239 INFO:teuthology.orchestra.run.smithi002.stdout: "public_bind_addr": "-", 2024-09-18T04:56:19.239 INFO:teuthology.orchestra.run.smithi002.stdout: "public_network": "", 2024-09-18T04:56:19.239 INFO:teuthology.orchestra.run.smithi002.stdout: "public_network_interface": "", 2024-09-18T04:56:19.239 INFO:teuthology.orchestra.run.smithi002.stdout: "qat_compressor_enabled": "false", 2024-09-18T04:56:19.239 INFO:teuthology.orchestra.run.smithi002.stdout: "qat_compressor_session_max_number": "256", 2024-09-18T04:56:19.239 INFO:teuthology.orchestra.run.smithi002.stdout: "rados_mon_op_timeout": "0", 2024-09-18T04:56:19.239 INFO:teuthology.orchestra.run.smithi002.stdout: "rados_osd_op_timeout": "0", 2024-09-18T04:56:19.239 INFO:teuthology.orchestra.run.smithi002.stdout: "rados_tracing": "false", 2024-09-18T04:56:19.240 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_atime_update_interval": "60", 2024-09-18T04:56:19.240 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-18T04:56:19.240 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_balance_parent_reads": "false", 2024-09-18T04:56:19.240 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_balance_snap_reads": "false", 2024-09-18T04:56:19.240 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_blkin_trace_all": "false", 2024-09-18T04:56:19.240 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-18T04:56:19.240 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-18T04:56:19.240 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache": "true", 2024-09-18T04:56:19.240 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-18T04:56:19.240 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-18T04:56:19.240 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-18T04:56:19.240 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-18T04:56:19.240 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache_policy": "writearound", 2024-09-18T04:56:19.240 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache_size": "33554432", 2024-09-18T04:56:19.241 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-18T04:56:19.241 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-18T04:56:19.241 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_clone_copy_on_read": "false", 2024-09-18T04:56:19.241 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_compression_hint": "none", 2024-09-18T04:56:19.241 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_concurrent_management_ops": "10", 2024-09-18T04:56:19.241 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-18T04:56:19.241 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_clone_format": "auto", 2024-09-18T04:56:19.241 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_data_pool": "", 2024-09-18T04:56:19.241 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_features": "61", 2024-09-18T04:56:19.241 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_format": "2", 2024-09-18T04:56:19.241 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_map_options": "", 2024-09-18T04:56:19.241 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_order": "22", 2024-09-18T04:56:19.241 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_pool": "rbd", 2024-09-18T04:56:19.242 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-18T04:56:19.242 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_stripe_count": "0", 2024-09-18T04:56:19.242 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_default_stripe_unit": "0", 2024-09-18T04:56:19.242 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-18T04:56:19.242 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-18T04:56:19.242 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-18T04:56:19.242 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_enable_alloc_hint": "true", 2024-09-18T04:56:19.242 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-18T04:56:19.242 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_io_scheduler": "simple", 2024-09-18T04:56:19.242 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-18T04:56:19.242 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-18T04:56:19.242 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-18T04:56:19.242 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-18T04:56:19.243 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-18T04:56:19.243 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-18T04:56:19.243 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-18T04:56:19.243 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-18T04:56:19.243 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-18T04:56:19.243 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_order": "24", 2024-09-18T04:56:19.243 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_pool": "", 2024-09-18T04:56:19.243 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_journal_splay_width": "4", 2024-09-18T04:56:19.243 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_localize_parent_reads": "false", 2024-09-18T04:56:19.243 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_localize_snap_reads": "false", 2024-09-18T04:56:19.243 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-18T04:56:19.243 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-18T04:56:19.243 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-18T04:56:19.243 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-18T04:56:19.244 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-18T04:56:19.244 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-18T04:56:19.244 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-18T04:56:19.244 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-18T04:56:19.244 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-18T04:56:19.244 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-18T04:56:19.244 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-18T04:56:19.244 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-18T04:56:19.244 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-18T04:56:19.244 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-18T04:56:19.244 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-18T04:56:19.244 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-18T04:56:19.244 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-18T04:56:19.245 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-18T04:56:19.245 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-18T04:56:19.245 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-18T04:56:19.245 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-18T04:56:19.245 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-18T04:56:19.245 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-18T04:56:19.245 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-18T04:56:19.245 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-18T04:56:19.245 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-18T04:56:19.245 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-18T04:56:19.245 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-18T04:56:19.245 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-18T04:56:19.245 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-18T04:56:19.245 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-18T04:56:19.246 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_mtime_update_interval": "60", 2024-09-18T04:56:19.246 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_non_blocking_aio": "true", 2024-09-18T04:56:19.246 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_op_thread_timeout": "60", 2024-09-18T04:56:19.246 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_op_threads": "1", 2024-09-18T04:56:19.246 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_parent_cache_enabled": "false", 2024-09-18T04:56:19.246 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-18T04:56:19.246 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-18T04:56:19.246 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-18T04:56:19.246 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_plugins": "", 2024-09-18T04:56:19.246 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_bps_burst": "0", 2024-09-18T04:56:19.246 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-18T04:56:19.246 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_bps_limit": "0", 2024-09-18T04:56:19.246 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_exclude_ops": "0", 2024-09-18T04:56:19.247 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_iops_burst": "0", 2024-09-18T04:56:19.247 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-18T04:56:19.247 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_iops_limit": "0", 2024-09-18T04:56:19.247 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-18T04:56:19.247 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-18T04:56:19.247 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-18T04:56:19.247 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-18T04:56:19.247 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-18T04:56:19.247 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-18T04:56:19.247 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-18T04:56:19.247 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-18T04:56:19.247 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-18T04:56:19.247 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-18T04:56:19.248 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-18T04:56:19.248 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-18T04:56:19.248 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-18T04:56:19.248 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-18T04:56:19.248 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_read_from_replica_policy": "default", 2024-09-18T04:56:19.248 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-18T04:56:19.248 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-18T04:56:19.248 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-18T04:56:19.248 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-18T04:56:19.248 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_skip_partial_discard": "true", 2024-09-18T04:56:19.248 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-18T04:56:19.248 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_tracing": "false", 2024-09-18T04:56:19.248 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_validate_names": "true", 2024-09-18T04:56:19.249 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd_validate_pool": "true", 2024-09-18T04:56:19.249 INFO:teuthology.orchestra.run.smithi002.stdout: "restapi_base_url": "", 2024-09-18T04:56:19.249 INFO:teuthology.orchestra.run.smithi002.stdout: "restapi_log_level": "", 2024-09-18T04:56:19.249 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_acl_grants_max_num": "100", 2024-09-18T04:56:19.249 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_admin_entry": "admin", 2024-09-18T04:56:19.249 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-18T04:56:19.249 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_backend_store": "rados", 2024-09-18T04:56:19.249 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_barbican_url": "", 2024-09-18T04:56:19.249 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_beast_enable_async": "true", 2024-09-18T04:56:19.249 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-18T04:56:19.249 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-18T04:56:19.249 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-18T04:56:19.249 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-18T04:56:19.249 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-18T04:56:19.250 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-18T04:56:19.250 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-18T04:56:19.250 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_cache_enabled": "true", 2024-09-18T04:56:19.250 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_cache_expiry_interval": "900", 2024-09-18T04:56:19.250 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_cache_lru_size": "10000", 2024-09-18T04:56:19.250 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_config_store": "rados", 2024-09-18T04:56:19.250 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_content_length_compat": "false", 2024-09-18T04:56:19.250 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_copy_obj_progress": "true", 2024-09-18T04:56:19.250 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-18T04:56:19.250 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_cors_rules_max_num": "100", 2024-09-18T04:56:19.250 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_cross_domain_policy": "", 2024-09-18T04:56:19.250 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-18T04:56:19.250 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_kmip_addr": "", 2024-09-18T04:56:19.251 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-18T04:56:19.251 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-18T04:56:19.251 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-18T04:56:19.251 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-18T04:56:19.251 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_kmip_password": "", 2024-09-18T04:56:19.251 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-18T04:56:19.251 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_kmip_username": "", 2024-09-18T04:56:19.251 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_require_ssl": "true", 2024-09-18T04:56:19.251 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-18T04:56:19.251 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-18T04:56:19.251 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-18T04:56:19.251 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-18T04:56:19.251 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-18T04:56:19.251 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-18T04:56:19.252 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-18T04:56:19.252 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-18T04:56:19.252 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-18T04:56:19.252 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-18T04:56:19.252 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-18T04:56:19.252 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-18T04:56:19.252 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-18T04:56:19.252 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-18T04:56:19.252 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-18T04:56:19.252 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_addr": "", 2024-09-18T04:56:19.253 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_auth": "token", 2024-09-18T04:56:19.253 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_namespace": "", 2024-09-18T04:56:19.253 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_prefix": "", 2024-09-18T04:56:19.253 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-18T04:56:19.253 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-18T04:56:19.253 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-18T04:56:19.254 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-18T04:56:19.254 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_token_file": "", 2024-09-18T04:56:19.254 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-18T04:56:19.254 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_curl_buffersize": "524288", 2024-09-18T04:56:19.254 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-18T04:56:19.254 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_curl_low_speed_time": "300", 2024-09-18T04:56:19.254 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-18T04:56:19.254 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-18T04:56:19.254 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-18T04:56:19.254 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-18T04:56:19.254 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-18T04:56:19.254 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-18T04:56:19.254 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-18T04:56:19.254 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-18T04:56:19.255 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-18T04:56:19.255 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-18T04:56:19.255 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-18T04:56:19.255 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_data_log_changes_size": "1000", 2024-09-18T04:56:19.255 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_data_log_num_shards": "128", 2024-09-18T04:56:19.255 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-18T04:56:19.255 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_data_log_window": "30", 2024-09-18T04:56:19.255 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-18T04:56:19.255 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-18T04:56:19.255 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-18T04:56:19.255 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-18T04:56:19.255 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-18T04:56:19.255 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-18T04:56:19.256 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-18T04:56:19.256 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-18T04:56:19.256 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-18T04:56:19.256 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-18T04:56:19.256 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-18T04:56:19.256 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-18T04:56:19.256 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-18T04:56:19.256 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-18T04:56:19.256 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-18T04:56:19.256 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-18T04:56:19.256 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-18T04:56:19.256 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-18T04:56:19.256 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-18T04:56:19.257 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-18T04:56:19.257 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-18T04:56:19.257 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-18T04:56:19.257 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-18T04:56:19.257 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-18T04:56:19.257 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dns_name": "", 2024-09-18T04:56:19.257 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dns_s3website_name": "", 2024-09-18T04:56:19.257 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_dynamic_resharding": "true", 2024-09-18T04:56:19.257 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-18T04:56:19.257 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_enable_gc_threads": "true", 2024-09-18T04:56:19.257 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_enable_lc_threads": "true", 2024-09-18T04:56:19.257 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_enable_ops_log": "false", 2024-09-18T04:56:19.257 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_enable_quota_threads": "true", 2024-09-18T04:56:19.258 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_enable_static_website": "false", 2024-09-18T04:56:19.258 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_enable_usage_log": "false", 2024-09-18T04:56:19.258 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_enforce_swift_acls": "true", 2024-09-18T04:56:19.258 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_exit_timeout_secs": "120", 2024-09-18T04:56:19.258 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_expose_bucket": "false", 2024-09-18T04:56:19.258 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_extended_http_attrs": "", 2024-09-18T04:56:19.258 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_filter": "none", 2024-09-18T04:56:19.258 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-18T04:56:19.258 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_frontends": "beast port=7480", 2024-09-18T04:56:19.258 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-18T04:56:19.258 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_max_deferred": "50", 2024-09-18T04:56:19.258 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-18T04:56:19.258 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_max_objs": "32", 2024-09-18T04:56:19.258 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-18T04:56:19.259 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-18T04:56:19.259 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-18T04:56:19.259 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-18T04:56:19.259 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_gc_processor_period": "3600", 2024-09-18T04:56:19.259 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-18T04:56:19.259 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-18T04:56:19.259 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-18T04:56:19.259 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-18T04:56:19.259 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_init_timeout": "300", 2024-09-18T04:56:19.259 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-18T04:56:19.259 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-18T04:56:19.259 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-18T04:56:19.259 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-18T04:56:19.260 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_admin_domain": "", 2024-09-18T04:56:19.260 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_admin_password": "", 2024-09-18T04:56:19.260 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_admin_password_path": "", 2024-09-18T04:56:19.260 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_admin_project": "", 2024-09-18T04:56:19.260 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_admin_tenant": "", 2024-09-18T04:56:19.260 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_admin_token": "", 2024-09-18T04:56:19.260 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_admin_token_path": "", 2024-09-18T04:56:19.260 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_admin_user": "", 2024-09-18T04:56:19.260 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_api_version": "2", 2024-09-18T04:56:19.260 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_barbican_domain": "", 2024-09-18T04:56:19.260 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_barbican_password": "", 2024-09-18T04:56:19.260 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_barbican_project": "", 2024-09-18T04:56:19.260 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-18T04:56:19.260 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_barbican_user": "", 2024-09-18T04:56:19.261 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-18T04:56:19.261 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-18T04:56:19.261 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-18T04:56:19.261 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-18T04:56:19.261 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-18T04:56:19.261 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_url": "", 2024-09-18T04:56:19.261 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-18T04:56:19.261 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_lc_debug_interval": "-1", 2024-09-18T04:56:19.261 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_lc_lock_max_time": "90", 2024-09-18T04:56:19.261 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_lc_max_objs": "32", 2024-09-18T04:56:19.261 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_lc_max_rules": "1000", 2024-09-18T04:56:19.261 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_lc_max_worker": "3", 2024-09-18T04:56:19.261 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-18T04:56:19.262 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_lc_thread_delay": "0", 2024-09-18T04:56:19.262 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-18T04:56:19.262 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ldap_dnattr": "uid", 2024-09-18T04:56:19.262 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-18T04:56:19.262 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ldap_searchfilter": "", 2024-09-18T04:56:19.262 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-18T04:56:19.262 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ldap_uri": "ldaps://", 2024-09-18T04:56:19.262 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-18T04:56:19.262 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-18T04:56:19.262 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-18T04:56:19.262 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_log_http_headers": "", 2024-09-18T04:56:19.262 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-18T04:56:19.262 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-18T04:56:19.262 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_log_object_name_utc": "false", 2024-09-18T04:56:19.263 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-09-18T04:56:19.263 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_attr_name_len": "0", 2024-09-18T04:56:19.263 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_attr_size": "0", 2024-09-18T04:56:19.263 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-18T04:56:19.263 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_chunk_size": "4194304", 2024-09-18T04:56:19.263 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-18T04:56:19.263 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-18T04:56:19.263 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-18T04:56:19.263 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_listing_results": "1000", 2024-09-18T04:56:19.263 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_notify_retries": "10", 2024-09-18T04:56:19.263 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-18T04:56:19.263 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_put_param_size": "1048576", 2024-09-18T04:56:19.263 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_put_size": "5368709120", 2024-09-18T04:56:19.264 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_max_slo_entries": "1000", 2024-09-18T04:56:19.264 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_md_log_max_shards": "64", 2024-09-18T04:56:19.264 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-18T04:56:19.264 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-18T04:56:19.264 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-18T04:56:19.264 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-18T04:56:19.264 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_mp_lock_max_time": "600", 2024-09-18T04:56:19.264 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-18T04:56:19.264 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-18T04:56:19.264 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-18T04:56:19.264 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-18T04:56:19.264 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-18T04:56:19.264 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-18T04:56:19.265 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-18T04:56:19.265 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-18T04:56:19.265 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_max_gc": "300", 2024-09-18T04:56:19.265 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-18T04:56:19.265 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-18T04:56:19.265 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-18T04:56:19.265 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-18T04:56:19.265 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-18T04:56:19.265 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-18T04:56:19.265 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-18T04:56:19.265 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_num_async_rados_threads": "32", 2024-09-18T04:56:19.265 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_num_control_oids": "8", 2024-09-18T04:56:19.265 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_numa_node": "-1", 2024-09-18T04:56:19.266 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-18T04:56:19.266 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-18T04:56:19.266 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_objexp_chunk_size": "100", 2024-09-18T04:56:19.266 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_objexp_gc_interval": "600", 2024-09-18T04:56:19.266 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-18T04:56:19.266 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-18T04:56:19.266 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-18T04:56:19.266 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_op_thread_timeout": "600", 2024-09-18T04:56:19.266 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_op_tracing": "false", 2024-09-18T04:56:19.266 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_opa_token": "", 2024-09-18T04:56:19.266 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_opa_url": "", 2024-09-18T04:56:19.266 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_opa_verify_ssl": "true", 2024-09-18T04:56:19.266 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-18T04:56:19.266 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-18T04:56:19.267 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ops_log_rados": "false", 2024-09-18T04:56:19.267 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_ops_log_socket_path": "", 2024-09-18T04:56:19.267 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-18T04:56:19.267 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-18T04:56:19.267 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-18T04:56:19.267 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_period_push_interval": "2.000000", 2024-09-18T04:56:19.267 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-18T04:56:19.267 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-18T04:56:19.267 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-18T04:56:19.267 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_print_continue": "true", 2024-09-18T04:56:19.267 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-18T04:56:19.267 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-18T04:56:19.267 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-18T04:56:19.267 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-18T04:56:19.268 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-18T04:56:19.268 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_rados_tracing": "false", 2024-09-18T04:56:19.268 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_realm": "", 2024-09-18T04:56:19.268 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_realm_id": "", 2024-09-18T04:56:19.268 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-18T04:56:19.268 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_region": "", 2024-09-18T04:56:19.268 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-18T04:56:19.268 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-18T04:56:19.268 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-18T04:56:19.268 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-18T04:56:19.268 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_request_uri": "", 2024-09-18T04:56:19.268 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_reshard_batch_size": "64", 2024-09-18T04:56:19.268 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-18T04:56:19.269 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_reshard_max_aio": "128", 2024-09-18T04:56:19.269 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_reshard_num_logs": "16", 2024-09-18T04:56:19.269 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_reshard_thread_interval": "600", 2024-09-18T04:56:19.269 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_resolve_cname": "false", 2024-09-18T04:56:19.269 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-18T04:56:19.269 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_run_sync_thread": "true", 2024-09-18T04:56:19.269 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-18T04:56:19.269 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-18T04:56:19.269 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-18T04:56:19.269 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-18T04:56:19.269 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-18T04:56:19.269 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-18T04:56:19.269 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-18T04:56:19.270 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-18T04:56:19.270 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_scheduler_type": "throttler", 2024-09-18T04:56:19.270 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_script_uri": "", 2024-09-18T04:56:19.270 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_service_provider_name": "", 2024-09-18T04:56:19.270 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-18T04:56:19.270 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sts_client_id": "", 2024-09-18T04:56:19.270 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sts_client_secret": "", 2024-09-18T04:56:19.270 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sts_entry": "sts", 2024-09-18T04:56:19.270 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sts_key": "sts", 2024-09-18T04:56:19.270 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-18T04:56:19.270 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sts_min_session_duration": "900", 2024-09-18T04:56:19.270 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sts_token_introspection_url": "", 2024-09-18T04:56:19.270 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_account_in_url": "false", 2024-09-18T04:56:19.270 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_auth_entry": "auth", 2024-09-18T04:56:19.271 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_auth_url": "", 2024-09-18T04:56:19.271 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_custom_header": "", 2024-09-18T04:56:19.271 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-18T04:56:19.271 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_need_stats": "true", 2024-09-18T04:56:19.271 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_tenant_name": "", 2024-09-18T04:56:19.271 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_token_expiration": "86400", 2024-09-18T04:56:19.271 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_url": "", 2024-09-18T04:56:19.271 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_url_prefix": "swift", 2024-09-18T04:56:19.271 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-18T04:56:19.271 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-18T04:56:19.271 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-18T04:56:19.271 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_lease_period": "120", 2024-09-18T04:56:19.271 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-18T04:56:19.271 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-18T04:56:19.272 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-18T04:56:19.272 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-18T04:56:19.272 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-18T04:56:19.272 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-18T04:56:19.272 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-18T04:56:19.272 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-18T04:56:19.272 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-18T04:56:19.272 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_thread_pool_size": "512", 2024-09-18T04:56:19.272 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_torrent_comment": "", 2024-09-18T04:56:19.272 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_torrent_createby": "", 2024-09-18T04:56:19.272 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_torrent_encoding": "", 2024-09-18T04:56:19.272 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_torrent_flag": "false", 2024-09-18T04:56:19.272 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_torrent_origin": "", 2024-09-18T04:56:19.273 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-18T04:56:19.273 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_torrent_tracker": "", 2024-09-18T04:56:19.273 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_trust_forwarded_https": "false", 2024-09-18T04:56:19.273 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-18T04:56:19.273 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-18T04:56:19.273 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_usage_max_shards": "32", 2024-09-18T04:56:19.273 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_usage_max_user_shards": "1", 2024-09-18T04:56:19.273 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_use_opa_authz": "false", 2024-09-18T04:56:19.273 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-18T04:56:19.273 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-18T04:56:19.273 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_max_buckets": "1000", 2024-09-18T04:56:19.273 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_policies_max_num": "100", 2024-09-18T04:56:19.273 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-18T04:56:19.273 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-18T04:56:19.274 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-18T04:56:19.274 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-18T04:56:19.274 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_user_unique_email": "true", 2024-09-18T04:56:19.274 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_verify_ssl": "true", 2024-09-18T04:56:19.274 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-18T04:56:19.274 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_zone": "", 2024-09-18T04:56:19.274 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_zone_id": "", 2024-09-18T04:56:19.274 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-18T04:56:19.274 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_zonegroup": "", 2024-09-18T04:56:19.274 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_zonegroup_id": "", 2024-09-18T04:56:19.274 INFO:teuthology.orchestra.run.smithi002.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-18T04:56:19.274 INFO:teuthology.orchestra.run.smithi002.stdout: "rgwlc_auto_session_clear": "true", 2024-09-18T04:56:19.274 INFO:teuthology.orchestra.run.smithi002.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-18T04:56:19.275 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_block_size": "4096", 2024-09-18T04:56:19.275 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-18T04:56:19.275 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-18T04:56:19.275 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-18T04:56:19.275 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-18T04:56:19.275 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-18T04:56:19.275 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cache_size": "536870912", 2024-09-18T04:56:19.275 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-18T04:56:19.275 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-18T04:56:19.275 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-18T04:56:19.275 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-18T04:56:19.275 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-18T04:56:19.275 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-18T04:56:19.275 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-18T04:56:19.276 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-18T04:56:19.276 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_index_type": "binary_search", 2024-09-18T04:56:19.276 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-18T04:56:19.276 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-18T04:56:19.276 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_partition_filters": "false", 2024-09-18T04:56:19.276 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_perf": "false", 2024-09-18T04:56:19.276 INFO:teuthology.orchestra.run.smithi002.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-18T04:56:19.276 INFO:teuthology.orchestra.run.smithi002.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-18T04:56:19.276 INFO:teuthology.orchestra.run.smithi002.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-18T04:56:19.276 INFO:teuthology.orchestra.run.smithi002.stdout: "run_dir": "/var/run/ceph", 2024-09-18T04:56:19.276 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_block_create": "true", 2024-09-18T04:56:19.276 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_cache_lru_size": "67108864", 2024-09-18T04:56:19.276 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-18T04:56:19.276 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_default_max_object_size": "16777216", 2024-09-18T04:56:19.277 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-18T04:56:19.277 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_device_size": "53687091200", 2024-09-18T04:56:19.277 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_journal_batch_capacity": "16", 2024-09-18T04:56:19.277 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-18T04:56:19.277 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-18T04:56:19.277 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-18T04:56:19.277 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_main_device_type": "SSD", 2024-09-18T04:56:19.277 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-18T04:56:19.277 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-18T04:56:19.277 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-18T04:56:19.277 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-18T04:56:19.277 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-18T04:56:19.278 INFO:teuthology.orchestra.run.smithi002.stdout: "seastore_segment_size": "67108864", 2024-09-18T04:56:19.278 INFO:teuthology.orchestra.run.smithi002.stdout: "set_keepcaps": "false", 2024-09-18T04:56:19.278 INFO:teuthology.orchestra.run.smithi002.stdout: "setgroup": "", 2024-09-18T04:56:19.278 INFO:teuthology.orchestra.run.smithi002.stdout: "setuser": "", 2024-09-18T04:56:19.278 INFO:teuthology.orchestra.run.smithi002.stdout: "setuser_match_path": "", 2024-09-18T04:56:19.278 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-18T04:56:19.278 INFO:teuthology.orchestra.run.smithi002.stdout: "thp": "false", 2024-09-18T04:56:19.278 INFO:teuthology.orchestra.run.smithi002.stdout: "threadpool_default_timeout": "60", 2024-09-18T04:56:19.278 INFO:teuthology.orchestra.run.smithi002.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-18T04:56:19.278 INFO:teuthology.orchestra.run.smithi002.stdout: "throttler_perf_counter": "true" 2024-09-18T04:56:19.278 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:19.280 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-18T04:56:19.396 INFO:teuthology.orchestra.run.smithi002.stdout:mon.a: {} 2024-09-18T04:56:19.397 INFO:teuthology.orchestra.run.smithi002.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T04:56:19.406 INFO:teuthology.orchestra.run.smithi002.stdout:mon.b: {} 2024-09-18T04:56:19.406 INFO:teuthology.orchestra.run.smithi002.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T04:56:19.415 INFO:teuthology.orchestra.run.smithi002.stdout:mon.c: {} 2024-09-18T04:56:19.415 INFO:teuthology.orchestra.run.smithi002.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T04:56:19.425 DEBUG:teuthology.orchestra.run.smithi002:> type ceph-objectstore-tool 2024-09-18T04:56:19.441 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-18T04:56:19.441 DEBUG:teuthology.orchestra.run.smithi099:> type ceph-objectstore-tool 2024-09-18T04:56:19.459 INFO:teuthology.orchestra.run.smithi099.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-09-18T04:56:19.460 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-18T04:56:19.473 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-18T04:56:19.473 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-18T04:56:19.473 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T04:56:19.473 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T04:56:19.474 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-18T04:56:19.474 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T04:56:19.474 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T04:56:19.474 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-18T04:56:19.474 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-18T04:56:19.475 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-18T04:56:19.475 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:56:19.475 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-18T04:56:19.475 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:56:19.476 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-18T04:56:19.476 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T04:56:19.476 DEBUG:teuthology.orchestra.run.smithi002:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-09-18T04:56:19.483 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:19.480+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:19.519 INFO:teuthology.orchestra.run.smithi002.stdout:mapped writes DISABLED 2024-09-18T04:56:19.519 INFO:teuthology.orchestra.run.smithi002.stdout:mapped reads DISABLED 2024-09-18T04:56:19.519 INFO:teuthology.orchestra.run.smithi002.stdout:Seed set to 108 2024-09-18T04:56:19.576 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:19.574+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:19.598 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:19.598 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-18T04:56:19.598 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:19.599 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:19.599 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:19.599 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:56:19.599 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:19.609 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:56:19.663 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:56:19.663 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:56:19.678 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:19.675+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:19.680 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:56:19.680 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:56:19.713 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:56:19.713 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:56:19.730 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:56:19.730 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:56:19.759 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:19.759 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:19.764 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:56:19.765 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:56:19.768 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:56:19.800 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:56:19.800 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:56:19.801 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:19.797+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:19.819 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:56:19.819 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:56:19.837 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:56:19.837 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:56:19.879 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:19.877+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:19.908 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:19.908 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:19.917 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:56:19.980 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:19.978+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:20.024 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:20.024 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:20.033 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:56:20.081 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:20.078+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:20.119 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-18T04:56:20.148 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:20.148 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:20.155 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:56:20.182 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:20.180+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:20.284 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:20.281+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:20.303 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:20.304 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:20.312 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:56:20.385 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:20.383+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:20.419 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:20.419 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:20.428 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:56:20.486 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:20.483+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:20.535 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:20.535 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:20.543 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:56:20.586 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:20.584+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:20.687 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:20.685+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:20.689 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:20.689 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:20.698 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:56:20.787 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:20.785+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:20.805 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:20.805 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:20.814 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:56:20.898 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:20.896+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:20.961 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:20.961 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:20.970 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:56:20.989 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:20.987+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:21.090 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:21.087+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:21.117 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:21.117 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:21.132 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:56:21.190 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:21.188+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:21.244 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:21.244 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:21.254 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:56:21.295 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:21.293+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:21.393 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:21.391+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:21.397 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:21.397 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:21.406 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:56:21.494 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:21.492+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:21.507 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:21.507 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:21.516 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:56:21.595 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:21.592+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:21.619 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:21.619 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:21.627 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:56:21.696 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:21.694+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:21.770 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:21.771 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:21.780 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:56:21.823 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:21.821+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:21.849 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:56:21.882 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:21.882 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:21.890 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:56:21.899 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:21.896+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:21.986 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:56:21.987 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:56:22.000 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:21.998+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:22.003 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:56:22.003 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:56:22.019 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:56:22.019 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:56:22.034 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:22.034 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:22.035 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:56:22.035 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:56:22.042 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:56:22.052 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:56:22.052 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:56:22.069 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:56:22.069 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:56:22.085 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:56:22.086 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:56:22.101 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:22.099+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:22.103 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:56:22.103 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:56:22.136 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T04:56:22.186 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:22.186 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:22.186 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:56:22.186 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:56:22.186 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:22.195 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:56:22.202 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:22.199+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:22.297 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:22.298 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:56:22.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:56:22.298 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:56:22.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.298 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4223.0:1 1.3 e15/14 1:c4c92e5a:::rbd_trash:head v 15'1, mlcod=0'0)", 2024-09-18T04:56:22.298 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:10.402353+0000", 2024-09-18T04:56:22.298 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.893654080999999, 2024-09-18T04:56:22.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047646060000000002, 2024-09-18T04:56:22.298 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:22.298 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:22.298 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:22.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:22.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.402353+0000", 2024-09-18T04:56:22.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:22.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.402353+0000", 2024-09-18T04:56:22.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:22.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.402353+0000", 2024-09-18T04:56:22.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:22.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:22.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.402357+0000", 2024-09-18T04:56:22.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3209999999999998e-06 2024-09-18T04:56:22.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:22.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.402372+0000", 2024-09-18T04:56:22.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5201999999999999e-05 2024-09-18T04:56:22.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:22.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.402382+0000", 2024-09-18T04:56:22.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8430000000000005e-06 2024-09-18T04:56:22.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:22.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.402417+0000", 2024-09-18T04:56:22.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4811999999999998e-05 2024-09-18T04:56:22.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:22.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.402434+0000", 2024-09-18T04:56:22.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6877000000000001e-05 2024-09-18T04:56:22.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:22.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.407087+0000", 2024-09-18T04:56:22.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.004653122 2024-09-18T04:56:22.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:22.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.407117+0000", 2024-09-18T04:56:22.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0738999999999998e-05 2024-09-18T04:56:22.303 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.303 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:22.303 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.303 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.303 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:22.303 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:22.304 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:22.302+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:22.306 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:56:22.405 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:22.403+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:22.449 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:22.449 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:22.449 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:56:22.449 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:22.457 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:56:22.506 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:22.504+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:22.599 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:22.599 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:22.599 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:56:22.599 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:56:22.599 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:22.607 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:22.604+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:22.608 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:56:22.707 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:22.706+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:22.711 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:22.712 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:56:22.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:56:22.712 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:56:22.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.712 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:56:22.712 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:56:22.712 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.314970333, 2024-09-18T04:56:22.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037718511000000003, 2024-09-18T04:56:22.712 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:22.712 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:22.712 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:22.712 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:56:22.713 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:56:22.713 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-18T04:56:22.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.713 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:22.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:22.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:56:22.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:22.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:56:22.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:22.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393125+0000", 2024-09-18T04:56:22.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:22.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:22.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393133+0000", 2024-09-18T04:56:22.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.225e-06 2024-09-18T04:56:22.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:22.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393144+0000", 2024-09-18T04:56:22.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1279e-05 2024-09-18T04:56:22.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:22.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393153+0000", 2024-09-18T04:56:22.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4540000000000007e-06 2024-09-18T04:56:22.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:22.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393203+0000", 2024-09-18T04:56:22.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9963000000000003e-05 2024-09-18T04:56:22.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:56:22.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393210+0000", 2024-09-18T04:56:22.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5689999999999999e-06 2024-09-18T04:56:22.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:22.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427399+0000", 2024-09-18T04:56:22.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034188784999999999 2024-09-18T04:56:22.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:22.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427532+0000", 2024-09-18T04:56:22.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013264700000000001 2024-09-18T04:56:22.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:22.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427685+0000", 2024-09-18T04:56:22.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015300599999999999 2024-09-18T04:56:22.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:22.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.428520+0000", 2024-09-18T04:56:22.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00083502499999999998 2024-09-18T04:56:22.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:22.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430772+0000", 2024-09-18T04:56:22.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022518880000000001 2024-09-18T04:56:22.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:22.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430817+0000", 2024-09-18T04:56:22.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5077000000000002e-05 2024-09-18T04:56:22.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:22.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430845+0000", 2024-09-18T04:56:22.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8036999999999999e-05 2024-09-18T04:56:22.719 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.719 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:22.719 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.719 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:32 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:56:22.719 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.464367+0000", 2024-09-18T04:56:22.719 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.243729638, 2024-09-18T04:56:22.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0067846240000000004, 2024-09-18T04:56:22.719 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:22.719 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:22.719 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:22.719 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:56:22.720 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:56:22.720 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 32 2024-09-18T04:56:22.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.720 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:22.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:22.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464367+0000", 2024-09-18T04:56:22.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:22.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464367+0000", 2024-09-18T04:56:22.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:22.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464366+0000", 2024-09-18T04:56:22.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:22.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:22.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464425+0000", 2024-09-18T04:56:22.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8925999999999998e-05 2024-09-18T04:56:22.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:22.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464434+0000", 2024-09-18T04:56:22.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2209999999999997e-06 2024-09-18T04:56:22.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:22.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464439+0000", 2024-09-18T04:56:22.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2870000000000004e-06 2024-09-18T04:56:22.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:22.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.465167+0000", 2024-09-18T04:56:22.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00072805999999999999 2024-09-18T04:56:22.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:22.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.465257+0000", 2024-09-18T04:56:22.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0000999999999994e-05 2024-09-18T04:56:22.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:22.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.465403+0000", 2024-09-18T04:56:22.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000145709 2024-09-18T04:56:22.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:22.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.467688+0000", 2024-09-18T04:56:22.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002285048 2024-09-18T04:56:22.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:22.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.471084+0000", 2024-09-18T04:56:22.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033959289999999998 2024-09-18T04:56:22.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:22.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.471113+0000", 2024-09-18T04:56:22.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8969e-05 2024-09-18T04:56:22.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:22.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.471152+0000", 2024-09-18T04:56:22.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8983999999999998e-05 2024-09-18T04:56:22.725 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.725 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:22.725 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.725 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:33 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:56:22.725 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.464450+0000", 2024-09-18T04:56:22.726 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.243646720999999, 2024-09-18T04:56:22.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.006768128, 2024-09-18T04:56:22.726 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:22.726 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:22.726 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:22.726 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:56:22.726 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:56:22.726 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 33 2024-09-18T04:56:22.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.726 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:22.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:22.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464450+0000", 2024-09-18T04:56:22.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:22.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464450+0000", 2024-09-18T04:56:22.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:22.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464450+0000", 2024-09-18T04:56:22.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:22.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:22.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464452+0000", 2024-09-18T04:56:22.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6070000000000002e-06 2024-09-18T04:56:22.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:22.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464456+0000", 2024-09-18T04:56:22.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3309999999999997e-06 2024-09-18T04:56:22.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:22.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464459+0000", 2024-09-18T04:56:22.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.903e-06 2024-09-18T04:56:22.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:22.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.465867+0000", 2024-09-18T04:56:22.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001407936 2024-09-18T04:56:22.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:22.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.465950+0000", 2024-09-18T04:56:22.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2310000000000003e-05 2024-09-18T04:56:22.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:22.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.466088+0000", 2024-09-18T04:56:22.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000138233 2024-09-18T04:56:22.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:22.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.467743+0000", 2024-09-18T04:56:22.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016549539999999999 2024-09-18T04:56:22.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:22.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.471158+0000", 2024-09-18T04:56:22.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034154150000000002 2024-09-18T04:56:22.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:22.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.471182+0000", 2024-09-18T04:56:22.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4000999999999999e-05 2024-09-18T04:56:22.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:22.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.471218+0000", 2024-09-18T04:56:22.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5836e-05 2024-09-18T04:56:22.731 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.731 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:22.731 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.731 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:34 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:56:22.732 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.464463+0000", 2024-09-18T04:56:22.732 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.243633933, 2024-09-18T04:56:22.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0068160529999999999, 2024-09-18T04:56:22.732 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:22.732 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:22.732 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:22.732 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:56:22.732 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:56:22.732 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 34 2024-09-18T04:56:22.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.732 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:22.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:22.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464463+0000", 2024-09-18T04:56:22.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:22.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464463+0000", 2024-09-18T04:56:22.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:22.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464462+0000", 2024-09-18T04:56:22.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:22.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:22.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464465+0000", 2024-09-18T04:56:22.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3099999999999999e-06 2024-09-18T04:56:22.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:22.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464468+0000", 2024-09-18T04:56:22.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5659999999999998e-06 2024-09-18T04:56:22.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:22.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464471+0000", 2024-09-18T04:56:22.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6390000000000001e-06 2024-09-18T04:56:22.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:22.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.466499+0000", 2024-09-18T04:56:22.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020283100000000002 2024-09-18T04:56:22.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:22.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.466584+0000", 2024-09-18T04:56:22.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4482999999999997e-05 2024-09-18T04:56:22.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:22.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.466717+0000", 2024-09-18T04:56:22.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013280200000000001 2024-09-18T04:56:22.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:22.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.468625+0000", 2024-09-18T04:56:22.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001908652 2024-09-18T04:56:22.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:22.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.471223+0000", 2024-09-18T04:56:22.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0025982560000000002 2024-09-18T04:56:22.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:22.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.471246+0000", 2024-09-18T04:56:22.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2631000000000001e-05 2024-09-18T04:56:22.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:22.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.471279+0000", 2024-09-18T04:56:22.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2639999999999999e-05 2024-09-18T04:56:22.737 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.737 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:22.737 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.738 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:35 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:56:22.738 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.464474+0000", 2024-09-18T04:56:22.738 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.243622853, 2024-09-18T04:56:22.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0068681310000000004, 2024-09-18T04:56:22.738 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:22.738 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:22.738 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:22.738 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:56:22.738 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:56:22.738 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 35 2024-09-18T04:56:22.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.738 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:22.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:22.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464474+0000", 2024-09-18T04:56:22.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:22.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464474+0000", 2024-09-18T04:56:22.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:22.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464474+0000", 2024-09-18T04:56:22.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:22.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:22.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464526+0000", 2024-09-18T04:56:22.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2602000000000002e-05 2024-09-18T04:56:22.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:22.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464532+0000", 2024-09-18T04:56:22.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6489999999999996e-06 2024-09-18T04:56:22.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:22.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464535+0000", 2024-09-18T04:56:22.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.208e-06 2024-09-18T04:56:22.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:22.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.467003+0000", 2024-09-18T04:56:22.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002467534 2024-09-18T04:56:22.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:22.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.467085+0000", 2024-09-18T04:56:22.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2262000000000006e-05 2024-09-18T04:56:22.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:22.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.467221+0000", 2024-09-18T04:56:22.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000135852 2024-09-18T04:56:22.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:22.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.468662+0000", 2024-09-18T04:56:22.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0014411579999999999 2024-09-18T04:56:22.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:22.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.471284+0000", 2024-09-18T04:56:22.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002622161 2024-09-18T04:56:22.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:22.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.471307+0000", 2024-09-18T04:56:22.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2830999999999999e-05 2024-09-18T04:56:22.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:22.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.471342+0000", 2024-09-18T04:56:22.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5083000000000003e-05 2024-09-18T04:56:22.743 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.743 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:22.743 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.744 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:36 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:56:22.744 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.464540+0000", 2024-09-18T04:56:22.744 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.243557117, 2024-09-18T04:56:22.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0068630690000000003, 2024-09-18T04:56:22.744 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:22.744 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:22.744 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:22.744 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:56:22.744 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:56:22.744 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 36 2024-09-18T04:56:22.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.744 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:22.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:22.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464540+0000", 2024-09-18T04:56:22.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:22.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464540+0000", 2024-09-18T04:56:22.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:22.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464539+0000", 2024-09-18T04:56:22.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:22.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:22.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464542+0000", 2024-09-18T04:56:22.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8320000000000002e-06 2024-09-18T04:56:22.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:22.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464546+0000", 2024-09-18T04:56:22.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6349999999999999e-06 2024-09-18T04:56:22.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:22.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464548+0000", 2024-09-18T04:56:22.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.79e-06 2024-09-18T04:56:22.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:22.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.467875+0000", 2024-09-18T04:56:22.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033261089999999998 2024-09-18T04:56:22.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:22.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.467944+0000", 2024-09-18T04:56:22.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9883999999999997e-05 2024-09-18T04:56:22.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:22.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.468084+0000", 2024-09-18T04:56:22.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013939099999999999 2024-09-18T04:56:22.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:22.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.468937+0000", 2024-09-18T04:56:22.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00085285999999999999 2024-09-18T04:56:22.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:22.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.471347+0000", 2024-09-18T04:56:22.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024105120000000001 2024-09-18T04:56:22.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:22.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.471370+0000", 2024-09-18T04:56:22.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2361000000000002e-05 2024-09-18T04:56:22.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:22.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.471403+0000", 2024-09-18T04:56:22.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3030000000000001e-05 2024-09-18T04:56:22.749 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.749 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:22.749 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.750 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:37 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:56:22.750 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.464552+0000", 2024-09-18T04:56:22.750 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.243545022999999, 2024-09-18T04:56:22.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0069146779999999996, 2024-09-18T04:56:22.750 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:22.750 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:22.750 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:22.750 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:56:22.750 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:56:22.750 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 37 2024-09-18T04:56:22.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.750 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:22.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:22.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464552+0000", 2024-09-18T04:56:22.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:22.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464552+0000", 2024-09-18T04:56:22.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:22.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464551+0000", 2024-09-18T04:56:22.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:22.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:22.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464554+0000", 2024-09-18T04:56:22.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0639999999999998e-06 2024-09-18T04:56:22.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:22.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464559+0000", 2024-09-18T04:56:22.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3259999999999997e-06 2024-09-18T04:56:22.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:22.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.464562+0000", 2024-09-18T04:56:22.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4460000000000001e-06 2024-09-18T04:56:22.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:22.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.468326+0000", 2024-09-18T04:56:22.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003763396 2024-09-18T04:56:22.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:22.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.468376+0000", 2024-09-18T04:56:22.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0757000000000002e-05 2024-09-18T04:56:22.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:22.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.468459+0000", 2024-09-18T04:56:22.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2694999999999993e-05 2024-09-18T04:56:22.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:22.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.469202+0000", 2024-09-18T04:56:22.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00074246999999999996 2024-09-18T04:56:22.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:22.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.471408+0000", 2024-09-18T04:56:22.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022062789999999998 2024-09-18T04:56:22.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:22.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.471431+0000", 2024-09-18T04:56:22.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3169000000000001e-05 2024-09-18T04:56:22.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:22.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.471466+0000", 2024-09-18T04:56:22.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5293999999999998e-05 2024-09-18T04:56:22.755 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.755 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:22.755 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.756 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:56:22.756 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.650006+0000", 2024-09-18T04:56:22.756 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.058090396000001, 2024-09-18T04:56:22.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.006927669, 2024-09-18T04:56:22.756 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:22.756 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:22.756 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:22.756 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:56:22.756 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:56:22.756 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 116 2024-09-18T04:56:22.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.757 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:22.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:22.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.650006+0000", 2024-09-18T04:56:22.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:22.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.650006+0000", 2024-09-18T04:56:22.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:22.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.650006+0000", 2024-09-18T04:56:22.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:22.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:22.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.650010+0000", 2024-09-18T04:56:22.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1860000000000002e-06 2024-09-18T04:56:22.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:22.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.650016+0000", 2024-09-18T04:56:22.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9410000000000004e-06 2024-09-18T04:56:22.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:22.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.650020+0000", 2024-09-18T04:56:22.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.515e-06 2024-09-18T04:56:22.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:22.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.655373+0000", 2024-09-18T04:56:22.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0053530770000000004 2024-09-18T04:56:22.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:22.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.655422+0000", 2024-09-18T04:56:22.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9011000000000002e-05 2024-09-18T04:56:22.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:22.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.655496+0000", 2024-09-18T04:56:22.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3362999999999995e-05 2024-09-18T04:56:22.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:22.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.656272+0000", 2024-09-18T04:56:22.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00077605600000000001 2024-09-18T04:56:22.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:22.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.656860+0000", 2024-09-18T04:56:22.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00058852600000000002 2024-09-18T04:56:22.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:22.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.656895+0000", 2024-09-18T04:56:22.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4839000000000003e-05 2024-09-18T04:56:22.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:22.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.656934+0000", 2024-09-18T04:56:22.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8689999999999997e-05 2024-09-18T04:56:22.762 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.762 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:22.762 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.762 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:56:22.762 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.650025+0000", 2024-09-18T04:56:22.762 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 11.058071209, 2024-09-18T04:56:22.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0077445789999999997, 2024-09-18T04:56:22.762 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:22.762 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:22.762 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:22.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:56:22.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:56:22.763 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117 2024-09-18T04:56:22.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.763 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:22.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:22.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.650025+0000", 2024-09-18T04:56:22.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:22.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.650025+0000", 2024-09-18T04:56:22.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:22.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.650025+0000", 2024-09-18T04:56:22.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:22.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:22.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.650030+0000", 2024-09-18T04:56:22.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4060000000000004e-06 2024-09-18T04:56:22.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:22.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.650038+0000", 2024-09-18T04:56:22.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9580000000000007e-06 2024-09-18T04:56:22.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:22.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.650043+0000", 2024-09-18T04:56:22.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8319999999999996e-06 2024-09-18T04:56:22.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:22.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.655932+0000", 2024-09-18T04:56:22.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0058888170000000002 2024-09-18T04:56:22.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:22.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.655984+0000", 2024-09-18T04:56:22.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1909000000000001e-05 2024-09-18T04:56:22.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:22.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.656075+0000", 2024-09-18T04:56:22.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0898999999999998e-05 2024-09-18T04:56:22.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:22.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.656698+0000", 2024-09-18T04:56:22.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00062278500000000005 2024-09-18T04:56:22.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:22.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.657695+0000", 2024-09-18T04:56:22.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00099742199999999998 2024-09-18T04:56:22.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:22.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.657730+0000", 2024-09-18T04:56:22.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4999999999999997e-05 2024-09-18T04:56:22.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:22.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.657770+0000", 2024-09-18T04:56:22.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9941999999999998e-05 2024-09-18T04:56:22.768 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.768 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:22.768 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.768 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:56:22.768 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:12.152599+0000", 2024-09-18T04:56:22.768 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.555497633, 2024-09-18T04:56:22.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0079008499999999992, 2024-09-18T04:56:22.768 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:22.768 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:22.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:22.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:56:22.769 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:56:22.769 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 137 2024-09-18T04:56:22.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.769 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:22.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:22.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152599+0000", 2024-09-18T04:56:22.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:22.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152599+0000", 2024-09-18T04:56:22.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:22.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152599+0000", 2024-09-18T04:56:22.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:22.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:22.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152641+0000", 2024-09-18T04:56:22.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2490000000000001e-05 2024-09-18T04:56:22.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:22.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152648+0000", 2024-09-18T04:56:22.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8789999999999997e-06 2024-09-18T04:56:22.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:22.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152652+0000", 2024-09-18T04:56:22.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.031e-06 2024-09-18T04:56:22.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:22.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.157119+0000", 2024-09-18T04:56:22.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0044668579999999998 2024-09-18T04:56:22.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:22.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.157166+0000", 2024-09-18T04:56:22.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6962000000000002e-05 2024-09-18T04:56:22.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:22.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.157245+0000", 2024-09-18T04:56:22.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9167999999999998e-05 2024-09-18T04:56:22.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:22.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.158706+0000", 2024-09-18T04:56:22.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001461045 2024-09-18T04:56:22.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:22.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.160448+0000", 2024-09-18T04:56:22.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001742272 2024-09-18T04:56:22.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:22.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.160476+0000", 2024-09-18T04:56:22.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7976000000000001e-05 2024-09-18T04:56:22.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:22.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.160500+0000", 2024-09-18T04:56:22.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3621e-05 2024-09-18T04:56:22.774 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.774 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:22.774 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.774 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:56:22.774 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:12.152657+0000", 2024-09-18T04:56:22.774 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.555439771, 2024-09-18T04:56:22.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0079748749999999993, 2024-09-18T04:56:22.775 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:22.775 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:22.775 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:22.775 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:56:22.775 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:56:22.775 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 138 2024-09-18T04:56:22.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.775 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:22.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:22.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152657+0000", 2024-09-18T04:56:22.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:22.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152657+0000", 2024-09-18T04:56:22.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:22.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152656+0000", 2024-09-18T04:56:22.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:22.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:22.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152660+0000", 2024-09-18T04:56:22.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.721e-06 2024-09-18T04:56:22.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:22.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152664+0000", 2024-09-18T04:56:22.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8759999999999998e-06 2024-09-18T04:56:22.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:22.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152667+0000", 2024-09-18T04:56:22.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.948e-06 2024-09-18T04:56:22.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:22.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.158048+0000", 2024-09-18T04:56:22.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.005380485 2024-09-18T04:56:22.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:22.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.158137+0000", 2024-09-18T04:56:22.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9958999999999997e-05 2024-09-18T04:56:22.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:22.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.158268+0000", 2024-09-18T04:56:22.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013034599999999999 2024-09-18T04:56:22.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:22.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.160442+0000", 2024-09-18T04:56:22.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002174652 2024-09-18T04:56:22.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:22.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.160587+0000", 2024-09-18T04:56:22.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014425399999999999 2024-09-18T04:56:22.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:22.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.160610+0000", 2024-09-18T04:56:22.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3048999999999999e-05 2024-09-18T04:56:22.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:22.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.160632+0000", 2024-09-18T04:56:22.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1970000000000001e-05 2024-09-18T04:56:22.780 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.780 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:22.780 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.780 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:56:22.780 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:12.152670+0000", 2024-09-18T04:56:22.780 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.555426801999999, 2024-09-18T04:56:22.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0085554020000000001, 2024-09-18T04:56:22.781 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:22.781 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:22.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:22.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:56:22.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:56:22.781 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 139 2024-09-18T04:56:22.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.781 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:22.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:22.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152670+0000", 2024-09-18T04:56:22.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:22.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152670+0000", 2024-09-18T04:56:22.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:22.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152670+0000", 2024-09-18T04:56:22.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:22.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:22.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152672+0000", 2024-09-18T04:56:22.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1490000000000001e-06 2024-09-18T04:56:22.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:22.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152677+0000", 2024-09-18T04:56:22.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2129999999999999e-06 2024-09-18T04:56:22.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:22.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152680+0000", 2024-09-18T04:56:22.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7379999999999999e-06 2024-09-18T04:56:22.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:22.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.158724+0000", 2024-09-18T04:56:22.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0060442270000000001 2024-09-18T04:56:22.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:22.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.158773+0000", 2024-09-18T04:56:22.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8791000000000001e-05 2024-09-18T04:56:22.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:22.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.158851+0000", 2024-09-18T04:56:22.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8566999999999995e-05 2024-09-18T04:56:22.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:22.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.160546+0000", 2024-09-18T04:56:22.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016949389999999999 2024-09-18T04:56:22.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:22.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.161180+0000", 2024-09-18T04:56:22.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00063336800000000004 2024-09-18T04:56:22.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:22.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.161205+0000", 2024-09-18T04:56:22.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5843999999999999e-05 2024-09-18T04:56:22.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:22.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.161225+0000", 2024-09-18T04:56:22.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9792999999999998e-05 2024-09-18T04:56:22.786 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.786 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:22.786 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.786 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:56:22.786 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:12.152686+0000", 2024-09-18T04:56:22.786 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.555410804999999, 2024-09-18T04:56:22.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0087926110000000005, 2024-09-18T04:56:22.787 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:22.787 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:22.787 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:22.787 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:56:22.787 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:56:22.787 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 140 2024-09-18T04:56:22.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.787 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:22.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:22.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152686+0000", 2024-09-18T04:56:22.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:22.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152686+0000", 2024-09-18T04:56:22.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:22.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152686+0000", 2024-09-18T04:56:22.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:22.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:22.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152727+0000", 2024-09-18T04:56:22.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1767999999999997e-05 2024-09-18T04:56:22.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:22.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152734+0000", 2024-09-18T04:56:22.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1809999999999998e-06 2024-09-18T04:56:22.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:22.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152739+0000", 2024-09-18T04:56:22.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0799999999999996e-06 2024-09-18T04:56:22.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:22.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.159157+0000", 2024-09-18T04:56:22.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0064182989999999997 2024-09-18T04:56:22.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:22.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.159239+0000", 2024-09-18T04:56:22.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1842999999999996e-05 2024-09-18T04:56:22.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:22.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.159369+0000", 2024-09-18T04:56:22.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000129927 2024-09-18T04:56:22.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:22.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.160582+0000", 2024-09-18T04:56:22.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0012130960000000001 2024-09-18T04:56:22.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:22.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.161437+0000", 2024-09-18T04:56:22.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00085517099999999995 2024-09-18T04:56:22.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:22.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.161460+0000", 2024-09-18T04:56:22.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.321e-05 2024-09-18T04:56:22.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:22.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.161478+0000", 2024-09-18T04:56:22.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8298999999999999e-05 2024-09-18T04:56:22.792 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.792 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:22.792 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.792 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:56:22.792 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:12.152746+0000", 2024-09-18T04:56:22.792 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.555351136000001, 2024-09-18T04:56:22.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0090239710000000004, 2024-09-18T04:56:22.793 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:22.793 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:22.793 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:22.793 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:56:22.793 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:56:22.793 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 141 2024-09-18T04:56:22.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.793 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:22.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:22.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152746+0000", 2024-09-18T04:56:22.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:22.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152746+0000", 2024-09-18T04:56:22.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:22.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152745+0000", 2024-09-18T04:56:22.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:22.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:22.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152749+0000", 2024-09-18T04:56:22.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.189e-06 2024-09-18T04:56:22.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:22.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152754+0000", 2024-09-18T04:56:22.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.42e-06 2024-09-18T04:56:22.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:22.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.152757+0000", 2024-09-18T04:56:22.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8489999999999999e-06 2024-09-18T04:56:22.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:22.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.160660+0000", 2024-09-18T04:56:22.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0079022650000000003 2024-09-18T04:56:22.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:22.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.160756+0000", 2024-09-18T04:56:22.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6186000000000004e-05 2024-09-18T04:56:22.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:22.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.160883+0000", 2024-09-18T04:56:22.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012761399999999999 2024-09-18T04:56:22.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:22.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.161664+0000", 2024-09-18T04:56:22.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00078091399999999996 2024-09-18T04:56:22.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:22.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.161728+0000", 2024-09-18T04:56:22.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3403000000000003e-05 2024-09-18T04:56:22.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:22.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.161751+0000", 2024-09-18T04:56:22.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3204e-05 2024-09-18T04:56:22.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:22.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.161769+0000", 2024-09-18T04:56:22.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8459000000000001e-05 2024-09-18T04:56:22.798 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.798 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:22.798 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.798 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4223.0:7 1.2 1:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T04:56:22.798 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:12.390266+0000", 2024-09-18T04:56:22.798 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.317830239999999, 2024-09-18T04:56:22.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0091548989999999993, 2024-09-18T04:56:22.799 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:22.799 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:22.799 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:22.799 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4223", 2024-09-18T04:56:22.799 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/880685202", 2024-09-18T04:56:22.799 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 7 2024-09-18T04:56:22.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.799 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:22.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:22.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.390266+0000", 2024-09-18T04:56:22.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:22.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.390266+0000", 2024-09-18T04:56:22.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:22.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.390256+0000", 2024-09-18T04:56:22.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999989 2024-09-18T04:56:22.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:22.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.390307+0000", 2024-09-18T04:56:22.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1280999999999999e-05 2024-09-18T04:56:22.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:22.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.390359+0000", 2024-09-18T04:56:22.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.215e-05 2024-09-18T04:56:22.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:22.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.390395+0000", 2024-09-18T04:56:22.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5802e-05 2024-09-18T04:56:22.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:22.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.390503+0000", 2024-09-18T04:56:22.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108557 2024-09-18T04:56:22.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-18T04:56:22.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.390514+0000", 2024-09-18T04:56:22.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0558e-05 2024-09-18T04:56:22.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:22.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.398078+0000", 2024-09-18T04:56:22.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0075643029999999997 2024-09-18T04:56:22.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:22.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.398148+0000", 2024-09-18T04:56:22.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0191999999999995e-05 2024-09-18T04:56:22.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:22.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.398253+0000", 2024-09-18T04:56:22.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104868 2024-09-18T04:56:22.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:22.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.398782+0000", 2024-09-18T04:56:22.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00052911799999999997 2024-09-18T04:56:22.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:22.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.399367+0000", 2024-09-18T04:56:22.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00058480999999999995 2024-09-18T04:56:22.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:22.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.399395+0000", 2024-09-18T04:56:22.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8299000000000001e-05 2024-09-18T04:56:22.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:22.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.399421+0000", 2024-09-18T04:56:22.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5797000000000001e-05 2024-09-18T04:56:22.805 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.805 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:22.805 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.805 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:155 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T04:56:22.805 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:12.660788+0000", 2024-09-18T04:56:22.805 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.047308148000001, 2024-09-18T04:56:22.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0065468879999999998, 2024-09-18T04:56:22.805 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:22.805 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:22.806 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:22.806 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:56:22.806 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:56:22.806 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 155 2024-09-18T04:56:22.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.806 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:22.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:22.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660788+0000", 2024-09-18T04:56:22.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:22.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660788+0000", 2024-09-18T04:56:22.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:22.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660788+0000", 2024-09-18T04:56:22.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:22.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:22.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660845+0000", 2024-09-18T04:56:22.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7457999999999998e-05 2024-09-18T04:56:22.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:22.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660852+0000", 2024-09-18T04:56:22.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9070000000000001e-06 2024-09-18T04:56:22.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:22.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660857+0000", 2024-09-18T04:56:22.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1889999999999998e-06 2024-09-18T04:56:22.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:22.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.661719+0000", 2024-09-18T04:56:22.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00086197499999999996 2024-09-18T04:56:22.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:22.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.661762+0000", 2024-09-18T04:56:22.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2876000000000001e-05 2024-09-18T04:56:22.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:22.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.661834+0000", 2024-09-18T04:56:22.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1355999999999999e-05 2024-09-18T04:56:22.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:22.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.663221+0000", 2024-09-18T04:56:22.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013870340000000001 2024-09-18T04:56:22.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:22.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.667272+0000", 2024-09-18T04:56:22.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040511410000000003 2024-09-18T04:56:22.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:22.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.667303+0000", 2024-09-18T04:56:22.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0716000000000002e-05 2024-09-18T04:56:22.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:22.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.667335+0000", 2024-09-18T04:56:22.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2852000000000003e-05 2024-09-18T04:56:22.811 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.811 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:22.811 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.811 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:156 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T04:56:22.811 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:12.660862+0000", 2024-09-18T04:56:22.811 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.047234378000001, 2024-09-18T04:56:22.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0065665230000000003, 2024-09-18T04:56:22.812 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:22.812 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:22.812 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:22.812 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:56:22.812 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:56:22.812 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 156 2024-09-18T04:56:22.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.812 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:22.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:22.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660862+0000", 2024-09-18T04:56:22.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:22.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660862+0000", 2024-09-18T04:56:22.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:22.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660862+0000", 2024-09-18T04:56:22.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:22.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:22.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660864+0000", 2024-09-18T04:56:22.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3860000000000001e-06 2024-09-18T04:56:22.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:22.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660869+0000", 2024-09-18T04:56:22.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3630000000000004e-06 2024-09-18T04:56:22.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:22.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660871+0000", 2024-09-18T04:56:22.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5349999999999999e-06 2024-09-18T04:56:22.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:22.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.662182+0000", 2024-09-18T04:56:22.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001310681 2024-09-18T04:56:22.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:22.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.662260+0000", 2024-09-18T04:56:22.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8101000000000003e-05 2024-09-18T04:56:22.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:22.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.662392+0000", 2024-09-18T04:56:22.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013190599999999999 2024-09-18T04:56:22.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:22.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.663257+0000", 2024-09-18T04:56:22.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00086536700000000002 2024-09-18T04:56:22.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:22.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.667345+0000", 2024-09-18T04:56:22.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040881509999999999 2024-09-18T04:56:22.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:22.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.667384+0000", 2024-09-18T04:56:22.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8383999999999997e-05 2024-09-18T04:56:22.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:22.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.667429+0000", 2024-09-18T04:56:22.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5065000000000003e-05 2024-09-18T04:56:22.817 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.817 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:22.817 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.818 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:157 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T04:56:22.818 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:12.660876+0000", 2024-09-18T04:56:22.818 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.04722078, 2024-09-18T04:56:22.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0066112369999999998, 2024-09-18T04:56:22.818 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:22.818 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:22.818 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:22.818 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:56:22.818 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:56:22.818 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 157 2024-09-18T04:56:22.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.818 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:22.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:22.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660876+0000", 2024-09-18T04:56:22.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:22.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660876+0000", 2024-09-18T04:56:22.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:22.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660876+0000", 2024-09-18T04:56:22.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:22.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:22.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660879+0000", 2024-09-18T04:56:22.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5439999999999999e-06 2024-09-18T04:56:22.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:22.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660884+0000", 2024-09-18T04:56:22.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1399999999999999e-06 2024-09-18T04:56:22.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:22.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660887+0000", 2024-09-18T04:56:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6759999999999999e-06 2024-09-18T04:56:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.662856+0000", 2024-09-18T04:56:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0019685829999999999 2024-09-18T04:56:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.662903+0000", 2024-09-18T04:56:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7385000000000001e-05 2024-09-18T04:56:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:22.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.662978+0000", 2024-09-18T04:56:22.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5075999999999997e-05 2024-09-18T04:56:22.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:22.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.664345+0000", 2024-09-18T04:56:22.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001366751 2024-09-18T04:56:22.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:22.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.667433+0000", 2024-09-18T04:56:22.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030880289999999999 2024-09-18T04:56:22.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:22.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.667456+0000", 2024-09-18T04:56:22.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2785e-05 2024-09-18T04:56:22.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:22.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.667487+0000", 2024-09-18T04:56:22.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1510000000000002e-05 2024-09-18T04:56:22.824 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.824 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:22.824 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.824 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:158 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T04:56:22.824 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:12.660890+0000", 2024-09-18T04:56:22.824 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.047206595, 2024-09-18T04:56:22.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0066560350000000002, 2024-09-18T04:56:22.824 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:22.824 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:22.824 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:22.824 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:56:22.824 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:56:22.825 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 158 2024-09-18T04:56:22.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.825 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:22.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:22.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660890+0000", 2024-09-18T04:56:22.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:22.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660890+0000", 2024-09-18T04:56:22.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:22.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660890+0000", 2024-09-18T04:56:22.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:22.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:22.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660925+0000", 2024-09-18T04:56:22.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5439999999999999e-05 2024-09-18T04:56:22.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:22.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660929+0000", 2024-09-18T04:56:22.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2069999999999997e-06 2024-09-18T04:56:22.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:22.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660932+0000", 2024-09-18T04:56:22.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7350000000000001e-06 2024-09-18T04:56:22.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:22.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.663288+0000", 2024-09-18T04:56:22.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0023553189999999998 2024-09-18T04:56:22.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:22.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.663368+0000", 2024-09-18T04:56:22.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0036000000000004e-05 2024-09-18T04:56:22.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:22.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.663496+0000", 2024-09-18T04:56:22.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012820799999999999 2024-09-18T04:56:22.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:22.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.664489+0000", 2024-09-18T04:56:22.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00099358900000000002 2024-09-18T04:56:22.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:22.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.667492+0000", 2024-09-18T04:56:22.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030026900000000001 2024-09-18T04:56:22.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:22.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.667514+0000", 2024-09-18T04:56:22.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1962000000000001e-05 2024-09-18T04:56:22.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:22.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.667546+0000", 2024-09-18T04:56:22.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2064000000000003e-05 2024-09-18T04:56:22.830 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.830 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:22.830 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.830 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-09-18T04:56:22.830 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:12.660936+0000", 2024-09-18T04:56:22.830 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.047160869000001, 2024-09-18T04:56:22.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0066701080000000001, 2024-09-18T04:56:22.830 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:22.830 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:22.830 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:22.830 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:56:22.831 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:56:22.831 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 159 2024-09-18T04:56:22.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.831 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:22.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:22.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660936+0000", 2024-09-18T04:56:22.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:22.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660936+0000", 2024-09-18T04:56:22.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:22.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:22.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660935+0000", 2024-09-18T04:56:22.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:22.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:22.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660938+0000", 2024-09-18T04:56:22.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2989999999999999e-06 2024-09-18T04:56:22.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:22.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660941+0000", 2024-09-18T04:56:22.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.501e-06 2024-09-18T04:56:22.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:22.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.660945+0000", 2024-09-18T04:56:22.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5379999999999998e-06 2024-09-18T04:56:22.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:22.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.663982+0000", 2024-09-18T04:56:22.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030377149999999999 2024-09-18T04:56:22.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:22.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.664030+0000", 2024-09-18T04:56:22.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7985000000000002e-05 2024-09-18T04:56:22.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:22.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.664104+0000", 2024-09-18T04:56:22.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3669000000000002e-05 2024-09-18T04:56:22.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:22.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.664738+0000", 2024-09-18T04:56:22.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00063424 2024-09-18T04:56:22.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:22.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.667551+0000", 2024-09-18T04:56:22.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002812607 2024-09-18T04:56:22.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:22.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.667573+0000", 2024-09-18T04:56:22.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2476999999999998e-05 2024-09-18T04:56:22.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:22.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:22.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:22.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:12.667606+0000", 2024-09-18T04:56:22.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2397999999999997e-05 2024-09-18T04:56:22.835 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.836 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:22.836 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.836 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:22.836 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:22.836 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:22.837 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:56:22.838 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:22.835+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:22.938 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:22.936+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:22.950 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:22.951 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:22.951 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:56:22.951 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:22.958 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:56:23.039 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:23.037+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:23.105 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:23.105 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:23.106 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:56:23.106 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:56:23.106 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:23.114 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:56:23.131 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-18T04:56:23.140 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:23.138+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:23.225 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:23.225 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:56:23.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:56:23.226 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:56:23.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.226 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4223.0:5 1.2 e16/14 1:5cea7035:::rbd_info:head v 16'1, mlcod=0'0)", 2024-09-18T04:56:23.226 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.387406+0000", 2024-09-18T04:56:23.226 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.836390668, 2024-09-18T04:56:23.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00089316000000000005, 2024-09-18T04:56:23.226 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:23.227 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:23.227 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:23.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:23.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.387406+0000", 2024-09-18T04:56:23.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:23.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.387406+0000", 2024-09-18T04:56:23.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:23.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.387405+0000", 2024-09-18T04:56:23.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:56:23.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:23.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.387418+0000", 2024-09-18T04:56:23.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2371999999999999e-05 2024-09-18T04:56:23.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:23.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.387437+0000", 2024-09-18T04:56:23.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9429999999999999e-05 2024-09-18T04:56:23.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:23.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.387446+0000", 2024-09-18T04:56:23.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0610000000000002e-06 2024-09-18T04:56:23.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:23.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.387498+0000", 2024-09-18T04:56:23.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1459000000000003e-05 2024-09-18T04:56:23.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:23.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.387529+0000", 2024-09-18T04:56:23.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1393999999999998e-05 2024-09-18T04:56:23.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:23.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.388253+0000", 2024-09-18T04:56:23.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00072375399999999996 2024-09-18T04:56:23.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:23.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.388299+0000", 2024-09-18T04:56:23.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5893999999999998e-05 2024-09-18T04:56:23.232 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.232 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:23.232 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.232 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4223.0:7 1.2 e17/14 1:5cea7035:::rbd_info:head v 17'2, mlcod=0'0)", 2024-09-18T04:56:23.233 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:12.398460+0000", 2024-09-18T04:56:23.233 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.825336262, 2024-09-18T04:56:23.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00081256500000000001, 2024-09-18T04:56:23.233 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:23.233 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:23.233 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:23.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:23.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.398460+0000", 2024-09-18T04:56:23.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:23.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.398460+0000", 2024-09-18T04:56:23.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:23.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.398460+0000", 2024-09-18T04:56:23.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:23.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:23.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.398471+0000", 2024-09-18T04:56:23.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1593000000000001e-05 2024-09-18T04:56:23.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:23.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.398479+0000", 2024-09-18T04:56:23.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9470000000000003e-06 2024-09-18T04:56:23.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:23.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.398495+0000", 2024-09-18T04:56:23.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5729000000000001e-05 2024-09-18T04:56:23.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:23.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.398551+0000", 2024-09-18T04:56:23.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5668999999999998e-05 2024-09-18T04:56:23.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:23.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.398566+0000", 2024-09-18T04:56:23.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5502000000000002e-05 2024-09-18T04:56:23.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:23.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.399241+0000", 2024-09-18T04:56:23.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067506100000000002 2024-09-18T04:56:23.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:23.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.399273+0000", 2024-09-18T04:56:23.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1304999999999999e-05 2024-09-18T04:56:23.238 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.238 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:23.238 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.238 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.239 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:23.239 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:23.240 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:56:23.241 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:23.239+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:23.342 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:23.340+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:23.347 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:23.347 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:23.347 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:56:23.347 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:23.356 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:56:23.443 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:23.442+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:23.462 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:23.462 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:23.462 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:56:23.462 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:56:23.463 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:23.471 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:56:23.544 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:23.542+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:23.579 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:23.579 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:56:23.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:56:23.579 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:56:23.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.579 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4223.0:1 1.3 1:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T04:56:23.580 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:10.382846+0000", 2024-09-18T04:56:23.580 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.193152026, 2024-09-18T04:56:23.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024488256999999999, 2024-09-18T04:56:23.580 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:23.580 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:23.580 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:23.580 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4223", 2024-09-18T04:56:23.580 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/880685202", 2024-09-18T04:56:23.580 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1 2024-09-18T04:56:23.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.580 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:23.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:23.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.382846+0000", 2024-09-18T04:56:23.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:23.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.382846+0000", 2024-09-18T04:56:23.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:23.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.382845+0000", 2024-09-18T04:56:23.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:23.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:23.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.382859+0000", 2024-09-18T04:56:23.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3929e-05 2024-09-18T04:56:23.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:23.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.382869+0000", 2024-09-18T04:56:23.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0190000000000001e-05 2024-09-18T04:56:23.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:23.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.382878+0000", 2024-09-18T04:56:23.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3649999999999993e-06 2024-09-18T04:56:23.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:23.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.382933+0000", 2024-09-18T04:56:23.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4802999999999997e-05 2024-09-18T04:56:23.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-18T04:56:23.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.382945+0000", 2024-09-18T04:56:23.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2469999999999999e-05 2024-09-18T04:56:23.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:23.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.401898+0000", 2024-09-18T04:56:23.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01895264 2024-09-18T04:56:23.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:23.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.402061+0000", 2024-09-18T04:56:23.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016263700000000001 2024-09-18T04:56:23.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-18T04:56:23.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.402206+0000", 2024-09-18T04:56:23.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014509200000000001 2024-09-18T04:56:23.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:23.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.402740+0000", 2024-09-18T04:56:23.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00053426199999999997 2024-09-18T04:56:23.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:23.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.407266+0000", 2024-09-18T04:56:23.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045261640000000001 2024-09-18T04:56:23.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:23.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.407295+0000", 2024-09-18T04:56:23.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8707000000000001e-05 2024-09-18T04:56:23.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:23.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.407335+0000", 2024-09-18T04:56:23.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9552999999999999e-05 2024-09-18T04:56:23.586 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.586 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:23.586 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.586 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:20 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'10, mlcod=16'8)", 2024-09-18T04:56:23.586 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.450524+0000", 2024-09-18T04:56:23.586 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 12.125474322000001, 2024-09-18T04:56:23.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00105641, 2024-09-18T04:56:23.587 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:23.587 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:23.587 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:23.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:23.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.450524+0000", 2024-09-18T04:56:23.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:23.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.450524+0000", 2024-09-18T04:56:23.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:23.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.450523+0000", 2024-09-18T04:56:23.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:23.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:23.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.450646+0000", 2024-09-18T04:56:23.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012245499999999999 2024-09-18T04:56:23.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:23.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.450658+0000", 2024-09-18T04:56:23.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1946e-05 2024-09-18T04:56:23.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:23.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.450663+0000", 2024-09-18T04:56:23.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6470000000000003e-06 2024-09-18T04:56:23.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:23.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.450701+0000", 2024-09-18T04:56:23.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7401000000000003e-05 2024-09-18T04:56:23.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:23.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.450722+0000", 2024-09-18T04:56:23.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1387999999999999e-05 2024-09-18T04:56:23.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:23.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.451546+0000", 2024-09-18T04:56:23.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00082379899999999995 2024-09-18T04:56:23.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:23.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.451580+0000", 2024-09-18T04:56:23.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4382999999999999e-05 2024-09-18T04:56:23.590 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.590 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:23.590 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.591 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:31 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-09-18T04:56:23.591 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.464964+0000", 2024-09-18T04:56:23.591 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 12.11103434, 2024-09-18T04:56:23.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010824389999999999, 2024-09-18T04:56:23.591 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:23.591 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:23.591 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:23.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:23.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.464964+0000", 2024-09-18T04:56:23.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:23.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.464964+0000", 2024-09-18T04:56:23.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:23.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.464963+0000", 2024-09-18T04:56:23.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:23.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:23.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.464970+0000", 2024-09-18T04:56:23.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.302e-06 2024-09-18T04:56:23.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:23.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.464979+0000", 2024-09-18T04:56:23.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2839999999999995e-06 2024-09-18T04:56:23.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:23.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.464985+0000", 2024-09-18T04:56:23.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8989999999999998e-06 2024-09-18T04:56:23.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:23.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.465022+0000", 2024-09-18T04:56:23.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7342000000000003e-05 2024-09-18T04:56:23.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:23.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.465042+0000", 2024-09-18T04:56:23.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9905999999999999e-05 2024-09-18T04:56:23.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:23.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466015+0000", 2024-09-18T04:56:23.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00097294000000000005 2024-09-18T04:56:23.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:23.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466046+0000", 2024-09-18T04:56:23.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1404e-05 2024-09-18T04:56:23.595 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.595 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:23.595 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.595 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:32 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-09-18T04:56:23.595 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.465488+0000", 2024-09-18T04:56:23.595 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 12.11051031, 2024-09-18T04:56:23.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001051197, 2024-09-18T04:56:23.595 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:23.595 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:23.596 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:23.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:23.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.465488+0000", 2024-09-18T04:56:23.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:23.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.465488+0000", 2024-09-18T04:56:23.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:23.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.465484+0000", 2024-09-18T04:56:23.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999957 2024-09-18T04:56:23.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:23.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.465559+0000", 2024-09-18T04:56:23.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5561000000000004e-05 2024-09-18T04:56:23.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:23.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.465571+0000", 2024-09-18T04:56:23.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2051e-05 2024-09-18T04:56:23.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:23.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.465576+0000", 2024-09-18T04:56:23.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0340000000000004e-06 2024-09-18T04:56:23.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:23.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.465614+0000", 2024-09-18T04:56:23.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7206999999999998e-05 2024-09-18T04:56:23.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:23.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.465635+0000", 2024-09-18T04:56:23.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1532999999999999e-05 2024-09-18T04:56:23.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:23.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466511+0000", 2024-09-18T04:56:23.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00087624800000000004 2024-09-18T04:56:23.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:23.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466539+0000", 2024-09-18T04:56:23.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7909999999999999e-05 2024-09-18T04:56:23.599 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.599 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:23.599 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.600 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:33 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'19, mlcod=16'14)", 2024-09-18T04:56:23.600 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.466169+0000", 2024-09-18T04:56:23.600 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 12.109829063999999, 2024-09-18T04:56:23.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000967938, 2024-09-18T04:56:23.600 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:23.600 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:23.600 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:23.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:23.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466169+0000", 2024-09-18T04:56:23.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:23.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466169+0000", 2024-09-18T04:56:23.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:23.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466169+0000", 2024-09-18T04:56:23.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:23.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:23.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466173+0000", 2024-09-18T04:56:23.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9110000000000003e-06 2024-09-18T04:56:23.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:23.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466182+0000", 2024-09-18T04:56:23.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.234e-06 2024-09-18T04:56:23.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:23.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466193+0000", 2024-09-18T04:56:23.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1289e-05 2024-09-18T04:56:23.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:23.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466239+0000", 2024-09-18T04:56:23.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5618999999999998e-05 2024-09-18T04:56:23.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:23.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466260+0000", 2024-09-18T04:56:23.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1339e-05 2024-09-18T04:56:23.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:23.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.467109+0000", 2024-09-18T04:56:23.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00084875700000000001 2024-09-18T04:56:23.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:23.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.467137+0000", 2024-09-18T04:56:23.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8248000000000001e-05 2024-09-18T04:56:23.604 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.604 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:23.604 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.604 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:34 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'14)", 2024-09-18T04:56:23.604 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.466788+0000", 2024-09-18T04:56:23.604 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 12.109210641000001, 2024-09-18T04:56:23.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010779489999999999, 2024-09-18T04:56:23.605 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:23.605 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:23.605 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:23.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:23.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466788+0000", 2024-09-18T04:56:23.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:23.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466788+0000", 2024-09-18T04:56:23.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:23.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466787+0000", 2024-09-18T04:56:23.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:23.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:23.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466792+0000", 2024-09-18T04:56:23.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9400000000000001e-06 2024-09-18T04:56:23.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:23.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466801+0000", 2024-09-18T04:56:23.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8610000000000004e-06 2024-09-18T04:56:23.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:23.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466805+0000", 2024-09-18T04:56:23.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2799999999999997e-06 2024-09-18T04:56:23.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:23.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466844+0000", 2024-09-18T04:56:23.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8902000000000001e-05 2024-09-18T04:56:23.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:23.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466865+0000", 2024-09-18T04:56:23.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1305999999999999e-05 2024-09-18T04:56:23.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:23.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.467835+0000", 2024-09-18T04:56:23.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00096979900000000003 2024-09-18T04:56:23.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:23.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.467866+0000", 2024-09-18T04:56:23.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0380999999999999e-05 2024-09-18T04:56:23.608 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.608 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:23.608 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.609 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:35 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'14)", 2024-09-18T04:56:23.609 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.467320+0000", 2024-09-18T04:56:23.609 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 12.108678632, 2024-09-18T04:56:23.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001134941, 2024-09-18T04:56:23.609 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:23.609 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:23.609 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:23.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:23.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.467320+0000", 2024-09-18T04:56:23.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:23.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.467320+0000", 2024-09-18T04:56:23.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:23.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.467319+0000", 2024-09-18T04:56:23.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:23.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:23.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.467377+0000", 2024-09-18T04:56:23.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7450000000000001e-05 2024-09-18T04:56:23.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:23.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.467386+0000", 2024-09-18T04:56:23.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3009999999999995e-06 2024-09-18T04:56:23.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:23.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.467390+0000", 2024-09-18T04:56:23.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.754e-06 2024-09-18T04:56:23.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:23.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.467424+0000", 2024-09-18T04:56:23.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4740999999999998e-05 2024-09-18T04:56:23.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:23.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.467443+0000", 2024-09-18T04:56:23.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8150000000000001e-05 2024-09-18T04:56:23.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:23.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.468417+0000", 2024-09-18T04:56:23.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00097475999999999997 2024-09-18T04:56:23.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:23.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.468455+0000", 2024-09-18T04:56:23.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7221999999999997e-05 2024-09-18T04:56:23.613 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.613 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:23.613 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.613 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:38 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'24, mlcod=16'22)", 2024-09-18T04:56:23.613 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.472058+0000", 2024-09-18T04:56:23.613 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 12.103940333000001, 2024-09-18T04:56:23.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00092610000000000001, 2024-09-18T04:56:23.614 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:23.614 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:23.614 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:23.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:23.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.472058+0000", 2024-09-18T04:56:23.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:23.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.472058+0000", 2024-09-18T04:56:23.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:23.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.472057+0000", 2024-09-18T04:56:23.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:23.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:23.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.472112+0000", 2024-09-18T04:56:23.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4645999999999998e-05 2024-09-18T04:56:23.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:23.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.472124+0000", 2024-09-18T04:56:23.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1904e-05 2024-09-18T04:56:23.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:23.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.472129+0000", 2024-09-18T04:56:23.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3859999999999998e-06 2024-09-18T04:56:23.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:23.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.472160+0000", 2024-09-18T04:56:23.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0443e-05 2024-09-18T04:56:23.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:23.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.472172+0000", 2024-09-18T04:56:23.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2856e-05 2024-09-18T04:56:23.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:23.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.472943+0000", 2024-09-18T04:56:23.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00077032000000000001 2024-09-18T04:56:23.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:23.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.472984+0000", 2024-09-18T04:56:23.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1307000000000003e-05 2024-09-18T04:56:23.617 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.617 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:23.618 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.618 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:54 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'32, mlcod=16'29)", 2024-09-18T04:56:23.618 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.487754+0000", 2024-09-18T04:56:23.618 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 12.088244777, 2024-09-18T04:56:23.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010036960000000001, 2024-09-18T04:56:23.618 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:23.618 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:23.618 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:23.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:23.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.487754+0000", 2024-09-18T04:56:23.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:23.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.487754+0000", 2024-09-18T04:56:23.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:23.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.487753+0000", 2024-09-18T04:56:23.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:23.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:23.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.487759+0000", 2024-09-18T04:56:23.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8710000000000002e-06 2024-09-18T04:56:23.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:23.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.487778+0000", 2024-09-18T04:56:23.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9749e-05 2024-09-18T04:56:23.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:23.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.487786+0000", 2024-09-18T04:56:23.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3309999999999998e-06 2024-09-18T04:56:23.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:23.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.487826+0000", 2024-09-18T04:56:23.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0632e-05 2024-09-18T04:56:23.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:23.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.487844+0000", 2024-09-18T04:56:23.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.753e-05 2024-09-18T04:56:23.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:23.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.488728+0000", 2024-09-18T04:56:23.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00088410799999999999 2024-09-18T04:56:23.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:23.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.488757+0000", 2024-09-18T04:56:23.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9178000000000001e-05 2024-09-18T04:56:23.622 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.622 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:23.622 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.622 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:55 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'33, mlcod=16'29)", 2024-09-18T04:56:23.622 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.488264+0000", 2024-09-18T04:56:23.622 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 12.087734730999999, 2024-09-18T04:56:23.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00095882300000000003, 2024-09-18T04:56:23.622 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:23.623 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:23.623 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:23.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:23.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.488264+0000", 2024-09-18T04:56:23.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:23.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.488264+0000", 2024-09-18T04:56:23.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:23.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.488263+0000", 2024-09-18T04:56:23.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:23.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:23.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.488339+0000", 2024-09-18T04:56:23.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5770000000000001e-05 2024-09-18T04:56:23.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:23.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.488348+0000", 2024-09-18T04:56:23.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8640000000000008e-06 2024-09-18T04:56:23.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:23.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.488354+0000", 2024-09-18T04:56:23.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9050000000000002e-06 2024-09-18T04:56:23.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:23.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.488393+0000", 2024-09-18T04:56:23.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8899000000000001e-05 2024-09-18T04:56:23.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:23.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.488413+0000", 2024-09-18T04:56:23.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9865e-05 2024-09-18T04:56:23.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:23.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.489175+0000", 2024-09-18T04:56:23.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076201700000000003 2024-09-18T04:56:23.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:23.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.489222+0000", 2024-09-18T04:56:23.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7247999999999999e-05 2024-09-18T04:56:23.627 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.627 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:23.627 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.627 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:57 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'35, mlcod=16'33)", 2024-09-18T04:56:23.628 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.490221+0000", 2024-09-18T04:56:23.628 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 12.085777732, 2024-09-18T04:56:23.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00094564200000000001, 2024-09-18T04:56:23.628 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:23.628 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:23.628 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:23.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:23.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.490221+0000", 2024-09-18T04:56:23.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:23.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.490221+0000", 2024-09-18T04:56:23.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:23.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.490220+0000", 2024-09-18T04:56:23.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:23.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:23.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.490289+0000", 2024-09-18T04:56:23.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8696999999999996e-05 2024-09-18T04:56:23.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:23.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.490299+0000", 2024-09-18T04:56:23.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9259999999999995e-06 2024-09-18T04:56:23.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:23.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.490304+0000", 2024-09-18T04:56:23.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8649999999999998e-06 2024-09-18T04:56:23.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:23.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.490328+0000", 2024-09-18T04:56:23.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3740999999999999e-05 2024-09-18T04:56:23.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:23.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.490348+0000", 2024-09-18T04:56:23.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9843e-05 2024-09-18T04:56:23.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:23.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.491134+0000", 2024-09-18T04:56:23.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00078551000000000001 2024-09-18T04:56:23.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:23.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.491166+0000", 2024-09-18T04:56:23.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2669999999999997e-05 2024-09-18T04:56:23.632 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.633 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:23.633 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.633 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:68 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-09-18T04:56:23.633 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.498459+0000", 2024-09-18T04:56:23.633 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 12.077539098999999, 2024-09-18T04:56:23.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00097680500000000008, 2024-09-18T04:56:23.633 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:23.633 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:23.633 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:23.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:23.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.498459+0000", 2024-09-18T04:56:23.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:23.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.498459+0000", 2024-09-18T04:56:23.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:23.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.498458+0000", 2024-09-18T04:56:23.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:23.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:23.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.498466+0000", 2024-09-18T04:56:23.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.841e-06 2024-09-18T04:56:23.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:23.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.498478+0000", 2024-09-18T04:56:23.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1734e-05 2024-09-18T04:56:23.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:23.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.498485+0000", 2024-09-18T04:56:23.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6560000000000003e-06 2024-09-18T04:56:23.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:23.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.498522+0000", 2024-09-18T04:56:23.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7465999999999997e-05 2024-09-18T04:56:23.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:23.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.498535+0000", 2024-09-18T04:56:23.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2537999999999999e-05 2024-09-18T04:56:23.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:23.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.499414+0000", 2024-09-18T04:56:23.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00087986100000000001 2024-09-18T04:56:23.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:23.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.499436+0000", 2024-09-18T04:56:23.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1554000000000001e-05 2024-09-18T04:56:23.637 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.637 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:23.637 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.638 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:72 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'46, mlcod=16'43)", 2024-09-18T04:56:23.638 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.501149+0000", 2024-09-18T04:56:23.638 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 12.074849082, 2024-09-18T04:56:23.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001105642, 2024-09-18T04:56:23.638 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:23.638 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:23.638 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:23.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:23.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.501149+0000", 2024-09-18T04:56:23.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:23.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.501149+0000", 2024-09-18T04:56:23.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:23.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.501149+0000", 2024-09-18T04:56:23.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:23.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:23.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.501215+0000", 2024-09-18T04:56:23.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6395000000000004e-05 2024-09-18T04:56:23.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:23.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.501234+0000", 2024-09-18T04:56:23.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8689999999999999e-05 2024-09-18T04:56:23.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:23.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.501241+0000", 2024-09-18T04:56:23.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2490000000000004e-06 2024-09-18T04:56:23.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:23.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.501273+0000", 2024-09-18T04:56:23.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1640999999999997e-05 2024-09-18T04:56:23.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:23.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.501301+0000", 2024-09-18T04:56:23.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8076e-05 2024-09-18T04:56:23.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:23.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.502222+0000", 2024-09-18T04:56:23.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00092150200000000004 2024-09-18T04:56:23.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:23.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.502255+0000", 2024-09-18T04:56:23.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2586999999999998e-05 2024-09-18T04:56:23.642 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.642 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:23.642 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.642 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:86 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'56, mlcod=16'52)", 2024-09-18T04:56:23.642 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.509513+0000", 2024-09-18T04:56:23.642 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 12.066485432, 2024-09-18T04:56:23.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00087199399999999996, 2024-09-18T04:56:23.643 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:23.643 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:23.643 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:23.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:23.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.509513+0000", 2024-09-18T04:56:23.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:23.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.509513+0000", 2024-09-18T04:56:23.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:23.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.509512+0000", 2024-09-18T04:56:23.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:23.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:23.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.509517+0000", 2024-09-18T04:56:23.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2540000000000003e-06 2024-09-18T04:56:23.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:23.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.509526+0000", 2024-09-18T04:56:23.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0580000000000004e-06 2024-09-18T04:56:23.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:23.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.509532+0000", 2024-09-18T04:56:23.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1029999999999998e-06 2024-09-18T04:56:23.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:23.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.509559+0000", 2024-09-18T04:56:23.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6758e-05 2024-09-18T04:56:23.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:23.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.509574+0000", 2024-09-18T04:56:23.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5268e-05 2024-09-18T04:56:23.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:23.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.510361+0000", 2024-09-18T04:56:23.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00078748200000000005 2024-09-18T04:56:23.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:23.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.510385+0000", 2024-09-18T04:56:23.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3555000000000001e-05 2024-09-18T04:56:23.646 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.646 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:23.647 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.647 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:87 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'57, mlcod=16'52)", 2024-09-18T04:56:23.647 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.509916+0000", 2024-09-18T04:56:23.647 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 12.066081909999999, 2024-09-18T04:56:23.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00093305799999999996, 2024-09-18T04:56:23.647 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:23.647 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:23.647 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:23.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:23.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.509916+0000", 2024-09-18T04:56:23.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:23.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.509916+0000", 2024-09-18T04:56:23.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:23.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.509916+0000", 2024-09-18T04:56:23.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:23.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:23.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.510003+0000", 2024-09-18T04:56:23.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7051000000000004e-05 2024-09-18T04:56:23.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:23.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.510011+0000", 2024-09-18T04:56:23.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5380000000000002e-06 2024-09-18T04:56:23.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:23.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.510017+0000", 2024-09-18T04:56:23.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5690000000000004e-06 2024-09-18T04:56:23.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:23.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.510040+0000", 2024-09-18T04:56:23.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3011999999999999e-05 2024-09-18T04:56:23.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:23.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.510050+0000", 2024-09-18T04:56:23.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9510000000000001e-06 2024-09-18T04:56:23.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:23.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.510826+0000", 2024-09-18T04:56:23.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00077587200000000004 2024-09-18T04:56:23.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:23.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.510849+0000", 2024-09-18T04:56:23.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3598999999999999e-05 2024-09-18T04:56:23.651 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.651 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:23.651 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.651 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:99 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'65, mlcod=16'63)", 2024-09-18T04:56:23.651 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.646928+0000", 2024-09-18T04:56:23.651 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.929070548, 2024-09-18T04:56:23.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00086536000000000002, 2024-09-18T04:56:23.652 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:23.652 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:23.652 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:23.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:23.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.646928+0000", 2024-09-18T04:56:23.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:23.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.646928+0000", 2024-09-18T04:56:23.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:23.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.646925+0000", 2024-09-18T04:56:23.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999971 2024-09-18T04:56:23.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:23.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.646944+0000", 2024-09-18T04:56:23.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8692999999999999e-05 2024-09-18T04:56:23.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:23.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.646976+0000", 2024-09-18T04:56:23.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2406000000000001e-05 2024-09-18T04:56:23.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:23.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.646993+0000", 2024-09-18T04:56:23.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7183000000000001e-05 2024-09-18T04:56:23.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:23.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.647069+0000", 2024-09-18T04:56:23.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5867999999999999e-05 2024-09-18T04:56:23.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:23.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.647095+0000", 2024-09-18T04:56:23.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5873000000000001e-05 2024-09-18T04:56:23.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:23.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.647752+0000", 2024-09-18T04:56:23.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00065679699999999998 2024-09-18T04:56:23.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:23.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.647793+0000", 2024-09-18T04:56:23.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1162999999999999e-05 2024-09-18T04:56:23.655 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.655 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:23.655 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.656 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:105 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'67, mlcod=16'64)", 2024-09-18T04:56:23.656 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.649887+0000", 2024-09-18T04:56:23.656 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.926111773000001, 2024-09-18T04:56:23.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00087080299999999996, 2024-09-18T04:56:23.656 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:23.656 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:23.656 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:23.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:23.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.649887+0000", 2024-09-18T04:56:23.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:23.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.649887+0000", 2024-09-18T04:56:23.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:23.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.649886+0000", 2024-09-18T04:56:23.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:23.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:23.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.649891+0000", 2024-09-18T04:56:23.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1540000000000001e-06 2024-09-18T04:56:23.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:23.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.649898+0000", 2024-09-18T04:56:23.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7530000000000004e-06 2024-09-18T04:56:23.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:23.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.649901+0000", 2024-09-18T04:56:23.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5370000000000002e-06 2024-09-18T04:56:23.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:23.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.649939+0000", 2024-09-18T04:56:23.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7511000000000001e-05 2024-09-18T04:56:23.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:23.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.649959+0000", 2024-09-18T04:56:23.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9704999999999999e-05 2024-09-18T04:56:23.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:23.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.650723+0000", 2024-09-18T04:56:23.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076422600000000001 2024-09-18T04:56:23.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:23.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.650757+0000", 2024-09-18T04:56:23.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4476e-05 2024-09-18T04:56:23.660 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.660 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:23.660 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.661 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:131 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-18T04:56:23.661 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:12.153368+0000", 2024-09-18T04:56:23.661 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.422630624, 2024-09-18T04:56:23.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00097627100000000002, 2024-09-18T04:56:23.661 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:23.661 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:23.662 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:23.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:23.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.153368+0000", 2024-09-18T04:56:23.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:23.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.153368+0000", 2024-09-18T04:56:23.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:23.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.153367+0000", 2024-09-18T04:56:23.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:23.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:23.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.153449+0000", 2024-09-18T04:56:23.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1291000000000005e-05 2024-09-18T04:56:23.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:23.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.153457+0000", 2024-09-18T04:56:23.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4190000000000002e-06 2024-09-18T04:56:23.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:23.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.153468+0000", 2024-09-18T04:56:23.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0949e-05 2024-09-18T04:56:23.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:23.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.153494+0000", 2024-09-18T04:56:23.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6047e-05 2024-09-18T04:56:23.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:23.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.153504+0000", 2024-09-18T04:56:23.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0560999999999999e-05 2024-09-18T04:56:23.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:23.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.154315+0000", 2024-09-18T04:56:23.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00081089499999999995 2024-09-18T04:56:23.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:23.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.154344+0000", 2024-09-18T04:56:23.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8560000000000001e-05 2024-09-18T04:56:23.665 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.665 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:23.665 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.666 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:142 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'100, mlcod=16'98)", 2024-09-18T04:56:23.666 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:12.162303+0000", 2024-09-18T04:56:23.666 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.413695062, 2024-09-18T04:56:23.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00091834900000000005, 2024-09-18T04:56:23.666 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:23.666 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:23.666 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:23.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:23.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.162303+0000", 2024-09-18T04:56:23.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:23.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.162303+0000", 2024-09-18T04:56:23.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:23.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.162303+0000", 2024-09-18T04:56:23.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:23.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:23.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.162356+0000", 2024-09-18T04:56:23.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3452000000000003e-05 2024-09-18T04:56:23.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:23.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.162367+0000", 2024-09-18T04:56:23.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.058e-05 2024-09-18T04:56:23.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:23.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.162372+0000", 2024-09-18T04:56:23.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3589999999999999e-06 2024-09-18T04:56:23.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:23.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.162415+0000", 2024-09-18T04:56:23.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3297000000000002e-05 2024-09-18T04:56:23.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:23.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.162428+0000", 2024-09-18T04:56:23.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3121e-05 2024-09-18T04:56:23.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:23.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.163193+0000", 2024-09-18T04:56:23.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076488999999999995 2024-09-18T04:56:23.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:23.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.163222+0000", 2024-09-18T04:56:23.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8311e-05 2024-09-18T04:56:23.670 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.670 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:23.671 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.671 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:9 3.1a e22/19 3:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-18T04:56:23.671 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:23.133017+0000", 2024-09-18T04:56:23.671 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.44298126399999999, 2024-09-18T04:56:23.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001015377, 2024-09-18T04:56:23.671 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:23.671 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:23.671 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:23.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:23.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:23.133017+0000", 2024-09-18T04:56:23.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:23.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:23.133017+0000", 2024-09-18T04:56:23.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:23.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:23.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:23.133017+0000", 2024-09-18T04:56:23.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:23.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:23.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:23.133027+0000", 2024-09-18T04:56:23.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0541e-05 2024-09-18T04:56:23.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:23.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:23.133050+0000", 2024-09-18T04:56:23.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2619000000000001e-05 2024-09-18T04:56:23.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:23.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:23.133061+0000", 2024-09-18T04:56:23.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1342000000000001e-05 2024-09-18T04:56:23.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:23.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:23.133107+0000", 2024-09-18T04:56:23.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6146000000000002e-05 2024-09-18T04:56:23.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:23.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:23.133127+0000", 2024-09-18T04:56:23.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.931e-05 2024-09-18T04:56:23.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:23.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:23.133991+0000", 2024-09-18T04:56:23.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00086470299999999998 2024-09-18T04:56:23.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:23.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:23.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:23.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:23.134032+0000", 2024-09-18T04:56:23.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0992000000000003e-05 2024-09-18T04:56:23.676 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.677 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:23.677 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.677 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:23.677 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:23.677 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:23.679 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:56:23.679 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:23.676+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:23.779 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:23.777+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:23.828 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:23.828 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:23.829 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:56:23.829 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:23.838 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:56:23.880 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:23.878+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:23.944 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:23.944 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:23.944 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:56:23.944 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:56:23.945 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:23.954 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:56:23.998 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:23.996+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:24.062 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:24.062 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:56:24.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:56:24.062 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [] 2024-09-18T04:56:24.062 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:24.071 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:56:24.081 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:24.080+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:24.115 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:56:24.183 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:24.181+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:24.219 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:24.220 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:24.220 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:56:24.220 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:24.229 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:56:24.281 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:56:24.281 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:56:24.283 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:24.281+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:24.298 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:56:24.298 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:56:24.331 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:56:24.331 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:56:24.352 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:56:24.353 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:56:24.353 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:24.353 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:24.362 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:56:24.370 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:56:24.370 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:56:24.384 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:24.380+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:24.387 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:56:24.388 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:56:24.405 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:56:24.405 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:56:24.422 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:56:24.422 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:56:24.470 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:24.471 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:24.479 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:56:24.520 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:24.517+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:24.586 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:24.583+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:24.594 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:24.594 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:24.604 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:56:24.611 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T04:56:24.611 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T04:56:24.611 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-18T04:56:24.611 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 0.05 2024-09-18T04:56:24.687 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:24.684+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:24.707 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:24.707 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:24.715 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:56:24.799 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:24.797+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:24.858 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:24.858 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:24.867 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:56:24.872 INFO:teuthology.orchestra.run.smithi002.stdout:skipping zero size read 2024-09-18T04:56:24.883 INFO:teuthology.orchestra.run.smithi002.stdout:2 write 0x3f2bcef thru 0x3f38ac6 (0xcdd8 bytes) 2024-09-18T04:56:24.887 INFO:teuthology.orchestra.run.smithi002.stdout:4 read 0x38f2e68 thru 0x38f7ba8 (0x4d41 bytes) 2024-09-18T04:56:24.888 INFO:teuthology.orchestra.run.smithi002.stdout:truncating to largest ever: 0x93ea2d7 2024-09-18T04:56:24.889 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:24.886+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:24.894 INFO:teuthology.orchestra.run.smithi002.stdout:5 trunc from 0x3f38ac7 to 0x93ea2d7 2024-09-18T04:56:24.901 INFO:teuthology.orchestra.run.smithi002.stdout:truncating to largest ever: 0xd269d1e 2024-09-18T04:56:24.902 INFO:teuthology.orchestra.run.smithi002.stdout:moved 0 / 82 (0%) 2024-09-18T04:56:24.902 INFO:teuthology.orchestra.run.smithi002.stdout:avg 10.25 2024-09-18T04:56:24.902 INFO:teuthology.orchestra.run.smithi002.stdout:stddev 3.56195 -> 3.56195 (expected baseline 2.99479) 2024-09-18T04:56:24.902 INFO:teuthology.orchestra.run.smithi002.stdout:min osd.2 with 4 -> 4 pgs (0.390244 -> 0.390244 * mean) 2024-09-18T04:56:24.902 INFO:teuthology.orchestra.run.smithi002.stdout:max osd.0 with 14 -> 14 pgs (1.36585 -> 1.36585 * mean) 2024-09-18T04:56:24.902 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:56:24.902 INFO:teuthology.orchestra.run.smithi002.stdout:oload 1000 2024-09-18T04:56:24.902 INFO:teuthology.orchestra.run.smithi002.stdout:max_change 0.05 2024-09-18T04:56:24.903 INFO:teuthology.orchestra.run.smithi002.stdout:max_change_osds 4 2024-09-18T04:56:24.903 INFO:teuthology.orchestra.run.smithi002.stdout:average_utilization 0.0003 2024-09-18T04:56:24.903 INFO:teuthology.orchestra.run.smithi002.stdout:overload_utilization 0.0031 2024-09-18T04:56:24.903 INFO:teuthology.orchestra.run.smithi002.stderr:no change 2024-09-18T04:56:24.905 INFO:teuthology.orchestra.run.smithi002.stdout:6 trunc from 0x93ea2d7 to 0xd269d1e 2024-09-18T04:56:24.913 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:24.913 INFO:teuthology.orchestra.run.smithi002.stdout:10 write 0xb30f44c thru 0xb31d637 (0xe1ec bytes) 2024-09-18T04:56:24.916 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-18T04:56:24.916 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2024-09-18T04:56:25.000 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:24.997+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:25.009 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:25.009 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:25.018 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:56:25.091 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:25.089+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:25.121 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:25.122 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:25.131 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:56:25.132 INFO:teuthology.orchestra.run.smithi002.stdout:12 trunc from 0xd269d1e to 0xcf45dd 2024-09-18T04:56:25.146 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T04:56:25.155 INFO:teuthology.orchestra.run.smithi002.stdout:13 write 0x4c24eb4 thru 0x4c27d54 (0x2ea1 bytes) 2024-09-18T04:56:25.158 INFO:teuthology.orchestra.run.smithi002.stdout:14 write 0x2920fcd thru 0x2925784 (0x47b8 bytes) 2024-09-18T04:56:25.173 INFO:teuthology.orchestra.run.smithi002.stdout:18 write 0x953f6da thru 0x9548d45 (0x966c bytes) 2024-09-18T04:56:25.185 INFO:teuthology.orchestra.run.smithi002.stdout:19 trunc from 0x9548d46 to 0x27e2487 2024-09-18T04:56:25.192 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:25.189+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:25.199 INFO:teuthology.orchestra.run.smithi002.stdout:20 clone 1 order 20 su 262144 sc 8 2024-09-18T04:56:25.248 INFO:teuthology.orchestra.run.smithi002.stdout:moved 0 / 82 (0%) 2024-09-18T04:56:25.248 INFO:teuthology.orchestra.run.smithi002.stdout:avg 10.25 2024-09-18T04:56:25.248 INFO:teuthology.orchestra.run.smithi002.stdout:stddev 3.56195 -> 3.56195 (expected baseline 2.99479) 2024-09-18T04:56:25.249 INFO:teuthology.orchestra.run.smithi002.stdout:min osd.2 with 4 -> 4 pgs (0.390244 -> 0.390244 * mean) 2024-09-18T04:56:25.249 INFO:teuthology.orchestra.run.smithi002.stdout:max osd.0 with 14 -> 14 pgs (1.36585 -> 1.36585 * mean) 2024-09-18T04:56:25.249 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:56:25.249 INFO:teuthology.orchestra.run.smithi002.stdout:oload 110 2024-09-18T04:56:25.249 INFO:teuthology.orchestra.run.smithi002.stdout:max_change 3 2024-09-18T04:56:25.249 INFO:teuthology.orchestra.run.smithi002.stdout:max_change_osds 4 2024-09-18T04:56:25.249 INFO:teuthology.orchestra.run.smithi002.stdout:average_utilization 0.0003 2024-09-18T04:56:25.249 INFO:teuthology.orchestra.run.smithi002.stdout:overload_utilization 0.0003 2024-09-18T04:56:25.249 INFO:teuthology.orchestra.run.smithi002.stderr:no change 2024-09-18T04:56:25.262 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-18T04:56:25.262 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2024-09-18T04:56:25.275 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:25.275 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:25.284 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:56:25.328 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:25.326+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:25.386 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:25.387 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:25.394 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:25.391+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:25.395 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:56:25.494 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:25.491+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:25.537 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:25.538 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:25.546 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:56:25.588 INFO:teuthology.orchestra.run.smithi002.stdout:moved 0 / 82 (0%) 2024-09-18T04:56:25.589 INFO:teuthology.orchestra.run.smithi002.stdout:avg 10.25 2024-09-18T04:56:25.589 INFO:teuthology.orchestra.run.smithi002.stdout:stddev 3.56195 -> 3.56195 (expected baseline 2.99479) 2024-09-18T04:56:25.589 INFO:teuthology.orchestra.run.smithi002.stdout:min osd.2 with 4 -> 4 pgs (0.390244 -> 0.390244 * mean) 2024-09-18T04:56:25.589 INFO:teuthology.orchestra.run.smithi002.stdout:max osd.0 with 14 -> 14 pgs (1.36585 -> 1.36585 * mean) 2024-09-18T04:56:25.589 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:56:25.589 INFO:teuthology.orchestra.run.smithi002.stdout:oload 110 2024-09-18T04:56:25.589 INFO:teuthology.orchestra.run.smithi002.stdout:max_change 3 2024-09-18T04:56:25.589 INFO:teuthology.orchestra.run.smithi002.stdout:max_change_osds 4 2024-09-18T04:56:25.590 INFO:teuthology.orchestra.run.smithi002.stdout:average_utilization 0.0003 2024-09-18T04:56:25.590 INFO:teuthology.orchestra.run.smithi002.stdout:overload_utilization 0.0003 2024-09-18T04:56:25.590 INFO:teuthology.orchestra.run.smithi002.stderr:no change 2024-09-18T04:56:25.595 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:25.593+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:25.601 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-18T04:56:25.601 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2024-09-18T04:56:25.648 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:25.648 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:25.656 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:56:25.696 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:25.693+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:25.797 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:25.794+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:25.799 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:25.799 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:25.807 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:56:25.898 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:25.895+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:25.910 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:25.910 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:25.919 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:56:25.927 INFO:teuthology.orchestra.run.smithi002.stdout:moved 0 / 82 (0%) 2024-09-18T04:56:25.927 INFO:teuthology.orchestra.run.smithi002.stdout:avg 10.25 2024-09-18T04:56:25.927 INFO:teuthology.orchestra.run.smithi002.stdout:stddev 3.56195 -> 3.56195 (expected baseline 2.99479) 2024-09-18T04:56:25.927 INFO:teuthology.orchestra.run.smithi002.stdout:min osd.2 with 4 -> 4 pgs (0.390244 -> 0.390244 * mean) 2024-09-18T04:56:25.927 INFO:teuthology.orchestra.run.smithi002.stdout:max osd.0 with 14 -> 14 pgs (1.36585 -> 1.36585 * mean) 2024-09-18T04:56:25.927 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:56:25.927 INFO:teuthology.orchestra.run.smithi002.stdout:oload 110 2024-09-18T04:56:25.928 INFO:teuthology.orchestra.run.smithi002.stdout:max_change 1 2024-09-18T04:56:25.928 INFO:teuthology.orchestra.run.smithi002.stdout:max_change_osds 4 2024-09-18T04:56:25.928 INFO:teuthology.orchestra.run.smithi002.stdout:average_utilization 0.0003 2024-09-18T04:56:25.928 INFO:teuthology.orchestra.run.smithi002.stdout:overload_utilization 0.0003 2024-09-18T04:56:25.928 INFO:teuthology.orchestra.run.smithi002.stderr:no change 2024-09-18T04:56:25.940 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-18T04:56:25.940 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2024-09-18T04:56:26.002 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:26.000+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:26.060 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:26.061 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:26.069 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:56:26.113 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:26.110+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:26.137 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-18T04:56:26.186 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:26.186 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:26.196 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:56:26.237 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:26.234+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:26.247 INFO:teuthology.orchestra.run.smithi002.stdout:22 write 0xecd2af3 thru 0xecd6079 (0x3587 bytes) 2024-09-18T04:56:26.264 INFO:teuthology.orchestra.run.smithi002.stdout:24 trunc from 0xecd607a to 0x266aafe 2024-09-18T04:56:26.285 INFO:teuthology.orchestra.run.smithi002.stdout:truncating to largest ever: 0xe9bfb60 2024-09-18T04:56:26.287 INFO:teuthology.orchestra.run.smithi002.stdout:moved 0 / 82 (0%) 2024-09-18T04:56:26.287 INFO:teuthology.orchestra.run.smithi002.stdout:avg 10.25 2024-09-18T04:56:26.288 INFO:teuthology.orchestra.run.smithi002.stdout:stddev 3.56195 -> 3.56195 (expected baseline 2.99479) 2024-09-18T04:56:26.288 INFO:teuthology.orchestra.run.smithi002.stdout:min osd.2 with 4 -> 4 pgs (0.390244 -> 0.390244 * mean) 2024-09-18T04:56:26.288 INFO:teuthology.orchestra.run.smithi002.stdout:max osd.0 with 14 -> 14 pgs (1.36585 -> 1.36585 * mean) 2024-09-18T04:56:26.288 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:56:26.288 INFO:teuthology.orchestra.run.smithi002.stdout:oload 110 2024-09-18T04:56:26.288 INFO:teuthology.orchestra.run.smithi002.stdout:max_change 3 2024-09-18T04:56:26.288 INFO:teuthology.orchestra.run.smithi002.stdout:max_change_osds 4 2024-09-18T04:56:26.288 INFO:teuthology.orchestra.run.smithi002.stdout:average_utilization 0.0003 2024-09-18T04:56:26.288 INFO:teuthology.orchestra.run.smithi002.stdout:overload_utilization 0.0003 2024-09-18T04:56:26.289 INFO:teuthology.orchestra.run.smithi002.stderr:no change 2024-09-18T04:56:26.299 INFO:teuthology.orchestra.run.smithi002.stdout:25 trunc from 0x266aafe to 0xe9bfb60 2024-09-18T04:56:26.300 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:26.298+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:26.304 INFO:teuthology.orchestra.run.smithi002.stdout:26 writesame 0x5eefd18 thru 0x5effbec data_size 0x1(0xfed5 bytes) 2024-09-18T04:56:26.336 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:26.336 INFO:teuthology.orchestra.run.smithi002.stdout:28 write 0x709c904 thru 0x70a98b3 (0xcfb0 bytes) 2024-09-18T04:56:26.339 INFO:teuthology.orchestra.run.smithi002.stdout:29 write 0xc8a0645 thru 0xc8b014f (0xfb0b bytes) 2024-09-18T04:56:26.342 INFO:teuthology.orchestra.run.smithi002.stdout:30 writesame 0x3cf61fc thru 0x3d0378e data_size 0x1b(0xd593 bytes) 2024-09-18T04:56:26.346 INFO:teuthology.orchestra.run.smithi002.stdout:31 read 0x2c0ccd2 thru 0x2c18766 (0xba95 bytes) 2024-09-18T04:56:26.346 INFO:teuthology.orchestra.run.smithi002.stdout:32 write 0x81f0fc6 thru 0x81f1699 (0x6d4 bytes) 2024-09-18T04:56:26.349 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:26.349 INFO:teuthology.orchestra.run.smithi002.stdout:36 write 0xadfe756 thru 0xae00184 (0x1a2f bytes) 2024-09-18T04:56:26.353 INFO:teuthology.orchestra.run.smithi002.stdout:37 write 0x7899c73 thru 0x78a2a56 (0x8de4 bytes) 2024-09-18T04:56:26.356 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:26.356 INFO:teuthology.orchestra.run.smithi002.stdout:39 write 0xe133a2a thru 0xe14382f (0xfe06 bytes) 2024-09-18T04:56:26.360 INFO:teuthology.orchestra.run.smithi002.stdout:41 read 0xab503da thru 0xab5a2c4 (0x9eeb bytes) 2024-09-18T04:56:26.367 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:26.367 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:26.369 INFO:teuthology.orchestra.run.smithi002.stdout:42 trunc from 0xe9bfb60 to 0x6ec1bec 2024-09-18T04:56:26.378 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:56:26.387 INFO:teuthology.orchestra.run.smithi002.stdout:43 trunc from 0x6ec1bec to 0x17db04c 2024-09-18T04:56:26.406 INFO:teuthology.orchestra.run.smithi002.stdout:44 read 0xe3a68 thru 0xec7ed (0x8d86 bytes) 2024-09-18T04:56:26.406 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:26.403+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:26.416 INFO:teuthology.orchestra.run.smithi002.stdout:46 write 0x5996670 thru 0x5999f2f (0x38c0 bytes) 2024-09-18T04:56:26.424 INFO:teuthology.orchestra.run.smithi002.stdout:47 write 0x69ab183 thru 0x69b0919 (0x5797 bytes) 2024-09-18T04:56:26.432 INFO:teuthology.orchestra.run.smithi002.stdout:48 trunc from 0x69b091a to 0xb2edced 2024-09-18T04:56:26.434 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:56:26.437 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:26.437 INFO:teuthology.orchestra.run.smithi002.stdout:50 read 0xa47e8cf thru 0xa489943 (0xb075 bytes) 2024-09-18T04:56:26.437 INFO:teuthology.orchestra.run.smithi002.stdout:52 write 0x6d53308 thru 0x6d57e2f (0x4b28 bytes) 2024-09-18T04:56:26.446 INFO:teuthology.orchestra.run.smithi002.stdout:53 trunc from 0xb2edced to 0x57867d7 2024-09-18T04:56:26.455 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:26.464 INFO:teuthology.orchestra.run.smithi002.stdout:56 write 0x89f3fcc thru 0x89f4e90 (0xec5 bytes) 2024-09-18T04:56:26.467 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:26.468 INFO:teuthology.orchestra.run.smithi002.stdout:58 writesame 0x7cdd6f8 thru 0x7cea47b data_size 0xe(0xcd84 bytes) 2024-09-18T04:56:26.471 INFO:teuthology.orchestra.run.smithi002.stdout:59 read 0x1c68d5d thru 0x1c6bbd0 (0x2e74 bytes) 2024-09-18T04:56:26.472 INFO:teuthology.orchestra.run.smithi002.stdout:60 write 0x7c2ab7a thru 0x7c34736 (0x9bbd bytes) 2024-09-18T04:56:26.478 INFO:teuthology.orchestra.run.smithi002.stdout:62 trunc from 0x89f4e91 to 0x61ffac0 2024-09-18T04:56:26.499 INFO:teuthology.orchestra.run.smithi002.stdout:63 trunc from 0x61ffac0 to 0xe68decc 2024-09-18T04:56:26.502 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:26.500+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:26.504 INFO:teuthology.orchestra.run.smithi002.stdout:64 writesame 0x85b009 thru 0x868d40 data_size 0x2(0xdd38 bytes) 2024-09-18T04:56:26.509 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:26.510 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:26.520 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:56:26.520 INFO:teuthology.orchestra.run.smithi002.stdout:65 read 0x21b7b1f thru 0x21bcee1 (0x53c3 bytes) 2024-09-18T04:56:26.520 INFO:teuthology.orchestra.run.smithi002.stdout:66 punch from 0x68e6aab to 0x68ea736, (0x3c8b bytes) 2024-09-18T04:56:26.529 INFO:teuthology.orchestra.run.smithi002.stdout:69 trunc from 0xe68decc to 0x61621dd 2024-09-18T04:56:26.535 INFO:teuthology.orchestra.run.smithi002.stdout:70 write 0x5795c9c thru 0x57a4007 (0xe36c bytes) 2024-09-18T04:56:26.538 INFO:teuthology.orchestra.run.smithi002.stdout:71 writesame 0x4921061 thru 0x492ce2f data_size 0x3(0xbdcf bytes) 2024-09-18T04:56:26.550 INFO:teuthology.orchestra.run.smithi002.stdout:72 read 0x5ffdfd4 thru 0x6008e0a (0xae37 bytes) 2024-09-18T04:56:26.550 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:26.550 INFO:teuthology.orchestra.run.smithi002.stdout:74 read 0x2145006 thru 0x214f333 (0xa32e bytes) 2024-09-18T04:56:26.550 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:26.555 INFO:teuthology.orchestra.run.smithi002.stdout:76 write 0x63daec3 thru 0x63e78c1 (0xc9ff bytes) 2024-09-18T04:56:26.560 INFO:teuthology.orchestra.run.smithi002.stdout:77 trunc from 0x63e78c2 to 0x7ed1384 2024-09-18T04:56:26.567 INFO:teuthology.orchestra.run.smithi002.stdout:78 trunc from 0x7ed1384 to 0x8de8f9e 2024-09-18T04:56:26.572 INFO:teuthology.orchestra.run.smithi002.stdout:79 punch from 0x4d327a3 to 0x4d32b1f, (0x37c bytes) 2024-09-18T04:56:26.574 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:56:26.574 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:56:26.578 INFO:teuthology.orchestra.run.smithi002.stdout:80 write 0x95bea68 thru 0x95c5047 (0x65e0 bytes) 2024-09-18T04:56:26.582 INFO:teuthology.orchestra.run.smithi002.stdout:81 write 0x5775fe4 thru 0x5778768 (0x2785 bytes) 2024-09-18T04:56:26.585 INFO:teuthology.orchestra.run.smithi002.stdout:82 write 0x79a80e3 thru 0x79b5a50 (0xd96e bytes) 2024-09-18T04:56:26.588 INFO:teuthology.orchestra.run.smithi002.stdout:85 read 0x6a4bf0d thru 0x6a5304d (0x7141 bytes) 2024-09-18T04:56:26.588 INFO:teuthology.orchestra.run.smithi002.stdout:86 punch from 0x76e20b9 to 0x76e3d05, (0x1c4c bytes) 2024-09-18T04:56:26.588 INFO:teuthology.orchestra.run.smithi002.stdout:87 read 0x2616428 thru 0x2622b25 (0xc6fe bytes) 2024-09-18T04:56:26.588 INFO:teuthology.orchestra.run.smithi002.stdout:88 punch from 0x243f060 to 0x2445397, (0x6337 bytes) 2024-09-18T04:56:26.596 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:56:26.596 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:56:26.597 INFO:teuthology.orchestra.run.smithi002.stdout:89 write 0xd2121f4 thru 0xd21cba5 (0xa9b2 bytes) 2024-09-18T04:56:26.601 INFO:teuthology.orchestra.run.smithi002.stdout:93 punch from 0x8dfdbaf to 0x8e080f9, (0xa54a bytes) 2024-09-18T04:56:26.601 INFO:teuthology.orchestra.run.smithi002.stdout:94 read 0x8e8f6a8 thru 0x8e97e71 (0x87ca bytes) 2024-09-18T04:56:26.601 INFO:teuthology.orchestra.run.smithi002.stdout:95 read 0x5534160 thru 0x553904c (0x4eed bytes) 2024-09-18T04:56:26.601 INFO:teuthology.orchestra.run.smithi002.stdout:96 read 0x28cd427 thru 0x28d63fb (0x8fd5 bytes) 2024-09-18T04:56:26.601 INFO:teuthology.orchestra.run.smithi002.stdout:97 read 0x1dc7c4e thru 0x1dd0ec0 (0x9273 bytes) 2024-09-18T04:56:26.601 INFO:teuthology.orchestra.run.smithi002.stdout:98 read 0xa32f6e7 thru 0xa33e37f (0xec99 bytes) 2024-09-18T04:56:26.601 INFO:teuthology.orchestra.run.smithi002.stdout:100 writesame 0x508ce7d thru 0x508d3e6 data_size 0x6(0x56a bytes) 2024-09-18T04:56:26.603 INFO:teuthology.orchestra.run.smithi002.stdout:101 punch from 0x2083bea to 0x208b0cc, (0x74e2 bytes) 2024-09-18T04:56:26.604 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:26.600+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:26.609 INFO:teuthology.orchestra.run.smithi002.stdout:103 trunc from 0xd21cba6 to 0x7dca8b3 2024-09-18T04:56:26.614 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:56:26.614 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:56:26.627 INFO:teuthology.orchestra.run.smithi002.stdout:104 write 0x805b62d thru 0x805bab8 (0x48c bytes) 2024-09-18T04:56:26.630 INFO:teuthology.orchestra.run.smithi002.stdout:105 read 0x5937763 thru 0x593b87e (0x411c bytes) 2024-09-18T04:56:26.630 INFO:teuthology.orchestra.run.smithi002.stdout:106 writesame 0x152619b thru 0x15332d4 data_size 0x6(0xd13a bytes) 2024-09-18T04:56:26.632 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:56:26.632 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:56:26.636 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:26.643 INFO:teuthology.orchestra.run.smithi002.stdout:108 write 0x9d878b0 thru 0x9d8c165 (0x48b6 bytes) 2024-09-18T04:56:26.646 INFO:teuthology.orchestra.run.smithi002.stdout:109 read 0x8375798 thru 0x83777cb (0x2034 bytes) 2024-09-18T04:56:26.646 INFO:teuthology.orchestra.run.smithi002.stdout:110 read 0x8f158e thru 0x90122a (0xfc9d bytes) 2024-09-18T04:56:26.646 INFO:teuthology.orchestra.run.smithi002.stdout:112 write 0x285271a thru 0x285cdf4 (0xa6db bytes) 2024-09-18T04:56:26.649 INFO:teuthology.orchestra.run.smithi002.stdout:113 clone 2 order 24 su 2097152 sc 12 2024-09-18T04:56:26.653 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:56:26.653 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:56:26.655 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:26.655 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:26.655 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:56:26.655 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:26.665 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:56:26.670 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:56:26.670 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:56:26.686 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:56:26.687 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:56:26.704 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:56:26.704 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:56:26.704 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:26.702+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:26.776 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:26.777 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:26.777 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:56:26.777 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:56:26.777 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:26.785 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:56:26.829 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:26.826+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:26.898 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:26.898 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:56:26.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:56:26.898 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:56:26.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.899 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4223.0:1 1.3 1:c4c92e5a:::rbd_trash:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-09-18T04:56:26.899 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:10.382846+0000", 2024-09-18T04:56:26.899 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.507208341999998, 2024-09-18T04:56:26.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024488256999999999, 2024-09-18T04:56:26.899 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:26.899 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:26.899 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:26.900 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4223", 2024-09-18T04:56:26.900 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/880685202", 2024-09-18T04:56:26.900 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1 2024-09-18T04:56:26.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.900 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:26.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:26.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.382846+0000", 2024-09-18T04:56:26.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:26.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.382846+0000", 2024-09-18T04:56:26.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:26.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.382845+0000", 2024-09-18T04:56:26.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:26.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:26.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.382859+0000", 2024-09-18T04:56:26.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3929e-05 2024-09-18T04:56:26.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:26.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.382869+0000", 2024-09-18T04:56:26.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0190000000000001e-05 2024-09-18T04:56:26.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:26.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.382878+0000", 2024-09-18T04:56:26.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3649999999999993e-06 2024-09-18T04:56:26.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:26.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.382933+0000", 2024-09-18T04:56:26.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4802999999999997e-05 2024-09-18T04:56:26.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-18T04:56:26.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.382945+0000", 2024-09-18T04:56:26.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2469999999999999e-05 2024-09-18T04:56:26.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:26.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.401898+0000", 2024-09-18T04:56:26.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01895264 2024-09-18T04:56:26.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:26.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.402061+0000", 2024-09-18T04:56:26.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016263700000000001 2024-09-18T04:56:26.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-18T04:56:26.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.402206+0000", 2024-09-18T04:56:26.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014509200000000001 2024-09-18T04:56:26.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:26.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.402740+0000", 2024-09-18T04:56:26.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00053426199999999997 2024-09-18T04:56:26.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:26.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.407266+0000", 2024-09-18T04:56:26.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0045261640000000001 2024-09-18T04:56:26.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:26.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.407295+0000", 2024-09-18T04:56:26.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8707000000000001e-05 2024-09-18T04:56:26.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:26.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:10.407335+0000", 2024-09-18T04:56:26.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9552999999999999e-05 2024-09-18T04:56:26.908 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.908 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:26.908 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.908 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:20 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'10, mlcod=16'8)", 2024-09-18T04:56:26.908 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.450524+0000", 2024-09-18T04:56:26.909 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.439530638000001, 2024-09-18T04:56:26.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00105641, 2024-09-18T04:56:26.909 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:26.909 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:26.909 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:26.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:26.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.450524+0000", 2024-09-18T04:56:26.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:26.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.450524+0000", 2024-09-18T04:56:26.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:26.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.450523+0000", 2024-09-18T04:56:26.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:26.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:26.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.450646+0000", 2024-09-18T04:56:26.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012245499999999999 2024-09-18T04:56:26.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:26.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.450658+0000", 2024-09-18T04:56:26.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1946e-05 2024-09-18T04:56:26.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:26.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.450663+0000", 2024-09-18T04:56:26.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6470000000000003e-06 2024-09-18T04:56:26.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:26.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.450701+0000", 2024-09-18T04:56:26.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7401000000000003e-05 2024-09-18T04:56:26.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:26.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.450722+0000", 2024-09-18T04:56:26.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1387999999999999e-05 2024-09-18T04:56:26.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:26.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.451546+0000", 2024-09-18T04:56:26.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00082379899999999995 2024-09-18T04:56:26.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:26.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.451580+0000", 2024-09-18T04:56:26.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4382999999999999e-05 2024-09-18T04:56:26.915 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.915 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:26.915 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.915 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:31 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-09-18T04:56:26.916 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.464964+0000", 2024-09-18T04:56:26.916 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.425090656, 2024-09-18T04:56:26.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010824389999999999, 2024-09-18T04:56:26.916 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:26.916 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:26.916 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:26.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:26.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.464964+0000", 2024-09-18T04:56:26.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:26.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.464964+0000", 2024-09-18T04:56:26.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:26.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.464963+0000", 2024-09-18T04:56:26.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:26.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:26.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.464970+0000", 2024-09-18T04:56:26.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.302e-06 2024-09-18T04:56:26.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:26.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.464979+0000", 2024-09-18T04:56:26.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2839999999999995e-06 2024-09-18T04:56:26.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:26.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.464985+0000", 2024-09-18T04:56:26.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8989999999999998e-06 2024-09-18T04:56:26.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:26.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.465022+0000", 2024-09-18T04:56:26.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7342000000000003e-05 2024-09-18T04:56:26.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:26.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.465042+0000", 2024-09-18T04:56:26.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9905999999999999e-05 2024-09-18T04:56:26.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:26.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466015+0000", 2024-09-18T04:56:26.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00097294000000000005 2024-09-18T04:56:26.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:26.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466046+0000", 2024-09-18T04:56:26.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1404e-05 2024-09-18T04:56:26.921 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.921 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:26.921 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.921 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:32 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-09-18T04:56:26.921 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.465488+0000", 2024-09-18T04:56:26.921 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.424566626000001, 2024-09-18T04:56:26.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001051197, 2024-09-18T04:56:26.921 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:26.921 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:26.921 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:26.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:26.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.465488+0000", 2024-09-18T04:56:26.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:26.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.465488+0000", 2024-09-18T04:56:26.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:26.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.465484+0000", 2024-09-18T04:56:26.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999957 2024-09-18T04:56:26.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:26.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.465559+0000", 2024-09-18T04:56:26.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5561000000000004e-05 2024-09-18T04:56:26.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:26.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.465571+0000", 2024-09-18T04:56:26.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2051e-05 2024-09-18T04:56:26.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:26.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.465576+0000", 2024-09-18T04:56:26.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0340000000000004e-06 2024-09-18T04:56:26.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:26.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.465614+0000", 2024-09-18T04:56:26.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7206999999999998e-05 2024-09-18T04:56:26.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:26.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.465635+0000", 2024-09-18T04:56:26.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1532999999999999e-05 2024-09-18T04:56:26.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:26.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466511+0000", 2024-09-18T04:56:26.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00087624800000000004 2024-09-18T04:56:26.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:26.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466539+0000", 2024-09-18T04:56:26.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7909999999999999e-05 2024-09-18T04:56:26.925 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.925 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:26.925 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.926 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:33 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'19, mlcod=16'14)", 2024-09-18T04:56:26.926 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.466169+0000", 2024-09-18T04:56:26.926 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.42388538, 2024-09-18T04:56:26.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000967938, 2024-09-18T04:56:26.926 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:26.926 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:26.926 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:26.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:26.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466169+0000", 2024-09-18T04:56:26.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:26.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466169+0000", 2024-09-18T04:56:26.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:26.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466169+0000", 2024-09-18T04:56:26.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:26.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:26.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466173+0000", 2024-09-18T04:56:26.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9110000000000003e-06 2024-09-18T04:56:26.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:26.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466182+0000", 2024-09-18T04:56:26.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.234e-06 2024-09-18T04:56:26.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:26.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466193+0000", 2024-09-18T04:56:26.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1289e-05 2024-09-18T04:56:26.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:26.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466239+0000", 2024-09-18T04:56:26.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5618999999999998e-05 2024-09-18T04:56:26.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:26.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466260+0000", 2024-09-18T04:56:26.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1339e-05 2024-09-18T04:56:26.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:26.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.467109+0000", 2024-09-18T04:56:26.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00084875700000000001 2024-09-18T04:56:26.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:26.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.467137+0000", 2024-09-18T04:56:26.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8248000000000001e-05 2024-09-18T04:56:26.930 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.930 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:26.930 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.930 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:34 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'20, mlcod=16'14)", 2024-09-18T04:56:26.930 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.466788+0000", 2024-09-18T04:56:26.930 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.423266956999999, 2024-09-18T04:56:26.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010779489999999999, 2024-09-18T04:56:26.930 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:26.930 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:26.930 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:26.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:26.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466788+0000", 2024-09-18T04:56:26.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:26.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466788+0000", 2024-09-18T04:56:26.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:26.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466787+0000", 2024-09-18T04:56:26.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:26.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:26.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466792+0000", 2024-09-18T04:56:26.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9400000000000001e-06 2024-09-18T04:56:26.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:26.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466801+0000", 2024-09-18T04:56:26.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8610000000000004e-06 2024-09-18T04:56:26.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:26.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466805+0000", 2024-09-18T04:56:26.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2799999999999997e-06 2024-09-18T04:56:26.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:26.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466844+0000", 2024-09-18T04:56:26.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8902000000000001e-05 2024-09-18T04:56:26.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:26.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.466865+0000", 2024-09-18T04:56:26.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1305999999999999e-05 2024-09-18T04:56:26.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:26.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.467835+0000", 2024-09-18T04:56:26.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00096979900000000003 2024-09-18T04:56:26.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:26.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.467866+0000", 2024-09-18T04:56:26.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0380999999999999e-05 2024-09-18T04:56:26.934 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.934 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:26.934 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.934 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:35 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'21, mlcod=16'14)", 2024-09-18T04:56:26.934 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.467320+0000", 2024-09-18T04:56:26.934 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.422734948, 2024-09-18T04:56:26.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001134941, 2024-09-18T04:56:26.935 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:26.935 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:26.935 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:26.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:26.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.467320+0000", 2024-09-18T04:56:26.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:26.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.467320+0000", 2024-09-18T04:56:26.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:26.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.467319+0000", 2024-09-18T04:56:26.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:26.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:26.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.467377+0000", 2024-09-18T04:56:26.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7450000000000001e-05 2024-09-18T04:56:26.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:26.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.467386+0000", 2024-09-18T04:56:26.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3009999999999995e-06 2024-09-18T04:56:26.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:26.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.467390+0000", 2024-09-18T04:56:26.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.754e-06 2024-09-18T04:56:26.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:26.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.467424+0000", 2024-09-18T04:56:26.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4740999999999998e-05 2024-09-18T04:56:26.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:26.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.467443+0000", 2024-09-18T04:56:26.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8150000000000001e-05 2024-09-18T04:56:26.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:26.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.468417+0000", 2024-09-18T04:56:26.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00097475999999999997 2024-09-18T04:56:26.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:26.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.468455+0000", 2024-09-18T04:56:26.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7221999999999997e-05 2024-09-18T04:56:26.939 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.939 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:26.939 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.940 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:38 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'24, mlcod=16'22)", 2024-09-18T04:56:26.940 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.472058+0000", 2024-09-18T04:56:26.940 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.417996648999999, 2024-09-18T04:56:26.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00092610000000000001, 2024-09-18T04:56:26.940 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:26.941 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:26.941 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:26.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:26.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.472058+0000", 2024-09-18T04:56:26.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:26.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.472058+0000", 2024-09-18T04:56:26.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:26.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.472057+0000", 2024-09-18T04:56:26.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:26.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:26.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.472112+0000", 2024-09-18T04:56:26.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4645999999999998e-05 2024-09-18T04:56:26.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:26.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.472124+0000", 2024-09-18T04:56:26.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1904e-05 2024-09-18T04:56:26.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:26.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.472129+0000", 2024-09-18T04:56:26.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3859999999999998e-06 2024-09-18T04:56:26.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:26.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.472160+0000", 2024-09-18T04:56:26.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0443e-05 2024-09-18T04:56:26.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:26.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.472172+0000", 2024-09-18T04:56:26.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2856e-05 2024-09-18T04:56:26.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:26.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.472943+0000", 2024-09-18T04:56:26.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00077032000000000001 2024-09-18T04:56:26.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:26.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.472984+0000", 2024-09-18T04:56:26.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1307000000000003e-05 2024-09-18T04:56:26.945 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.945 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:26.945 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.946 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:54 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'32, mlcod=16'29)", 2024-09-18T04:56:26.946 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.487754+0000", 2024-09-18T04:56:26.946 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.402301093, 2024-09-18T04:56:26.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010036960000000001, 2024-09-18T04:56:26.946 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:26.946 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:26.946 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:26.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:26.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.487754+0000", 2024-09-18T04:56:26.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:26.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.487754+0000", 2024-09-18T04:56:26.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:26.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.487753+0000", 2024-09-18T04:56:26.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:26.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:26.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.487759+0000", 2024-09-18T04:56:26.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8710000000000002e-06 2024-09-18T04:56:26.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:26.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.487778+0000", 2024-09-18T04:56:26.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9749e-05 2024-09-18T04:56:26.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:26.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.487786+0000", 2024-09-18T04:56:26.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3309999999999998e-06 2024-09-18T04:56:26.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:26.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.487826+0000", 2024-09-18T04:56:26.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0632e-05 2024-09-18T04:56:26.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:26.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.487844+0000", 2024-09-18T04:56:26.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.753e-05 2024-09-18T04:56:26.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:26.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.488728+0000", 2024-09-18T04:56:26.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00088410799999999999 2024-09-18T04:56:26.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:26.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.488757+0000", 2024-09-18T04:56:26.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9178000000000001e-05 2024-09-18T04:56:26.950 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.950 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:26.950 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.950 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:55 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'33, mlcod=16'29)", 2024-09-18T04:56:26.951 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.488264+0000", 2024-09-18T04:56:26.951 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.401791047, 2024-09-18T04:56:26.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00095882300000000003, 2024-09-18T04:56:26.951 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:26.951 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:26.951 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:26.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:26.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.488264+0000", 2024-09-18T04:56:26.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:26.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.488264+0000", 2024-09-18T04:56:26.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:26.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.488263+0000", 2024-09-18T04:56:26.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:26.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:26.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.488339+0000", 2024-09-18T04:56:26.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5770000000000001e-05 2024-09-18T04:56:26.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:26.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.488348+0000", 2024-09-18T04:56:26.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8640000000000008e-06 2024-09-18T04:56:26.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:26.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.488354+0000", 2024-09-18T04:56:26.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9050000000000002e-06 2024-09-18T04:56:26.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:26.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.488393+0000", 2024-09-18T04:56:26.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8899000000000001e-05 2024-09-18T04:56:26.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:26.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.488413+0000", 2024-09-18T04:56:26.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9865e-05 2024-09-18T04:56:26.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:26.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.489175+0000", 2024-09-18T04:56:26.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076201700000000003 2024-09-18T04:56:26.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:26.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.489222+0000", 2024-09-18T04:56:26.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7247999999999999e-05 2024-09-18T04:56:26.955 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.955 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:26.955 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.955 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:57 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'35, mlcod=16'33)", 2024-09-18T04:56:26.955 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.490221+0000", 2024-09-18T04:56:26.955 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.399834048000001, 2024-09-18T04:56:26.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00094564200000000001, 2024-09-18T04:56:26.955 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:26.955 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:26.955 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:26.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:26.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.490221+0000", 2024-09-18T04:56:26.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:26.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.490221+0000", 2024-09-18T04:56:26.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:26.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.490220+0000", 2024-09-18T04:56:26.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:26.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:26.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.490289+0000", 2024-09-18T04:56:26.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8696999999999996e-05 2024-09-18T04:56:26.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:26.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.490299+0000", 2024-09-18T04:56:26.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9259999999999995e-06 2024-09-18T04:56:26.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:26.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.490304+0000", 2024-09-18T04:56:26.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8649999999999998e-06 2024-09-18T04:56:26.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:26.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.490328+0000", 2024-09-18T04:56:26.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3740999999999999e-05 2024-09-18T04:56:26.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:26.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.490348+0000", 2024-09-18T04:56:26.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9843e-05 2024-09-18T04:56:26.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:26.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.491134+0000", 2024-09-18T04:56:26.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00078551000000000001 2024-09-18T04:56:26.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:26.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.491166+0000", 2024-09-18T04:56:26.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2669999999999997e-05 2024-09-18T04:56:26.961 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.961 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:26.961 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.962 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:68 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-09-18T04:56:26.962 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.498459+0000", 2024-09-18T04:56:26.962 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.391595414999999, 2024-09-18T04:56:26.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00097680500000000008, 2024-09-18T04:56:26.962 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:26.962 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:26.962 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:26.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:26.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.498459+0000", 2024-09-18T04:56:26.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:26.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.498459+0000", 2024-09-18T04:56:26.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:26.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.498458+0000", 2024-09-18T04:56:26.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:26.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:26.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.498466+0000", 2024-09-18T04:56:26.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.841e-06 2024-09-18T04:56:26.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:26.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.498478+0000", 2024-09-18T04:56:26.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1734e-05 2024-09-18T04:56:26.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:26.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.498485+0000", 2024-09-18T04:56:26.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6560000000000003e-06 2024-09-18T04:56:26.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:26.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.498522+0000", 2024-09-18T04:56:26.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7465999999999997e-05 2024-09-18T04:56:26.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:26.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.498535+0000", 2024-09-18T04:56:26.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2537999999999999e-05 2024-09-18T04:56:26.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:26.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.499414+0000", 2024-09-18T04:56:26.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00087986100000000001 2024-09-18T04:56:26.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:26.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.499436+0000", 2024-09-18T04:56:26.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1554000000000001e-05 2024-09-18T04:56:26.968 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.968 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:26.968 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.968 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:72 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'46, mlcod=16'43)", 2024-09-18T04:56:26.969 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.501149+0000", 2024-09-18T04:56:26.969 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.388905398, 2024-09-18T04:56:26.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001105642, 2024-09-18T04:56:26.969 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:26.969 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:26.969 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:26.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:26.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.501149+0000", 2024-09-18T04:56:26.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:26.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.501149+0000", 2024-09-18T04:56:26.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:26.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.501149+0000", 2024-09-18T04:56:26.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:26.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:26.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.501215+0000", 2024-09-18T04:56:26.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6395000000000004e-05 2024-09-18T04:56:26.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:26.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.501234+0000", 2024-09-18T04:56:26.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8689999999999999e-05 2024-09-18T04:56:26.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:26.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.501241+0000", 2024-09-18T04:56:26.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2490000000000004e-06 2024-09-18T04:56:26.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:26.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.501273+0000", 2024-09-18T04:56:26.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1640999999999997e-05 2024-09-18T04:56:26.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:26.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.501301+0000", 2024-09-18T04:56:26.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8076e-05 2024-09-18T04:56:26.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:26.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.502222+0000", 2024-09-18T04:56:26.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00092150200000000004 2024-09-18T04:56:26.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:26.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.502255+0000", 2024-09-18T04:56:26.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2586999999999998e-05 2024-09-18T04:56:26.973 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.973 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:26.973 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.973 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:87 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'57, mlcod=16'52)", 2024-09-18T04:56:26.973 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:11.509916+0000", 2024-09-18T04:56:26.973 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.380138226, 2024-09-18T04:56:26.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00093305799999999996, 2024-09-18T04:56:26.974 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:26.974 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:26.974 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:26.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:26.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.509916+0000", 2024-09-18T04:56:26.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:26.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.509916+0000", 2024-09-18T04:56:26.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:26.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.509916+0000", 2024-09-18T04:56:26.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:26.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:26.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.510003+0000", 2024-09-18T04:56:26.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7051000000000004e-05 2024-09-18T04:56:26.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:26.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.510011+0000", 2024-09-18T04:56:26.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5380000000000002e-06 2024-09-18T04:56:26.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:26.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.510017+0000", 2024-09-18T04:56:26.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5690000000000004e-06 2024-09-18T04:56:26.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:26.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.510040+0000", 2024-09-18T04:56:26.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3011999999999999e-05 2024-09-18T04:56:26.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:26.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.510050+0000", 2024-09-18T04:56:26.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9510000000000001e-06 2024-09-18T04:56:26.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:26.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.510826+0000", 2024-09-18T04:56:26.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00077587200000000004 2024-09-18T04:56:26.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:26.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:11.510849+0000", 2024-09-18T04:56:26.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3598999999999999e-05 2024-09-18T04:56:26.979 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.979 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:26.979 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.980 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:131 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'89, mlcod=16'85)", 2024-09-18T04:56:26.980 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:12.153368+0000", 2024-09-18T04:56:26.980 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.73668694, 2024-09-18T04:56:26.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00097627100000000002, 2024-09-18T04:56:26.980 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:26.980 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:26.981 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:26.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:26.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.153368+0000", 2024-09-18T04:56:26.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:26.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.153368+0000", 2024-09-18T04:56:26.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:26.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.153367+0000", 2024-09-18T04:56:26.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:26.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:26.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.153449+0000", 2024-09-18T04:56:26.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1291000000000005e-05 2024-09-18T04:56:26.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:26.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.153457+0000", 2024-09-18T04:56:26.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4190000000000002e-06 2024-09-18T04:56:26.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:26.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.153468+0000", 2024-09-18T04:56:26.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0949e-05 2024-09-18T04:56:26.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:26.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.153494+0000", 2024-09-18T04:56:26.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6047e-05 2024-09-18T04:56:26.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:26.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.153504+0000", 2024-09-18T04:56:26.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0560999999999999e-05 2024-09-18T04:56:26.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:26.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.154315+0000", 2024-09-18T04:56:26.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00081089499999999995 2024-09-18T04:56:26.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:26.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.154344+0000", 2024-09-18T04:56:26.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8560000000000001e-05 2024-09-18T04:56:26.986 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.986 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:26.987 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.987 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4204.0:142 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'100, mlcod=16'98)", 2024-09-18T04:56:26.987 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:12.162303+0000", 2024-09-18T04:56:26.987 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.727751378000001, 2024-09-18T04:56:26.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00091834900000000005, 2024-09-18T04:56:26.987 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:26.987 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:26.987 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:26.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:26.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.162303+0000", 2024-09-18T04:56:26.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:26.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.162303+0000", 2024-09-18T04:56:26.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:26.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.162303+0000", 2024-09-18T04:56:26.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:26.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:26.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.162356+0000", 2024-09-18T04:56:26.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3452000000000003e-05 2024-09-18T04:56:26.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:26.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.162367+0000", 2024-09-18T04:56:26.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.058e-05 2024-09-18T04:56:26.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:26.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.162372+0000", 2024-09-18T04:56:26.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3589999999999999e-06 2024-09-18T04:56:26.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:26.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.162415+0000", 2024-09-18T04:56:26.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3297000000000002e-05 2024-09-18T04:56:26.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:26.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.162428+0000", 2024-09-18T04:56:26.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3121e-05 2024-09-18T04:56:26.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:26.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.163193+0000", 2024-09-18T04:56:26.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00076488999999999995 2024-09-18T04:56:26.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:26.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:12.163222+0000", 2024-09-18T04:56:26.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8311e-05 2024-09-18T04:56:26.991 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.991 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:26.991 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.991 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:9 3.1a e22/19 3:5cea7035:::rbd_info:head v 22'2, mlcod=0'0)", 2024-09-18T04:56:26.991 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:23.133017+0000", 2024-09-18T04:56:26.991 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.75703758, 2024-09-18T04:56:26.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001015377, 2024-09-18T04:56:26.991 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:26.992 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:26.992 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:26.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:26.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:23.133017+0000", 2024-09-18T04:56:26.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:26.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:23.133017+0000", 2024-09-18T04:56:26.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:26.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:23.133017+0000", 2024-09-18T04:56:26.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:26.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:26.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:23.133027+0000", 2024-09-18T04:56:26.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0541e-05 2024-09-18T04:56:26.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:26.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:23.133050+0000", 2024-09-18T04:56:26.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2619000000000001e-05 2024-09-18T04:56:26.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:26.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:23.133061+0000", 2024-09-18T04:56:26.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1342000000000001e-05 2024-09-18T04:56:26.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:26.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:23.133107+0000", 2024-09-18T04:56:26.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6146000000000002e-05 2024-09-18T04:56:26.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:26.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:23.133127+0000", 2024-09-18T04:56:26.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.931e-05 2024-09-18T04:56:26.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:26.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:23.133991+0000", 2024-09-18T04:56:26.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00086470299999999998 2024-09-18T04:56:26.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:26.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:23.134032+0000", 2024-09-18T04:56:26.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0992000000000003e-05 2024-09-18T04:56:26.995 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.995 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:26.995 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:26.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.996 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:544 3.8 e23/19 3:1447b1cc:::rbd_data.11acdc0eab99.0000000000000014:head v 23'3, mlcod=23'1)", 2024-09-18T04:56:26.996 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:26.632587+0000", 2024-09-18T04:56:26.996 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.25746783699999998, 2024-09-18T04:56:26.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00092819599999999995, 2024-09-18T04:56:26.996 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:26.996 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:26.996 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:26.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:26.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.632587+0000", 2024-09-18T04:56:26.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:26.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.632587+0000", 2024-09-18T04:56:26.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:26.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:26.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.632585+0000", 2024-09-18T04:56:26.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:26.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:26.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.632682+0000", 2024-09-18T04:56:26.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6399999999999999e-05 2024-09-18T04:56:26.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:26.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.632698+0000", 2024-09-18T04:56:26.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6030000000000001e-05 2024-09-18T04:56:26.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:26.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.632705+0000", 2024-09-18T04:56:26.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4100000000000002e-06 2024-09-18T04:56:26.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:26.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.632752+0000", 2024-09-18T04:56:26.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7283000000000001e-05 2024-09-18T04:56:26.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:26.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.632786+0000", 2024-09-18T04:56:26.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3238999999999998e-05 2024-09-18T04:56:26.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:26.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.633477+0000", 2024-09-18T04:56:26.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00069183299999999999 2024-09-18T04:56:26.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:26.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:26.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:27.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.633515+0000", 2024-09-18T04:56:27.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7490999999999997e-05 2024-09-18T04:56:27.000 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.000 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:27.000 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.000 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:555 3.0 3:0124d2fa:::rbd_data.11acdc0eab99.000000000000009e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 555184~18614] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T04:56:27.000 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:26.643676+0000", 2024-09-18T04:56:27.000 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.24637832000000001, 2024-09-18T04:56:27.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014957359999999999, 2024-09-18T04:56:27.000 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:27.000 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:27.000 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:27.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:27.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:27.001 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 555 2024-09-18T04:56:27.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.001 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:27.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:27.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.643676+0000", 2024-09-18T04:56:27.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:27.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.643676+0000", 2024-09-18T04:56:27.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:27.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.643674+0000", 2024-09-18T04:56:27.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999981 2024-09-18T04:56:27.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:27.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.643712+0000", 2024-09-18T04:56:27.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8180999999999999e-05 2024-09-18T04:56:27.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:27.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.643731+0000", 2024-09-18T04:56:27.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8216e-05 2024-09-18T04:56:27.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:27.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.643740+0000", 2024-09-18T04:56:27.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8810000000000002e-06 2024-09-18T04:56:27.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:27.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.643807+0000", 2024-09-18T04:56:27.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7339e-05 2024-09-18T04:56:27.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:27.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.643954+0000", 2024-09-18T04:56:27.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014672700000000001 2024-09-18T04:56:27.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-18T04:56:27.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.644102+0000", 2024-09-18T04:56:27.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014831800000000001 2024-09-18T04:56:27.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:27.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.645068+0000", 2024-09-18T04:56:27.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00096612599999999999 2024-09-18T04:56:27.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:27.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.645118+0000", 2024-09-18T04:56:27.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9628e-05 2024-09-18T04:56:27.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:27.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.645146+0000", 2024-09-18T04:56:27.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8129000000000001e-05 2024-09-18T04:56:27.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:27.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.645172+0000", 2024-09-18T04:56:27.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6191999999999999e-05 2024-09-18T04:56:27.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.006 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:27.006 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.006 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:557 3.4 3:21ae0c4f:::rbd_data.11acdc0eab99.0000000000000029:head [set-alloc-hint object_size 1048576 write_size 1048576,write 75546~42715] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-09-18T04:56:27.006 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:26.647139+0000", 2024-09-18T04:56:27.006 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.24291590099999999, 2024-09-18T04:56:27.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014030550000000001, 2024-09-18T04:56:27.006 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:27.006 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:27.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:27.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:27.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:27.007 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 557 2024-09-18T04:56:27.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.007 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:27.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:27.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.647139+0000", 2024-09-18T04:56:27.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:27.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.647139+0000", 2024-09-18T04:56:27.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:27.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.647138+0000", 2024-09-18T04:56:27.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:27.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:27.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.647168+0000", 2024-09-18T04:56:27.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0518000000000002e-05 2024-09-18T04:56:27.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:27.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.647179+0000", 2024-09-18T04:56:27.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0811e-05 2024-09-18T04:56:27.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:27.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.647184+0000", 2024-09-18T04:56:27.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8199999999999996e-06 2024-09-18T04:56:27.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:27.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.647229+0000", 2024-09-18T04:56:27.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5435000000000001e-05 2024-09-18T04:56:27.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:27.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.647313+0000", 2024-09-18T04:56:27.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3349000000000004e-05 2024-09-18T04:56:27.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 2", 2024-09-18T04:56:27.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.647400+0000", 2024-09-18T04:56:27.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7117e-05 2024-09-18T04:56:27.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:27.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.648119+0000", 2024-09-18T04:56:27.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00071912300000000001 2024-09-18T04:56:27.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:27.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.648489+0000", 2024-09-18T04:56:27.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00037002399999999999 2024-09-18T04:56:27.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:27.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.648518+0000", 2024-09-18T04:56:27.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8764000000000001e-05 2024-09-18T04:56:27.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:27.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.648542+0000", 2024-09-18T04:56:27.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4026999999999999e-05 2024-09-18T04:56:27.011 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.011 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:27.012 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.012 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.012 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:27.012 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:27.013 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:56:27.014 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:27.010+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:27.114 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:27.111+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:27.167 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:27.167 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:27.167 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:56:27.167 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:27.177 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:56:27.203 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone1 from 0x9d8c166 (overlap 0x17db04c) to 0x0 2024-09-18T04:56:27.216 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:27.213+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:27.260 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-18T04:56:27.306 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:27.306 INFO:teuthology.orchestra.run.smithi002.stdout:115 write 0x21bf79f thru 0x21c3a0c (0x426e bytes) 2024-09-18T04:56:27.317 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:27.315+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:27.320 INFO:teuthology.orchestra.run.smithi002.stdout:116 punch from 0x875ec42 to 0x876c6a6, (0xda64 bytes) 2024-09-18T04:56:27.321 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:27.321 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:27.322 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:56:27.322 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:56:27.322 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:27.329 INFO:teuthology.orchestra.run.smithi002.stdout:117 write 0xd5ef48f thru 0xd5fac4c (0xb7be bytes) 2024-09-18T04:56:27.334 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:56:27.335 INFO:teuthology.orchestra.run.smithi002.stdout:118 writesame 0x4cf2ab9 thru 0x4cfbf85 data_size 0x1(0x94cd bytes) 2024-09-18T04:56:27.353 INFO:teuthology.orchestra.run.smithi002.stdout:119 punch from 0xa30eb84 to 0xa3163ef, (0x786b bytes) 2024-09-18T04:56:27.353 INFO:teuthology.orchestra.run.smithi002.stdout:120 read 0x9c072bc thru 0x9c07672 (0x3b7 bytes) 2024-09-18T04:56:27.364 INFO:teuthology.orchestra.run.smithi002.stdout:122 trunc from 0xd5fac4d to 0x2f207c9 2024-09-18T04:56:27.405 INFO:teuthology.orchestra.run.smithi002.stdout:123 trunc from 0x2f207c9 to 0x81678a9 2024-09-18T04:56:27.410 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:27.416 INFO:teuthology.orchestra.run.smithi002.stdout:125 trunc from 0x81678a9 to 0xccb8e32 2024-09-18T04:56:27.418 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:27.416+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:27.421 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:27.428 INFO:teuthology.orchestra.run.smithi002.stdout:127 write 0xea89d1e thru 0xea92f91 (0x9274 bytes) 2024-09-18T04:56:27.431 INFO:teuthology.orchestra.run.smithi002.stdout:128 writesame 0x7f1f390 thru 0x7f23c78 data_size 0x1(0x48e9 bytes) 2024-09-18T04:56:27.441 INFO:teuthology.orchestra.run.smithi002.stdout:130 read 0x2f8f5ab thru 0x2f97d07 (0x875d bytes) 2024-09-18T04:56:27.441 INFO:teuthology.orchestra.run.smithi002.stdout:131 read 0x6926424 thru 0x69272b4 (0xe91 bytes) 2024-09-18T04:56:27.442 INFO:teuthology.orchestra.run.smithi002.stdout:132 read 0x6423c6e thru 0x6425436 (0x17c9 bytes) 2024-09-18T04:56:27.442 INFO:teuthology.orchestra.run.smithi002.stdout:133 write 0xe9fe521 thru 0xea0954e (0xb02e bytes) 2024-09-18T04:56:27.445 INFO:teuthology.orchestra.run.smithi002.stdout:135 read 0x846451e thru 0x8468285 (0x3d68 bytes) 2024-09-18T04:56:27.445 INFO:teuthology.orchestra.run.smithi002.stdout:137 write 0x7130cc thru 0x714929 (0x185e bytes) 2024-09-18T04:56:27.458 INFO:teuthology.orchestra.run.smithi002.stdout:138 trunc from 0xea92f92 to 0x4b0a400 2024-09-18T04:56:27.460 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:27.460 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:56:27.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:56:27.460 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:56:27.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.461 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:555 3.0 e23/19 3:0124d2fa:::rbd_data.11acdc0eab99.000000000000009e:head v 23'1, mlcod=0'0)", 2024-09-18T04:56:27.461 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:26.644209+0000", 2024-09-18T04:56:27.461 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.81403672800000004, 2024-09-18T04:56:27.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00081704399999999995, 2024-09-18T04:56:27.461 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:27.461 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:27.461 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:27.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:27.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.644209+0000", 2024-09-18T04:56:27.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:27.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.644209+0000", 2024-09-18T04:56:27.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:27.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.644207+0000", 2024-09-18T04:56:27.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:27.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:27.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.644231+0000", 2024-09-18T04:56:27.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3686e-05 2024-09-18T04:56:27.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:27.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.644248+0000", 2024-09-18T04:56:27.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6988000000000001e-05 2024-09-18T04:56:27.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:27.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.644261+0000", 2024-09-18T04:56:27.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2924e-05 2024-09-18T04:56:27.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:27.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.644310+0000", 2024-09-18T04:56:27.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8619000000000003e-05 2024-09-18T04:56:27.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:27.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.644325+0000", 2024-09-18T04:56:27.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5107e-05 2024-09-18T04:56:27.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:27.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.644982+0000", 2024-09-18T04:56:27.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00065705600000000005 2024-09-18T04:56:27.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:27.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.645026+0000", 2024-09-18T04:56:27.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3977000000000003e-05 2024-09-18T04:56:27.466 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.466 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:27.466 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.466 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:557 3.4 e23/19 3:21ae0c4f:::rbd_data.11acdc0eab99.0000000000000029:head v 23'5, mlcod=23'3)", 2024-09-18T04:56:27.467 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:26.647465+0000", 2024-09-18T04:56:27.467 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.81078053400000005, 2024-09-18T04:56:27.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00093274199999999997, 2024-09-18T04:56:27.467 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:27.467 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:27.467 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:27.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:27.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.647465+0000", 2024-09-18T04:56:27.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:27.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.647465+0000", 2024-09-18T04:56:27.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:27.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.647464+0000", 2024-09-18T04:56:27.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:27.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:27.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.647506+0000", 2024-09-18T04:56:27.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2378e-05 2024-09-18T04:56:27.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:27.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.647522+0000", 2024-09-18T04:56:27.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5344000000000001e-05 2024-09-18T04:56:27.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:27.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.647529+0000", 2024-09-18T04:56:27.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2910000000000002e-06 2024-09-18T04:56:27.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:27.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.647578+0000", 2024-09-18T04:56:27.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8776000000000002e-05 2024-09-18T04:56:27.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:27.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.647599+0000", 2024-09-18T04:56:27.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1382999999999998e-05 2024-09-18T04:56:27.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:27.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.648359+0000", 2024-09-18T04:56:27.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00075958099999999995 2024-09-18T04:56:27.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:27.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.648398+0000", 2024-09-18T04:56:27.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8911000000000001e-05 2024-09-18T04:56:27.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.472 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:27.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.473 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:609 3.0 e24/19 3:04479396:::rbd_id.image_client.0-clone2:head v 24'2, mlcod=0'0)", 2024-09-18T04:56:27.473 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.179433+0000", 2024-09-18T04:56:27.473 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.27881275500000002, 2024-09-18T04:56:27.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00083956000000000005, 2024-09-18T04:56:27.473 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:27.473 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:27.473 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:27.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:27.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.179433+0000", 2024-09-18T04:56:27.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:27.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.179433+0000", 2024-09-18T04:56:27.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:27.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.179433+0000", 2024-09-18T04:56:27.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:56:27.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:27.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.179438+0000", 2024-09-18T04:56:27.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5419999999999996e-06 2024-09-18T04:56:27.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:27.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.179456+0000", 2024-09-18T04:56:27.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7938e-05 2024-09-18T04:56:27.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:27.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.179463+0000", 2024-09-18T04:56:27.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2790000000000001e-06 2024-09-18T04:56:27.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:27.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.179525+0000", 2024-09-18T04:56:27.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1458999999999996e-05 2024-09-18T04:56:27.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:27.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.179554+0000", 2024-09-18T04:56:27.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8804999999999999e-05 2024-09-18T04:56:27.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:27.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.180233+0000", 2024-09-18T04:56:27.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067958600000000004 2024-09-18T04:56:27.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:27.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.180272+0000", 2024-09-18T04:56:27.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9106000000000001e-05 2024-09-18T04:56:27.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.478 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:27.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.478 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "pg_update_log_missing(3.0 epoch 24/19 rep_tid 13 entries 24'3 (0'0) error 3:6a357650:::rbd_data.11acdc0eab99.0000000000000012:head by client.4524.0:675 0.000000 -2 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: 1, new_object: 0 trim_to 0'0 roll_forward_to 23'1)", 2024-09-18T04:56:27.478 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.209827+0000", 2024-09-18T04:56:27.478 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.24841818199999999, 2024-09-18T04:56:27.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00065784000000000003, 2024-09-18T04:56:27.479 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:27.479 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:27.479 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:27.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:27.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.209827+0000", 2024-09-18T04:56:27.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:27.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.209827+0000", 2024-09-18T04:56:27.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:27.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.209826+0000", 2024-09-18T04:56:27.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:27.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:27.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.209839+0000", 2024-09-18T04:56:27.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2761e-05 2024-09-18T04:56:27.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:27.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.209868+0000", 2024-09-18T04:56:27.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8966e-05 2024-09-18T04:56:27.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:27.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.209885+0000", 2024-09-18T04:56:27.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7365000000000001e-05 2024-09-18T04:56:27.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:27.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.209954+0000", 2024-09-18T04:56:27.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8986000000000007e-05 2024-09-18T04:56:27.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:27.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.210485+0000", 2024-09-18T04:56:27.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00053107999999999996 2024-09-18T04:56:27.482 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.482 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:27.482 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.482 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:685 3.0 e24/19 3:06a35765:::rbd_data.11acdc0eab99.0000000000000012:head v 24'4, mlcod=24'2)", 2024-09-18T04:56:27.482 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.212212+0000", 2024-09-18T04:56:27.482 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.24603295999999999, 2024-09-18T04:56:27.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00065424600000000004, 2024-09-18T04:56:27.483 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:27.483 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:27.483 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:27.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:27.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.212212+0000", 2024-09-18T04:56:27.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:27.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.212212+0000", 2024-09-18T04:56:27.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:27.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.212211+0000", 2024-09-18T04:56:27.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:27.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:27.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.212219+0000", 2024-09-18T04:56:27.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9219999999999997e-06 2024-09-18T04:56:27.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:27.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.212235+0000", 2024-09-18T04:56:27.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6254999999999999e-05 2024-09-18T04:56:27.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:27.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.212243+0000", 2024-09-18T04:56:27.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7910000000000005e-06 2024-09-18T04:56:27.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:27.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.212288+0000", 2024-09-18T04:56:27.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5383000000000002e-05 2024-09-18T04:56:27.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:27.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.212310+0000", 2024-09-18T04:56:27.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1324000000000001e-05 2024-09-18T04:56:27.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:27.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.212829+0000", 2024-09-18T04:56:27.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00051917100000000002 2024-09-18T04:56:27.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:27.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.212867+0000", 2024-09-18T04:56:27.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7722000000000002e-05 2024-09-18T04:56:27.488 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.488 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:27.488 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.488 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:697 3.0 e24/19 3:0124d2fa:::rbd_data.11acdc0eab99.000000000000009e:head v 24'6, mlcod=24'3)", 2024-09-18T04:56:27.488 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.214917+0000", 2024-09-18T04:56:27.488 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.243328087, 2024-09-18T04:56:27.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011006900000000001, 2024-09-18T04:56:27.488 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:27.488 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:27.488 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:27.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:27.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214917+0000", 2024-09-18T04:56:27.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:27.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214917+0000", 2024-09-18T04:56:27.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:27.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214916+0000", 2024-09-18T04:56:27.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:27.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:27.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214924+0000", 2024-09-18T04:56:27.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0749999999999998e-06 2024-09-18T04:56:27.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:27.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214940+0000", 2024-09-18T04:56:27.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5644000000000001e-05 2024-09-18T04:56:27.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:27.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214949+0000", 2024-09-18T04:56:27.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5180000000000004e-06 2024-09-18T04:56:27.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:27.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.215005+0000", 2024-09-18T04:56:27.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6187999999999998e-05 2024-09-18T04:56:27.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:27.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.215027+0000", 2024-09-18T04:56:27.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1815999999999999e-05 2024-09-18T04:56:27.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:27.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.215983+0000", 2024-09-18T04:56:27.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00095677499999999999 2024-09-18T04:56:27.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:27.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.216018+0000", 2024-09-18T04:56:27.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4604000000000003e-05 2024-09-18T04:56:27.492 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.492 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:27.492 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.493 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:689 3.4 e24/19 3:21ae0c4f:::rbd_data.11acdc0eab99.0000000000000029:head v 24'7, mlcod=23'4)", 2024-09-18T04:56:27.493 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.214992+0000", 2024-09-18T04:56:27.493 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.243253783, 2024-09-18T04:56:27.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001477602, 2024-09-18T04:56:27.493 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:27.493 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:27.493 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:27.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:27.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214992+0000", 2024-09-18T04:56:27.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:27.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214992+0000", 2024-09-18T04:56:27.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:27.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214991+0000", 2024-09-18T04:56:27.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:27.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:27.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214997+0000", 2024-09-18T04:56:27.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1709999999999999e-06 2024-09-18T04:56:27.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:27.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.215006+0000", 2024-09-18T04:56:27.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8799999999999997e-06 2024-09-18T04:56:27.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:27.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.215012+0000", 2024-09-18T04:56:27.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0870000000000003e-06 2024-09-18T04:56:27.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:27.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.215068+0000", 2024-09-18T04:56:27.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5350999999999999e-05 2024-09-18T04:56:27.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:27.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.215087+0000", 2024-09-18T04:56:27.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9565e-05 2024-09-18T04:56:27.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:27.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.216432+0000", 2024-09-18T04:56:27.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013447439999999999 2024-09-18T04:56:27.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:27.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.216469+0000", 2024-09-18T04:56:27.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7370999999999998e-05 2024-09-18T04:56:27.499 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.499 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:27.499 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.499 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.499 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:27.499 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:27.501 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:56:27.501 INFO:teuthology.orchestra.run.smithi002.stdout:139 write 0xe72a8bd thru 0xe7360e8 (0xb82c bytes) 2024-09-18T04:56:27.501 INFO:teuthology.orchestra.run.smithi002.stdout:141 write 0xbbbf36f thru 0xbbc6b62 (0x77f4 bytes) 2024-09-18T04:56:27.504 INFO:teuthology.orchestra.run.smithi002.stdout:143 trunc from 0xe7360e9 to 0x3864e30 2024-09-18T04:56:27.520 INFO:teuthology.orchestra.run.smithi002.stdout:144 read 0x4cec83 thru 0x4d85f0 (0x996e bytes) 2024-09-18T04:56:27.520 INFO:teuthology.orchestra.run.smithi002.stdout:145 read 0x12bb9f9 thru 0x12c8b15 (0xd11d bytes) 2024-09-18T04:56:27.521 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:27.516+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:27.522 INFO:teuthology.orchestra.run.smithi002.stdout:146 read 0xadef32 thru 0xaeba30 (0xcaff bytes) 2024-09-18T04:56:27.522 INFO:teuthology.orchestra.run.smithi002.stdout:truncating to largest ever: 0xeb6262d 2024-09-18T04:56:27.532 INFO:teuthology.orchestra.run.smithi002.stdout:147 trunc from 0x3864e30 to 0xeb6262d 2024-09-18T04:56:27.538 INFO:teuthology.orchestra.run.smithi002.stdout:148 read 0x1b24698 thru 0x1b2d6f5 (0x905e bytes) 2024-09-18T04:56:27.538 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:27.538 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:27.538 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:27.539 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:27.539 INFO:teuthology.orchestra.run.smithi002.stdout:154 read 0x3100030 thru 0x310f422 (0xf3f3 bytes) 2024-09-18T04:56:27.539 INFO:teuthology.orchestra.run.smithi002.stdout:155 read 0x6f672ff thru 0x6f6a5ef (0x32f1 bytes) 2024-09-18T04:56:27.540 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:27.540 INFO:teuthology.orchestra.run.smithi002.stdout:157 write 0xd6d4818 thru 0xd6dfd41 (0xb52a bytes) 2024-09-18T04:56:27.543 INFO:teuthology.orchestra.run.smithi002.stdout:158 read 0xcff7af7 thru 0xd002662 (0xab6c bytes) 2024-09-18T04:56:27.543 INFO:teuthology.orchestra.run.smithi002.stdout:159 read 0x2c51983 thru 0x2c5d919 (0xbf97 bytes) 2024-09-18T04:56:27.544 INFO:teuthology.orchestra.run.smithi002.stdout:161 punch from 0xc2d16eb to 0xc2e1057, (0xf96c bytes) 2024-09-18T04:56:27.544 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:27.544 INFO:teuthology.orchestra.run.smithi002.stdout:164 write 0x4cf0433 thru 0x4cf0a93 (0x661 bytes) 2024-09-18T04:56:27.544 INFO:teuthology.orchestra.run.smithi002.stdout:166 write 0xa292bff thru 0xa29884a (0x5c4c bytes) 2024-09-18T04:56:27.545 INFO:teuthology.orchestra.run.smithi002.stdout:167 write 0x1a6920d thru 0x1a6e8a2 (0x5696 bytes) 2024-09-18T04:56:27.546 INFO:teuthology.orchestra.run.smithi002.stdout:169 read 0x87cd2eb thru 0x87d0ed9 (0x3bef bytes) 2024-09-18T04:56:27.547 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:27.547 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:27.547 INFO:teuthology.orchestra.run.smithi002.stdout:172 read 0xb7372b thru 0xb7c97a (0x9250 bytes) 2024-09-18T04:56:27.548 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:27.548 INFO:teuthology.orchestra.run.smithi002.stdout:175 punch from 0xd3538d1 to 0xd361fce, (0xe6fd bytes) 2024-09-18T04:56:27.548 INFO:teuthology.orchestra.run.smithi002.stdout:178 read 0xbd59f5e thru 0xbd5ef2f (0x4fd2 bytes) 2024-09-18T04:56:27.548 INFO:teuthology.orchestra.run.smithi002.stdout:179 write 0x714b2e3 thru 0x71538e8 (0x8606 bytes) 2024-09-18T04:56:27.549 INFO:teuthology.orchestra.run.smithi002.stdout:180 read 0x83f69f6 thru 0x83f80a8 (0x16b3 bytes) 2024-09-18T04:56:27.558 INFO:teuthology.orchestra.run.smithi002.stdout:183 trunc from 0xeb6262d to 0x7069e70 2024-09-18T04:56:27.575 INFO:teuthology.orchestra.run.smithi002.stdout:184 read 0x4666e50 thru 0x466f503 (0x86b4 bytes) 2024-09-18T04:56:27.575 INFO:teuthology.orchestra.run.smithi002.stdout:185 writesame 0x534f2eb thru 0x53519a9 data_size 0xd(0x26bf bytes) 2024-09-18T04:56:27.577 INFO:teuthology.orchestra.run.smithi002.stdout:186 read 0x4785e75 thru 0x478694a (0xad6 bytes) 2024-09-18T04:56:27.577 INFO:teuthology.orchestra.run.smithi002.stdout:187 clone 3 order 18 su 32768 sc 14 2024-09-18T04:56:27.622 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:27.622 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:27.622 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:56:27.623 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:27.623 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:27.618+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:27.630 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:56:27.723 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:27.720+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:27.775 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:27.776 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:27.776 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:56:27.776 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:56:27.776 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:27.785 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:56:27.825 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:27.822+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:27.926 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:27.923+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:27.933 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:27.933 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:56:27.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:56:27.933 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:56:27.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.933 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:562 3.1e 3:78b10053:::rbd_object_map.11acdc0eab99:head [read 0~74] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:27.933 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.143910+0000", 2024-09-18T04:56:27.933 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.78539497199999997, 2024-09-18T04:56:27.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021030770000000001, 2024-09-18T04:56:27.933 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:27.934 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:27.934 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:27.934 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:27.934 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:27.934 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 562 2024-09-18T04:56:27.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.934 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:27.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:27.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.143910+0000", 2024-09-18T04:56:27.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:27.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.143910+0000", 2024-09-18T04:56:27.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:27.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.143908+0000", 2024-09-18T04:56:27.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999976 2024-09-18T04:56:27.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:27.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.143919+0000", 2024-09-18T04:56:27.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1283e-05 2024-09-18T04:56:27.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:27.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.143939+0000", 2024-09-18T04:56:27.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9936999999999999e-05 2024-09-18T04:56:27.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:27.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.143950+0000", 2024-09-18T04:56:27.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0631999999999999e-05 2024-09-18T04:56:27.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:27.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.143998+0000", 2024-09-18T04:56:27.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8053999999999998e-05 2024-09-18T04:56:27.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op must wait for map", 2024-09-18T04:56:27.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.144004+0000", 2024-09-18T04:56:27.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1319999999999999e-06 2024-09-18T04:56:27.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:27.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.145860+0000", 2024-09-18T04:56:27.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018560079999999999 2024-09-18T04:56:27.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:27.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.145944+0000", 2024-09-18T04:56:27.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4271999999999995e-05 2024-09-18T04:56:27.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:27.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.146013+0000", 2024-09-18T04:56:27.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8936999999999994e-05 2024-09-18T04:56:27.939 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.939 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:27.939 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.939 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:644 3.7 3:e2b0a2ae:::rbd_data.11acdc0eab99.0000000000000000:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:27.939 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.203497+0000", 2024-09-18T04:56:27.939 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.72580786500000005, 2024-09-18T04:56:27.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002495658, 2024-09-18T04:56:27.939 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:27.939 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:27.939 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:27.939 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:27.940 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:27.940 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 644 2024-09-18T04:56:27.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.940 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:27.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:27.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.203497+0000", 2024-09-18T04:56:27.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:27.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.203497+0000", 2024-09-18T04:56:27.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:27.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.203497+0000", 2024-09-18T04:56:27.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:27.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:27.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.203501+0000", 2024-09-18T04:56:27.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.014e-06 2024-09-18T04:56:27.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:27.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.203514+0000", 2024-09-18T04:56:27.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3397000000000001e-05 2024-09-18T04:56:27.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:27.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.203522+0000", 2024-09-18T04:56:27.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2120000000000003e-06 2024-09-18T04:56:27.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:27.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.203570+0000", 2024-09-18T04:56:27.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7818999999999997e-05 2024-09-18T04:56:27.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:27.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.203706+0000", 2024-09-18T04:56:27.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013603399999999999 2024-09-18T04:56:27.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:27.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.205993+0000", 2024-09-18T04:56:27.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022868900000000002 2024-09-18T04:56:27.944 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.944 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:27.944 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.944 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:649 3.7 3:e4f572d7:::rbd_data.11acdc0eab99.0000000000000005:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:27.944 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.203578+0000", 2024-09-18T04:56:27.944 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.72572674500000001, 2024-09-18T04:56:27.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026390530000000001, 2024-09-18T04:56:27.945 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:27.945 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:27.945 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:27.945 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:27.945 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:27.945 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 649 2024-09-18T04:56:27.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.945 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:27.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:27.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.203578+0000", 2024-09-18T04:56:27.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:27.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.203578+0000", 2024-09-18T04:56:27.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:27.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.203578+0000", 2024-09-18T04:56:27.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:27.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:27.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.203582+0000", 2024-09-18T04:56:27.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5419999999999999e-06 2024-09-18T04:56:27.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:27.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.203590+0000", 2024-09-18T04:56:27.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.661e-06 2024-09-18T04:56:27.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:27.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.203595+0000", 2024-09-18T04:56:27.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1460000000000004e-06 2024-09-18T04:56:27.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:27.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.204268+0000", 2024-09-18T04:56:27.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067269699999999999 2024-09-18T04:56:27.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:27.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.204394+0000", 2024-09-18T04:56:27.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000125809 2024-09-18T04:56:27.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:27.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.206218+0000", 2024-09-18T04:56:27.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001823978 2024-09-18T04:56:27.948 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.948 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:27.949 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.949 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:661 3.7 3:e2b0a2ae:::rbd_data.11acdc0eab99.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:27.949 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.206184+0000", 2024-09-18T04:56:27.949 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.72312115899999996, 2024-09-18T04:56:27.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028519190000000001, 2024-09-18T04:56:27.949 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:27.949 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:27.949 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:27.949 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:27.949 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:27.949 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 661 2024-09-18T04:56:27.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.950 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:27.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:27.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.206184+0000", 2024-09-18T04:56:27.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:27.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.206184+0000", 2024-09-18T04:56:27.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:27.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.206183+0000", 2024-09-18T04:56:27.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:27.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:27.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.206189+0000", 2024-09-18T04:56:27.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.801e-06 2024-09-18T04:56:27.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:27.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.206201+0000", 2024-09-18T04:56:27.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2130000000000001e-05 2024-09-18T04:56:27.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:27.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.206208+0000", 2024-09-18T04:56:27.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3030000000000002e-06 2024-09-18T04:56:27.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:27.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.206247+0000", 2024-09-18T04:56:27.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8974000000000003e-05 2024-09-18T04:56:27.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:27.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.206324+0000", 2024-09-18T04:56:27.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7143000000000003e-05 2024-09-18T04:56:27.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-18T04:56:27.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.206523+0000", 2024-09-18T04:56:27.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00019891400000000001 2024-09-18T04:56:27.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:27.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.207695+0000", 2024-09-18T04:56:27.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001171247 2024-09-18T04:56:27.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:27.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.208939+0000", 2024-09-18T04:56:27.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012441500000000001 2024-09-18T04:56:27.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:27.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.208985+0000", 2024-09-18T04:56:27.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5769999999999997e-05 2024-09-18T04:56:27.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:27.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.209036+0000", 2024-09-18T04:56:27.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1419999999999999e-05 2024-09-18T04:56:27.955 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.955 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:27.955 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.955 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:662 3.7 3:e4f572d7:::rbd_data.11acdc0eab99.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:27.955 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.206374+0000", 2024-09-18T04:56:27.955 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.72293120200000005, 2024-09-18T04:56:27.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002884958, 2024-09-18T04:56:27.955 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:27.956 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:27.956 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:27.956 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:27.956 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:27.956 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 662 2024-09-18T04:56:27.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.956 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:27.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:27.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.206374+0000", 2024-09-18T04:56:27.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:27.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.206374+0000", 2024-09-18T04:56:27.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:27.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.206373+0000", 2024-09-18T04:56:27.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:27.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:27.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.206378+0000", 2024-09-18T04:56:27.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.101e-06 2024-09-18T04:56:27.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:27.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.206387+0000", 2024-09-18T04:56:27.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5900000000000008e-06 2024-09-18T04:56:27.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:27.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.206392+0000", 2024-09-18T04:56:27.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8690000000000003e-06 2024-09-18T04:56:27.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:27.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.206787+0000", 2024-09-18T04:56:27.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00039529399999999997 2024-09-18T04:56:27.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:27.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.206862+0000", 2024-09-18T04:56:27.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4609000000000003e-05 2024-09-18T04:56:27.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-18T04:56:27.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.207050+0000", 2024-09-18T04:56:27.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018868300000000001 2024-09-18T04:56:27.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:27.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.207960+0000", 2024-09-18T04:56:27.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00091009299999999995 2024-09-18T04:56:27.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:27.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.209185+0000", 2024-09-18T04:56:27.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00122474 2024-09-18T04:56:27.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:27.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.209225+0000", 2024-09-18T04:56:27.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0139000000000003e-05 2024-09-18T04:56:27.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:27.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.209259+0000", 2024-09-18T04:56:27.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3661999999999997e-05 2024-09-18T04:56:27.961 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.961 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:27.961 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.961 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:671 3.18 3:1c8846cf:::rbd_data.11acdc0eab99.000000000000000d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:27.961 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.208610+0000", 2024-09-18T04:56:27.961 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.720695262, 2024-09-18T04:56:27.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002666772, 2024-09-18T04:56:27.962 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:27.962 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:27.962 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:27.962 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:27.962 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:27.962 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 671 2024-09-18T04:56:27.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.962 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:27.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:27.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.208610+0000", 2024-09-18T04:56:27.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:27.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.208610+0000", 2024-09-18T04:56:27.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:27.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.208609+0000", 2024-09-18T04:56:27.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:27.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:27.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.208614+0000", 2024-09-18T04:56:27.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3820000000000003e-06 2024-09-18T04:56:27.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:27.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.208627+0000", 2024-09-18T04:56:27.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2996000000000001e-05 2024-09-18T04:56:27.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:27.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.208635+0000", 2024-09-18T04:56:27.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1899999999999998e-06 2024-09-18T04:56:27.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:27.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.208684+0000", 2024-09-18T04:56:27.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9249000000000002e-05 2024-09-18T04:56:27.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:27.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.208732+0000", 2024-09-18T04:56:27.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8640000000000002e-05 2024-09-18T04:56:27.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:27.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.208853+0000", 2024-09-18T04:56:27.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120017 2024-09-18T04:56:27.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:27.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.209593+0000", 2024-09-18T04:56:27.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00074024099999999999 2024-09-18T04:56:27.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:27.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.211189+0000", 2024-09-18T04:56:27.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001595993 2024-09-18T04:56:27.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:27.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.211241+0000", 2024-09-18T04:56:27.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2266999999999999e-05 2024-09-18T04:56:27.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:27.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.211277+0000", 2024-09-18T04:56:27.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5735999999999998e-05 2024-09-18T04:56:27.967 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.967 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:27.967 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.967 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:672 3.18 3:1f3a3809:::rbd_data.11acdc0eab99.0000000000000011:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:27.967 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.209178+0000", 2024-09-18T04:56:27.967 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.72012696200000004, 2024-09-18T04:56:27.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021998590000000002, 2024-09-18T04:56:27.968 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:27.968 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:27.968 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:27.968 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:27.968 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:27.968 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 672 2024-09-18T04:56:27.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.968 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:27.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:27.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.209178+0000", 2024-09-18T04:56:27.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:27.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.209178+0000", 2024-09-18T04:56:27.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:27.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.209177+0000", 2024-09-18T04:56:27.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:27.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:27.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.209182+0000", 2024-09-18T04:56:27.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9239999999999998e-06 2024-09-18T04:56:27.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:27.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.209194+0000", 2024-09-18T04:56:27.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1192999999999999e-05 2024-09-18T04:56:27.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:27.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.209202+0000", 2024-09-18T04:56:27.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3489999999999992e-06 2024-09-18T04:56:27.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:27.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.209235+0000", 2024-09-18T04:56:27.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3479999999999998e-05 2024-09-18T04:56:27.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:27.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.209343+0000", 2024-09-18T04:56:27.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108057 2024-09-18T04:56:27.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:27.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.211378+0000", 2024-09-18T04:56:27.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0020346549999999998 2024-09-18T04:56:27.972 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.972 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:27.972 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.972 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:679 3.8 3:1447b1cc:::rbd_data.11acdc0eab99.0000000000000014:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:27.972 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.210375+0000", 2024-09-18T04:56:27.972 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.71892978100000005, 2024-09-18T04:56:27.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002622657, 2024-09-18T04:56:27.972 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:27.972 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:27.973 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:27.973 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:27.973 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:27.973 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 679 2024-09-18T04:56:27.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.973 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:27.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:27.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.210375+0000", 2024-09-18T04:56:27.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:27.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.210375+0000", 2024-09-18T04:56:27.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:27.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.210374+0000", 2024-09-18T04:56:27.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:27.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:27.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.210380+0000", 2024-09-18T04:56:27.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8869999999999997e-06 2024-09-18T04:56:27.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:27.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.210394+0000", 2024-09-18T04:56:27.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3628000000000001e-05 2024-09-18T04:56:27.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:27.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.210402+0000", 2024-09-18T04:56:27.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2460000000000003e-06 2024-09-18T04:56:27.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:27.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.210458+0000", 2024-09-18T04:56:27.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6403999999999997e-05 2024-09-18T04:56:27.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:27.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.210544+0000", 2024-09-18T04:56:27.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6038000000000005e-05 2024-09-18T04:56:27.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:27.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.210909+0000", 2024-09-18T04:56:27.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00036474000000000001 2024-09-18T04:56:27.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:27.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.212720+0000", 2024-09-18T04:56:27.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001810868 2024-09-18T04:56:27.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:27.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.212921+0000", 2024-09-18T04:56:27.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000200609 2024-09-18T04:56:27.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:27.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.212962+0000", 2024-09-18T04:56:27.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0794000000000003e-05 2024-09-18T04:56:27.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:27.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.212998+0000", 2024-09-18T04:56:27.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6599e-05 2024-09-18T04:56:27.979 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.979 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:27.980 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.980 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:683 3.7 3:e2382fb5:::rbd_data.11acdc0eab99.0000000000000017:head [stat,delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:27.980 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.211427+0000", 2024-09-18T04:56:27.980 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.71787833899999998, 2024-09-18T04:56:27.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001856857, 2024-09-18T04:56:27.980 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:27.980 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:27.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:27.981 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:27.981 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:27.981 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 683 2024-09-18T04:56:27.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.981 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:27.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:27.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.211427+0000", 2024-09-18T04:56:27.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:27.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.211427+0000", 2024-09-18T04:56:27.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:27.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.211426+0000", 2024-09-18T04:56:27.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:27.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:27.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.211433+0000", 2024-09-18T04:56:27.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8719999999999996e-06 2024-09-18T04:56:27.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:27.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.211444+0000", 2024-09-18T04:56:27.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0942e-05 2024-09-18T04:56:27.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:27.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.211456+0000", 2024-09-18T04:56:27.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1895999999999999e-05 2024-09-18T04:56:27.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:27.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.211502+0000", 2024-09-18T04:56:27.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6799999999999999e-05 2024-09-18T04:56:27.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:27.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.211576+0000", 2024-09-18T04:56:27.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3061999999999999e-05 2024-09-18T04:56:27.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-18T04:56:27.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.211746+0000", 2024-09-18T04:56:27.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000170073 2024-09-18T04:56:27.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:27.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.212895+0000", 2024-09-18T04:56:27.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011494750000000001 2024-09-18T04:56:27.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:27.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.213218+0000", 2024-09-18T04:56:27.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032297499999999998 2024-09-18T04:56:27.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:27.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.213255+0000", 2024-09-18T04:56:27.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7196000000000001e-05 2024-09-18T04:56:27.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:27.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.213284+0000", 2024-09-18T04:56:27.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8521999999999999e-05 2024-09-18T04:56:27.986 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.986 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:27.986 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.986 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:698 3.1e 3:78b10053:::rbd_object_map.11acdc0eab99:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:27.986 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.220184+0000", 2024-09-18T04:56:27.986 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.70912110100000003, 2024-09-18T04:56:27.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0024422599999999999, 2024-09-18T04:56:27.987 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:27.987 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:27.987 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:27.987 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:27.987 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:27.987 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 698 2024-09-18T04:56:27.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.987 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:27.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:27.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.220184+0000", 2024-09-18T04:56:27.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:27.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.220184+0000", 2024-09-18T04:56:27.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:27.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.220183+0000", 2024-09-18T04:56:27.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:27.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:27.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.220190+0000", 2024-09-18T04:56:27.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1010000000000003e-06 2024-09-18T04:56:27.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:27.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.220201+0000", 2024-09-18T04:56:27.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1627000000000001e-05 2024-09-18T04:56:27.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:27.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.220209+0000", 2024-09-18T04:56:27.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5900000000000002e-06 2024-09-18T04:56:27.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:27.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.220253+0000", 2024-09-18T04:56:27.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3838999999999998e-05 2024-09-18T04:56:27.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:27.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.220318+0000", 2024-09-18T04:56:27.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5376000000000006e-05 2024-09-18T04:56:27.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-18T04:56:27.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.220560+0000", 2024-09-18T04:56:27.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000242217 2024-09-18T04:56:27.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:27.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.221138+0000", 2024-09-18T04:56:27.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000577555 2024-09-18T04:56:27.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:27.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.222571+0000", 2024-09-18T04:56:27.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001433164 2024-09-18T04:56:27.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:27.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.222604+0000", 2024-09-18T04:56:27.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2728000000000002e-05 2024-09-18T04:56:27.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:27.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.222626+0000", 2024-09-18T04:56:27.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2591000000000001e-05 2024-09-18T04:56:27.992 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.992 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:27.992 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.992 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:849 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:27.992 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.371372+0000", 2024-09-18T04:56:27.992 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.55793350500000005, 2024-09-18T04:56:27.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023172940000000001, 2024-09-18T04:56:27.993 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:27.993 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:27.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:27.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:27.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:27.993 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 849 2024-09-18T04:56:27.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.993 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:27.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:27.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.371372+0000", 2024-09-18T04:56:27.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:27.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.371372+0000", 2024-09-18T04:56:27.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:27.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:27.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.371370+0000", 2024-09-18T04:56:27.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999981 2024-09-18T04:56:27.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:27.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.371380+0000", 2024-09-18T04:56:27.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8349999999999999e-06 2024-09-18T04:56:27.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:27.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.371401+0000", 2024-09-18T04:56:27.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1756999999999999e-05 2024-09-18T04:56:27.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:27.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.371411+0000", 2024-09-18T04:56:27.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6479999999999998e-06 2024-09-18T04:56:27.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:27.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.371470+0000", 2024-09-18T04:56:27.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9194999999999998e-05 2024-09-18T04:56:27.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:27.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.371582+0000", 2024-09-18T04:56:27.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111334 2024-09-18T04:56:27.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:27.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.372035+0000", 2024-09-18T04:56:27.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00045340099999999998 2024-09-18T04:56:27.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:27.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.373555+0000", 2024-09-18T04:56:27.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015198569999999999 2024-09-18T04:56:27.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:27.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.373591+0000", 2024-09-18T04:56:27.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6631000000000002e-05 2024-09-18T04:56:27.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:27.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.373642+0000", 2024-09-18T04:56:27.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0457999999999997e-05 2024-09-18T04:56:27.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:27.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.373689+0000", 2024-09-18T04:56:27.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7129000000000002e-05 2024-09-18T04:56:27.998 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.998 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:27.998 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:27.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:27.999 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:850 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:27.999 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.372538+0000", 2024-09-18T04:56:27.999 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.55676695700000001, 2024-09-18T04:56:27.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002356478, 2024-09-18T04:56:27.999 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:27.999 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:27.999 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:27.999 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:27.999 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:27.999 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 850 2024-09-18T04:56:27.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:27.999 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:27.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:28.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.372538+0000", 2024-09-18T04:56:28.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:28.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:28.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.372538+0000", 2024-09-18T04:56:28.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:28.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:28.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.372537+0000", 2024-09-18T04:56:28.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999981 2024-09-18T04:56:28.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:28.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.372546+0000", 2024-09-18T04:56:28.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3929999999999997e-06 2024-09-18T04:56:28.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:28.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.372562+0000", 2024-09-18T04:56:28.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.56e-05 2024-09-18T04:56:28.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.372570+0000", 2024-09-18T04:56:28.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2549999999999997e-06 2024-09-18T04:56:28.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:28.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.372633+0000", 2024-09-18T04:56:28.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2706999999999994e-05 2024-09-18T04:56:28.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:28.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.372736+0000", 2024-09-18T04:56:28.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010320800000000001 2024-09-18T04:56:28.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:28.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.373189+0000", 2024-09-18T04:56:28.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00045317600000000001 2024-09-18T04:56:28.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:28.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.374458+0000", 2024-09-18T04:56:28.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012694640000000001 2024-09-18T04:56:28.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:28.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.374793+0000", 2024-09-18T04:56:28.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033461999999999999 2024-09-18T04:56:28.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:28.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.374846+0000", 2024-09-18T04:56:28.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3177999999999998e-05 2024-09-18T04:56:28.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:28.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.374895+0000", 2024-09-18T04:56:28.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8578000000000001e-05 2024-09-18T04:56:28.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:28.005 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:28.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:28.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.005 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:851 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:28.005 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.373749+0000", 2024-09-18T04:56:28.005 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.55555639000000001, 2024-09-18T04:56:28.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0036956670000000001, 2024-09-18T04:56:28.005 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:28.005 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:28.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:28.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:28.006 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 851 2024-09-18T04:56:28.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.006 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:28.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:28.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.373749+0000", 2024-09-18T04:56:28.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:28.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:28.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.373749+0000", 2024-09-18T04:56:28.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:28.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:28.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.373748+0000", 2024-09-18T04:56:28.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:28.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:28.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.373753+0000", 2024-09-18T04:56:28.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4199999999999998e-06 2024-09-18T04:56:28.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:28.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.373773+0000", 2024-09-18T04:56:28.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9168e-05 2024-09-18T04:56:28.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.373780+0000", 2024-09-18T04:56:28.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.79e-06 2024-09-18T04:56:28.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:28.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.373830+0000", 2024-09-18T04:56:28.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9298000000000001e-05 2024-09-18T04:56:28.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:28.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.373912+0000", 2024-09-18T04:56:28.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2365000000000002e-05 2024-09-18T04:56:28.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:28.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.374128+0000", 2024-09-18T04:56:28.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000216147 2024-09-18T04:56:28.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:28.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.377299+0000", 2024-09-18T04:56:28.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003170829 2024-09-18T04:56:28.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:28.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.377349+0000", 2024-09-18T04:56:28.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.045e-05 2024-09-18T04:56:28.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:28.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.377400+0000", 2024-09-18T04:56:28.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0849000000000001e-05 2024-09-18T04:56:28.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:28.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.377445+0000", 2024-09-18T04:56:28.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4261000000000002e-05 2024-09-18T04:56:28.011 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:28.011 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:28.011 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:28.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.011 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:852 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:28.011 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.374744+0000", 2024-09-18T04:56:28.011 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.55456139500000001, 2024-09-18T04:56:28.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028696659999999999, 2024-09-18T04:56:28.012 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:28.012 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.012 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:28.012 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:28.012 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:28.012 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 852 2024-09-18T04:56:28.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.012 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:28.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:28.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.374744+0000", 2024-09-18T04:56:28.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:28.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:28.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.374744+0000", 2024-09-18T04:56:28.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:28.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:28.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.374743+0000", 2024-09-18T04:56:28.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:28.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:28.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.374748+0000", 2024-09-18T04:56:28.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8420000000000001e-06 2024-09-18T04:56:28.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:28.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.374759+0000", 2024-09-18T04:56:28.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0997e-05 2024-09-18T04:56:28.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.374775+0000", 2024-09-18T04:56:28.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5913000000000002e-05 2024-09-18T04:56:28.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:28.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.374929+0000", 2024-09-18T04:56:28.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00015396499999999999 2024-09-18T04:56:28.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:28.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375027+0000", 2024-09-18T04:56:28.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7979000000000005e-05 2024-09-18T04:56:28.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:28.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375345+0000", 2024-09-18T04:56:28.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00031734200000000003 2024-09-18T04:56:28.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:28.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.377533+0000", 2024-09-18T04:56:28.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021882949999999998 2024-09-18T04:56:28.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:28.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.377543+0000", 2024-09-18T04:56:28.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0261999999999999e-05 2024-09-18T04:56:28.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:28.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.377591+0000", 2024-09-18T04:56:28.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7763999999999999e-05 2024-09-18T04:56:28.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:28.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.377614+0000", 2024-09-18T04:56:28.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2527999999999998e-05 2024-09-18T04:56:28.017 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:28.017 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:28.017 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:28.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.018 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:853 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:28.018 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.374789+0000", 2024-09-18T04:56:28.018 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.55451655600000005, 2024-09-18T04:56:28.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029257100000000002, 2024-09-18T04:56:28.018 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:28.018 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.018 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:28.018 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:28.018 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:28.018 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 853 2024-09-18T04:56:28.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.019 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:28.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:28.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.374789+0000", 2024-09-18T04:56:28.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:28.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:28.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.374789+0000", 2024-09-18T04:56:28.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:28.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:28.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.374788+0000", 2024-09-18T04:56:28.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:28.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:28.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.374792+0000", 2024-09-18T04:56:28.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3429999999999998e-06 2024-09-18T04:56:28.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:28.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.374800+0000", 2024-09-18T04:56:28.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6359999999999994e-06 2024-09-18T04:56:28.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.374805+0000", 2024-09-18T04:56:28.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1120000000000004e-06 2024-09-18T04:56:28.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:28.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375680+0000", 2024-09-18T04:56:28.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00087473800000000001 2024-09-18T04:56:28.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:28.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375731+0000", 2024-09-18T04:56:28.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1162999999999998e-05 2024-09-18T04:56:28.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:28.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.376046+0000", 2024-09-18T04:56:28.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00031527999999999998 2024-09-18T04:56:28.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:28.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.377625+0000", 2024-09-18T04:56:28.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001578636 2024-09-18T04:56:28.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:28.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.377667+0000", 2024-09-18T04:56:28.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2098999999999998e-05 2024-09-18T04:56:28.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:28.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.377694+0000", 2024-09-18T04:56:28.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7209e-05 2024-09-18T04:56:28.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:28.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.377714+0000", 2024-09-18T04:56:28.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0333e-05 2024-09-18T04:56:28.024 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:28.024 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:28.024 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:28.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.024 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:854 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:28.024 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.374978+0000", 2024-09-18T04:56:28.024 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.55432763900000004, 2024-09-18T04:56:28.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0046422130000000001, 2024-09-18T04:56:28.025 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:28.025 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.025 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:28.025 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:28.025 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:28.025 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 854 2024-09-18T04:56:28.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.025 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:28.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:28.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.374978+0000", 2024-09-18T04:56:28.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:28.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:28.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.374978+0000", 2024-09-18T04:56:28.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:28.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:28.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.374977+0000", 2024-09-18T04:56:28.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:28.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:28.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.374982+0000", 2024-09-18T04:56:28.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4450000000000004e-06 2024-09-18T04:56:28.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:28.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.374992+0000", 2024-09-18T04:56:28.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8649999999999996e-06 2024-09-18T04:56:28.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.374998+0000", 2024-09-18T04:56:28.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1290000000000001e-06 2024-09-18T04:56:28.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:28.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.376580+0000", 2024-09-18T04:56:28.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001581839 2024-09-18T04:56:28.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:28.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.376684+0000", 2024-09-18T04:56:28.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010376499999999999 2024-09-18T04:56:28.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:28.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.377104+0000", 2024-09-18T04:56:28.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00042055800000000002 2024-09-18T04:56:28.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:28.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.379481+0000", 2024-09-18T04:56:28.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023770660000000002 2024-09-18T04:56:28.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:28.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.379539+0000", 2024-09-18T04:56:28.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7379000000000001e-05 2024-09-18T04:56:28.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:28.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.379585+0000", 2024-09-18T04:56:28.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6174999999999998e-05 2024-09-18T04:56:28.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:28.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.379620+0000", 2024-09-18T04:56:28.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5058999999999998e-05 2024-09-18T04:56:28.030 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:28.030 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:28.030 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:28.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.030 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:855 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:28.030 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.375028+0000", 2024-09-18T04:56:28.030 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.55427764199999996, 2024-09-18T04:56:28.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0063269789999999999, 2024-09-18T04:56:28.031 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:28.031 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.031 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:28.031 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:28.031 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:28.031 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 855 2024-09-18T04:56:28.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.031 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:28.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:28.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375028+0000", 2024-09-18T04:56:28.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:28.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:28.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375028+0000", 2024-09-18T04:56:28.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:28.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:28.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375027+0000", 2024-09-18T04:56:28.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:28.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:28.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375033+0000", 2024-09-18T04:56:28.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9869999999999996e-06 2024-09-18T04:56:28.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:28.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375042+0000", 2024-09-18T04:56:28.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3899999999999999e-06 2024-09-18T04:56:28.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375051+0000", 2024-09-18T04:56:28.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2839999999999995e-06 2024-09-18T04:56:28.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:28.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.377760+0000", 2024-09-18T04:56:28.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027089869999999999 2024-09-18T04:56:28.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:28.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.377862+0000", 2024-09-18T04:56:28.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010184200000000001 2024-09-18T04:56:28.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:28.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.378261+0000", 2024-09-18T04:56:28.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00039838900000000002 2024-09-18T04:56:28.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:28.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.380468+0000", 2024-09-18T04:56:28.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002207156 2024-09-18T04:56:28.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:28.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381284+0000", 2024-09-18T04:56:28.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00081592199999999996 2024-09-18T04:56:28.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:28.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381332+0000", 2024-09-18T04:56:28.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8047000000000003e-05 2024-09-18T04:56:28.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:28.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381355+0000", 2024-09-18T04:56:28.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2802999999999998e-05 2024-09-18T04:56:28.036 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:28.036 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:28.036 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:28.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.036 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:856 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:28.036 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.375180+0000", 2024-09-18T04:56:28.036 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.55412479199999998, 2024-09-18T04:56:28.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0063488080000000001, 2024-09-18T04:56:28.037 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:28.037 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.037 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:28.037 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:28.037 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:28.037 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 856 2024-09-18T04:56:28.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.037 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:28.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:28.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375180+0000", 2024-09-18T04:56:28.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:28.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:28.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375180+0000", 2024-09-18T04:56:28.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:28.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:28.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375180+0000", 2024-09-18T04:56:28.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:28.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:28.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375184+0000", 2024-09-18T04:56:28.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6530000000000002e-06 2024-09-18T04:56:28.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:28.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375202+0000", 2024-09-18T04:56:28.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8099000000000001e-05 2024-09-18T04:56:28.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375209+0000", 2024-09-18T04:56:28.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7469999999999999e-06 2024-09-18T04:56:28.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:28.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.378708+0000", 2024-09-18T04:56:28.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034990329999999999 2024-09-18T04:56:28.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:28.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.378801+0000", 2024-09-18T04:56:28.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2967999999999995e-05 2024-09-18T04:56:28.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:28.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.379196+0000", 2024-09-18T04:56:28.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00039495699999999998 2024-09-18T04:56:28.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:28.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381369+0000", 2024-09-18T04:56:28.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021726559999999998 2024-09-18T04:56:28.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:28.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381422+0000", 2024-09-18T04:56:28.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3523000000000002e-05 2024-09-18T04:56:28.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:28.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381469+0000", 2024-09-18T04:56:28.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7107999999999997e-05 2024-09-18T04:56:28.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:28.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381529+0000", 2024-09-18T04:56:28.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9846000000000003e-05 2024-09-18T04:56:28.045 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:28.045 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:28.045 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:28.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.046 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:857 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:28.046 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.375218+0000", 2024-09-18T04:56:28.046 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.55408674499999999, 2024-09-18T04:56:28.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006463809, 2024-09-18T04:56:28.046 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:28.046 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:28.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:28.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:28.047 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 857 2024-09-18T04:56:28.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.047 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:28.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:28.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375218+0000", 2024-09-18T04:56:28.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:28.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:28.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375218+0000", 2024-09-18T04:56:28.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:28.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:28.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375218+0000", 2024-09-18T04:56:28.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:28.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:28.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375222+0000", 2024-09-18T04:56:28.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.704e-06 2024-09-18T04:56:28.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:28.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375231+0000", 2024-09-18T04:56:28.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3999999999999998e-06 2024-09-18T04:56:28.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375237+0000", 2024-09-18T04:56:28.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5169999999999999e-06 2024-09-18T04:56:28.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:28.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.379666+0000", 2024-09-18T04:56:28.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044297390000000002 2024-09-18T04:56:28.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:28.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.379747+0000", 2024-09-18T04:56:28.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0316999999999997e-05 2024-09-18T04:56:28.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:28.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.380138+0000", 2024-09-18T04:56:28.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00039156599999999998 2024-09-18T04:56:28.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:28.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381578+0000", 2024-09-18T04:56:28.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001439423 2024-09-18T04:56:28.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:28.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381587+0000", 2024-09-18T04:56:28.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2499999999999995e-06 2024-09-18T04:56:28.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:28.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381641+0000", 2024-09-18T04:56:28.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3961000000000001e-05 2024-09-18T04:56:28.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:28.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381682+0000", 2024-09-18T04:56:28.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1454e-05 2024-09-18T04:56:28.055 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:28.055 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:28.055 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:28.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.056 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:860 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:28.056 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.375377+0000", 2024-09-18T04:56:28.056 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.55392845999999996, 2024-09-18T04:56:28.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0066062229999999996, 2024-09-18T04:56:28.056 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:28.056 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:28.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:28.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:28.056 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 860 2024-09-18T04:56:28.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.056 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:28.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:28.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375377+0000", 2024-09-18T04:56:28.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:28.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:28.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375377+0000", 2024-09-18T04:56:28.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:28.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:28.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375375+0000", 2024-09-18T04:56:28.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:28.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:28.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375382+0000", 2024-09-18T04:56:28.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0280000000000003e-06 2024-09-18T04:56:28.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:28.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375393+0000", 2024-09-18T04:56:28.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0518e-05 2024-09-18T04:56:28.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375401+0000", 2024-09-18T04:56:28.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1680000000000004e-06 2024-09-18T04:56:28.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:28.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.380505+0000", 2024-09-18T04:56:28.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051037920000000002 2024-09-18T04:56:28.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:28.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.380589+0000", 2024-09-18T04:56:28.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4259999999999996e-05 2024-09-18T04:56:28.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:28.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.380912+0000", 2024-09-18T04:56:28.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000322763 2024-09-18T04:56:28.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:28.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381694+0000", 2024-09-18T04:56:28.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00078214099999999998 2024-09-18T04:56:28.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:28.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381927+0000", 2024-09-18T04:56:28.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000233248 2024-09-18T04:56:28.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:28.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381961+0000", 2024-09-18T04:56:28.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3691e-05 2024-09-18T04:56:28.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:28.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:28.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:28.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381983+0000", 2024-09-18T04:56:28.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1968999999999999e-05 2024-09-18T04:56:28.062 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:28.062 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:28.062 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:28.062 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:28.062 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:28.062 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:28.063 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:56:28.064 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:28.061+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:28.075 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone2 from 0x7069e70 (overlap 0x2f207c9) to 0x2a6a09b 2024-09-18T04:56:28.157 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T04:56:28.161 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-18T04:56:28.163 INFO:teuthology.orchestra.run.smithi002.stdout:188 read 0x5e947c7 thru 0x5e95c57 (0x1491 bytes) 2024-09-18T04:56:28.163 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:28.160+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:28.164 INFO:teuthology.orchestra.run.smithi002.stdout:189 write 0x62629e thru 0x62f67d (0x93e0 bytes) 2024-09-18T04:56:28.172 INFO:teuthology.orchestra.run.smithi002.stdout:191 writesame 0x4009846 thru 0x4015aab data_size 0x2(0xc266 bytes) 2024-09-18T04:56:28.173 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:28.173 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:28.173 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:56:28.173 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:28.183 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:56:28.191 INFO:teuthology.orchestra.run.smithi002.stdout:192 write 0xad6fa14 thru 0xad7dd6d (0xe35a bytes) 2024-09-18T04:56:28.195 INFO:teuthology.orchestra.run.smithi002.stdout:193 read 0x7212fcc thru 0x721e667 (0xb69c bytes) 2024-09-18T04:56:28.196 INFO:teuthology.orchestra.run.smithi002.stdout:194 trunc from 0xad7dd6e to 0xbbbf7d8 2024-09-18T04:56:28.205 INFO:teuthology.orchestra.run.smithi002.stdout:197 write 0xbbe2ef9 thru 0xbbf0500 (0xd608 bytes) 2024-09-18T04:56:28.209 INFO:teuthology.orchestra.run.smithi002.stdout:198 punch from 0x3bd7617 to 0x3bd9e96, (0x287f bytes) 2024-09-18T04:56:28.210 INFO:teuthology.orchestra.run.smithi002.stdout:199 writesame 0xa01c0b thru 0xa0b6fb data_size 0x1(0x9af1 bytes) 2024-09-18T04:56:28.237 INFO:teuthology.orchestra.run.smithi002.stdout:200 trunc from 0xbbf0501 to 0x33d8035 2024-09-18T04:56:28.264 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:28.262+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:28.266 INFO:teuthology.orchestra.run.smithi002.stdout:201 punch from 0x1f0cf to 0x2f0ca, (0xfffb bytes) 2024-09-18T04:56:28.267 INFO:teuthology.orchestra.run.smithi002.stdout:202 punch from 0x2fc98f0 to 0x2fcd9a2, (0x40b2 bytes) 2024-09-18T04:56:28.267 INFO:teuthology.orchestra.run.smithi002.stdout:203 read 0x30ece08 thru 0x30fb30a (0xe503 bytes) 2024-09-18T04:56:28.267 INFO:teuthology.orchestra.run.smithi002.stdout:204 read 0xa91c3 thru 0xab8b2 (0x26f0 bytes) 2024-09-18T04:56:28.277 INFO:teuthology.orchestra.run.smithi002.stdout:207 trunc from 0x33d8035 to 0xbe3e0f9 2024-09-18T04:56:28.292 INFO:teuthology.orchestra.run.smithi002.stdout:208 writesame 0xc956d29 thru 0xc960170 data_size 0x28(0x9448 bytes) 2024-09-18T04:56:28.292 INFO:teuthology.orchestra.run.smithi002.stdout:210 read 0xad3290e thru 0xad34a26 (0x2119 bytes) 2024-09-18T04:56:28.292 INFO:teuthology.orchestra.run.smithi002.stdout:211 writesame 0x2418c2b thru 0x241d26a data_size 0x20(0x4640 bytes) 2024-09-18T04:56:28.299 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:56:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:56:28.299 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:28.304 INFO:teuthology.orchestra.run.smithi002.stdout:212 write 0xea25f05 thru 0xea2ee73 (0x8f6f bytes) 2024-09-18T04:56:28.310 INFO:teuthology.orchestra.run.smithi002.stdout:213 punch from 0x7ca5c70 to 0x7ca60e2, (0x472 bytes) 2024-09-18T04:56:28.310 INFO:teuthology.orchestra.run.smithi002.stdout:215 read 0xa6901f1 thru 0xa6902de (0xee bytes) 2024-09-18T04:56:28.310 INFO:teuthology.orchestra.run.smithi002.stdout:216 read 0x685964b thru 0x68608bc (0x7272 bytes) 2024-09-18T04:56:28.310 INFO:teuthology.orchestra.run.smithi002.stdout:217 read 0x709de2a thru 0x70a1fa7 (0x417e bytes) 2024-09-18T04:56:28.310 INFO:teuthology.orchestra.run.smithi002.stdout:218 write 0xd10b41a thru 0xd117b74 (0xc75b bytes) 2024-09-18T04:56:28.311 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:56:28.312 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:28.312 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:28.312 INFO:teuthology.orchestra.run.smithi002.stdout:223 read 0xa7933eb thru 0xa7950ec (0x1d02 bytes) 2024-09-18T04:56:28.312 INFO:teuthology.orchestra.run.smithi002.stdout:225 writesame 0x3abff26 thru 0x3ac79b6 data_size 0x1(0x7a91 bytes) 2024-09-18T04:56:28.328 INFO:teuthology.orchestra.run.smithi002.stdout:226 read 0x4ee37df thru 0x4ee62a4 (0x2ac6 bytes) 2024-09-18T04:56:28.328 INFO:teuthology.orchestra.run.smithi002.stdout:228 writesame 0xa09c8d1 thru 0xa0a1ae0 data_size 0x65(0x5210 bytes) 2024-09-18T04:56:28.332 INFO:teuthology.orchestra.run.smithi002.stdout:229 read 0x912571a thru 0x912db8e (0x8475 bytes) 2024-09-18T04:56:28.339 INFO:teuthology.orchestra.run.smithi002.stdout:230 trunc from 0xea2ee74 to 0x88eaf46 2024-09-18T04:56:28.353 INFO:teuthology.orchestra.run.smithi002.stdout:231 trunc from 0x88eaf46 to 0x9a26a4b 2024-09-18T04:56:28.365 INFO:teuthology.orchestra.run.smithi002.stdout:233 trunc from 0x9a26a4b to 0x59f49c5 2024-09-18T04:56:28.365 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:28.362+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:28.382 INFO:teuthology.orchestra.run.smithi002.stdout:234 writesame 0xa47ca54 thru 0xa483992 data_size 0xb(0x6f3f bytes) 2024-09-18T04:56:28.382 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:28.382 INFO:teuthology.orchestra.run.smithi002.stdout:236 punch from 0x3890c99 to 0x38946e9, (0x3a50 bytes) 2024-09-18T04:56:28.389 INFO:teuthology.orchestra.run.smithi002.stdout:237 write 0xc7efee6 thru 0xc7ffdca (0xfee5 bytes) 2024-09-18T04:56:28.397 INFO:teuthology.orchestra.run.smithi002.stdout:238 read 0xb0f3fc9 thru 0xb0f4376 (0x3ae bytes) 2024-09-18T04:56:28.397 INFO:teuthology.orchestra.run.smithi002.stdout:239 writesame 0x8bc3c90 thru 0x8bc6490 data_size 0x31(0x2801 bytes) 2024-09-18T04:56:28.400 INFO:teuthology.orchestra.run.smithi002.stdout:240 read 0x9d55f90 thru 0x9d5aae9 (0x4b5a bytes) 2024-09-18T04:56:28.401 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:28.401 INFO:teuthology.orchestra.run.smithi002.stdout:244 read 0x8a8a1af thru 0x8a8c6ae (0x2500 bytes) 2024-09-18T04:56:28.401 INFO:teuthology.orchestra.run.smithi002.stdout:245 read 0x23c656c thru 0x23ce1ba (0x7c4f bytes) 2024-09-18T04:56:28.404 INFO:teuthology.orchestra.run.smithi002.stdout:246 trunc from 0xc7ffdcb to 0x92ba4e5 2024-09-18T04:56:28.413 INFO:teuthology.orchestra.run.smithi002.stdout:247 write 0xe7a25f thru 0xe7c994 (0x2736 bytes) 2024-09-18T04:56:28.417 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:28.417 INFO:teuthology.orchestra.run.smithi002.stdout:249 read 0x16ae4e thru 0x171ca2 (0x6e55 bytes) 2024-09-18T04:56:28.418 INFO:teuthology.orchestra.run.smithi002.stdout:250 write 0x7d960 thru 0x850cb (0x776c bytes) 2024-09-18T04:56:28.424 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:28.432 INFO:teuthology.orchestra.run.smithi002.stdout:252 write 0xae939bf thru 0xaea0427 (0xca69 bytes) 2024-09-18T04:56:28.444 INFO:teuthology.orchestra.run.smithi002.stdout:253 trunc from 0xaea0428 to 0x6eca2e7 2024-09-18T04:56:28.454 INFO:teuthology.orchestra.run.smithi002.stdout:254 read 0x13ec1cc thru 0x13f7a3d (0xb872 bytes) 2024-09-18T04:56:28.461 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:28.461 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:56:28.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:56:28.461 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:56:28.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.462 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:7 3.1a 3:5cea7035:::rbd_info:head [create,write 0~8] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-09-18T04:56:28.462 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:22.121517+0000", 2024-09-18T04:56:28.462 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.3369223439999995, 2024-09-18T04:56:28.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035231910000000002, 2024-09-18T04:56:28.462 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:28.462 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.462 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:28.462 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:28.462 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:28.462 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 7 2024-09-18T04:56:28.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.463 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:28.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:28.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:22.121517+0000", 2024-09-18T04:56:28.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:28.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:22.121517+0000", 2024-09-18T04:56:28.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:28.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:22.121516+0000", 2024-09-18T04:56:28.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:28.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:28.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:22.121525+0000", 2024-09-18T04:56:28.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3039999999999994e-06 2024-09-18T04:56:28.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:28.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:22.121541+0000", 2024-09-18T04:56:28.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6209999999999999e-05 2024-09-18T04:56:28.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:22.121551+0000", 2024-09-18T04:56:28.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6390000000000004e-06 2024-09-18T04:56:28.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:28.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:22.121592+0000", 2024-09-18T04:56:28.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0698999999999998e-05 2024-09-18T04:56:28.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-18T04:56:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:22.121597+0000", 2024-09-18T04:56:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1710000000000001e-06 2024-09-18T04:56:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:22.123794+0000", 2024-09-18T04:56:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002196796 2024-09-18T04:56:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:22.123914+0000", 2024-09-18T04:56:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012005 2024-09-18T04:56:28.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:28.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:22.124054+0000", 2024-09-18T04:56:28.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00014041300000000001 2024-09-18T04:56:28.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:28.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:22.124640+0000", 2024-09-18T04:56:28.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00058561099999999999 2024-09-18T04:56:28.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:28.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:22.124948+0000", 2024-09-18T04:56:28.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030871300000000001 2024-09-18T04:56:28.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:28.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:22.125003+0000", 2024-09-18T04:56:28.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4719000000000002e-05 2024-09-18T04:56:28.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:28.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:22.125040+0000", 2024-09-18T04:56:28.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7394999999999997e-05 2024-09-18T04:56:28.468 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.468 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:28.468 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.469 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e22)", 2024-09-18T04:56:28.469 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:23.130894+0000", 2024-09-18T04:56:28.469 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.3275459959999996, 2024-09-18T04:56:28.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032993800000000002, 2024-09-18T04:56:28.469 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:28.469 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.469 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:28.469 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:28.469 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:28.469 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 9 2024-09-18T04:56:28.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.469 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:23.130894+0000", 2024-09-18T04:56:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:23.130894+0000", 2024-09-18T04:56:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:23.130892+0000", 2024-09-18T04:56:28.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:56:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:23.130903+0000", 2024-09-18T04:56:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1483e-05 2024-09-18T04:56:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:23.130919+0000", 2024-09-18T04:56:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5738000000000001e-05 2024-09-18T04:56:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:23.130930+0000", 2024-09-18T04:56:28.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.075e-05 2024-09-18T04:56:28.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:28.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:23.130982+0000", 2024-09-18T04:56:28.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2398000000000002e-05 2024-09-18T04:56:28.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-18T04:56:28.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:23.130987+0000", 2024-09-18T04:56:28.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8110000000000002e-06 2024-09-18T04:56:28.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:28.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:23.132627+0000", 2024-09-18T04:56:28.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016398529999999999 2024-09-18T04:56:28.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:28.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:23.132708+0000", 2024-09-18T04:56:28.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1426000000000003e-05 2024-09-18T04:56:28.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:23.132843+0000", 2024-09-18T04:56:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013473000000000001 2024-09-18T04:56:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:23.133585+0000", 2024-09-18T04:56:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00074198500000000002 2024-09-18T04:56:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:23.134137+0000", 2024-09-18T04:56:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00055250099999999995 2024-09-18T04:56:28.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:23.134168+0000", 2024-09-18T04:56:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0595000000000001e-05 2024-09-18T04:56:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:23.134193+0000", 2024-09-18T04:56:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4987e-05 2024-09-18T04:56:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.476 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:693 3.f e24/19 3:f0e4723c:::rbd_data.11acdc0eab99.0000000000000056:head v 24'35, mlcod=24'32)", 2024-09-18T04:56:28.476 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.215714+0000", 2024-09-18T04:56:28.476 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.2427254679999999, 2024-09-18T04:56:28.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.004129916, 2024-09-18T04:56:28.476 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:28.476 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.476 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:28.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:28.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215714+0000", 2024-09-18T04:56:28.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:28.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215714+0000", 2024-09-18T04:56:28.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:28.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215714+0000", 2024-09-18T04:56:28.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:28.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:28.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215734+0000", 2024-09-18T04:56:28.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0386000000000001e-05 2024-09-18T04:56:28.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:28.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215743+0000", 2024-09-18T04:56:28.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2550000000000003e-06 2024-09-18T04:56:28.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215763+0000", 2024-09-18T04:56:28.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9239e-05 2024-09-18T04:56:28.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215816+0000", 2024-09-18T04:56:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3121999999999997e-05 2024-09-18T04:56:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215836+0000", 2024-09-18T04:56:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9811999999999999e-05 2024-09-18T04:56:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.219810+0000", 2024-09-18T04:56:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039743620000000004 2024-09-18T04:56:28.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:28.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.219844+0000", 2024-09-18T04:56:28.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4205999999999997e-05 2024-09-18T04:56:28.480 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.480 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:28.480 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.480 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:694 3.a e24/19 3:54bf2b96:::rbd_data.11acdc0eab99.0000000000000067:head v 24'8, mlcod=23'3)", 2024-09-18T04:56:28.480 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.215805+0000", 2024-09-18T04:56:28.481 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.2426343470000001, 2024-09-18T04:56:28.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040608659999999998, 2024-09-18T04:56:28.481 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:28.481 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.481 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:28.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:28.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215805+0000", 2024-09-18T04:56:28.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:28.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215805+0000", 2024-09-18T04:56:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215805+0000", 2024-09-18T04:56:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215810+0000", 2024-09-18T04:56:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3990000000000003e-06 2024-09-18T04:56:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:28.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215819+0000", 2024-09-18T04:56:28.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6289999999999999e-06 2024-09-18T04:56:28.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215825+0000", 2024-09-18T04:56:28.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8939999999999998e-06 2024-09-18T04:56:28.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:28.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215863+0000", 2024-09-18T04:56:28.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8313999999999999e-05 2024-09-18T04:56:28.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:28.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215882+0000", 2024-09-18T04:56:28.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8930999999999999e-05 2024-09-18T04:56:28.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:28.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.219834+0000", 2024-09-18T04:56:28.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039519170000000001 2024-09-18T04:56:28.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:28.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.219866+0000", 2024-09-18T04:56:28.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2314000000000003e-05 2024-09-18T04:56:28.485 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.485 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:28.485 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.485 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:691 3.13 e24/19 3:cc0a27f0:::rbd_data.11acdc0eab99.0000000000000052:head v 24'42, mlcod=23'39)", 2024-09-18T04:56:28.485 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.216010+0000", 2024-09-18T04:56:28.485 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.2424293419999999, 2024-09-18T04:56:28.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0038708919999999999, 2024-09-18T04:56:28.486 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:28.486 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.486 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:28.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:28.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.216010+0000", 2024-09-18T04:56:28.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:28.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.216010+0000", 2024-09-18T04:56:28.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:28.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.216010+0000", 2024-09-18T04:56:28.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:28.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:28.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.216015+0000", 2024-09-18T04:56:28.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6500000000000001e-06 2024-09-18T04:56:28.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:28.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.216025+0000", 2024-09-18T04:56:28.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3570000000000004e-06 2024-09-18T04:56:28.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.216039+0000", 2024-09-18T04:56:28.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4234e-05 2024-09-18T04:56:28.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:28.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.216078+0000", 2024-09-18T04:56:28.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9317999999999999e-05 2024-09-18T04:56:28.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:28.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.216097+0000", 2024-09-18T04:56:28.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8512000000000002e-05 2024-09-18T04:56:28.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:28.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.219850+0000", 2024-09-18T04:56:28.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037528330000000001 2024-09-18T04:56:28.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:28.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.219881+0000", 2024-09-18T04:56:28.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1538000000000002e-05 2024-09-18T04:56:28.490 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.490 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:28.490 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.490 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:865 3.11 e24/19 3:8b581e29:::rbd_data.11ac33624de9.0000000000000008:head v 24'6, mlcod=23'3)", 2024-09-18T04:56:28.490 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.384443+0000", 2024-09-18T04:56:28.490 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.073996671, 2024-09-18T04:56:28.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0053766509999999997, 2024-09-18T04:56:28.491 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:28.491 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.491 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:28.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:28.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.384443+0000", 2024-09-18T04:56:28.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:28.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.384443+0000", 2024-09-18T04:56:28.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:28.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.384442+0000", 2024-09-18T04:56:28.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:28.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:28.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.385656+0000", 2024-09-18T04:56:28.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001213626 2024-09-18T04:56:28.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:28.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.385675+0000", 2024-09-18T04:56:28.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9765000000000001e-05 2024-09-18T04:56:28.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.385686+0000", 2024-09-18T04:56:28.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0468999999999999e-05 2024-09-18T04:56:28.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:28.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.385738+0000", 2024-09-18T04:56:28.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2552000000000001e-05 2024-09-18T04:56:28.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:28.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.385762+0000", 2024-09-18T04:56:28.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3061000000000002e-05 2024-09-18T04:56:28.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:28.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.389782+0000", 2024-09-18T04:56:28.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040200749999999997 2024-09-18T04:56:28.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:28.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.389820+0000", 2024-09-18T04:56:28.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7991999999999998e-05 2024-09-18T04:56:28.495 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.495 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:28.495 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.495 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:867 3.13 e24/19 3:c80b79e3:::rbd_data.11ac33624de9.000000000000000a:head v 24'45, mlcod=24'43)", 2024-09-18T04:56:28.495 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.386953+0000", 2024-09-18T04:56:28.495 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.071486554, 2024-09-18T04:56:28.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0051944479999999999, 2024-09-18T04:56:28.495 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:28.495 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.495 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:28.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:28.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.386953+0000", 2024-09-18T04:56:28.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:28.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.386953+0000", 2024-09-18T04:56:28.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:28.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.386952+0000", 2024-09-18T04:56:28.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:28.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:28.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.387720+0000", 2024-09-18T04:56:28.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00076791999999999995 2024-09-18T04:56:28.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:28.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.387731+0000", 2024-09-18T04:56:28.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1454000000000001e-05 2024-09-18T04:56:28.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.387737+0000", 2024-09-18T04:56:28.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6919999999999999e-06 2024-09-18T04:56:28.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:28.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.387779+0000", 2024-09-18T04:56:28.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2382999999999997e-05 2024-09-18T04:56:28.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:28.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.387804+0000", 2024-09-18T04:56:28.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4742000000000001e-05 2024-09-18T04:56:28.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:28.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.392114+0000", 2024-09-18T04:56:28.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.004309846 2024-09-18T04:56:28.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:28.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.392148+0000", 2024-09-18T04:56:28.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3494000000000002e-05 2024-09-18T04:56:28.502 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.502 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:28.502 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.503 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:868 3.11 e24/19 3:888e89d2:::rbd_data.11ac33624de9.000000000000000b:head v 24'7, mlcod=23'3)", 2024-09-18T04:56:28.503 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.387746+0000", 2024-09-18T04:56:28.503 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.0706936659999999, 2024-09-18T04:56:28.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0043760630000000003, 2024-09-18T04:56:28.504 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:28.504 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.504 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:28.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:28.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.387746+0000", 2024-09-18T04:56:28.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:28.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.387746+0000", 2024-09-18T04:56:28.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:28.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.387746+0000", 2024-09-18T04:56:28.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:28.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:28.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.387750+0000", 2024-09-18T04:56:28.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7419999999999997e-06 2024-09-18T04:56:28.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:28.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.387754+0000", 2024-09-18T04:56:28.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1169999999999997e-06 2024-09-18T04:56:28.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.387758+0000", 2024-09-18T04:56:28.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5439999999999999e-06 2024-09-18T04:56:28.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:28.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.387795+0000", 2024-09-18T04:56:28.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7239e-05 2024-09-18T04:56:28.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:28.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.387815+0000", 2024-09-18T04:56:28.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9381e-05 2024-09-18T04:56:28.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:28.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.392095+0000", 2024-09-18T04:56:28.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0042801430000000001 2024-09-18T04:56:28.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:28.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.392122+0000", 2024-09-18T04:56:28.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7317e-05 2024-09-18T04:56:28.511 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.511 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:28.511 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.512 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:926 3.f e24/19 3:f33a2580:::rbd_data.11ac33624de9.0000000000000006:head v 24'40, mlcod=24'38)", 2024-09-18T04:56:28.512 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.461527+0000", 2024-09-18T04:56:28.512 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.99691233700000004, 2024-09-18T04:56:28.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037370049999999998, 2024-09-18T04:56:28.512 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:28.513 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.513 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:28.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:28.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461527+0000", 2024-09-18T04:56:28.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:28.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461527+0000", 2024-09-18T04:56:28.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:28.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461526+0000", 2024-09-18T04:56:28.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:28.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:28.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461533+0000", 2024-09-18T04:56:28.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7070000000000003e-06 2024-09-18T04:56:28.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:28.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461548+0000", 2024-09-18T04:56:28.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4728e-05 2024-09-18T04:56:28.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461556+0000", 2024-09-18T04:56:28.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6230000000000003e-06 2024-09-18T04:56:28.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:28.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461742+0000", 2024-09-18T04:56:28.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018596699999999999 2024-09-18T04:56:28.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:28.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461763+0000", 2024-09-18T04:56:28.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1097000000000001e-05 2024-09-18T04:56:28.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:28.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.465226+0000", 2024-09-18T04:56:28.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034625960000000001 2024-09-18T04:56:28.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:28.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.465264+0000", 2024-09-18T04:56:28.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8219000000000001e-05 2024-09-18T04:56:28.520 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.520 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:28.520 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.520 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:930 3.13 e24/19 3:c80b79e3:::rbd_data.11ac33624de9.000000000000000a:head v 24'46, mlcod=24'44)", 2024-09-18T04:56:28.521 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.461837+0000", 2024-09-18T04:56:28.521 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.99660267700000005, 2024-09-18T04:56:28.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034310740000000001, 2024-09-18T04:56:28.521 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:28.521 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.521 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:28.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:28.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461837+0000", 2024-09-18T04:56:28.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:28.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461837+0000", 2024-09-18T04:56:28.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:28.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461836+0000", 2024-09-18T04:56:28.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:28.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:28.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461841+0000", 2024-09-18T04:56:28.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9790000000000001e-06 2024-09-18T04:56:28.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:28.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461852+0000", 2024-09-18T04:56:28.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0175e-05 2024-09-18T04:56:28.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461859+0000", 2024-09-18T04:56:28.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1840000000000002e-06 2024-09-18T04:56:28.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:28.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461901+0000", 2024-09-18T04:56:28.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2722999999999997e-05 2024-09-18T04:56:28.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:28.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461922+0000", 2024-09-18T04:56:28.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0471000000000001e-05 2024-09-18T04:56:28.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:28.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.465230+0000", 2024-09-18T04:56:28.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033083040000000002 2024-09-18T04:56:28.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:28.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.465268+0000", 2024-09-18T04:56:28.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7799999999999997e-05 2024-09-18T04:56:28.526 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.526 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:28.526 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.526 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:928 3.11 e24/19 3:8b581e29:::rbd_data.11ac33624de9.0000000000000008:head v 24'8, mlcod=24'6)", 2024-09-18T04:56:28.526 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.462033+0000", 2024-09-18T04:56:28.526 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.99640627299999995, 2024-09-18T04:56:28.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032323019999999998, 2024-09-18T04:56:28.526 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:28.527 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.527 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:28.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:28.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.462033+0000", 2024-09-18T04:56:28.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:28.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.462033+0000", 2024-09-18T04:56:28.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:28.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.462033+0000", 2024-09-18T04:56:28.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:28.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:28.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.462038+0000", 2024-09-18T04:56:28.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3399999999999997e-06 2024-09-18T04:56:28.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:28.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.462047+0000", 2024-09-18T04:56:28.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4290000000000001e-06 2024-09-18T04:56:28.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.462052+0000", 2024-09-18T04:56:28.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5720000000000002e-06 2024-09-18T04:56:28.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:28.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.462094+0000", 2024-09-18T04:56:28.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1325000000000002e-05 2024-09-18T04:56:28.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:28.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.462112+0000", 2024-09-18T04:56:28.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.874e-05 2024-09-18T04:56:28.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:28.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.465236+0000", 2024-09-18T04:56:28.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031237359999999998 2024-09-18T04:56:28.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:28.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.465266+0000", 2024-09-18T04:56:28.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9567999999999999e-05 2024-09-18T04:56:28.535 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.535 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:28.535 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.535 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:966 3.13 e24/19 3:c80b79e3:::rbd_data.11ac33624de9.000000000000000a:head v 24'47, mlcod=24'45)", 2024-09-18T04:56:28.536 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.508697+0000", 2024-09-18T04:56:28.536 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.94974281900000002, 2024-09-18T04:56:28.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003363624, 2024-09-18T04:56:28.536 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:28.536 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.536 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:28.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:28.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.508697+0000", 2024-09-18T04:56:28.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:28.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.508697+0000", 2024-09-18T04:56:28.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:28.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.508696+0000", 2024-09-18T04:56:28.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:28.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:28.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.508703+0000", 2024-09-18T04:56:28.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.224e-06 2024-09-18T04:56:28.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:28.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.508711+0000", 2024-09-18T04:56:28.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4169999999999992e-06 2024-09-18T04:56:28.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.508717+0000", 2024-09-18T04:56:28.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6999999999999996e-06 2024-09-18T04:56:28.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:28.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.508752+0000", 2024-09-18T04:56:28.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5617000000000001e-05 2024-09-18T04:56:28.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:28.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.508770+0000", 2024-09-18T04:56:28.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8087000000000001e-05 2024-09-18T04:56:28.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:28.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.512029+0000", 2024-09-18T04:56:28.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032581279999999999 2024-09-18T04:56:28.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:28.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.512060+0000", 2024-09-18T04:56:28.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1881999999999997e-05 2024-09-18T04:56:28.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.540 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:28.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.540 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:967 3.11 e24/19 3:888e89d2:::rbd_data.11ac33624de9.000000000000000b:head v 24'11, mlcod=24'8)", 2024-09-18T04:56:28.540 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.508809+0000", 2024-09-18T04:56:28.541 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.94963023800000002, 2024-09-18T04:56:28.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032690689999999999, 2024-09-18T04:56:28.541 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:28.541 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.541 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:28.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:28.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.508809+0000", 2024-09-18T04:56:28.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:28.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.508809+0000", 2024-09-18T04:56:28.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:28.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.508809+0000", 2024-09-18T04:56:28.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:28.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:28.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.508815+0000", 2024-09-18T04:56:28.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6579999999999996e-06 2024-09-18T04:56:28.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:28.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.508827+0000", 2024-09-18T04:56:28.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1596e-05 2024-09-18T04:56:28.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.508833+0000", 2024-09-18T04:56:28.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3929999999999996e-06 2024-09-18T04:56:28.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:28.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.508924+0000", 2024-09-18T04:56:28.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1135000000000001e-05 2024-09-18T04:56:28.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:28.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.508943+0000", 2024-09-18T04:56:28.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8921999999999999e-05 2024-09-18T04:56:28.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:28.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.512056+0000", 2024-09-18T04:56:28.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031130620000000002 2024-09-18T04:56:28.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:28.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.512078+0000", 2024-09-18T04:56:28.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2087999999999999e-05 2024-09-18T04:56:28.547 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.547 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:28.547 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.547 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:1012 3.f e24/19 3:f33a2580:::rbd_data.11ac33624de9.0000000000000006:head v 24'42, mlcod=24'40)", 2024-09-18T04:56:28.547 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.562398+0000", 2024-09-18T04:56:28.547 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.89604211099999997, 2024-09-18T04:56:28.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034003710000000001, 2024-09-18T04:56:28.548 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:28.548 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.548 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:28.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:28.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562398+0000", 2024-09-18T04:56:28.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:28.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562398+0000", 2024-09-18T04:56:28.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:28.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562397+0000", 2024-09-18T04:56:28.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:28.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:28.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562403+0000", 2024-09-18T04:56:28.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1140000000000002e-06 2024-09-18T04:56:28.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:28.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562416+0000", 2024-09-18T04:56:28.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3473e-05 2024-09-18T04:56:28.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562423+0000", 2024-09-18T04:56:28.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5409999999999997e-06 2024-09-18T04:56:28.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:28.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562456+0000", 2024-09-18T04:56:28.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2979999999999999e-05 2024-09-18T04:56:28.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:28.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562474+0000", 2024-09-18T04:56:28.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8082999999999999e-05 2024-09-18T04:56:28.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:28.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.565740+0000", 2024-09-18T04:56:28.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032657839999999999 2024-09-18T04:56:28.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:28.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.565798+0000", 2024-09-18T04:56:28.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8091999999999999e-05 2024-09-18T04:56:28.554 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.554 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:28.554 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.555 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:1016 3.13 e24/19 3:c80b79e3:::rbd_data.11ac33624de9.000000000000000a:head v 24'48, mlcod=24'46)", 2024-09-18T04:56:28.555 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.562434+0000", 2024-09-18T04:56:28.555 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.89600537199999997, 2024-09-18T04:56:28.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033753889999999999, 2024-09-18T04:56:28.555 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:28.555 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.555 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:28.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:28.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562434+0000", 2024-09-18T04:56:28.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:28.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562434+0000", 2024-09-18T04:56:28.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:28.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562434+0000", 2024-09-18T04:56:28.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:28.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:28.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562438+0000", 2024-09-18T04:56:28.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4809999999999998e-06 2024-09-18T04:56:28.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:28.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562444+0000", 2024-09-18T04:56:28.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6880000000000002e-06 2024-09-18T04:56:28.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562448+0000", 2024-09-18T04:56:28.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2289999999999997e-06 2024-09-18T04:56:28.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:28.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562483+0000", 2024-09-18T04:56:28.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4799999999999999e-05 2024-09-18T04:56:28.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:28.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562500+0000", 2024-09-18T04:56:28.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6597999999999999e-05 2024-09-18T04:56:28.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:28.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.565789+0000", 2024-09-18T04:56:28.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0032888510000000002 2024-09-18T04:56:28.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:28.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.565810+0000", 2024-09-18T04:56:28.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0981000000000001e-05 2024-09-18T04:56:28.559 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.560 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:28.560 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.560 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:1014 3.11 e24/19 3:8b581e29:::rbd_data.11ac33624de9.0000000000000008:head v 24'13, mlcod=24'11)", 2024-09-18T04:56:28.560 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.562742+0000", 2024-09-18T04:56:28.560 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.89569784600000002, 2024-09-18T04:56:28.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00567346, 2024-09-18T04:56:28.560 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:28.560 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.560 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:28.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:28.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562742+0000", 2024-09-18T04:56:28.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:28.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562742+0000", 2024-09-18T04:56:28.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:28.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562741+0000", 2024-09-18T04:56:28.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:28.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:28.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562748+0000", 2024-09-18T04:56:28.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.409e-06 2024-09-18T04:56:28.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:28.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562761+0000", 2024-09-18T04:56:28.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3406e-05 2024-09-18T04:56:28.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562768+0000", 2024-09-18T04:56:28.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1489999999999997e-06 2024-09-18T04:56:28.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:28.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562817+0000", 2024-09-18T04:56:28.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8714000000000001e-05 2024-09-18T04:56:28.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:28.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562837+0000", 2024-09-18T04:56:28.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0352000000000001e-05 2024-09-18T04:56:28.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:28.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.568385+0000", 2024-09-18T04:56:28.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0055482960000000003 2024-09-18T04:56:28.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:28.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.568415+0000", 2024-09-18T04:56:28.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9838000000000001e-05 2024-09-18T04:56:28.564 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.564 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:28.564 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.564 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:1017 3.11 e24/19 3:888e89d2:::rbd_data.11ac33624de9.000000000000000b:head v 24'14, mlcod=24'11)", 2024-09-18T04:56:28.565 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.563208+0000", 2024-09-18T04:56:28.565 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.89523167999999997, 2024-09-18T04:56:28.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0052333680000000004, 2024-09-18T04:56:28.565 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:28.565 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.565 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:28.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:28.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.563208+0000", 2024-09-18T04:56:28.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:28.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.563208+0000", 2024-09-18T04:56:28.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:28.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.563207+0000", 2024-09-18T04:56:28.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:28.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:28.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.563214+0000", 2024-09-18T04:56:28.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1920000000000002e-06 2024-09-18T04:56:28.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:28.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.563223+0000", 2024-09-18T04:56:28.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7229999999999999e-06 2024-09-18T04:56:28.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.563229+0000", 2024-09-18T04:56:28.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7060000000000001e-06 2024-09-18T04:56:28.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:28.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.563261+0000", 2024-09-18T04:56:28.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2094000000000001e-05 2024-09-18T04:56:28.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:28.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.563278+0000", 2024-09-18T04:56:28.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7442e-05 2024-09-18T04:56:28.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:28.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.568418+0000", 2024-09-18T04:56:28.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0051392670000000003 2024-09-18T04:56:28.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:28.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.568441+0000", 2024-09-18T04:56:28.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3570999999999998e-05 2024-09-18T04:56:28.568 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.569 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:28.569 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.569 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:1339 3.3 3:c3fae2b0:::rbd_data.11ac99060bb1.0000000000000029:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:28.569 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.212068+0000", 2024-09-18T04:56:28.569 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.246371795, 2024-09-18T04:56:28.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0096840729999999996, 2024-09-18T04:56:28.569 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:28.569 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.569 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:28.569 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:28.569 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:28.569 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1339 2024-09-18T04:56:28.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.570 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:28.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:28.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.212068+0000", 2024-09-18T04:56:28.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:28.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.212068+0000", 2024-09-18T04:56:28.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:28.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.212066+0000", 2024-09-18T04:56:28.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:28.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:28.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.212080+0000", 2024-09-18T04:56:28.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3875e-05 2024-09-18T04:56:28.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:28.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.212094+0000", 2024-09-18T04:56:28.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3474e-05 2024-09-18T04:56:28.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.212100+0000", 2024-09-18T04:56:28.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2809999999999997e-06 2024-09-18T04:56:28.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:28.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.212147+0000", 2024-09-18T04:56:28.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7441999999999997e-05 2024-09-18T04:56:28.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:28.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.212284+0000", 2024-09-18T04:56:28.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013625700000000001 2024-09-18T04:56:28.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:28.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.213890+0000", 2024-09-18T04:56:28.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016066660000000001 2024-09-18T04:56:28.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:28.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.221662+0000", 2024-09-18T04:56:28.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0077715750000000002 2024-09-18T04:56:28.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:28.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.221671+0000", 2024-09-18T04:56:28.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0180000000000007e-06 2024-09-18T04:56:28.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:28.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.221709+0000", 2024-09-18T04:56:28.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8380000000000002e-05 2024-09-18T04:56:28.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:28.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.221752+0000", 2024-09-18T04:56:28.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2697999999999997e-05 2024-09-18T04:56:28.575 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.575 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:28.575 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.575 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop_reply(client.4524.0:1339 3.3 e25/19 ondisk, result = 0)", 2024-09-18T04:56:28.575 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.217671+0000", 2024-09-18T04:56:28.575 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.24076829199999999, 2024-09-18T04:56:28.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039927449999999998, 2024-09-18T04:56:28.576 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:28.576 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:28.576 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:28.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:28.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.217671+0000", 2024-09-18T04:56:28.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:28.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.217671+0000", 2024-09-18T04:56:28.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:28.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.217670+0000", 2024-09-18T04:56:28.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:28.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:28.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.217676+0000", 2024-09-18T04:56:28.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8699999999999997e-06 2024-09-18T04:56:28.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:28.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.217690+0000", 2024-09-18T04:56:28.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3591000000000001e-05 2024-09-18T04:56:28.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.217698+0000", 2024-09-18T04:56:28.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4670000000000004e-06 2024-09-18T04:56:28.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:28.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.221649+0000", 2024-09-18T04:56:28.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039510559999999997 2024-09-18T04:56:28.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:28.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.221658+0000", 2024-09-18T04:56:28.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3990000000000004e-06 2024-09-18T04:56:28.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:28.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.221664+0000", 2024-09-18T04:56:28.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5910000000000001e-06 2024-09-18T04:56:28.579 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.579 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:28.579 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.579 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:1487 3.f e25/19 3:f43f8cc3:::rbd_data.11ac99060bb1.00000000000000cf:head v 25'47, mlcod=25'45)", 2024-09-18T04:56:28.579 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.256403+0000", 2024-09-18T04:56:28.579 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.20203684399999999, 2024-09-18T04:56:28.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0038258939999999998, 2024-09-18T04:56:28.580 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:28.580 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:28.580 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:28.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:28.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.256403+0000", 2024-09-18T04:56:28.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:28.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.256403+0000", 2024-09-18T04:56:28.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:28.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:28.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.256402+0000", 2024-09-18T04:56:28.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:28.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:28.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.256408+0000", 2024-09-18T04:56:28.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3230000000000003e-06 2024-09-18T04:56:28.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:28.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.256418+0000", 2024-09-18T04:56:28.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7399999999999999e-06 2024-09-18T04:56:28.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:28.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.256425+0000", 2024-09-18T04:56:28.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0620000000000003e-06 2024-09-18T04:56:28.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:28.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.256480+0000", 2024-09-18T04:56:28.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5197e-05 2024-09-18T04:56:28.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:28.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.256512+0000", 2024-09-18T04:56:28.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1133000000000002e-05 2024-09-18T04:56:28.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:28.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.260205+0000", 2024-09-18T04:56:28.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003693034 2024-09-18T04:56:28.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:28.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:28.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:28.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.260229+0000", 2024-09-18T04:56:28.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4141000000000001e-05 2024-09-18T04:56:28.583 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.583 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:28.583 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.583 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:28.583 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:28.583 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:28.586 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:56:28.586 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:28.582+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:28.586 INFO:teuthology.orchestra.run.smithi002.stdout:257 trunc from 0x6eca2e7 to 0xd425739 2024-09-18T04:56:28.586 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:28.586 INFO:teuthology.orchestra.run.smithi002.stdout:260 trunc from 0xd425739 to 0x4dbbe7 2024-09-18T04:56:28.586 INFO:teuthology.orchestra.run.smithi002.stdout:261 write 0x471f432 thru 0x47260e7 (0x6cb6 bytes) 2024-09-18T04:56:28.586 INFO:teuthology.orchestra.run.smithi002.stdout:262 write 0x5e26f48 thru 0x5e2ba17 (0x4ad0 bytes) 2024-09-18T04:56:28.586 INFO:teuthology.orchestra.run.smithi002.stdout:263 trunc from 0x5e2ba18 to 0x12530b 2024-09-18T04:56:28.587 INFO:teuthology.orchestra.run.smithi002.stdout:264 write 0x5e6e3d4 thru 0x5e7250b (0x4138 bytes) 2024-09-18T04:56:28.599 INFO:teuthology.orchestra.run.smithi002.stdout:265 trunc from 0x5e7250c to 0xd259c92 2024-09-18T04:56:28.607 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:28.607 INFO:teuthology.orchestra.run.smithi002.stdout:270 punch from 0xbe274bf to 0xbe296d7, (0x2218 bytes) 2024-09-18T04:56:28.607 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:28.607 INFO:teuthology.orchestra.run.smithi002.stdout:272 write 0xa69dc41 thru 0xa6a222a (0x45ea bytes) 2024-09-18T04:56:28.614 INFO:teuthology.orchestra.run.smithi002.stdout:273 write 0x1bf77af thru 0x1c04ea1 (0xd6f3 bytes) 2024-09-18T04:56:28.622 INFO:teuthology.orchestra.run.smithi002.stdout:274 write 0xab20446 thru 0xab25baa (0x5765 bytes) 2024-09-18T04:56:28.628 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:28.629 INFO:teuthology.orchestra.run.smithi002.stdout:276 trunc from 0xd259c92 to 0xd4bc434 2024-09-18T04:56:28.635 INFO:teuthology.orchestra.run.smithi002.stdout:277 write 0xe2f737 thru 0xe38b83 (0x944d bytes) 2024-09-18T04:56:28.643 INFO:teuthology.orchestra.run.smithi002.stdout:279 read 0x5883760 thru 0x588e3c4 (0xac65 bytes) 2024-09-18T04:56:28.643 INFO:teuthology.orchestra.run.smithi002.stdout:281 read 0xa95c726 thru 0xa96219e (0x5a79 bytes) 2024-09-18T04:56:28.657 INFO:teuthology.orchestra.run.smithi002.stdout:284 trunc from 0xd4bc434 to 0x9cd322 2024-09-18T04:56:28.676 INFO:teuthology.orchestra.run.smithi002.stdout:285 read 0x326e78 thru 0x3351cc (0xe355 bytes) 2024-09-18T04:56:28.686 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:28.684+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:28.699 INFO:teuthology.orchestra.run.smithi002.stdout:287 writesame 0xe230ce7 thru 0xe230d9e data_size 0x4(0xb8 bytes) 2024-09-18T04:56:28.699 INFO:teuthology.orchestra.run.smithi002.stdout:288 read 0x4fe5e1 thru 0x507edd (0x98fd bytes) 2024-09-18T04:56:28.699 INFO:teuthology.orchestra.run.smithi002.stdout:289 write 0x2ff184a thru 0x2ffa5e5 (0x8d9c bytes) 2024-09-18T04:56:28.704 INFO:teuthology.orchestra.run.smithi002.stdout:291 read 0xbf587ac thru 0xbf5d749 (0x4f9e bytes) 2024-09-18T04:56:28.705 INFO:teuthology.orchestra.run.smithi002.stdout:293 writesame 0x4505e8d thru 0x45108ca data_size 0x4d(0xaa3e bytes) 2024-09-18T04:56:28.713 INFO:teuthology.orchestra.run.smithi002.stdout:294 writesame 0x316a11 thru 0x3203b4 data_size 0x2(0x99a4 bytes) 2024-09-18T04:56:28.716 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:56:28.721 INFO:teuthology.orchestra.run.smithi002.stdout:296 write 0x52a8c0e thru 0x52aa589 (0x197c bytes) 2024-09-18T04:56:28.725 INFO:teuthology.orchestra.run.smithi002.stdout:297 writesame 0x7551ae7 thru 0x755c1c1 data_size 0x1(0xa6db bytes) 2024-09-18T04:56:28.735 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:28.736 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:28.736 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:56:28.736 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:28.740 INFO:teuthology.orchestra.run.smithi002.stdout:298 read 0xd4c00f5 thru 0xd4c597f (0x588b bytes) 2024-09-18T04:56:28.740 INFO:teuthology.orchestra.run.smithi002.stdout:300 read 0x4b14d2f thru 0x4b1dd27 (0x8ff9 bytes) 2024-09-18T04:56:28.740 INFO:teuthology.orchestra.run.smithi002.stdout:303 read 0x960c113 thru 0x960cd47 (0xc35 bytes) 2024-09-18T04:56:28.741 INFO:teuthology.orchestra.run.smithi002.stdout:304 write 0x2898cf2 thru 0x28992ce (0x5dd bytes) 2024-09-18T04:56:28.743 INFO:teuthology.orchestra.run.smithi002.stdout:307 writesame 0x65cb08d thru 0x65d8e75 data_size 0x1(0xdde9 bytes) 2024-09-18T04:56:28.745 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:56:28.762 INFO:teuthology.orchestra.run.smithi002.stdout:308 writesame 0x24ef306 thru 0x24f8b9a data_size 0x35(0x9895 bytes) 2024-09-18T04:56:28.766 INFO:teuthology.orchestra.run.smithi002.stdout:truncating to largest ever: 0xeda7706 2024-09-18T04:56:28.767 INFO:teuthology.orchestra.run.smithi002.stdout:309 trunc from 0xe230d9f to 0xeda7706 2024-09-18T04:56:28.772 INFO:teuthology.orchestra.run.smithi002.stdout:310 write 0x584b857 thru 0x584d254 (0x19fe bytes) 2024-09-18T04:56:28.775 INFO:teuthology.orchestra.run.smithi002.stdout:312 write 0xa0bbec8 thru 0xa0cbc55 (0xfd8e bytes) 2024-09-18T04:56:28.786 INFO:teuthology.orchestra.run.smithi002.stdout:314 trunc from 0xeda7706 to 0x99e2ca7 2024-09-18T04:56:28.787 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:28.784+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:28.795 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:28.795 INFO:teuthology.orchestra.run.smithi002.stdout:317 read 0x4d7e84e thru 0x4d8cb34 (0xe2e7 bytes) 2024-09-18T04:56:28.795 INFO:teuthology.orchestra.run.smithi002.stdout:318 write 0x80ba20c thru 0x80c3b2a (0x991f bytes) 2024-09-18T04:56:28.799 INFO:teuthology.orchestra.run.smithi002.stdout:320 punch from 0x915a29f to 0x91617f3, (0x7554 bytes) 2024-09-18T04:56:28.804 INFO:teuthology.orchestra.run.smithi002.stdout:321 trunc from 0x99e2ca7 to 0xd7f31a4 2024-09-18T04:56:28.809 INFO:teuthology.orchestra.run.smithi002.stdout:322 write 0xaee2e99 thru 0xaee5289 (0x23f1 bytes) 2024-09-18T04:56:28.818 INFO:teuthology.orchestra.run.smithi002.stdout:324 write 0xe490745 thru 0xe4938b0 (0x316c bytes) 2024-09-18T04:56:28.832 INFO:teuthology.orchestra.run.smithi002.stdout:325 trunc from 0xe4938b1 to 0x4536a86 2024-09-18T04:56:28.850 INFO:teuthology.orchestra.run.smithi002.stdout:326 writesame 0x4edb33f thru 0x4edd6d2 data_size 0x9(0x2394 bytes) 2024-09-18T04:56:28.850 INFO:teuthology.orchestra.run.smithi002.stdout:328 punch from 0x2af64db to 0x2afdcc5, (0x77ea bytes) 2024-09-18T04:56:28.850 INFO:teuthology.orchestra.run.smithi002.stdout:330 read 0x19eabd6 thru 0x19f22bf (0x76ea bytes) 2024-09-18T04:56:28.851 INFO:teuthology.orchestra.run.smithi002.stdout:331 writesame 0x14cc893 thru 0x14d9ba6 data_size 0xab(0xd314 bytes) 2024-09-18T04:56:28.854 INFO:teuthology.orchestra.run.smithi002.stdout:332 write 0x2d5b185 thru 0x2d683d7 (0xd253 bytes) 2024-09-18T04:56:28.856 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:56:28.856 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:56:28.858 INFO:teuthology.orchestra.run.smithi002.stdout:333 write 0x4eba3f3 thru 0x4ec802e (0xdc3c bytes) 2024-09-18T04:56:28.862 INFO:teuthology.orchestra.run.smithi002.stdout:335 write 0x1a18b16 thru 0x1a1d762 (0x4c4d bytes) 2024-09-18T04:56:28.865 INFO:teuthology.orchestra.run.smithi002.stdout:336 read 0x7bfd1c thru 0x7c1565 (0x184a bytes) 2024-09-18T04:56:28.865 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:28.865 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:28.865 INFO:teuthology.orchestra.run.smithi002.stdout:339 punch from 0x285218c to 0x2853064, (0xed8 bytes) 2024-09-18T04:56:28.865 INFO:teuthology.orchestra.run.smithi002.stdout:341 write 0x2f6ab31 thru 0x2f75264 (0xa734 bytes) 2024-09-18T04:56:28.875 INFO:teuthology.orchestra.run.smithi002.stdout:342 trunc from 0x4edd6d3 to 0xadaaf5b 2024-09-18T04:56:28.877 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:56:28.877 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:56:28.883 INFO:teuthology.orchestra.run.smithi002.stdout:345 trunc from 0xadaaf5b to 0xd2cc621 2024-09-18T04:56:28.887 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:28.884+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:28.888 INFO:teuthology.orchestra.run.smithi002.stdout:346 read 0x9001022 thru 0x9003458 (0x2437 bytes) 2024-09-18T04:56:28.888 INFO:teuthology.orchestra.run.smithi002.stdout:347 punch from 0x7d64551 to 0x7d68972, (0x4421 bytes) 2024-09-18T04:56:28.888 INFO:teuthology.orchestra.run.smithi002.stdout:349 write 0xae9fb10 thru 0xaea6668 (0x6b59 bytes) 2024-09-18T04:56:28.892 INFO:teuthology.orchestra.run.smithi002.stdout:350 punch from 0x15bf036 to 0x15bf0a7, (0x71 bytes) 2024-09-18T04:56:28.892 INFO:teuthology.orchestra.run.smithi002.stdout:352 read 0x7dbf859 thru 0x7dcbd3e (0xc4e6 bytes) 2024-09-18T04:56:28.892 INFO:teuthology.orchestra.run.smithi002.stdout:353 clone 4 order 24 su 1048576 sc 6 2024-09-18T04:56:28.894 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:56:28.894 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:56:28.899 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:28.899 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:28.899 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:56:28.899 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:56:28.899 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:28.908 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:56:28.910 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:56:28.910 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:56:28.926 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:56:28.926 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:56:28.944 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:56:28.945 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:56:28.961 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:56:28.961 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:56:28.977 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:56:28.978 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:56:28.988 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:28.985+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:29.013 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:29.013 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:29.021 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:56:29.105 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:29.103+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:29.150 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-18T04:56:29.167 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:29.167 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:29.177 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:56:29.190 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:29.187+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:29.227 INFO:teuthology.orchestra.run.smithi002.stdout:flattening image image_client.0-clone3 2024-09-18T04:56:29.292 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:29.289+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:29.319 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:29.319 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:29.328 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:56:29.386 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-18T04:56:29.392 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:29.389+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:29.444 INFO:teuthology.orchestra.run.smithi002.stdout:355 trunc from 0xd2cc621 to 0xe25cf7a 2024-09-18T04:56:29.452 INFO:teuthology.orchestra.run.smithi002.stdout:356 read 0xf4b77b thru 0xf58e72 (0xd6f8 bytes) 2024-09-18T04:56:29.452 INFO:teuthology.orchestra.run.smithi002.stdout:358 write 0x2c230e7 thru 0x2c25b88 (0x2aa2 bytes) 2024-09-18T04:56:29.470 INFO:teuthology.orchestra.run.smithi002.stdout:359 trunc from 0xe25cf7a to 0x1c16beb 2024-09-18T04:56:29.477 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:29.477 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:29.487 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:56:29.490 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:29.493 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:29.490+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:29.506 INFO:teuthology.orchestra.run.smithi002.stdout:363 write 0xb87a565 thru 0xb888edb (0xe977 bytes) 2024-09-18T04:56:29.509 INFO:teuthology.orchestra.run.smithi002.stdout:365 writesame 0x210e658 thru 0x2117178 data_size 0x1(0x8b21 bytes) 2024-09-18T04:56:29.526 INFO:teuthology.orchestra.run.smithi002.stdout:366 punch from 0xaa92ffd to 0xaa9fb69, (0xcb6c bytes) 2024-09-18T04:56:29.526 INFO:teuthology.orchestra.run.smithi002.stdout:367 read 0x15652f2 thru 0x1572607 (0xd316 bytes) 2024-09-18T04:56:29.526 INFO:teuthology.orchestra.run.smithi002.stdout:368 write 0x7d37419 thru 0x7d4660e (0xf1f6 bytes) 2024-09-18T04:56:29.536 INFO:teuthology.orchestra.run.smithi002.stdout:370 writesame 0xc377f33 thru 0xc37f76f data_size 0x1(0x783d bytes) 2024-09-18T04:56:29.536 INFO:teuthology.orchestra.run.smithi002.stdout:371 punch from 0x33d9053 to 0x33ddb74, (0x4b21 bytes) 2024-09-18T04:56:29.536 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:29.536 INFO:teuthology.orchestra.run.smithi002.stdout:373 write 0x8fe8ec6 thru 0x8ff321f (0xa35a bytes) 2024-09-18T04:56:29.538 INFO:teuthology.orchestra.run.smithi002.stdout:374 punch from 0x559d177 to 0x55aa629, (0xd4b2 bytes) 2024-09-18T04:56:29.538 INFO:teuthology.orchestra.run.smithi002.stdout:375 read 0xc073879 thru 0xc077016 (0x379e bytes) 2024-09-18T04:56:29.538 INFO:teuthology.orchestra.run.smithi002.stdout:376 writesame 0xaad7160 thru 0xaadc2a1 data_size 0x3(0x5142 bytes) 2024-09-18T04:56:29.542 INFO:teuthology.orchestra.run.smithi002.stdout:377 read 0x7c98d1f thru 0x7ca6b34 (0xde16 bytes) 2024-09-18T04:56:29.543 INFO:teuthology.orchestra.run.smithi002.stdout:378 punch from 0xa308728 to 0xa316d62, (0xe63a bytes) 2024-09-18T04:56:29.543 INFO:teuthology.orchestra.run.smithi002.stdout:380 read 0x9079b3e thru 0x90808a1 (0x6d64 bytes) 2024-09-18T04:56:29.543 INFO:teuthology.orchestra.run.smithi002.stdout:382 read 0x1ad2250 thru 0x1adf929 (0xd6da bytes) 2024-09-18T04:56:29.544 INFO:teuthology.orchestra.run.smithi002.stdout:383 read 0x63c9b95 thru 0x63d358e (0x99fa bytes) 2024-09-18T04:56:29.544 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:29.544 INFO:teuthology.orchestra.run.smithi002.stdout:386 read 0xb2b7bd6 thru 0xb2be29b (0x66c6 bytes) 2024-09-18T04:56:29.544 INFO:teuthology.orchestra.run.smithi002.stdout:388 write 0x856e3b thru 0x857edc (0x10a2 bytes) 2024-09-18T04:56:29.545 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:29.555 INFO:teuthology.orchestra.run.smithi002.stdout:390 trunc from 0xc37f770 to 0x3537e0b 2024-09-18T04:56:29.582 INFO:teuthology.orchestra.run.smithi002.stdout:393 write 0xa917858 thru 0xa9180a8 (0x851 bytes) 2024-09-18T04:56:29.593 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:29.591+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:29.594 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:29.594 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:29.595 INFO:teuthology.orchestra.run.smithi002.stdout:394 trunc from 0xa9180a9 to 0x2217f5c 2024-09-18T04:56:29.603 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:56:29.605 INFO:teuthology.orchestra.run.smithi002.stdout:395 read 0x1dd923c thru 0x1ddcab7 (0x387c bytes) 2024-09-18T04:56:29.605 INFO:teuthology.orchestra.run.smithi002.stdout:396 read 0x1fff86d thru 0x2005477 (0x5c0b bytes) 2024-09-18T04:56:29.606 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:29.606 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:29.606 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:29.609 INFO:teuthology.orchestra.run.smithi002.stdout:401 trunc from 0x2217f5c to 0x4c29b37 2024-09-18T04:56:29.616 INFO:teuthology.orchestra.run.smithi002.stdout:402 trunc from 0x4c29b37 to 0x638c570 2024-09-18T04:56:29.633 INFO:teuthology.orchestra.run.smithi002.stdout:403 write 0xc090684 thru 0xc0957c0 (0x513d bytes) 2024-09-18T04:56:29.636 INFO:teuthology.orchestra.run.smithi002.stdout:405 write 0x934b960 thru 0x934d1fe (0x189f bytes) 2024-09-18T04:56:29.639 INFO:teuthology.orchestra.run.smithi002.stdout:407 write 0xa34dd9b thru 0xa34fb0a (0x1d70 bytes) 2024-09-18T04:56:29.641 INFO:teuthology.orchestra.run.smithi002.stdout:408 punch from 0xb061da1 to 0xb0660c2, (0x4321 bytes) 2024-09-18T04:56:29.641 INFO:teuthology.orchestra.run.smithi002.stdout:409 read 0x7509e09 thru 0x75107bd (0x69b5 bytes) 2024-09-18T04:56:29.642 INFO:teuthology.orchestra.run.smithi002.stdout:410 writesame 0x1893c77 thru 0x1895471 data_size 0x1(0x17fb bytes) 2024-09-18T04:56:29.645 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:29.652 INFO:teuthology.orchestra.run.smithi002.stdout:412 trunc from 0xc0957c1 to 0x683ba72 2024-09-18T04:56:29.670 INFO:teuthology.orchestra.run.smithi002.stdout:413 writesame 0x945356f thru 0x945390f data_size 0x1(0x3a1 bytes) 2024-09-18T04:56:29.672 INFO:teuthology.orchestra.run.smithi002.stdout:414 trunc from 0x9453910 to 0xab98afb 2024-09-18T04:56:29.680 INFO:teuthology.orchestra.run.smithi002.stdout:415 trunc from 0xab98afb to 0x8e6e688 2024-09-18T04:56:29.686 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:29.686 INFO:teuthology.orchestra.run.smithi002.stdout:417 write 0x5b3c1eb thru 0x5b440ae (0x7ec4 bytes) 2024-09-18T04:56:29.689 INFO:teuthology.orchestra.run.smithi002.stdout:418 write 0x4628364 thru 0x46293c2 (0x105f bytes) 2024-09-18T04:56:29.689 INFO:teuthology.orchestra.run.smithi002.stdout:419 read 0x705ba25 thru 0x705dd22 (0x22fe bytes) 2024-09-18T04:56:29.695 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:29.693+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:29.699 INFO:teuthology.orchestra.run.smithi002.stdout:422 writesame 0xd0feeaf thru 0xd0ff1f1 data_size 0x1(0x343 bytes) 2024-09-18T04:56:29.699 INFO:teuthology.orchestra.run.smithi002.stdout:423 punch from 0x4e63b3b to 0x4e6f763, (0xbc28 bytes) 2024-09-18T04:56:29.699 INFO:teuthology.orchestra.run.smithi002.stdout:424 writesame 0xa01f48d thru 0xa01f8d8 data_size 0x16(0x44c bytes) 2024-09-18T04:56:29.701 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:29.701 INFO:teuthology.orchestra.run.smithi002.stdout:426 write 0x6d51465 thru 0x6d54df2 (0x398e bytes) 2024-09-18T04:56:29.703 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:29.703 INFO:teuthology.orchestra.run.smithi002.stdout:429 read 0x5653538 thru 0x565ef3a (0xba03 bytes) 2024-09-18T04:56:29.703 INFO:teuthology.orchestra.run.smithi002.stdout:430 read 0xdf6039 thru 0xe004b5 (0xa47d bytes) 2024-09-18T04:56:29.704 INFO:teuthology.orchestra.run.smithi002.stdout:433 write 0xb1e5f73 thru 0xb1ef775 (0x9803 bytes) 2024-09-18T04:56:29.705 INFO:teuthology.orchestra.run.smithi002.stdout:436 write 0x7711483 thru 0x7711b82 (0x700 bytes) 2024-09-18T04:56:29.708 INFO:teuthology.orchestra.run.smithi002.stdout:437 write 0x340166e thru 0x3407505 (0x5e98 bytes) 2024-09-18T04:56:29.709 INFO:teuthology.orchestra.run.smithi002.stdout:438 read 0x8face98 thru 0x8fb2cec (0x5e55 bytes) 2024-09-18T04:56:29.709 INFO:teuthology.orchestra.run.smithi002.stdout:439 write 0x5341326 thru 0x534793a (0x6615 bytes) 2024-09-18T04:56:29.711 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:29.711 INFO:teuthology.orchestra.run.smithi002.stdout:442 read 0x95307fd thru 0x9535669 (0x4e6d bytes) 2024-09-18T04:56:29.711 INFO:teuthology.orchestra.run.smithi002.stdout:443 writesame 0x3bed7d thru 0x3c7370 data_size 0x2(0x85f4 bytes) 2024-09-18T04:56:29.720 INFO:teuthology.orchestra.run.smithi002.stdout:445 read 0x86380e2 thru 0x86457ff (0xd71e bytes) 2024-09-18T04:56:29.720 INFO:teuthology.orchestra.run.smithi002.stdout:446 read 0x1975921 thru 0x1983707 (0xdde7 bytes) 2024-09-18T04:56:29.720 INFO:teuthology.orchestra.run.smithi002.stdout:447 write 0x71aaa60 thru 0x71b0730 (0x5cd1 bytes) 2024-09-18T04:56:29.727 INFO:teuthology.orchestra.run.smithi002.stdout:449 writesame 0xd6d036e thru 0xd6d427b data_size 0x7(0x3f0e bytes) 2024-09-18T04:56:29.727 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:29.727 INFO:teuthology.orchestra.run.smithi002.stdout:451 punch from 0x40b50cb to 0x40ba2df, (0x5214 bytes) 2024-09-18T04:56:29.732 INFO:teuthology.orchestra.run.smithi002.stdout:452 write 0xdcb6a39 thru 0xdcc4dd5 (0xe39d bytes) 2024-09-18T04:56:29.735 INFO:teuthology.orchestra.run.smithi002.stdout:454 write 0xb6898fb thru 0xb68bbba (0x22c0 bytes) 2024-09-18T04:56:29.744 INFO:teuthology.orchestra.run.smithi002.stdout:455 trunc from 0xdcc4dd6 to 0x8470a3f 2024-09-18T04:56:29.753 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:29.754 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:29.754 INFO:teuthology.orchestra.run.smithi002.stdout:456 punch from 0x65dffe3 to 0x65e4a21, (0x4a3e bytes) 2024-09-18T04:56:29.757 INFO:teuthology.orchestra.run.smithi002.stdout:457 trunc from 0x8470a3f to 0xac12552 2024-09-18T04:56:29.763 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:56:29.763 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:29.764 INFO:teuthology.orchestra.run.smithi002.stdout:459 write 0x8b97fbd thru 0x8b9fea9 (0x7eed bytes) 2024-09-18T04:56:29.764 INFO:teuthology.orchestra.run.smithi002.stdout:462 read 0x6b8dc2b thru 0x6b956d1 (0x7aa7 bytes) 2024-09-18T04:56:29.764 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:29.765 INFO:teuthology.orchestra.run.smithi002.stdout:464 read 0x777d3ba thru 0x7785969 (0x85b0 bytes) 2024-09-18T04:56:29.765 INFO:teuthology.orchestra.run.smithi002.stdout:465 read 0x2b7b302 thru 0x2b849cf (0x96ce bytes) 2024-09-18T04:56:29.765 INFO:teuthology.orchestra.run.smithi002.stdout:466 write 0xabd41cd thru 0xabde0a3 (0x9ed7 bytes) 2024-09-18T04:56:29.767 INFO:teuthology.orchestra.run.smithi002.stdout:467 read 0x21973bf thru 0x21a1dba (0xa9fc bytes) 2024-09-18T04:56:29.772 INFO:teuthology.orchestra.run.smithi002.stdout:469 trunc from 0xac12552 to 0x6a86deb 2024-09-18T04:56:29.778 INFO:teuthology.orchestra.run.smithi002.stdout:470 read 0x2a83f16 thru 0x2a85c7d (0x1d68 bytes) 2024-09-18T04:56:29.778 INFO:teuthology.orchestra.run.smithi002.stdout:471 read 0x2c4844f thru 0x2c4fdae (0x7960 bytes) 2024-09-18T04:56:29.778 INFO:teuthology.orchestra.run.smithi002.stdout:472 writesame 0x6980971 thru 0x6989e83 data_size 0x1(0x9513 bytes) 2024-09-18T04:56:29.796 INFO:teuthology.orchestra.run.smithi002.stdout:474 punch from 0x13f88b3 to 0x13fa46e, (0x1bbb bytes) 2024-09-18T04:56:29.802 INFO:teuthology.orchestra.run.smithi002.stdout:475 writesame 0x75dcb16 thru 0x75dffaf data_size 0x1(0x349a bytes) 2024-09-18T04:56:29.804 INFO:teuthology.orchestra.run.smithi002.stdout:476 read 0x57258c6 thru 0x572746f (0x1baa bytes) 2024-09-18T04:56:29.804 INFO:teuthology.orchestra.run.smithi002.stdout:477 write 0x5bea8f0 thru 0x5beaef8 (0x609 bytes) 2024-09-18T04:56:29.805 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:29.802+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:29.818 INFO:teuthology.orchestra.run.smithi002.stdout:478 writesame 0xdfcf6fc thru 0xdfd4d02 data_size 0x3(0x5607 bytes) 2024-09-18T04:56:29.818 INFO:teuthology.orchestra.run.smithi002.stdout:480 write 0xd12691 thru 0xd207f4 (0xe164 bytes) 2024-09-18T04:56:29.819 INFO:teuthology.orchestra.run.smithi002.stdout:482 punch from 0xdaa8e0f to 0xdab7fd1, (0xf1c2 bytes) 2024-09-18T04:56:29.819 INFO:teuthology.orchestra.run.smithi002.stdout:483 read 0xb6bdf9 thru 0xb7b8ec (0xfaf4 bytes) 2024-09-18T04:56:29.823 INFO:teuthology.orchestra.run.smithi002.stdout:484 trunc from 0xdfd4d03 to 0xb4311f9 2024-09-18T04:56:29.830 INFO:teuthology.orchestra.run.smithi002.stdout:486 writesame 0xab9b106 thru 0xaba09a8 data_size 0x1(0x58a3 bytes) 2024-09-18T04:56:29.840 INFO:teuthology.orchestra.run.smithi002.stdout:487 punch from 0x9e6146d to 0x9e6252b, (0x10be bytes) 2024-09-18T04:56:29.851 INFO:teuthology.orchestra.run.smithi002.stdout:488 trunc from 0xb4311f9 to 0x1721199 2024-09-18T04:56:29.863 INFO:teuthology.orchestra.run.smithi002.stdout:489 punch from 0x7c83d1 to 0x7d769a, (0xf2c9 bytes) 2024-09-18T04:56:29.871 INFO:teuthology.orchestra.run.smithi002.stdout:490 trunc from 0x1721199 to 0x83c6237 2024-09-18T04:56:29.876 INFO:teuthology.orchestra.run.smithi002.stdout:491 punch from 0x6df18fa to 0x6dfbff2, (0xa6f8 bytes) 2024-09-18T04:56:29.876 INFO:teuthology.orchestra.run.smithi002.stdout:492 write 0xdc574c thru 0xdd4da5 (0xf65a bytes) 2024-09-18T04:56:29.878 INFO:teuthology.orchestra.run.smithi002.stdout:494 write 0x6eb5dc6 thru 0x6ec48dc (0xeb17 bytes) 2024-09-18T04:56:29.881 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:29.881 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:29.881 INFO:teuthology.orchestra.run.smithi002.stdout:497 write 0x6805537 thru 0x680d040 (0x7b0a bytes) 2024-09-18T04:56:29.882 INFO:teuthology.orchestra.run.smithi002.stdout:498 read 0x5b8ddfd thru 0x5b9b892 (0xda96 bytes) 2024-09-18T04:56:29.882 INFO:teuthology.orchestra.run.smithi002.stdout:499 read 0x1da8cf9 thru 0x1daec96 (0x5f9e bytes) 2024-09-18T04:56:29.882 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:29.885 INFO:teuthology.orchestra.run.smithi002.stdout:501 trunc from 0x83c6237 to 0x9c7bb87 2024-09-18T04:56:29.891 INFO:teuthology.orchestra.run.smithi002.stdout:503 trunc from 0x9c7bb87 to 0xb42bd53 2024-09-18T04:56:29.896 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:29.896 INFO:teuthology.orchestra.run.smithi002.stdout:506 write 0x9905449 thru 0x9907b7a (0x2732 bytes) 2024-09-18T04:56:29.897 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:29.893+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:29.899 INFO:teuthology.orchestra.run.smithi002.stdout:507 clone 5 order 22 su 1048576 sc 3 2024-09-18T04:56:29.911 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:29.911 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:29.920 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:56:29.997 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:29.995+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:30.027 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:30.027 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:30.036 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:56:30.098 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:30.096+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:30.142 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:30.143 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:30.152 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:56:30.198 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:30.195+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:30.209 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone4 from 0xb42bd53 (overlap 0x1721199) to 0x14d0fd6 2024-09-18T04:56:30.269 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:30.269 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:30.277 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-18T04:56:30.277 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:56:30.300 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:30.298+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:30.401 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:30.398+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:30.437 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:30.437 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:30.447 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:56:30.522 INFO:teuthology.orchestra.run.smithi002.stdout:508 punch from 0x72f1778 to 0x72f6c83, (0x550b bytes) 2024-09-18T04:56:30.522 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:30.519+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:30.527 INFO:teuthology.orchestra.run.smithi002.stdout:509 write 0x6b406ab thru 0x6b4b10a (0xaa60 bytes) 2024-09-18T04:56:30.531 INFO:teuthology.orchestra.run.smithi002.stdout:510 writesame 0x3086083 thru 0x3095882 data_size 0x80(0xf800 bytes) 2024-09-18T04:56:30.535 INFO:teuthology.orchestra.run.smithi002.stdout:512 write 0x1e41f81 thru 0x1e47b15 (0x5b95 bytes) 2024-09-18T04:56:30.539 INFO:teuthology.orchestra.run.smithi002.stdout:513 read 0x306fc64 thru 0x30739d8 (0x3d75 bytes) 2024-09-18T04:56:30.540 INFO:teuthology.orchestra.run.smithi002.stdout:514 writesame 0x5018422 thru 0x5019e12 data_size 0x1(0x19f1 bytes) 2024-09-18T04:56:30.545 INFO:teuthology.orchestra.run.smithi002.stdout:515 punch from 0x51d197b to 0x51d4f68, (0x35ed bytes) 2024-09-18T04:56:30.545 INFO:teuthology.orchestra.run.smithi002.stdout:516 punch from 0x3842172 to 0x384539e, (0x322c bytes) 2024-09-18T04:56:30.545 INFO:teuthology.orchestra.run.smithi002.stdout:517 writesame 0x78a04c3 thru 0x78afedd data_size 0x1(0xfa1b bytes) 2024-09-18T04:56:30.577 INFO:teuthology.orchestra.run.smithi002.stdout:518 write 0x6a313e9 thru 0x6a3f97e (0xe596 bytes) 2024-09-18T04:56:30.580 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:30.580 INFO:teuthology.orchestra.run.smithi002.stdout:520 trunc from 0xb42bd53 to 0xb0e2416 2024-09-18T04:56:30.594 INFO:teuthology.orchestra.run.smithi002.stdout:522 read 0x5a767b thru 0x5b12ac (0x9c32 bytes) 2024-09-18T04:56:30.594 INFO:teuthology.orchestra.run.smithi002.stdout:524 read 0x91f5958 thru 0x920340e (0xdab7 bytes) 2024-09-18T04:56:30.594 INFO:teuthology.orchestra.run.smithi002.stdout:527 read 0x72e2c05 thru 0x72e9816 (0x6c12 bytes) 2024-09-18T04:56:30.595 INFO:teuthology.orchestra.run.smithi002.stdout:528 trunc from 0xb0e2416 to 0xc0014d0 2024-09-18T04:56:30.601 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:30.601 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:30.603 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:30.601+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:30.609 INFO:teuthology.orchestra.run.smithi002.stdout:529 writesame 0xd2452f2 thru 0xd254955 data_size 0x4(0xf664 bytes) 2024-09-18T04:56:30.611 INFO:teuthology.orchestra.run.smithi002.stdout:530 trunc from 0xd254956 to 0xec025d7 2024-09-18T04:56:30.612 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:56:30.627 INFO:teuthology.orchestra.run.smithi002.stdout:531 trunc from 0xec025d7 to 0x6252c4c 2024-09-18T04:56:30.645 INFO:teuthology.orchestra.run.smithi002.stdout:533 trunc from 0x6252c4c to 0x24cb6d2 2024-09-18T04:56:30.661 INFO:teuthology.orchestra.run.smithi002.stdout:534 trunc from 0x24cb6d2 to 0x693af36 2024-09-18T04:56:30.673 INFO:teuthology.orchestra.run.smithi002.stdout:535 trunc from 0x693af36 to 0xad2fe8 2024-09-18T04:56:30.701 INFO:teuthology.orchestra.run.smithi002.stdout:536 write 0xa3beb65 thru 0xa3bf7ca (0xc66 bytes) 2024-09-18T04:56:30.704 INFO:teuthology.orchestra.run.smithi002.stdout:537 writesame 0x684bad9 thru 0x6857dc2 data_size 0x3d(0xc2ea bytes) 2024-09-18T04:56:30.704 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:30.700+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:30.706 INFO:teuthology.orchestra.run.smithi002.stdout:539 read 0x21bfd13 thru 0x21cc0c8 (0xc3b6 bytes) 2024-09-18T04:56:30.706 INFO:teuthology.orchestra.run.smithi002.stdout:540 write 0x55e07d5 thru 0x55eed38 (0xe564 bytes) 2024-09-18T04:56:30.710 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:30.710 INFO:teuthology.orchestra.run.smithi002.stdout:542 punch from 0x67f0a61 to 0x67fdbde, (0xd17d bytes) 2024-09-18T04:56:30.710 INFO:teuthology.orchestra.run.smithi002.stdout:543 punch from 0x2d5f6f9 to 0x2d6d963, (0xe26a bytes) 2024-09-18T04:56:30.721 INFO:teuthology.orchestra.run.smithi002.stdout:544 trunc from 0xa3bf7cb to 0x22964e 2024-09-18T04:56:30.733 INFO:teuthology.orchestra.run.smithi002.stdout:545 read 0x2960d thru 0x2e221 (0x4c15 bytes) 2024-09-18T04:56:30.733 INFO:teuthology.orchestra.run.smithi002.stdout:546 read 0x1ab88c thru 0x1b21cc (0x6941 bytes) 2024-09-18T04:56:30.745 INFO:teuthology.orchestra.run.smithi002.stdout:547 write 0x539df0a thru 0x539ed73 (0xe6a bytes) 2024-09-18T04:56:30.748 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:30.748 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:30.759 INFO:teuthology.orchestra.run.smithi002.stdout:548 write 0xa282e74 thru 0xa287ab4 (0x4c41 bytes) 2024-09-18T04:56:30.760 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:56:30.761 INFO:teuthology.orchestra.run.smithi002.stdout:549 writesame 0x7ae7546 thru 0x7af0447 data_size 0x23(0x8f02 bytes) 2024-09-18T04:56:30.763 INFO:teuthology.orchestra.run.smithi002.stdout:551 punch from 0x5d32b7f to 0x5d3608b, (0x350c bytes) 2024-09-18T04:56:30.763 INFO:teuthology.orchestra.run.smithi002.stdout:552 writesame 0x7799551 thru 0x779d4a8 data_size 0x4(0x3f58 bytes) 2024-09-18T04:56:30.767 INFO:teuthology.orchestra.run.smithi002.stdout:553 writesame 0x27ce41b thru 0x27d117a data_size 0x16(0x2d60 bytes) 2024-09-18T04:56:30.772 INFO:teuthology.orchestra.run.smithi002.stdout:554 read 0x745104 thru 0x74acfe (0x5bfb bytes) 2024-09-18T04:56:30.772 INFO:teuthology.orchestra.run.smithi002.stdout:555 write 0x5b5a158 thru 0x5b5bcfd (0x1ba6 bytes) 2024-09-18T04:56:30.775 INFO:teuthology.orchestra.run.smithi002.stdout:559 clone 6 order 25 su 262144 sc 11 2024-09-18T04:56:30.805 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:30.801+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:30.867 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:30.868 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:30.876 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:56:30.905 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:30.903+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:30.988 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:56:31.006 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:31.004+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:31.023 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:31.023 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:31.033 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:56:31.115 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:56:31.115 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:56:31.116 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:31.112+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:31.132 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:56:31.133 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:56:31.148 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:56:31.149 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:56:31.165 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T04:56:31.168 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:56:31.168 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:56:31.185 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:56:31.186 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:56:31.194 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:31.194 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:56:31.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:56:31.194 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:56:31.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.194 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:555 3.0 e23/19 3:0124d2fa:::rbd_data.11acdc0eab99.000000000000009e:head v 23'1, mlcod=0'0)", 2024-09-18T04:56:31.194 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:26.644209+0000", 2024-09-18T04:56:31.195 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.5428310170000001, 2024-09-18T04:56:31.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00081704399999999995, 2024-09-18T04:56:31.195 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.195 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.195 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.644209+0000", 2024-09-18T04:56:31.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.644209+0000", 2024-09-18T04:56:31.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.644207+0000", 2024-09-18T04:56:31.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:31.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.644231+0000", 2024-09-18T04:56:31.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3686e-05 2024-09-18T04:56:31.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.644248+0000", 2024-09-18T04:56:31.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6988000000000001e-05 2024-09-18T04:56:31.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.644261+0000", 2024-09-18T04:56:31.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2924e-05 2024-09-18T04:56:31.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.644310+0000", 2024-09-18T04:56:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8619000000000003e-05 2024-09-18T04:56:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.644325+0000", 2024-09-18T04:56:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5107e-05 2024-09-18T04:56:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.644982+0000", 2024-09-18T04:56:31.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00065705600000000005 2024-09-18T04:56:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.645026+0000", 2024-09-18T04:56:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3977000000000003e-05 2024-09-18T04:56:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:557 3.4 e23/19 3:21ae0c4f:::rbd_data.11acdc0eab99.0000000000000029:head v 23'5, mlcod=23'3)", 2024-09-18T04:56:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:26.647465+0000", 2024-09-18T04:56:31.199 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.5395748229999997, 2024-09-18T04:56:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00093274199999999997, 2024-09-18T04:56:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.647465+0000", 2024-09-18T04:56:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.647465+0000", 2024-09-18T04:56:31.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.647464+0000", 2024-09-18T04:56:31.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:31.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.647506+0000", 2024-09-18T04:56:31.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2378e-05 2024-09-18T04:56:31.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.647522+0000", 2024-09-18T04:56:31.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5344000000000001e-05 2024-09-18T04:56:31.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.647529+0000", 2024-09-18T04:56:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2910000000000002e-06 2024-09-18T04:56:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.647578+0000", 2024-09-18T04:56:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8776000000000002e-05 2024-09-18T04:56:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.647599+0000", 2024-09-18T04:56:31.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1382999999999998e-05 2024-09-18T04:56:31.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.648359+0000", 2024-09-18T04:56:31.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00075958099999999995 2024-09-18T04:56:31.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:26.648398+0000", 2024-09-18T04:56:31.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8911000000000001e-05 2024-09-18T04:56:31.204 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.204 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.204 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:609 3.0 e24/19 3:04479396:::rbd_id.image_client.0-clone2:head v 24'2, mlcod=0'0)", 2024-09-18T04:56:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.179433+0000", 2024-09-18T04:56:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.0076070440000002, 2024-09-18T04:56:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00083956000000000005, 2024-09-18T04:56:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.179433+0000", 2024-09-18T04:56:31.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.179433+0000", 2024-09-18T04:56:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.179433+0000", 2024-09-18T04:56:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:56:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.179438+0000", 2024-09-18T04:56:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5419999999999996e-06 2024-09-18T04:56:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.179456+0000", 2024-09-18T04:56:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7938e-05 2024-09-18T04:56:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.179463+0000", 2024-09-18T04:56:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2790000000000001e-06 2024-09-18T04:56:31.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.179525+0000", 2024-09-18T04:56:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1458999999999996e-05 2024-09-18T04:56:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.179554+0000", 2024-09-18T04:56:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8804999999999999e-05 2024-09-18T04:56:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.180233+0000", 2024-09-18T04:56:31.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067958600000000004 2024-09-18T04:56:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.180272+0000", 2024-09-18T04:56:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9106000000000001e-05 2024-09-18T04:56:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:697 3.0 e24/19 3:0124d2fa:::rbd_data.11acdc0eab99.000000000000009e:head v 24'6, mlcod=24'3)", 2024-09-18T04:56:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.214917+0000", 2024-09-18T04:56:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.9721223759999997, 2024-09-18T04:56:31.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011006900000000001, 2024-09-18T04:56:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214917+0000", 2024-09-18T04:56:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214917+0000", 2024-09-18T04:56:31.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214916+0000", 2024-09-18T04:56:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214924+0000", 2024-09-18T04:56:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0749999999999998e-06 2024-09-18T04:56:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214940+0000", 2024-09-18T04:56:31.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5644000000000001e-05 2024-09-18T04:56:31.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214949+0000", 2024-09-18T04:56:31.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5180000000000004e-06 2024-09-18T04:56:31.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.215005+0000", 2024-09-18T04:56:31.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6187999999999998e-05 2024-09-18T04:56:31.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.215027+0000", 2024-09-18T04:56:31.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1815999999999999e-05 2024-09-18T04:56:31.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.215983+0000", 2024-09-18T04:56:31.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00095677499999999999 2024-09-18T04:56:31.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.216018+0000", 2024-09-18T04:56:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4604000000000003e-05 2024-09-18T04:56:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:689 3.4 e24/19 3:21ae0c4f:::rbd_data.11acdc0eab99.0000000000000029:head v 24'7, mlcod=23'4)", 2024-09-18T04:56:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.214992+0000", 2024-09-18T04:56:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.9720480719999998, 2024-09-18T04:56:31.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001477602, 2024-09-18T04:56:31.215 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.215 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.215 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214992+0000", 2024-09-18T04:56:31.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214992+0000", 2024-09-18T04:56:31.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214991+0000", 2024-09-18T04:56:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214997+0000", 2024-09-18T04:56:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1709999999999999e-06 2024-09-18T04:56:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.215006+0000", 2024-09-18T04:56:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8799999999999997e-06 2024-09-18T04:56:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.215012+0000", 2024-09-18T04:56:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0870000000000003e-06 2024-09-18T04:56:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.215068+0000", 2024-09-18T04:56:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5350999999999999e-05 2024-09-18T04:56:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.215087+0000", 2024-09-18T04:56:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9565e-05 2024-09-18T04:56:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.216432+0000", 2024-09-18T04:56:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013447439999999999 2024-09-18T04:56:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.216469+0000", 2024-09-18T04:56:31.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7370999999999998e-05 2024-09-18T04:56:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:923 3.0 e24/19 3:0546939f:::rbd_data.11ac33624de9.0000000000000003:head v 24'11, mlcod=24'9)", 2024-09-18T04:56:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.461453+0000", 2024-09-18T04:56:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.7255866499999999, 2024-09-18T04:56:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010622870000000001, 2024-09-18T04:56:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.461453+0000", 2024-09-18T04:56:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.461453+0000", 2024-09-18T04:56:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.461452+0000", 2024-09-18T04:56:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:31.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.461461+0000", 2024-09-18T04:56:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.431e-06 2024-09-18T04:56:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.461481+0000", 2024-09-18T04:56:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9558000000000001e-05 2024-09-18T04:56:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.461489+0000", 2024-09-18T04:56:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6969999999999999e-06 2024-09-18T04:56:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.461551+0000", 2024-09-18T04:56:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1869e-05 2024-09-18T04:56:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.461577+0000", 2024-09-18T04:56:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5969000000000002e-05 2024-09-18T04:56:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.462480+0000", 2024-09-18T04:56:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00090271600000000002 2024-09-18T04:56:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.462515+0000", 2024-09-18T04:56:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5336999999999997e-05 2024-09-18T04:56:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.224 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:951 3.9 e24/19 3:936a2561:::rbd_data.11ac33624de9.0000000000000013:head v 24'11, mlcod=23'9)", 2024-09-18T04:56:31.224 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.488534+0000", 2024-09-18T04:56:31.224 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.6985052540000001, 2024-09-18T04:56:31.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00090539099999999996, 2024-09-18T04:56:31.224 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.224 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.224 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.488534+0000", 2024-09-18T04:56:31.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.488534+0000", 2024-09-18T04:56:31.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.488533+0000", 2024-09-18T04:56:31.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:31.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.488661+0000", 2024-09-18T04:56:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000127389 2024-09-18T04:56:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.488676+0000", 2024-09-18T04:56:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.525e-05 2024-09-18T04:56:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.488683+0000", 2024-09-18T04:56:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0600000000000002e-06 2024-09-18T04:56:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.488719+0000", 2024-09-18T04:56:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6164999999999997e-05 2024-09-18T04:56:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.488735+0000", 2024-09-18T04:56:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.624e-05 2024-09-18T04:56:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.489412+0000", 2024-09-18T04:56:31.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067661499999999999 2024-09-18T04:56:31.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.489440+0000", 2024-09-18T04:56:31.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7781e-05 2024-09-18T04:56:31.228 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.228 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.228 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.229 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:965 3.0 e24/19 3:006abf32:::rbd_data.11ac33624de9.0000000000000009:head v 24'15, mlcod=24'12)", 2024-09-18T04:56:31.229 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.508428+0000", 2024-09-18T04:56:31.229 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.6786115390000003, 2024-09-18T04:56:31.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00095802500000000004, 2024-09-18T04:56:31.229 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.229 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.229 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.508428+0000", 2024-09-18T04:56:31.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.508428+0000", 2024-09-18T04:56:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.508428+0000", 2024-09-18T04:56:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.508434+0000", 2024-09-18T04:56:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1700000000000002e-06 2024-09-18T04:56:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.508446+0000", 2024-09-18T04:56:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2703e-05 2024-09-18T04:56:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.508453+0000", 2024-09-18T04:56:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2060000000000004e-06 2024-09-18T04:56:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.508493+0000", 2024-09-18T04:56:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0475000000000001e-05 2024-09-18T04:56:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.508513+0000", 2024-09-18T04:56:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9843999999999998e-05 2024-09-18T04:56:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.509352+0000", 2024-09-18T04:56:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00083858700000000001 2024-09-18T04:56:31.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.509386+0000", 2024-09-18T04:56:31.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4694999999999999e-05 2024-09-18T04:56:31.233 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.233 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.233 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.233 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:1143 3.0 e25/19 3:0546939f:::rbd_data.11ac33624de9.0000000000000003:head v 25'20, mlcod=24'17)", 2024-09-18T04:56:31.233 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.079430+0000", 2024-09-18T04:56:31.233 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.107610083, 2024-09-18T04:56:31.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010410370000000001, 2024-09-18T04:56:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.079430+0000", 2024-09-18T04:56:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.079430+0000", 2024-09-18T04:56:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.079429+0000", 2024-09-18T04:56:31.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:31.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.079448+0000", 2024-09-18T04:56:31.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8508999999999998e-05 2024-09-18T04:56:31.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.079460+0000", 2024-09-18T04:56:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2412000000000001e-05 2024-09-18T04:56:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.079468+0000", 2024-09-18T04:56:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.011e-06 2024-09-18T04:56:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.079521+0000", 2024-09-18T04:56:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2939000000000003e-05 2024-09-18T04:56:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.079542+0000", 2024-09-18T04:56:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1089000000000001e-05 2024-09-18T04:56:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.080436+0000", 2024-09-18T04:56:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00089435000000000003 2024-09-18T04:56:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.080471+0000", 2024-09-18T04:56:31.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4199000000000002e-05 2024-09-18T04:56:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:1149 3.0 e25/19 3:006abf32:::rbd_data.11ac33624de9.0000000000000009:head v 25'22, mlcod=24'18)", 2024-09-18T04:56:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.084697+0000", 2024-09-18T04:56:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.102342438, 2024-09-18T04:56:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00082441199999999995, 2024-09-18T04:56:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084697+0000", 2024-09-18T04:56:31.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084697+0000", 2024-09-18T04:56:31.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084696+0000", 2024-09-18T04:56:31.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084716+0000", 2024-09-18T04:56:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.914e-05 2024-09-18T04:56:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084731+0000", 2024-09-18T04:56:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5449999999999999e-05 2024-09-18T04:56:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084739+0000", 2024-09-18T04:56:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8369999999999997e-06 2024-09-18T04:56:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084785+0000", 2024-09-18T04:56:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6448e-05 2024-09-18T04:56:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084807+0000", 2024-09-18T04:56:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1892999999999999e-05 2024-09-18T04:56:31.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.085487+0000", 2024-09-18T04:56:31.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00068033700000000004 2024-09-18T04:56:31.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.085522+0000", 2024-09-18T04:56:31.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4205999999999997e-05 2024-09-18T04:56:31.242 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.242 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.243 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.244 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:1293 3.4 e25/19 3:23600ee1:::rbd_data.11ac99060bb1.00000000000002b3:head v 25'8, mlcod=23'5)", 2024-09-18T04:56:31.244 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.192707+0000", 2024-09-18T04:56:31.244 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 2.9943328519999999, 2024-09-18T04:56:31.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00090253299999999996, 2024-09-18T04:56:31.244 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.244 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.244 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.192707+0000", 2024-09-18T04:56:31.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.192707+0000", 2024-09-18T04:56:31.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.192706+0000", 2024-09-18T04:56:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.192717+0000", 2024-09-18T04:56:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.118e-05 2024-09-18T04:56:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.192737+0000", 2024-09-18T04:56:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0645999999999999e-05 2024-09-18T04:56:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.192760+0000", 2024-09-18T04:56:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2342999999999999e-05 2024-09-18T04:56:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.192840+0000", 2024-09-18T04:56:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0067000000000004e-05 2024-09-18T04:56:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.192861+0000", 2024-09-18T04:56:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1433e-05 2024-09-18T04:56:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.193571+0000", 2024-09-18T04:56:31.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00070978899999999995 2024-09-18T04:56:31.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.193609+0000", 2024-09-18T04:56:31.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8404e-05 2024-09-18T04:56:31.248 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.248 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.248 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:1709 3.4 e25/19 3:27c0b102:::rbd_data.11ac99060bb1.0000000000000039:head v 25'12, mlcod=25'10)", 2024-09-18T04:56:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.483598+0000", 2024-09-18T04:56:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 2.7034415890000001, 2024-09-18T04:56:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00084758000000000001, 2024-09-18T04:56:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.483598+0000", 2024-09-18T04:56:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.483598+0000", 2024-09-18T04:56:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.483597+0000", 2024-09-18T04:56:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.483610+0000", 2024-09-18T04:56:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3209e-05 2024-09-18T04:56:31.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.483630+0000", 2024-09-18T04:56:31.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0508000000000001e-05 2024-09-18T04:56:31.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.483640+0000", 2024-09-18T04:56:31.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1789999999999997e-06 2024-09-18T04:56:31.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.483699+0000", 2024-09-18T04:56:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.94e-05 2024-09-18T04:56:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.483722+0000", 2024-09-18T04:56:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3419999999999999e-05 2024-09-18T04:56:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.484409+0000", 2024-09-18T04:56:31.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000686435 2024-09-18T04:56:31.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.484446+0000", 2024-09-18T04:56:31.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6826999999999998e-05 2024-09-18T04:56:31.254 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.254 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.254 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.254 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:1898 3.9 e25/19 3:943e339c:::rbd_data.11ac99060bb1.000000000000001a:head v 25'15, mlcod=25'13)", 2024-09-18T04:56:31.254 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.541312+0000", 2024-09-18T04:56:31.254 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 2.6457273890000002, 2024-09-18T04:56:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00081594800000000004, 2024-09-18T04:56:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.541312+0000", 2024-09-18T04:56:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.541312+0000", 2024-09-18T04:56:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.541311+0000", 2024-09-18T04:56:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.541321+0000", 2024-09-18T04:56:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0062e-05 2024-09-18T04:56:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.541341+0000", 2024-09-18T04:56:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0434000000000001e-05 2024-09-18T04:56:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.541351+0000", 2024-09-18T04:56:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3039999999999994e-06 2024-09-18T04:56:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.541405+0000", 2024-09-18T04:56:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3844999999999997e-05 2024-09-18T04:56:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.541426+0000", 2024-09-18T04:56:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1452000000000001e-05 2024-09-18T04:56:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.542093+0000", 2024-09-18T04:56:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00066649500000000002 2024-09-18T04:56:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.542128+0000", 2024-09-18T04:56:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5676999999999997e-05 2024-09-18T04:56:31.258 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.259 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.259 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.259 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:1900 3.4 e25/19 3:200073bd:::rbd_data.11ac99060bb1.000000000000011f:head v 25'15, mlcod=25'13)", 2024-09-18T04:56:31.259 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.541542+0000", 2024-09-18T04:56:31.259 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 2.6454974330000001, 2024-09-18T04:56:31.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00088303700000000004, 2024-09-18T04:56:31.259 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.259 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.259 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.541542+0000", 2024-09-18T04:56:31.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.541542+0000", 2024-09-18T04:56:31.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.541541+0000", 2024-09-18T04:56:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.541549+0000", 2024-09-18T04:56:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0539999999999995e-06 2024-09-18T04:56:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.541563+0000", 2024-09-18T04:56:31.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3157e-05 2024-09-18T04:56:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.541569+0000", 2024-09-18T04:56:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7909999999999999e-06 2024-09-18T04:56:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.541619+0000", 2024-09-18T04:56:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9679999999999999e-05 2024-09-18T04:56:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.541639+0000", 2024-09-18T04:56:31.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9714000000000001e-05 2024-09-18T04:56:31.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.542390+0000", 2024-09-18T04:56:31.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00075118600000000004 2024-09-18T04:56:31.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.542425+0000", 2024-09-18T04:56:31.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5417000000000003e-05 2024-09-18T04:56:31.263 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:2026 3.4 e25/19 3:20e92754:::rbd_data.11ac99060bb1.000000000000000d:head v 25'16, mlcod=25'14)", 2024-09-18T04:56:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.562187+0000", 2024-09-18T04:56:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 2.624853087, 2024-09-18T04:56:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00085861100000000001, 2024-09-18T04:56:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.562187+0000", 2024-09-18T04:56:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.562187+0000", 2024-09-18T04:56:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.562186+0000", 2024-09-18T04:56:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.562194+0000", 2024-09-18T04:56:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0849999999999997e-06 2024-09-18T04:56:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.562206+0000", 2024-09-18T04:56:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1969e-05 2024-09-18T04:56:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.562230+0000", 2024-09-18T04:56:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4286000000000001e-05 2024-09-18T04:56:31.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.562275+0000", 2024-09-18T04:56:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4771000000000002e-05 2024-09-18T04:56:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.562297+0000", 2024-09-18T04:56:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1866e-05 2024-09-18T04:56:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.563015+0000", 2024-09-18T04:56:31.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00071832000000000005 2024-09-18T04:56:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.563045+0000", 2024-09-18T04:56:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0207999999999999e-05 2024-09-18T04:56:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:2134 3.0 e25/19 3:04026e77:::rbd_data.11ac99060bb1.000000000000014f:head v 25'25, mlcod=25'23)", 2024-09-18T04:56:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.723277+0000", 2024-09-18T04:56:31.268 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 2.4637624759999999, 2024-09-18T04:56:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00087620299999999999, 2024-09-18T04:56:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.723277+0000", 2024-09-18T04:56:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.723277+0000", 2024-09-18T04:56:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.723276+0000", 2024-09-18T04:56:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.723298+0000", 2024-09-18T04:56:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2138999999999999e-05 2024-09-18T04:56:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.723314+0000", 2024-09-18T04:56:31.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5916000000000001e-05 2024-09-18T04:56:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.723321+0000", 2024-09-18T04:56:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7379999999999997e-06 2024-09-18T04:56:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.723372+0000", 2024-09-18T04:56:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1675000000000002e-05 2024-09-18T04:56:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.723399+0000", 2024-09-18T04:56:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6874999999999999e-05 2024-09-18T04:56:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.724121+0000", 2024-09-18T04:56:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00072172899999999995 2024-09-18T04:56:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.724153+0000", 2024-09-18T04:56:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2382000000000003e-05 2024-09-18T04:56:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.272 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.273 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.273 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:2150 3.9 e25/19 3:933b0650:::rbd_data.11ac99060bb1.0000000000000091:head v 25'19, mlcod=25'17)", 2024-09-18T04:56:31.273 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.763718+0000", 2024-09-18T04:56:31.273 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 2.4233216340000001, 2024-09-18T04:56:31.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00092042100000000004, 2024-09-18T04:56:31.273 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.273 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.273 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.763718+0000", 2024-09-18T04:56:31.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.763718+0000", 2024-09-18T04:56:31.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.763717+0000", 2024-09-18T04:56:31.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:31.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.763730+0000", 2024-09-18T04:56:31.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3804e-05 2024-09-18T04:56:31.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.763751+0000", 2024-09-18T04:56:31.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0420000000000001e-05 2024-09-18T04:56:31.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.763772+0000", 2024-09-18T04:56:31.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1229999999999998e-05 2024-09-18T04:56:31.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.763822+0000", 2024-09-18T04:56:31.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9703000000000002e-05 2024-09-18T04:56:31.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.763846+0000", 2024-09-18T04:56:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4136999999999999e-05 2024-09-18T04:56:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.764599+0000", 2024-09-18T04:56:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00075293099999999998 2024-09-18T04:56:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.764638+0000", 2024-09-18T04:56:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9601999999999998e-05 2024-09-18T04:56:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.277 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:2151 3.0 e25/19 3:052499af:::rbd_data.11ac99060bb1.0000000000000092:head v 25'26, mlcod=25'24)", 2024-09-18T04:56:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.763909+0000", 2024-09-18T04:56:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 2.4231306610000001, 2024-09-18T04:56:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00094393000000000005, 2024-09-18T04:56:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.763909+0000", 2024-09-18T04:56:31.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.763909+0000", 2024-09-18T04:56:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.763908+0000", 2024-09-18T04:56:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.763951+0000", 2024-09-18T04:56:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2826e-05 2024-09-18T04:56:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.763966+0000", 2024-09-18T04:56:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5400000000000002e-05 2024-09-18T04:56:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.763974+0000", 2024-09-18T04:56:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8709999999999997e-06 2024-09-18T04:56:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.764034+0000", 2024-09-18T04:56:31.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9935999999999997e-05 2024-09-18T04:56:31.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.764058+0000", 2024-09-18T04:56:31.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3842e-05 2024-09-18T04:56:31.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.764817+0000", 2024-09-18T04:56:31.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00075939499999999995 2024-09-18T04:56:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.764853+0000", 2024-09-18T04:56:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5583000000000001e-05 2024-09-18T04:56:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:2198 3.9 e25/19 3:97f0c2d6:::rbd_data.11ac99060bb1.00000000000002b8:head v 25'20, mlcod=25'18)", 2024-09-18T04:56:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.809602+0000", 2024-09-18T04:56:31.282 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 2.3774372430000001, 2024-09-18T04:56:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00090549399999999996, 2024-09-18T04:56:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.809602+0000", 2024-09-18T04:56:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.809602+0000", 2024-09-18T04:56:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.809601+0000", 2024-09-18T04:56:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999981 2024-09-18T04:56:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.809628+0000", 2024-09-18T04:56:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7362999999999999e-05 2024-09-18T04:56:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.809644+0000", 2024-09-18T04:56:31.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5943999999999998e-05 2024-09-18T04:56:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.809651+0000", 2024-09-18T04:56:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.131e-06 2024-09-18T04:56:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.809700+0000", 2024-09-18T04:56:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8838999999999998e-05 2024-09-18T04:56:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.809727+0000", 2024-09-18T04:56:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6506999999999999e-05 2024-09-18T04:56:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.810476+0000", 2024-09-18T04:56:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00074915099999999998 2024-09-18T04:56:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.810508+0000", 2024-09-18T04:56:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2246999999999998e-05 2024-09-18T04:56:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.286 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.287 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.287 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:2211 3.0 e25/19 3:04026e77:::rbd_data.11ac99060bb1.000000000000014f:head v 25'27, mlcod=25'25)", 2024-09-18T04:56:31.287 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.833366+0000", 2024-09-18T04:56:31.287 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 2.3536736830000002, 2024-09-18T04:56:31.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00087809400000000005, 2024-09-18T04:56:31.287 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.287 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.287 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.833366+0000", 2024-09-18T04:56:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.833366+0000", 2024-09-18T04:56:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.833365+0000", 2024-09-18T04:56:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.833377+0000", 2024-09-18T04:56:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2870999999999999e-05 2024-09-18T04:56:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.833400+0000", 2024-09-18T04:56:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2597000000000001e-05 2024-09-18T04:56:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.833409+0000", 2024-09-18T04:56:31.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9239999999999996e-06 2024-09-18T04:56:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.833457+0000", 2024-09-18T04:56:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8572000000000002e-05 2024-09-18T04:56:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.833479+0000", 2024-09-18T04:56:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1926999999999999e-05 2024-09-18T04:56:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.834209+0000", 2024-09-18T04:56:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00072937000000000002 2024-09-18T04:56:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.834244+0000", 2024-09-18T04:56:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5333999999999997e-05 2024-09-18T04:56:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.291 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.292 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:31.293 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:56:31.294 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:56:31.294 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:56:31.295 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:56:31.295 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:56:31.295 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone5 from 0xa287ab5 (overlap 0x22964e) to 0x114b27 2024-09-18T04:56:31.296 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:56:31.296 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:56:31.297 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:31.290+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:31.300 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-18T04:56:31.304 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T04:56:31.304 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T04:56:31.304 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-18T04:56:31.304 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T04:56:31.329 INFO:teuthology.orchestra.run.smithi002.stdout:561 write 0x5312a69 thru 0x531ef0a (0xc4a2 bytes) 2024-09-18T04:56:31.337 INFO:teuthology.orchestra.run.smithi002.stdout:565 clone 7 order 25 su 2097152 sc 9 2024-09-18T04:56:31.394 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:31.391+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:31.413 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:31.413 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:31.413 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:56:31.413 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:31.422 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:56:31.495 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:31.493+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:31.569 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:31.569 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:56:31.570 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:56:31.570 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:31.579 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:56:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:56:31.620 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:31.620 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 28, 2024-09-18T04:56:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "fsid": "c0761d0c-69d5-4b64-b8b9-c7d098b2d06d", 2024-09-18T04:56:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "created": "2024-09-18T04:55:58.216538+0000", 2024-09-18T04:56:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "modified": "2024-09-18T04:56:31.151107+0000", 2024-09-18T04:56:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "last_up_change": "2024-09-18T04:56:08.369924+0000", 2024-09-18T04:56:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "last_in_change": "2024-09-18T04:56:00.333800+0000", 2024-09-18T04:56:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T04:56:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_num": 5799936, 2024-09-18T04:56:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_set": [ 2024-09-18T04:56:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "pglog_hardlimit", 2024-09-18T04:56:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "purged_snapdirs", 2024-09-18T04:56:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_deletes", 2024-09-18T04:56:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: "sortbitwise" 2024-09-18T04:56:31.621 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:56:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_version": 4, 2024-09-18T04:56:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T04:56:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T04:56:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T04:56:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_snapshot": "", 2024-09-18T04:56:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_max": 3, 2024-09-18T04:56:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd": 8, 2024-09-18T04:56:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "require_min_compat_client": "luminous", 2024-09-18T04:56:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "min_compat_client": "jewel", 2024-09-18T04:56:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "require_osd_release": "reef", 2024-09-18T04:56:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_crimson": false, 2024-09-18T04:56:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: "pools": [ 2024-09-18T04:56:31.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 1, 2024-09-18T04:56:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "rbd", 2024-09-18T04:56:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-18T04:56:08.983344+0000", 2024-09-18T04:56:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-18T04:56:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T04:56:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-18T04:56:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-18T04:56:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-18T04:56:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-18T04:56:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-18T04:56:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-18T04:56:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T04:56:31.623 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T04:56:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-18T04:56:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-18T04:56:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 8, 2024-09-18T04:56:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 8, 2024-09-18T04:56:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 8, 2024-09-18T04:56:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 8, 2024-09-18T04:56:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 8, 2024-09-18T04:56:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-18T04:56:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-18T04:56:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-18T04:56:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-18T04:56:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-18T04:56:31.624 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-18T04:56:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-18T04:56:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "17", 2024-09-18T04:56:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-18T04:56:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T04:56:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T04:56:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-18T04:56:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-18T04:56:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 3, 2024-09-18T04:56:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 17, 2024-09-18T04:56:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-18T04:56:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-18T04:56:31.625 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-18T04:56:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-18T04:56:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-18T04:56:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-18T04:56:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-18T04:56:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-18T04:56:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-18T04:56:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-18T04:56:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-18T04:56:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T04:56:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T04:56:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T04:56:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-18T04:56:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-18T04:56:31.626 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-18T04:56:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-18T04:56:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-18T04:56:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-18T04:56:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-18T04:56:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-18T04:56:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-18T04:56:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-18T04:56:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T04:56:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-18T04:56:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-18T04:56:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-18T04:56:31.627 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-18T04:56:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-18T04:56:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-18T04:56:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-18T04:56:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-18T04:56:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-18T04:56:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 2, 2024-09-18T04:56:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 2, 2024-09-18T04:56:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 1, 2024-09-18T04:56:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 2, 2024-09-18T04:56:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 2, 2024-09-18T04:56:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-18T04:56:31.628 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-18T04:56:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-18T04:56:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 2, 2024-09-18T04:56:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": ".mgr", 2024-09-18T04:56:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-18T04:56:10.313988+0000", 2024-09-18T04:56:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-18T04:56:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-18T04:56:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-18T04:56:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-18T04:56:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-18T04:56:31.629 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-18T04:56:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-18T04:56:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-18T04:56:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T04:56:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T04:56:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-18T04:56:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-18T04:56:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 1, 2024-09-18T04:56:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 1, 2024-09-18T04:56:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 1, 2024-09-18T04:56:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 1, 2024-09-18T04:56:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 1, 2024-09-18T04:56:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-18T04:56:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-18T04:56:31.630 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-18T04:56:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-18T04:56:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-18T04:56:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-18T04:56:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-18T04:56:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "17", 2024-09-18T04:56:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-18T04:56:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T04:56:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T04:56:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-18T04:56:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-18T04:56:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-18T04:56:31.631 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-18T04:56:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-18T04:56:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-18T04:56:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-18T04:56:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-18T04:56:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-18T04:56:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-18T04:56:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-18T04:56:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-18T04:56:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-18T04:56:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-18T04:56:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-18T04:56:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T04:56:31.632 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T04:56:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T04:56:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-18T04:56:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-18T04:56:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-18T04:56:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-18T04:56:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-18T04:56:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-18T04:56:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-18T04:56:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-18T04:56:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-18T04:56:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-18T04:56:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T04:56:31.633 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-18T04:56:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-18T04:56:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-18T04:56:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-18T04:56:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-18T04:56:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-18T04:56:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_max": 32, 2024-09-18T04:56:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 1 2024-09-18T04:56:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-18T04:56:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": {} 2024-09-18T04:56:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-18T04:56:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 8, 2024-09-18T04:56:31.634 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 8, 2024-09-18T04:56:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.25, 2024-09-18T04:56:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 2, 2024-09-18T04:56:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 2, 2024-09-18T04:56:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-18T04:56:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-18T04:56:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-18T04:56:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-18T04:56:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "pool_client.0", 2024-09-18T04:56:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-18T04:56:19.562579+0000", 2024-09-18T04:56:31.635 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-18T04:56:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T04:56:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-18T04:56:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-18T04:56:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-18T04:56:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-18T04:56:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-18T04:56:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-18T04:56:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T04:56:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T04:56:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-18T04:56:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-18T04:56:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 32, 2024-09-18T04:56:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 32, 2024-09-18T04:56:31.636 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 32, 2024-09-18T04:56:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 32, 2024-09-18T04:56:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 32, 2024-09-18T04:56:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-18T04:56:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-18T04:56:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-18T04:56:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-18T04:56:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-18T04:56:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-18T04:56:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-18T04:56:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "28", 2024-09-18T04:56:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-18T04:56:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T04:56:31.637 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T04:56:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-18T04:56:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-18T04:56:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 9, 2024-09-18T04:56:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 28, 2024-09-18T04:56:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-18T04:56:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-18T04:56:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-18T04:56:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-18T04:56:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-18T04:56:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-18T04:56:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-18T04:56:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-18T04:56:31.638 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-18T04:56:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-18T04:56:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-18T04:56:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T04:56:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T04:56:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T04:56:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-18T04:56:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-18T04:56:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-18T04:56:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-18T04:56:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-18T04:56:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-18T04:56:31.639 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-18T04:56:31.640 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-18T04:56:31.640 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-18T04:56:31.640 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-18T04:56:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T04:56:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-18T04:56:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-18T04:56:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-18T04:56:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-18T04:56:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-18T04:56:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-18T04:56:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-18T04:56:31.641 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-18T04:56:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-18T04:56:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 1.5, 2024-09-18T04:56:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 1.5, 2024-09-18T04:56:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 1, 2024-09-18T04:56:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 1.5, 2024-09-18T04:56:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 1.5, 2024-09-18T04:56:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-18T04:56:31.642 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-18T04:56:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-18T04:56:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:56:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-18T04:56:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-18T04:56:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "c43f8024-3a3e-424e-aa54-398310ceb70d", 2024-09-18T04:56:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:31.643 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 19, 2024-09-18T04:56:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:31.644 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6824", 2024-09-18T04:56:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6825", 2024-09-18T04:56:31.645 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6826", 2024-09-18T04:56:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6827", 2024-09-18T04:56:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.647 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6830", 2024-09-18T04:56:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6831", 2024-09-18T04:56:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:31.648 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6828", 2024-09-18T04:56:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6829", 2024-09-18T04:56:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.649 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6825/4106704527", 2024-09-18T04:56:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6827/4106704527", 2024-09-18T04:56:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6831/4106704527", 2024-09-18T04:56:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6829/4106704527", 2024-09-18T04:56:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-18T04:56:31.650 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "c664b970-145c-4b6f-8852-2f6dcf1fa30d", 2024-09-18T04:56:31.651 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:31.651 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:31.651 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:31.651 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:31.651 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:31.651 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:31.651 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:31.651 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 19, 2024-09-18T04:56:31.651 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:31.651 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:31.651 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:31.652 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.652 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.652 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6808", 2024-09-18T04:56:31.652 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:31.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.652 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.652 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6809", 2024-09-18T04:56:31.652 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:31.652 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.652 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6810", 2024-09-18T04:56:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6811", 2024-09-18T04:56:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6814", 2024-09-18T04:56:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6815", 2024-09-18T04:56:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.654 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6812", 2024-09-18T04:56:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6813", 2024-09-18T04:56:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6809/1781959259", 2024-09-18T04:56:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6811/1781959259", 2024-09-18T04:56:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6815/1781959259", 2024-09-18T04:56:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6813/1781959259", 2024-09-18T04:56:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-18T04:56:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "0609ea8c-d975-490e-8844-04a3e4512efd", 2024-09-18T04:56:31.656 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 0, 2024-09-18T04:56:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.657 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6800", 2024-09-18T04:56:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6801", 2024-09-18T04:56:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.658 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6802", 2024-09-18T04:56:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6803", 2024-09-18T04:56:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.659 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6806", 2024-09-18T04:56:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6807", 2024-09-18T04:56:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6804", 2024-09-18T04:56:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6805", 2024-09-18T04:56:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.661 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6801/1153570802", 2024-09-18T04:56:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6803/1153570802", 2024-09-18T04:56:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6807/1153570802", 2024-09-18T04:56:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6805/1153570802", 2024-09-18T04:56:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-18T04:56:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "c6bd5ac8-ab89-4fa2-b843-3b166d1b238b", 2024-09-18T04:56:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 19, 2024-09-18T04:56:31.663 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6816", 2024-09-18T04:56:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6817", 2024-09-18T04:56:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6818", 2024-09-18T04:56:31.665 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6819", 2024-09-18T04:56:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.666 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6822", 2024-09-18T04:56:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.667 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6823", 2024-09-18T04:56:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6820", 2024-09-18T04:56:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6821", 2024-09-18T04:56:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.669 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6817/2008321375", 2024-09-18T04:56:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6819/2008321375", 2024-09-18T04:56:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6823/2008321375", 2024-09-18T04:56:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6821/2008321375", 2024-09-18T04:56:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.670 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-18T04:56:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "89d6a371-4f4e-465e-a023-d5083cce6e6c", 2024-09-18T04:56:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 19, 2024-09-18T04:56:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:31.671 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6824", 2024-09-18T04:56:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.672 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6825", 2024-09-18T04:56:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.673 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6826", 2024-09-18T04:56:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6827", 2024-09-18T04:56:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:31.674 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6830", 2024-09-18T04:56:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6831", 2024-09-18T04:56:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.677 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.677 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6828", 2024-09-18T04:56:31.677 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:31.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.677 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.677 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6829", 2024-09-18T04:56:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6825/2939896598", 2024-09-18T04:56:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6827/2939896598", 2024-09-18T04:56:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6831/2939896598", 2024-09-18T04:56:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6829/2939896598", 2024-09-18T04:56:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:31.678 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-18T04:56:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "da99686b-d979-4753-b4aa-a3b26e9a0497", 2024-09-18T04:56:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:31.679 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 19, 2024-09-18T04:56:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6816", 2024-09-18T04:56:31.680 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6817", 2024-09-18T04:56:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6818", 2024-09-18T04:56:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6819", 2024-09-18T04:56:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6822", 2024-09-18T04:56:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6823", 2024-09-18T04:56:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6820", 2024-09-18T04:56:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6821", 2024-09-18T04:56:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.684 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6817/3440879618", 2024-09-18T04:56:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6819/3440879618", 2024-09-18T04:56:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6823/3440879618", 2024-09-18T04:56:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6821/3440879618", 2024-09-18T04:56:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-18T04:56:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "ceec6aa6-603d-4706-bfea-3c07dbdaed3a", 2024-09-18T04:56:31.685 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 19, 2024-09-18T04:56:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.686 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6800", 2024-09-18T04:56:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6801", 2024-09-18T04:56:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.687 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6802", 2024-09-18T04:56:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.688 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6803", 2024-09-18T04:56:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6806", 2024-09-18T04:56:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:31.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6807", 2024-09-18T04:56:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6804", 2024-09-18T04:56:31.690 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6805", 2024-09-18T04:56:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6801/2493830504", 2024-09-18T04:56:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6803/2493830504", 2024-09-18T04:56:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6807/2493830504", 2024-09-18T04:56:31.691 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6805/2493830504", 2024-09-18T04:56:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-18T04:56:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "6779dfad-f60b-41f4-b666-a99db6dc1814", 2024-09-18T04:56:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:31.692 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 19, 2024-09-18T04:56:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6808", 2024-09-18T04:56:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6809", 2024-09-18T04:56:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6810", 2024-09-18T04:56:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:31.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6811", 2024-09-18T04:56:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.695 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6814", 2024-09-18T04:56:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6815", 2024-09-18T04:56:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:31.696 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6812", 2024-09-18T04:56:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:31.697 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6813", 2024-09-18T04:56:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6809/91038858", 2024-09-18T04:56:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6811/91038858", 2024-09-18T04:56:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6815/91038858", 2024-09-18T04:56:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6813/91038858", 2024-09-18T04:56:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:31.698 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:56:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_xinfo": [ 2024-09-18T04:56:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-18T04:56:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:31.699 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.468213+0000", 2024-09-18T04:56:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-18T04:56:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:31.700 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.538610+0000", 2024-09-18T04:56:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-18T04:56:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:31.701 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.500881+0000", 2024-09-18T04:56:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-18T04:56:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:31.702 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.474179+0000", 2024-09-18T04:56:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-18T04:56:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:31.703 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.483737+0000", 2024-09-18T04:56:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-18T04:56:31.704 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.460693+0000", 2024-09-18T04:56:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-18T04:56:31.705 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:31.706 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:31.706 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:31.706 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:31.706 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:31.706 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.475970+0000", 2024-09-18T04:56:31.706 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:31.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.707 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-18T04:56:31.707 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:31.707 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:31.707 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:31.707 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:31.707 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:31.707 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.519287+0000", 2024-09-18T04:56:31.707 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:31.707 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:56:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap": [], 2024-09-18T04:56:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_items": [], 2024-09-18T04:56:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_primaries": [], 2024-09-18T04:56:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_temp": [], 2024-09-18T04:56:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_temp": [], 2024-09-18T04:56:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: "blocklist": {}, 2024-09-18T04:56:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: "range_blocklist": {}, 2024-09-18T04:56:31.708 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profiles": { 2024-09-18T04:56:31.709 INFO:teuthology.orchestra.run.smithi002.stdout: "default": { 2024-09-18T04:56:31.709 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-failure-domain": "osd", 2024-09-18T04:56:31.709 INFO:teuthology.orchestra.run.smithi002.stdout: "k": "2", 2024-09-18T04:56:31.709 INFO:teuthology.orchestra.run.smithi002.stdout: "m": "1", 2024-09-18T04:56:31.709 INFO:teuthology.orchestra.run.smithi002.stdout: "plugin": "jerasure", 2024-09-18T04:56:31.709 INFO:teuthology.orchestra.run.smithi002.stdout: "technique": "reed_sol_van" 2024-09-18T04:56:31.709 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.710 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps_queue": [], 2024-09-18T04:56:31.710 INFO:teuthology.orchestra.run.smithi002.stdout: "new_removed_snaps": [], 2024-09-18T04:56:31.710 INFO:teuthology.orchestra.run.smithi002.stdout: "new_purged_snaps": [], 2024-09-18T04:56:31.710 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_node_flags": {}, 2024-09-18T04:56:31.710 INFO:teuthology.orchestra.run.smithi002.stdout: "device_class_flags": {}, 2024-09-18T04:56:31.711 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode": { 2024-09-18T04:56:31.711 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_enabled": false, 2024-09-18T04:56:31.711 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_bucket_count": 0, 2024-09-18T04:56:31.711 INFO:teuthology.orchestra.run.smithi002.stdout: "degraded_stretch_mode": 0, 2024-09-18T04:56:31.711 INFO:teuthology.orchestra.run.smithi002.stdout: "recovering_stretch_mode": 0, 2024-09-18T04:56:31.711 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_bucket": 0 2024-09-18T04:56:31.711 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.712 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:31.712 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:31.617+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:31.714 INFO:tasks.thrashosds.thrasher:j is {'epoch': 28, 'fsid': 'c0761d0c-69d5-4b64-b8b9-c7d098b2d06d', 'created': '2024-09-18T04:55:58.216538+0000', 'modified': '2024-09-18T04:56:31.151107+0000', 'last_up_change': '2024-09-18T04:56:08.369924+0000', 'last_in_change': '2024-09-18T04:56:00.333800+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-18T04:56:08.983344+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-18T04:56:10.313988+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T04:56:19.562579+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '28', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 9, 'snap_epoch': 28, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'c43f8024-3a3e-424e-aa54-398310ceb70d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6824', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6825', 'nonce': 4106704527}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6826', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6827', 'nonce': 4106704527}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6830', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6831', 'nonce': 4106704527}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6828', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6829', 'nonce': 4106704527}]}, 'public_addr': '172.21.15.2:6825/4106704527', 'cluster_addr': '172.21.15.2:6827/4106704527', 'heartbeat_back_addr': '172.21.15.2:6831/4106704527', 'heartbeat_front_addr': '172.21.15.2:6829/4106704527', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c664b970-145c-4b6f-8852-2f6dcf1fa30d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6808', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6809', 'nonce': 1781959259}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6810', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6811', 'nonce': 1781959259}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6814', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6815', 'nonce': 1781959259}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6812', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6813', 'nonce': 1781959259}]}, 'public_addr': '172.21.15.2:6809/1781959259', 'cluster_addr': '172.21.15.2:6811/1781959259', 'heartbeat_back_addr': '172.21.15.2:6815/1781959259', 'heartbeat_front_addr': '172.21.15.2:6813/1781959259', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0609ea8c-d975-490e-8844-04a3e4512efd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6800', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6801', 'nonce': 1153570802}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6802', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6803', 'nonce': 1153570802}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6806', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6807', 'nonce': 1153570802}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6804', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6805', 'nonce': 1153570802}]}, 'public_addr': '172.21.15.2:6801/1153570802', 'cluster_addr': '172.21.15.2:6803/1153570802', 'heartbeat_back_addr': '172.21.15.2:6807/1153570802', 'heartbeat_front_addr': '172.21.15.2:6805/1153570802', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'c6bd5ac8-ab89-4fa2-b843-3b166d1b238b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6816', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6817', 'nonce': 2008321375}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6818', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6819', 'nonce': 2008321375}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6822', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6823', 'nonce': 2008321375}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6820', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6821', 'nonce': 2008321375}]}, 'public_addr': '172.21.15.2:6817/2008321375', 'cluster_addr': '172.21.15.2:6819/2008321375', 'heartbeat_back_addr': '172.21.15.2:6823/2008321375', 'heartbeat_front_addr': '172.21.15.2:6821/2008321375', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '89d6a371-4f4e-465e-a023-d5083cce6e6c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6824', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6825', 'nonce': 2939896598}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6826', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6827', 'nonce': 2939896598}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6830', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6831', 'nonce': 2939896598}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6828', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6829', 'nonce': 2939896598}]}, 'public_addr': '172.21.15.99:6825/2939896598', 'cluster_addr': '172.21.15.99:6827/2939896598', 'heartbeat_back_addr': '172.21.15.99:6831/2939896598', 'heartbeat_front_addr': '172.21.15.99:6829/2939896598', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'da99686b-d979-4753-b4aa-a3b26e9a0497', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6816', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6817', 'nonce': 3440879618}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6818', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6819', 'nonce': 3440879618}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6822', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6823', 'nonce': 3440879618}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6820', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6821', 'nonce': 3440879618}]}, 'public_addr': '172.21.15.99:6817/3440879618', 'cluster_addr': '172.21.15.99:6819/3440879618', 'heartbeat_back_addr': '172.21.15.99:6823/3440879618', 'heartbeat_front_addr': '172.21.15.99:6821/3440879618', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ceec6aa6-603d-4706-bfea-3c07dbdaed3a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6800', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6801', 'nonce': 2493830504}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6802', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6803', 'nonce': 2493830504}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6806', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6807', 'nonce': 2493830504}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6804', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6805', 'nonce': 2493830504}]}, 'public_addr': '172.21.15.99:6801/2493830504', 'cluster_addr': '172.21.15.99:6803/2493830504', 'heartbeat_back_addr': '172.21.15.99:6807/2493830504', 'heartbeat_front_addr': '172.21.15.99:6805/2493830504', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '6779dfad-f60b-41f4-b666-a99db6dc1814', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6808', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6809', 'nonce': 91038858}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6810', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6811', 'nonce': 91038858}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6814', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6815', 'nonce': 91038858}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6812', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6813', 'nonce': 91038858}]}, 'public_addr': '172.21.15.99:6809/91038858', 'cluster_addr': '172.21.15.99:6811/91038858', 'heartbeat_back_addr': '172.21.15.99:6815/91038858', 'heartbeat_front_addr': '172.21.15.99:6813/91038858', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.468213+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.538610+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.500881+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.474179+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.483737+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.460693+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.475970+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.519287+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T04:56:31.714 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T04:56:31.716 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:56:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:56:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:56:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:855 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.375028+0000", 2024-09-18T04:56:31.716 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.3092548910000001, 2024-09-18T04:56:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0063269789999999999, 2024-09-18T04:56:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 855 2024-09-18T04:56:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375028+0000", 2024-09-18T04:56:31.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375028+0000", 2024-09-18T04:56:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375027+0000", 2024-09-18T04:56:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375033+0000", 2024-09-18T04:56:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9869999999999996e-06 2024-09-18T04:56:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375042+0000", 2024-09-18T04:56:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3899999999999999e-06 2024-09-18T04:56:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375051+0000", 2024-09-18T04:56:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2839999999999995e-06 2024-09-18T04:56:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.377760+0000", 2024-09-18T04:56:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027089869999999999 2024-09-18T04:56:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.377862+0000", 2024-09-18T04:56:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010184200000000001 2024-09-18T04:56:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.378261+0000", 2024-09-18T04:56:31.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00039838900000000002 2024-09-18T04:56:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.380468+0000", 2024-09-18T04:56:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002207156 2024-09-18T04:56:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381284+0000", 2024-09-18T04:56:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00081592199999999996 2024-09-18T04:56:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381332+0000", 2024-09-18T04:56:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8047000000000003e-05 2024-09-18T04:56:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381355+0000", 2024-09-18T04:56:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2802999999999998e-05 2024-09-18T04:56:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:856 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.375180+0000", 2024-09-18T04:56:31.722 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.309102041, 2024-09-18T04:56:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0063488080000000001, 2024-09-18T04:56:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 856 2024-09-18T04:56:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375180+0000", 2024-09-18T04:56:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375180+0000", 2024-09-18T04:56:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375180+0000", 2024-09-18T04:56:31.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375184+0000", 2024-09-18T04:56:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6530000000000002e-06 2024-09-18T04:56:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375202+0000", 2024-09-18T04:56:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8099000000000001e-05 2024-09-18T04:56:31.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375209+0000", 2024-09-18T04:56:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7469999999999999e-06 2024-09-18T04:56:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.378708+0000", 2024-09-18T04:56:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034990329999999999 2024-09-18T04:56:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.378801+0000", 2024-09-18T04:56:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2967999999999995e-05 2024-09-18T04:56:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.379196+0000", 2024-09-18T04:56:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00039495699999999998 2024-09-18T04:56:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381369+0000", 2024-09-18T04:56:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021726559999999998 2024-09-18T04:56:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381422+0000", 2024-09-18T04:56:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3523000000000002e-05 2024-09-18T04:56:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381469+0000", 2024-09-18T04:56:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7107999999999997e-05 2024-09-18T04:56:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381529+0000", 2024-09-18T04:56:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9846000000000003e-05 2024-09-18T04:56:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:857 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.375218+0000", 2024-09-18T04:56:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.3090639939999997, 2024-09-18T04:56:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006463809, 2024-09-18T04:56:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 857 2024-09-18T04:56:31.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375218+0000", 2024-09-18T04:56:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375218+0000", 2024-09-18T04:56:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375218+0000", 2024-09-18T04:56:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375222+0000", 2024-09-18T04:56:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.704e-06 2024-09-18T04:56:31.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375231+0000", 2024-09-18T04:56:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3999999999999998e-06 2024-09-18T04:56:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375237+0000", 2024-09-18T04:56:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5169999999999999e-06 2024-09-18T04:56:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.379666+0000", 2024-09-18T04:56:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044297390000000002 2024-09-18T04:56:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.379747+0000", 2024-09-18T04:56:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0316999999999997e-05 2024-09-18T04:56:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.380138+0000", 2024-09-18T04:56:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00039156599999999998 2024-09-18T04:56:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:31.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381578+0000", 2024-09-18T04:56:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001439423 2024-09-18T04:56:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381587+0000", 2024-09-18T04:56:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2499999999999995e-06 2024-09-18T04:56:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381641+0000", 2024-09-18T04:56:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3961000000000001e-05 2024-09-18T04:56:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381682+0000", 2024-09-18T04:56:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1454e-05 2024-09-18T04:56:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.737 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:860 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.375377+0000", 2024-09-18T04:56:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.3089057090000003, 2024-09-18T04:56:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0066062229999999996, 2024-09-18T04:56:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.738 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 860 2024-09-18T04:56:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375377+0000", 2024-09-18T04:56:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375377+0000", 2024-09-18T04:56:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375375+0000", 2024-09-18T04:56:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375382+0000", 2024-09-18T04:56:31.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0280000000000003e-06 2024-09-18T04:56:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375393+0000", 2024-09-18T04:56:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0518e-05 2024-09-18T04:56:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375401+0000", 2024-09-18T04:56:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1680000000000004e-06 2024-09-18T04:56:31.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.380505+0000", 2024-09-18T04:56:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051037920000000002 2024-09-18T04:56:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.380589+0000", 2024-09-18T04:56:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4259999999999996e-05 2024-09-18T04:56:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:31.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.380912+0000", 2024-09-18T04:56:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000322763 2024-09-18T04:56:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381694+0000", 2024-09-18T04:56:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00078214099999999998 2024-09-18T04:56:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381927+0000", 2024-09-18T04:56:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000233248 2024-09-18T04:56:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381961+0000", 2024-09-18T04:56:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3691e-05 2024-09-18T04:56:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381983+0000", 2024-09-18T04:56:31.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1968999999999999e-05 2024-09-18T04:56:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:1132 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.074574+0000", 2024-09-18T04:56:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.60970835, 2024-09-18T04:56:31.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0059714329999999999, 2024-09-18T04:56:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1132 2024-09-18T04:56:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.747 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074574+0000", 2024-09-18T04:56:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074574+0000", 2024-09-18T04:56:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074574+0000", 2024-09-18T04:56:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:56:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074575+0000", 2024-09-18T04:56:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4869999999999999e-06 2024-09-18T04:56:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074579+0000", 2024-09-18T04:56:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.14e-06 2024-09-18T04:56:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074581+0000", 2024-09-18T04:56:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4549999999999998e-06 2024-09-18T04:56:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.077782+0000", 2024-09-18T04:56:31.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0032005779999999999 2024-09-18T04:56:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.077867+0000", 2024-09-18T04:56:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5494999999999994e-05 2024-09-18T04:56:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.078233+0000", 2024-09-18T04:56:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00036551800000000001 2024-09-18T04:56:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.080412+0000", 2024-09-18T04:56:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021793630000000001 2024-09-18T04:56:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.080464+0000", 2024-09-18T04:56:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2024000000000002e-05 2024-09-18T04:56:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.080510+0000", 2024-09-18T04:56:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6393000000000001e-05 2024-09-18T04:56:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.080546+0000", 2024-09-18T04:56:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5206000000000001e-05 2024-09-18T04:56:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:1133 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.074583+0000", 2024-09-18T04:56:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.6096992110000001, 2024-09-18T04:56:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0060199470000000003, 2024-09-18T04:56:31.753 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1133 2024-09-18T04:56:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074583+0000", 2024-09-18T04:56:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074583+0000", 2024-09-18T04:56:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074583+0000", 2024-09-18T04:56:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:56:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074585+0000", 2024-09-18T04:56:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7209999999999999e-06 2024-09-18T04:56:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074588+0000", 2024-09-18T04:56:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0060000000000001e-06 2024-09-18T04:56:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074591+0000", 2024-09-18T04:56:31.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1259999999999998e-06 2024-09-18T04:56:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.078883+0000", 2024-09-18T04:56:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004291614 2024-09-18T04:56:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.078949+0000", 2024-09-18T04:56:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6367999999999999e-05 2024-09-18T04:56:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.079191+0000", 2024-09-18T04:56:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000242112 2024-09-18T04:56:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.080456+0000", 2024-09-18T04:56:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001265033 2024-09-18T04:56:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:31.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.080553+0000", 2024-09-18T04:56:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7374999999999995e-05 2024-09-18T04:56:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.080583+0000", 2024-09-18T04:56:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9677999999999999e-05 2024-09-18T04:56:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.080603+0000", 2024-09-18T04:56:31.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.014e-05 2024-09-18T04:56:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:1134 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.074594+0000", 2024-09-18T04:56:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.6096883399999999, 2024-09-18T04:56:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0076433650000000001, 2024-09-18T04:56:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.760 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1134 2024-09-18T04:56:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074594+0000", 2024-09-18T04:56:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074594+0000", 2024-09-18T04:56:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074594+0000", 2024-09-18T04:56:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:56:31.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074596+0000", 2024-09-18T04:56:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.86e-06 2024-09-18T04:56:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074599+0000", 2024-09-18T04:56:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1240000000000001e-06 2024-09-18T04:56:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074601+0000", 2024-09-18T04:56:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.407e-06 2024-09-18T04:56:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.079611+0000", 2024-09-18T04:56:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0050095080000000002 2024-09-18T04:56:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.079702+0000", 2024-09-18T04:56:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0915000000000006e-05 2024-09-18T04:56:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.080046+0000", 2024-09-18T04:56:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00034459399999999999 2024-09-18T04:56:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.082150+0000", 2024-09-18T04:56:31.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00210392 2024-09-18T04:56:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.082161+0000", 2024-09-18T04:56:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0383999999999999e-05 2024-09-18T04:56:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.082203+0000", 2024-09-18T04:56:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2542e-05 2024-09-18T04:56:31.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.082238+0000", 2024-09-18T04:56:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4345999999999999e-05 2024-09-18T04:56:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:1135 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:31.767 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.074614+0000", 2024-09-18T04:56:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.6096689610000001, 2024-09-18T04:56:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0093099629999999992, 2024-09-18T04:56:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1135 2024-09-18T04:56:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.768 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074614+0000", 2024-09-18T04:56:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074614+0000", 2024-09-18T04:56:31.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074613+0000", 2024-09-18T04:56:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074615+0000", 2024-09-18T04:56:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0030000000000001e-06 2024-09-18T04:56:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074619+0000", 2024-09-18T04:56:31.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8870000000000002e-06 2024-09-18T04:56:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074624+0000", 2024-09-18T04:56:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8339999999999998e-06 2024-09-18T04:56:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.080635+0000", 2024-09-18T04:56:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0060110900000000002 2024-09-18T04:56:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.080721+0000", 2024-09-18T04:56:31.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5717000000000006e-05 2024-09-18T04:56:31.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:31.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.080956+0000", 2024-09-18T04:56:31.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000234859 2024-09-18T04:56:31.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.083773+0000", 2024-09-18T04:56:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028177580000000001 2024-09-18T04:56:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.083831+0000", 2024-09-18T04:56:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8118999999999997e-05 2024-09-18T04:56:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.083881+0000", 2024-09-18T04:56:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9845000000000001e-05 2024-09-18T04:56:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.083923+0000", 2024-09-18T04:56:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2160999999999998e-05 2024-09-18T04:56:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.775 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:1136 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.074627+0000", 2024-09-18T04:56:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.6096558189999999, 2024-09-18T04:56:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0094502679999999995, 2024-09-18T04:56:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1136 2024-09-18T04:56:31.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074627+0000", 2024-09-18T04:56:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074627+0000", 2024-09-18T04:56:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074626+0000", 2024-09-18T04:56:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:56:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074628+0000", 2024-09-18T04:56:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.951e-06 2024-09-18T04:56:31.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074632+0000", 2024-09-18T04:56:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2150000000000001e-06 2024-09-18T04:56:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074634+0000", 2024-09-18T04:56:31.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4159999999999998e-06 2024-09-18T04:56:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.081336+0000", 2024-09-18T04:56:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067015340000000003 2024-09-18T04:56:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.081411+0000", 2024-09-18T04:56:31.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5067999999999993e-05 2024-09-18T04:56:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.081740+0000", 2024-09-18T04:56:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032918800000000001 2024-09-18T04:56:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:31.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.083985+0000", 2024-09-18T04:56:31.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022451189999999998 2024-09-18T04:56:31.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:31.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.083990+0000", 2024-09-18T04:56:31.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.356e-06 2024-09-18T04:56:31.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084040+0000", 2024-09-18T04:56:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9492e-05 2024-09-18T04:56:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084077+0000", 2024-09-18T04:56:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7156999999999997e-05 2024-09-18T04:56:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:1137 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.074636+0000", 2024-09-18T04:56:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.6096463160000001, 2024-09-18T04:56:31.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0095487100000000002, 2024-09-18T04:56:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1137 2024-09-18T04:56:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074636+0000", 2024-09-18T04:56:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074636+0000", 2024-09-18T04:56:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074636+0000", 2024-09-18T04:56:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:56:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074639+0000", 2024-09-18T04:56:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7010000000000001e-06 2024-09-18T04:56:31.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074642+0000", 2024-09-18T04:56:31.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0410000000000002e-06 2024-09-18T04:56:31.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074644+0000", 2024-09-18T04:56:31.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4099999999999998e-06 2024-09-18T04:56:31.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.082287+0000", 2024-09-18T04:56:31.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0076424140000000002 2024-09-18T04:56:31.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.082372+0000", 2024-09-18T04:56:31.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5289000000000003e-05 2024-09-18T04:56:31.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:31.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.082705+0000", 2024-09-18T04:56:31.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000332951 2024-09-18T04:56:31.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:31.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084087+0000", 2024-09-18T04:56:31.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013827030000000001 2024-09-18T04:56:31.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:31.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084119+0000", 2024-09-18T04:56:31.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1139000000000001e-05 2024-09-18T04:56:31.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084158+0000", 2024-09-18T04:56:31.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9421000000000001e-05 2024-09-18T04:56:31.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084185+0000", 2024-09-18T04:56:31.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6855999999999998e-05 2024-09-18T04:56:31.793 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.793 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.793 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.793 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:1138 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:31.793 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.074647+0000", 2024-09-18T04:56:31.793 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.6096358550000001, 2024-09-18T04:56:31.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010428664000000001, 2024-09-18T04:56:31.794 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.794 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.794 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:31.794 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:31.794 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:31.795 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1138 2024-09-18T04:56:31.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.795 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074647+0000", 2024-09-18T04:56:31.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074647+0000", 2024-09-18T04:56:31.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074646+0000", 2024-09-18T04:56:31.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:31.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074648+0000", 2024-09-18T04:56:31.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6479999999999999e-06 2024-09-18T04:56:31.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074655+0000", 2024-09-18T04:56:31.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8410000000000002e-06 2024-09-18T04:56:31.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074658+0000", 2024-09-18T04:56:31.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.678e-06 2024-09-18T04:56:31.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.083112+0000", 2024-09-18T04:56:31.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0084543150000000004 2024-09-18T04:56:31.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.083182+0000", 2024-09-18T04:56:31.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9782000000000004e-05 2024-09-18T04:56:31.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:31.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.083469+0000", 2024-09-18T04:56:31.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028766099999999998 2024-09-18T04:56:31.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:31.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084836+0000", 2024-09-18T04:56:31.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001366826 2024-09-18T04:56:31.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:31.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084997+0000", 2024-09-18T04:56:31.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016116500000000001 2024-09-18T04:56:31.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.085043+0000", 2024-09-18T04:56:31.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5234000000000001e-05 2024-09-18T04:56:31.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.085075+0000", 2024-09-18T04:56:31.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2753999999999998e-05 2024-09-18T04:56:31.802 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.802 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.802 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.802 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:1139 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:31.802 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.074660+0000", 2024-09-18T04:56:31.802 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.6096223589999998, 2024-09-18T04:56:31.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011003634999999999, 2024-09-18T04:56:31.802 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.802 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.802 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:31.803 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:31.803 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:31.803 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1139 2024-09-18T04:56:31.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.803 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074660+0000", 2024-09-18T04:56:31.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074660+0000", 2024-09-18T04:56:31.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074660+0000", 2024-09-18T04:56:31.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:56:31.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074661+0000", 2024-09-18T04:56:31.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.528e-06 2024-09-18T04:56:31.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074664+0000", 2024-09-18T04:56:31.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.039e-06 2024-09-18T04:56:31.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074667+0000", 2024-09-18T04:56:31.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6029999999999999e-06 2024-09-18T04:56:31.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084216+0000", 2024-09-18T04:56:31.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0095484650000000008 2024-09-18T04:56:31.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084281+0000", 2024-09-18T04:56:31.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5046e-05 2024-09-18T04:56:31.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:31.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084527+0000", 2024-09-18T04:56:31.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024690600000000002 2024-09-18T04:56:31.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:31.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.085206+0000", 2024-09-18T04:56:31.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067853800000000001 2024-09-18T04:56:31.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:31.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.085581+0000", 2024-09-18T04:56:31.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00037512 2024-09-18T04:56:31.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.085629+0000", 2024-09-18T04:56:31.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7620000000000001e-05 2024-09-18T04:56:31.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.085664+0000", 2024-09-18T04:56:31.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4987999999999998e-05 2024-09-18T04:56:31.810 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.810 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.810 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.811 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:1279 3.18 3:1b790e41:::rbd_data.11ac99060bb1.0000000000000101:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:31.811 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.174431+0000", 2024-09-18T04:56:31.811 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.5098510689999998, 2024-09-18T04:56:31.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0053931070000000003, 2024-09-18T04:56:31.811 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.811 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.811 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:31.811 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:31.811 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:31.812 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1279 2024-09-18T04:56:31.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.812 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.174431+0000", 2024-09-18T04:56:31.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.174431+0000", 2024-09-18T04:56:31.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.174431+0000", 2024-09-18T04:56:31.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:31.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.174437+0000", 2024-09-18T04:56:31.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2580000000000001e-06 2024-09-18T04:56:31.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.174446+0000", 2024-09-18T04:56:31.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.448e-06 2024-09-18T04:56:31.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.174451+0000", 2024-09-18T04:56:31.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3800000000000004e-06 2024-09-18T04:56:31.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.174483+0000", 2024-09-18T04:56:31.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2190999999999997e-05 2024-09-18T04:56:31.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.174584+0000", 2024-09-18T04:56:31.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101208 2024-09-18T04:56:31.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:31.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.175626+0000", 2024-09-18T04:56:31.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001041859 2024-09-18T04:56:31.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:31.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.179711+0000", 2024-09-18T04:56:31.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00408545 2024-09-18T04:56:31.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:31.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.179723+0000", 2024-09-18T04:56:31.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1258e-05 2024-09-18T04:56:31.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.179785+0000", 2024-09-18T04:56:31.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2150999999999995e-05 2024-09-18T04:56:31.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.179825+0000", 2024-09-18T04:56:31.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9647000000000002e-05 2024-09-18T04:56:31.819 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.819 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.819 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.820 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:1467 3.7 3:e6f33ce6:::rbd_object_map.11ac99060bb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:31.820 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.247138+0000", 2024-09-18T04:56:31.820 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.4371448459999998, 2024-09-18T04:56:31.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058561139999999999, 2024-09-18T04:56:31.820 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.820 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.820 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:31.820 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:31.820 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:31.821 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1467 2024-09-18T04:56:31.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.821 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.247138+0000", 2024-09-18T04:56:31.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.247138+0000", 2024-09-18T04:56:31.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.247137+0000", 2024-09-18T04:56:31.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:31.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.247142+0000", 2024-09-18T04:56:31.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9010000000000002e-06 2024-09-18T04:56:31.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.247153+0000", 2024-09-18T04:56:31.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1032e-05 2024-09-18T04:56:31.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.247159+0000", 2024-09-18T04:56:31.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8370000000000002e-06 2024-09-18T04:56:31.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.248305+0000", 2024-09-18T04:56:31.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001146453 2024-09-18T04:56:31.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.248375+0000", 2024-09-18T04:56:31.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9748999999999999e-05 2024-09-18T04:56:31.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-18T04:56:31.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.248728+0000", 2024-09-18T04:56:31.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00035267600000000001 2024-09-18T04:56:31.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:31.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.250710+0000", 2024-09-18T04:56:31.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019824869999999998 2024-09-18T04:56:31.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:31.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.252938+0000", 2024-09-18T04:56:31.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022282439999999999 2024-09-18T04:56:31.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.252968+0000", 2024-09-18T04:56:31.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9386999999999998e-05 2024-09-18T04:56:31.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.252994+0000", 2024-09-18T04:56:31.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6092e-05 2024-09-18T04:56:31.828 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.828 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.828 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.828 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:1611 3.7 3:e6f33ce6:::rbd_object_map.11ac99060bb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:31.829 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.388383+0000", 2024-09-18T04:56:31.829 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.2958998049999999, 2024-09-18T04:56:31.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006597538, 2024-09-18T04:56:31.829 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.829 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.829 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:31.829 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:31.829 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:31.829 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1611 2024-09-18T04:56:31.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.829 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.388383+0000", 2024-09-18T04:56:31.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.388383+0000", 2024-09-18T04:56:31.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.388382+0000", 2024-09-18T04:56:31.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:31.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.388387+0000", 2024-09-18T04:56:31.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6569999999999996e-06 2024-09-18T04:56:31.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.388392+0000", 2024-09-18T04:56:31.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.673e-06 2024-09-18T04:56:31.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.388395+0000", 2024-09-18T04:56:31.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9220000000000001e-06 2024-09-18T04:56:31.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.389502+0000", 2024-09-18T04:56:31.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011077400000000001 2024-09-18T04:56:31.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.389552+0000", 2024-09-18T04:56:31.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9861999999999998e-05 2024-09-18T04:56:31.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-18T04:56:31.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.389775+0000", 2024-09-18T04:56:31.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022317699999999999 2024-09-18T04:56:31.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:31.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.390312+0000", 2024-09-18T04:56:31.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00053678000000000005 2024-09-18T04:56:31.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:31.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.394931+0000", 2024-09-18T04:56:31.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0046185100000000001 2024-09-18T04:56:31.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.394959+0000", 2024-09-18T04:56:31.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7994999999999999e-05 2024-09-18T04:56:31.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.394980+0000", 2024-09-18T04:56:31.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1472999999999999e-05 2024-09-18T04:56:31.835 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.835 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.835 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.835 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:2020 3.7 3:e6f33ce6:::rbd_object_map.11ac99060bb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:31.836 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.554716+0000", 2024-09-18T04:56:31.836 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.1295665779999999, 2024-09-18T04:56:31.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0069705610000000001, 2024-09-18T04:56:31.836 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.836 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.836 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:31.836 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:31.836 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:31.836 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2020 2024-09-18T04:56:31.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.836 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.554716+0000", 2024-09-18T04:56:31.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.554716+0000", 2024-09-18T04:56:31.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.554715+0000", 2024-09-18T04:56:31.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:31.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.554721+0000", 2024-09-18T04:56:31.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8379999999999998e-06 2024-09-18T04:56:31.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.554751+0000", 2024-09-18T04:56:31.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9896e-05 2024-09-18T04:56:31.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.554758+0000", 2024-09-18T04:56:31.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3599999999999998e-06 2024-09-18T04:56:31.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.555664+0000", 2024-09-18T04:56:31.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00090563299999999998 2024-09-18T04:56:31.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.555752+0000", 2024-09-18T04:56:31.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8358999999999999e-05 2024-09-18T04:56:31.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-18T04:56:31.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.556088+0000", 2024-09-18T04:56:31.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033591099999999999 2024-09-18T04:56:31.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:31.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.557300+0000", 2024-09-18T04:56:31.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012117969999999999 2024-09-18T04:56:31.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:31.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.561500+0000", 2024-09-18T04:56:31.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0042000639999999999 2024-09-18T04:56:31.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.561605+0000", 2024-09-18T04:56:31.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010563100000000001 2024-09-18T04:56:31.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.561686+0000", 2024-09-18T04:56:31.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0965999999999997e-05 2024-09-18T04:56:31.841 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.841 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.841 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.842 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:2021 3.7 3:e6f33ce6:::rbd_object_map.11ac99060bb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:31.842 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.555332+0000", 2024-09-18T04:56:31.842 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.1289507969999999, 2024-09-18T04:56:31.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0064605660000000001, 2024-09-18T04:56:31.842 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.842 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.842 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:31.842 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:31.842 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:31.842 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2021 2024-09-18T04:56:31.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.842 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.555332+0000", 2024-09-18T04:56:31.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.555332+0000", 2024-09-18T04:56:31.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.555331+0000", 2024-09-18T04:56:31.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:31.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.555336+0000", 2024-09-18T04:56:31.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0150000000000003e-06 2024-09-18T04:56:31.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.555345+0000", 2024-09-18T04:56:31.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0310000000000004e-06 2024-09-18T04:56:31.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.555350+0000", 2024-09-18T04:56:31.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5280000000000003e-06 2024-09-18T04:56:31.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.556568+0000", 2024-09-18T04:56:31.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012179339999999999 2024-09-18T04:56:31.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.556642+0000", 2024-09-18T04:56:31.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3280000000000003e-05 2024-09-18T04:56:31.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-18T04:56:31.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.556926+0000", 2024-09-18T04:56:31.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000284739 2024-09-18T04:56:31.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:31.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.557667+0000", 2024-09-18T04:56:31.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00074095300000000003 2024-09-18T04:56:31.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:31.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.561722+0000", 2024-09-18T04:56:31.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040548270000000004 2024-09-18T04:56:31.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.561750+0000", 2024-09-18T04:56:31.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7440000000000002e-05 2024-09-18T04:56:31.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.561792+0000", 2024-09-18T04:56:31.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2592999999999997e-05 2024-09-18T04:56:31.849 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.849 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.849 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.850 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:2064 3.7 3:e6f33ce6:::rbd_object_map.11ac99060bb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:31.850 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.613367+0000", 2024-09-18T04:56:31.850 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.0709158740000002, 2024-09-18T04:56:31.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0052065430000000001, 2024-09-18T04:56:31.850 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.850 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.850 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:31.850 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:31.851 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:31.851 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2064 2024-09-18T04:56:31.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.851 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.613367+0000", 2024-09-18T04:56:31.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.613367+0000", 2024-09-18T04:56:31.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.613366+0000", 2024-09-18T04:56:31.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:31.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.613368+0000", 2024-09-18T04:56:31.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.886e-06 2024-09-18T04:56:31.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.613373+0000", 2024-09-18T04:56:31.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8949999999999997e-06 2024-09-18T04:56:31.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.613376+0000", 2024-09-18T04:56:31.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0199999999999999e-06 2024-09-18T04:56:31.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.614462+0000", 2024-09-18T04:56:31.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001085761 2024-09-18T04:56:31.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.614510+0000", 2024-09-18T04:56:31.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7991999999999997e-05 2024-09-18T04:56:31.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-18T04:56:31.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.614695+0000", 2024-09-18T04:56:31.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018487199999999999 2024-09-18T04:56:31.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:31.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.615346+0000", 2024-09-18T04:56:31.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00065114100000000005 2024-09-18T04:56:31.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:31.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.618531+0000", 2024-09-18T04:56:31.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0031846159999999999 2024-09-18T04:56:31.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.618554+0000", 2024-09-18T04:56:31.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3054e-05 2024-09-18T04:56:31.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.618573+0000", 2024-09-18T04:56:31.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9556e-05 2024-09-18T04:56:31.860 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.860 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.860 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.860 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:2138 3.7 3:e50557b3:::rbd_data.11ac99060bb1.00000000000001d1:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:31.860 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.728591+0000", 2024-09-18T04:56:31.860 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 2.9556910690000002, 2024-09-18T04:56:31.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0083980019999999999, 2024-09-18T04:56:31.860 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.860 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.861 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:31.861 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:31.861 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:31.861 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2138 2024-09-18T04:56:31.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.861 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.728591+0000", 2024-09-18T04:56:31.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.728591+0000", 2024-09-18T04:56:31.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.728591+0000", 2024-09-18T04:56:31.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:31.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.728595+0000", 2024-09-18T04:56:31.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.746e-06 2024-09-18T04:56:31.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.728606+0000", 2024-09-18T04:56:31.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0448999999999999e-05 2024-09-18T04:56:31.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.728613+0000", 2024-09-18T04:56:31.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4789999999999999e-06 2024-09-18T04:56:31.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.728642+0000", 2024-09-18T04:56:31.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9141000000000001e-05 2024-09-18T04:56:31.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.728722+0000", 2024-09-18T04:56:31.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9268000000000001e-05 2024-09-18T04:56:31.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-18T04:56:31.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.730157+0000", 2024-09-18T04:56:31.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014354179999999999 2024-09-18T04:56:31.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:31.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.736871+0000", 2024-09-18T04:56:31.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067139469999999996 2024-09-18T04:56:31.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:31.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.736886+0000", 2024-09-18T04:56:31.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5056e-05 2024-09-18T04:56:31.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.736942+0000", 2024-09-18T04:56:31.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6153999999999998e-05 2024-09-18T04:56:31.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.736989+0000", 2024-09-18T04:56:31.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7184000000000001e-05 2024-09-18T04:56:31.866 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.866 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.867 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.867 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:2144 3.18 3:1d491e03:::rbd_data.11ac99060bb1.000000000000019f:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:31.867 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.744894+0000", 2024-09-18T04:56:31.867 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 2.9393882360000001, 2024-09-18T04:56:31.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073157459999999997, 2024-09-18T04:56:31.867 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:31.867 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:31.867 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:31.867 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:31.867 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:31.867 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2144 2024-09-18T04:56:31.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.868 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:31.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:31.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.744894+0000", 2024-09-18T04:56:31.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:31.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.744894+0000", 2024-09-18T04:56:31.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:31.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:31.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.744893+0000", 2024-09-18T04:56:31.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:31.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:31.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.744898+0000", 2024-09-18T04:56:31.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9609999999999996e-06 2024-09-18T04:56:31.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:31.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.744908+0000", 2024-09-18T04:56:31.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2560000000000008e-06 2024-09-18T04:56:31.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:31.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.744912+0000", 2024-09-18T04:56:31.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3970000000000004e-06 2024-09-18T04:56:31.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:31.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.744957+0000", 2024-09-18T04:56:31.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5241999999999998e-05 2024-09-18T04:56:31.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:31.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.745068+0000", 2024-09-18T04:56:31.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111118 2024-09-18T04:56:31.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:31.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.746476+0000", 2024-09-18T04:56:31.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014079489999999999 2024-09-18T04:56:31.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:31.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.752138+0000", 2024-09-18T04:56:31.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0056614259999999998 2024-09-18T04:56:31.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:31.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.752146+0000", 2024-09-18T04:56:31.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7520000000000002e-06 2024-09-18T04:56:31.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:31.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.752182+0000", 2024-09-18T04:56:31.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6003000000000001e-05 2024-09-18T04:56:31.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:31.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:31.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:31.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.752210+0000", 2024-09-18T04:56:31.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7529e-05 2024-09-18T04:56:31.872 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.873 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.873 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.873 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:31.873 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:31.873 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:31.873 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:56:31.875 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:31.872+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:31.976 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:31.973+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:31.978 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:31.978 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:31.978 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:56:31.978 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:31.987 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:56:32.077 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:32.075+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:32.089 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:32.089 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:32.089 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:56:32.089 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:56:32.089 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:32.097 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:56:32.158 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:56:32.159 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-18T04:56:32.163 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-18T04:56:32.173 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-09-18T04:56:31.155403+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":130583496,"num_objects":132,"num_object_clones":24,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":33,"num_read":5124,"num_read_kb":17030,"num_write":1290,"num_write_kb":11124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1048,"ondisk_log_size":1048,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240624,"kb_used_data":3956,"kb_used_omap":12,"kb_used_meta":236531,"kb_avail":754734096,"statfs":{"total":773094113280,"available":772847714304,"internally_reserved":0,"allocated":4050944,"data_stored":23827635,"data_compressed":110080,"data_compressed_allocated":770048,"data_compressed_original":21934080,"omap_allocated":12742,"internal_metadata":242208314},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":129993044,"num_objects":123,"num_object_clones":24,"num_object_copies":246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":123,"num_whiteouts":33,"num_read":4974,"num_read_kb":16904,"num_write":1123,"num_write_kb":8700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016802"},"pg_stats":[{"pgid":"3.1f","version":"25'14","reported_seq":38,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.856629+0000","last_change":"2024-09-18T04:56:23.130295+0000","last_active":"2024-09-18T04:56:28.856629+0000","last_peered":"2024-09-18T04:56:28.856629+0000","last_clean":"2024-09-18T04:56:28.856629+0000","last_became_active":"2024-09-18T04:56:21.132342+0000","last_became_peered":"2024-09-18T04:56:21.132342+0000","last_unstale":"2024-09-18T04:56:28.856629+0000","last_undegraded":"2024-09-18T04:56:28.856629+0000","last_fullsized":"2024-09-18T04:56:28.856629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:43:47.128460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011070279999999999,"stat_sum":{"num_bytes":262160,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":12,"num_read_kb":2,"num_write":13,"num_write_kb":64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"25'94","reported_seq":116,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.787882+0000","last_change":"2024-09-18T04:56:23.582447+0000","last_active":"2024-09-18T04:56:28.787882+0000","last_peered":"2024-09-18T04:56:28.787882+0000","last_clean":"2024-09-18T04:56:28.787882+0000","last_became_active":"2024-09-18T04:56:21.126431+0000","last_became_peered":"2024-09-18T04:56:21.126431+0000","last_unstale":"2024-09-18T04:56:28.787882+0000","last_undegraded":"2024-09-18T04:56:28.787882+0000","last_fullsized":"2024-09-18T04:56:28.787882+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":94,"log_dups_size":0,"ondisk_log_size":94,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:59.671751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062644600000000001,"stat_sum":{"num_bytes":65566,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":401,"num_read_kb":319,"num_write":147,"num_write_kb":295,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"25'13","reported_seq":135,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.555070+0000","last_change":"2024-09-18T04:56:23.137064+0000","last_active":"2024-09-18T04:56:28.555070+0000","last_peered":"2024-09-18T04:56:28.555070+0000","last_clean":"2024-09-18T04:56:28.555070+0000","last_became_active":"2024-09-18T04:56:21.131613+0000","last_became_peered":"2024-09-18T04:56:21.131613+0000","last_unstale":"2024-09-18T04:56:28.555070+0000","last_undegraded":"2024-09-18T04:56:28.555070+0000","last_fullsized":"2024-09-18T04:56:28.555070+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:15:27.419409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000455248,"stat_sum":{"num_bytes":10485760,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":3230,"num_write":12,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"26'14","reported_seq":60,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T04:56:29.198482+0000","last_change":"2024-09-18T04:56:23.137422+0000","last_active":"2024-09-18T04:56:29.198482+0000","last_peered":"2024-09-18T04:56:29.198482+0000","last_clean":"2024-09-18T04:56:29.198482+0000","last_became_active":"2024-09-18T04:56:21.129551+0000","last_became_peered":"2024-09-18T04:56:21.129551+0000","last_unstale":"2024-09-18T04:56:29.198482+0000","last_undegraded":"2024-09-18T04:56:29.198482+0000","last_fullsized":"2024-09-18T04:56:29.198482+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:38:38.231233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081678499999999999,"stat_sum":{"num_bytes":196682,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":44,"num_read_kb":36,"num_write":13,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"26'15","reported_seq":37,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T04:56:29.177594+0000","last_change":"2024-09-18T04:56:23.130841+0000","last_active":"2024-09-18T04:56:29.177594+0000","last_peered":"2024-09-18T04:56:29.177594+0000","last_clean":"2024-09-18T04:56:29.177594+0000","last_became_active":"2024-09-18T04:56:21.129660+0000","last_became_peered":"2024-09-18T04:56:21.129660+0000","last_unstale":"2024-09-18T04:56:29.177594+0000","last_undegraded":"2024-09-18T04:56:29.177594+0000","last_fullsized":"2024-09-18T04:56:29.177594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:43.007677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014550170000000001,"stat_sum":{"num_bytes":185347,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":9,"num_read_kb":2,"num_write":13,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"25'9","reported_seq":155,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T04:56:29.224908+0000","last_change":"2024-09-18T04:56:23.133311+0000","last_active":"2024-09-18T04:56:29.224908+0000","last_peered":"2024-09-18T04:56:29.224908+0000","last_clean":"2024-09-18T04:56:29.224908+0000","last_became_active":"2024-09-18T04:56:21.127888+0000","last_became_peered":"2024-09-18T04:56:21.127888+0000","last_unstale":"2024-09-18T04:56:29.224908+0000","last_undegraded":"2024-09-18T04:56:29.224908+0000","last_fullsized":"2024-09-18T04:56:29.224908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:35:20.652713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087073300000000005,"stat_sum":{"num_bytes":167252,"num_objects":4,"num_object_clones":1,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":130,"num_read_kb":179,"num_write":7,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"25'26","reported_seq":190,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T04:56:29.184431+0000","last_change":"2024-09-18T04:56:23.130039+0000","last_active":"2024-09-18T04:56:29.184431+0000","last_peered":"2024-09-18T04:56:29.184431+0000","last_clean":"2024-09-18T04:56:29.184431+0000","last_became_active":"2024-09-18T04:56:21.125153+0000","last_became_peered":"2024-09-18T04:56:21.125153+0000","last_unstale":"2024-09-18T04:56:29.184431+0000","last_undegraded":"2024-09-18T04:56:29.184431+0000","last_fullsized":"2024-09-18T04:56:29.184431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:54:02.903679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012370669999999999,"stat_sum":{"num_bytes":283237,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":433,"num_read_kb":369,"num_write":26,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"25'72","reported_seq":94,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.834662+0000","last_change":"2024-09-18T04:56:23.582074+0000","last_active":"2024-09-18T04:56:28.834662+0000","last_peered":"2024-09-18T04:56:28.834662+0000","last_clean":"2024-09-18T04:56:28.834662+0000","last_became_active":"2024-09-18T04:56:21.127959+0000","last_became_peered":"2024-09-18T04:56:21.127959+0000","last_unstale":"2024-09-18T04:56:28.834662+0000","last_undegraded":"2024-09-18T04:56:28.834662+0000","last_fullsized":"2024-09-18T04:56:28.834662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:17:32.019845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053040699999999999,"stat_sum":{"num_bytes":37,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":269,"num_read_kb":212,"num_write":107,"num_write_kb":229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"25'4","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.540809+0000","last_change":"2024-09-18T04:56:23.130832+0000","last_active":"2024-09-18T04:56:28.540809+0000","last_peered":"2024-09-18T04:56:28.540809+0000","last_clean":"2024-09-18T04:56:28.540809+0000","last_became_active":"2024-09-18T04:56:21.130316+0000","last_became_peered":"2024-09-18T04:56:21.130316+0000","last_unstale":"2024-09-18T04:56:28.540809+0000","last_undegraded":"2024-09-18T04:56:28.540809+0000","last_fullsized":"2024-09-18T04:56:28.540809+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:12:05.411040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001612023,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"25'10","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.239623+0000","last_change":"2024-09-18T04:56:23.136846+0000","last_active":"2024-09-18T04:56:28.239623+0000","last_peered":"2024-09-18T04:56:28.239623+0000","last_clean":"2024-09-18T04:56:28.239623+0000","last_became_active":"2024-09-18T04:56:21.131133+0000","last_became_peered":"2024-09-18T04:56:21.131133+0000","last_unstale":"2024-09-18T04:56:28.239623+0000","last_undegraded":"2024-09-18T04:56:28.239623+0000","last_fullsized":"2024-09-18T04:56:28.239623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:01:57.036904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025847199999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2,"num_read_kb":0,"num_write":8,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"25'27","reported_seq":263,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T04:56:29.154785+0000","last_change":"2024-09-18T04:56:23.142657+0000","last_active":"2024-09-18T04:56:29.154785+0000","last_peered":"2024-09-18T04:56:29.154785+0000","last_clean":"2024-09-18T04:56:29.154785+0000","last_became_active":"2024-09-18T04:56:21.131979+0000","last_became_peered":"2024-09-18T04:56:21.131979+0000","last_unstale":"2024-09-18T04:56:29.154785+0000","last_undegraded":"2024-09-18T04:56:29.154785+0000","last_fullsized":"2024-09-18T04:56:29.154785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:16:43.303511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029355000000000001,"stat_sum":{"num_bytes":21084894,"num_objects":8,"num_object_clones":3,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":238,"num_read_kb":6738,"num_write":23,"num_write_kb":120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"25'8","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.890474+0000","last_change":"2024-09-18T04:56:23.132876+0000","last_active":"2024-09-18T04:56:28.890474+0000","last_peered":"2024-09-18T04:56:28.890474+0000","last_clean":"2024-09-18T04:56:28.890474+0000","last_became_active":"2024-09-18T04:56:21.134979+0000","last_became_peered":"2024-09-18T04:56:21.134979+0000","last_unstale":"2024-09-18T04:56:28.890474+0000","last_undegraded":"2024-09-18T04:56:28.890474+0000","last_fullsized":"2024-09-18T04:56:28.890474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:36:24.881782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038004100000000002,"stat_sum":{"num_bytes":429135,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.066322+0000","last_change":"2024-09-18T04:56:12.390222+0000","last_active":"2024-09-18T04:56:28.066322+0000","last_peered":"2024-09-18T04:56:28.066322+0000","last_clean":"2024-09-18T04:56:28.066322+0000","last_became_active":"2024-09-18T04:56:10.401272+0000","last_became_peered":"2024-09-18T04:56:10.401272+0000","last_unstale":"2024-09-18T04:56:28.066322+0000","last_undegraded":"2024-09-18T04:56:28.066322+0000","last_fullsized":"2024-09-18T04:56:28.066322+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:33:50.692096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063436799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"25'5","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.662454+0000","last_change":"2024-09-18T04:56:23.137022+0000","last_active":"2024-09-18T04:56:28.662454+0000","last_peered":"2024-09-18T04:56:28.662454+0000","last_clean":"2024-09-18T04:56:28.662454+0000","last_became_active":"2024-09-18T04:56:21.129964+0000","last_became_peered":"2024-09-18T04:56:21.129964+0000","last_unstale":"2024-09-18T04:56:28.662454+0000","last_undegraded":"2024-09-18T04:56:28.662454+0000","last_fullsized":"2024-09-18T04:56:28.662454+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:29:32.843883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043204099999999998,"stat_sum":{"num_bytes":16,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":2,"num_write":5,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.066490+0000","last_change":"2024-09-18T04:56:12.389971+0000","last_active":"2024-09-18T04:56:28.066490+0000","last_peered":"2024-09-18T04:56:28.066490+0000","last_clean":"2024-09-18T04:56:28.066490+0000","last_became_active":"2024-09-18T04:56:10.412595+0000","last_became_peered":"2024-09-18T04:56:10.412595+0000","last_unstale":"2024-09-18T04:56:28.066490+0000","last_undegraded":"2024-09-18T04:56:28.066490+0000","last_fullsized":"2024-09-18T04:56:28.066490+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:45.638420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038576799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"25'16","reported_seq":39,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.563157+0000","last_change":"2024-09-18T04:56:23.129727+0000","last_active":"2024-09-18T04:56:28.563157+0000","last_peered":"2024-09-18T04:56:28.563157+0000","last_clean":"2024-09-18T04:56:28.563157+0000","last_became_active":"2024-09-18T04:56:21.122488+0000","last_became_peered":"2024-09-18T04:56:21.122488+0000","last_unstale":"2024-09-18T04:56:28.563157+0000","last_undegraded":"2024-09-18T04:56:28.563157+0000","last_fullsized":"2024-09-18T04:56:28.563157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:06:03.886036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072197,"stat_sum":{"num_bytes":183797,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":8,"num_read_kb":42,"num_write":15,"num_write_kb":59,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.017503+0000","last_change":"2024-09-18T04:56:12.412298+0000","last_active":"2024-09-18T04:56:28.017503+0000","last_peered":"2024-09-18T04:56:28.017503+0000","last_clean":"2024-09-18T04:56:28.017503+0000","last_became_active":"2024-09-18T04:56:10.399718+0000","last_became_peered":"2024-09-18T04:56:10.399718+0000","last_unstale":"2024-09-18T04:56:28.017503+0000","last_undegraded":"2024-09-18T04:56:28.017503+0000","last_fullsized":"2024-09-18T04:56:28.017503+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:41:31.956685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024059099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"26'204","reported_seq":230,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T04:56:29.156777+0000","last_change":"2024-09-18T04:56:23.582182+0000","last_active":"2024-09-18T04:56:29.156777+0000","last_peered":"2024-09-18T04:56:29.156777+0000","last_clean":"2024-09-18T04:56:29.156777+0000","last_became_active":"2024-09-18T04:56:21.125785+0000","last_became_peered":"2024-09-18T04:56:21.125785+0000","last_unstale":"2024-09-18T04:56:29.156777+0000","last_undegraded":"2024-09-18T04:56:29.156777+0000","last_fullsized":"2024-09-18T04:56:29.156777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:35:40.684580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048263400000000001,"stat_sum":{"num_bytes":1787848,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":957,"num_read_kb":765,"num_write":340,"num_write_kb":598,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.002464+0000","last_change":"2024-09-18T04:56:12.413007+0000","last_active":"2024-09-18T04:56:28.002464+0000","last_peered":"2024-09-18T04:56:28.002464+0000","last_clean":"2024-09-18T04:56:28.002464+0000","last_became_active":"2024-09-18T04:56:10.399076+0000","last_became_peered":"2024-09-18T04:56:10.399076+0000","last_unstale":"2024-09-18T04:56:28.002464+0000","last_undegraded":"2024-09-18T04:56:28.002464+0000","last_fullsized":"2024-09-18T04:56:28.002464+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:54:29.826665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045144100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"25'16","reported_seq":54,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T04:56:29.158609+0000","last_change":"2024-09-18T04:56:23.129577+0000","last_active":"2024-09-18T04:56:29.158609+0000","last_peered":"2024-09-18T04:56:29.158609+0000","last_clean":"2024-09-18T04:56:29.158609+0000","last_became_active":"2024-09-18T04:56:21.127750+0000","last_became_peered":"2024-09-18T04:56:21.127750+0000","last_unstale":"2024-09-18T04:56:29.158609+0000","last_undegraded":"2024-09-18T04:56:29.158609+0000","last_fullsized":"2024-09-18T04:56:29.158609+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:58:18.075339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039168900000000002,"stat_sum":{"num_bytes":10485797,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":33,"num_read_kb":320,"num_write":15,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"25'10","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.838961+0000","last_change":"2024-09-18T04:56:23.130445+0000","last_active":"2024-09-18T04:56:28.838961+0000","last_peered":"2024-09-18T04:56:28.838961+0000","last_clean":"2024-09-18T04:56:28.838961+0000","last_became_active":"2024-09-18T04:56:21.130514+0000","last_became_peered":"2024-09-18T04:56:21.130514+0000","last_unstale":"2024-09-18T04:56:28.838961+0000","last_undegraded":"2024-09-18T04:56:28.838961+0000","last_fullsized":"2024-09-18T04:56:28.838961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:06:37.491009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001360497,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":3,"num_read_kb":0,"num_write":8,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":169,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.005126+0000","last_change":"2024-09-18T04:56:11.428051+0000","last_active":"2024-09-18T04:56:28.005126+0000","last_peered":"2024-09-18T04:56:28.005126+0000","last_clean":"2024-09-18T04:56:28.005126+0000","last_became_active":"2024-09-18T04:56:11.427170+0000","last_became_peered":"2024-09-18T04:56:11.427170+0000","last_unstale":"2024-09-18T04:56:28.005126+0000","last_undegraded":"2024-09-18T04:56:28.005126+0000","last_fullsized":"2024-09-18T04:56:28.005126+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_clean_scrub_stamp":"2024-09-18T04:56:10.373785+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:16:08.102627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.001007+0000","last_change":"2024-09-18T04:56:12.413125+0000","last_active":"2024-09-18T04:56:28.001007+0000","last_peered":"2024-09-18T04:56:28.001007+0000","last_clean":"2024-09-18T04:56:28.001007+0000","last_became_active":"2024-09-18T04:56:10.401641+0000","last_became_peered":"2024-09-18T04:56:10.401641+0000","last_unstale":"2024-09-18T04:56:28.001007+0000","last_undegraded":"2024-09-18T04:56:28.001007+0000","last_fullsized":"2024-09-18T04:56:28.001007+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:19:46.375505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000622431,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"26'32","reported_seq":106,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T04:56:29.215625+0000","last_change":"2024-09-18T04:56:23.129808+0000","last_active":"2024-09-18T04:56:29.215625+0000","last_peered":"2024-09-18T04:56:29.215625+0000","last_clean":"2024-09-18T04:56:29.215625+0000","last_became_active":"2024-09-18T04:56:21.122785+0000","last_became_peered":"2024-09-18T04:56:21.122785+0000","last_unstale":"2024-09-18T04:56:29.215625+0000","last_undegraded":"2024-09-18T04:56:29.215625+0000","last_fullsized":"2024-09-18T04:56:29.215625+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:05:30.391004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010536390000000001,"stat_sum":{"num_bytes":19874166,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":105,"num_read_kb":1397,"num_write":27,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.005162+0000","last_change":"2024-09-18T04:56:12.398631+0000","last_active":"2024-09-18T04:56:28.005162+0000","last_peered":"2024-09-18T04:56:28.005162+0000","last_clean":"2024-09-18T04:56:28.005162+0000","last_became_active":"2024-09-18T04:56:10.412614+0000","last_became_peered":"2024-09-18T04:56:10.412614+0000","last_unstale":"2024-09-18T04:56:28.005162+0000","last_undegraded":"2024-09-18T04:56:28.005162+0000","last_fullsized":"2024-09-18T04:56:28.005162+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:44:13.401320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075126299999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"25'11","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.675083+0000","last_change":"2024-09-18T04:56:23.133345+0000","last_active":"2024-09-18T04:56:28.675083+0000","last_peered":"2024-09-18T04:56:28.675083+0000","last_clean":"2024-09-18T04:56:28.675083+0000","last_became_active":"2024-09-18T04:56:21.128793+0000","last_became_peered":"2024-09-18T04:56:21.128793+0000","last_unstale":"2024-09-18T04:56:28.675083+0000","last_undegraded":"2024-09-18T04:56:28.675083+0000","last_fullsized":"2024-09-18T04:56:28.675083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:10:36.658665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067161900000000004,"stat_sum":{"num_bytes":98304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":5,"num_read_kb":0,"num_write":10,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.001281+0000","last_change":"2024-09-18T04:56:12.390044+0000","last_active":"2024-09-18T04:56:28.001281+0000","last_peered":"2024-09-18T04:56:28.001281+0000","last_clean":"2024-09-18T04:56:28.001281+0000","last_became_active":"2024-09-18T04:56:10.413042+0000","last_became_peered":"2024-09-18T04:56:10.413042+0000","last_unstale":"2024-09-18T04:56:28.001281+0000","last_undegraded":"2024-09-18T04:56:28.001281+0000","last_fullsized":"2024-09-18T04:56:28.001281+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:46:11.074243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"25'11","reported_seq":38,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.867049+0000","last_change":"2024-09-18T04:56:23.581937+0000","last_active":"2024-09-18T04:56:28.867049+0000","last_peered":"2024-09-18T04:56:28.867049+0000","last_clean":"2024-09-18T04:56:28.867049+0000","last_became_active":"2024-09-18T04:56:21.130465+0000","last_became_peered":"2024-09-18T04:56:21.130465+0000","last_unstale":"2024-09-18T04:56:28.867049+0000","last_undegraded":"2024-09-18T04:56:28.867049+0000","last_fullsized":"2024-09-18T04:56:28.867049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:38:38.231233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037357100000000001,"stat_sum":{"num_bytes":262160,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":3,"num_write":11,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.005154+0000","last_change":"2024-09-18T04:56:12.398299+0000","last_active":"2024-09-18T04:56:28.005154+0000","last_peered":"2024-09-18T04:56:28.005154+0000","last_clean":"2024-09-18T04:56:28.005154+0000","last_became_active":"2024-09-18T04:56:10.413140+0000","last_became_peered":"2024-09-18T04:56:10.413140+0000","last_unstale":"2024-09-18T04:56:28.005154+0000","last_undegraded":"2024-09-18T04:56:28.005154+0000","last_fullsized":"2024-09-18T04:56:28.005154+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:37:11.125495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041592799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"26'18","reported_seq":43,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T04:56:29.210296+0000","last_change":"2024-09-18T04:56:23.582318+0000","last_active":"2024-09-18T04:56:29.210296+0000","last_peered":"2024-09-18T04:56:29.210296+0000","last_clean":"2024-09-18T04:56:29.210296+0000","last_became_active":"2024-09-18T04:56:21.125295+0000","last_became_peered":"2024-09-18T04:56:21.125295+0000","last_unstale":"2024-09-18T04:56:29.210296+0000","last_undegraded":"2024-09-18T04:56:29.210296+0000","last_fullsized":"2024-09-18T04:56:29.210296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:29:59.427218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053349799999999998,"stat_sum":{"num_bytes":864997,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":2,"num_read":17,"num_read_kb":54,"num_write":16,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"25'22","reported_seq":42,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.838343+0000","last_change":"2024-09-18T04:56:23.133113+0000","last_active":"2024-09-18T04:56:28.838343+0000","last_peered":"2024-09-18T04:56:28.838343+0000","last_clean":"2024-09-18T04:56:28.838343+0000","last_became_active":"2024-09-18T04:56:21.128135+0000","last_became_peered":"2024-09-18T04:56:21.128135+0000","last_unstale":"2024-09-18T04:56:28.838343+0000","last_undegraded":"2024-09-18T04:56:28.838343+0000","last_fullsized":"2024-09-18T04:56:28.838343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:46:04.362749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000513702,"stat_sum":{"num_bytes":425984,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":22,"num_write_kb":308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"26'21","reported_seq":47,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T04:56:29.216905+0000","last_change":"2024-09-18T04:56:23.641225+0000","last_active":"2024-09-18T04:56:29.216905+0000","last_peered":"2024-09-18T04:56:29.216905+0000","last_clean":"2024-09-18T04:56:29.216905+0000","last_became_active":"2024-09-18T04:56:21.127832+0000","last_became_peered":"2024-09-18T04:56:21.127832+0000","last_unstale":"2024-09-18T04:56:29.216905+0000","last_undegraded":"2024-09-18T04:56:29.216905+0000","last_fullsized":"2024-09-18T04:56:29.216905+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:23:15.821734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029947799999999999,"stat_sum":{"num_bytes":826965,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":2,"num_read":16,"num_read_kb":104,"num_write":19,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"25'8","reported_seq":28,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.860254+0000","last_change":"2024-09-18T04:56:23.582528+0000","last_active":"2024-09-18T04:56:28.860254+0000","last_peered":"2024-09-18T04:56:28.860254+0000","last_clean":"2024-09-18T04:56:28.860254+0000","last_became_active":"2024-09-18T04:56:21.127444+0000","last_became_peered":"2024-09-18T04:56:21.127444+0000","last_unstale":"2024-09-18T04:56:28.860254+0000","last_undegraded":"2024-09-18T04:56:28.860254+0000","last_fullsized":"2024-09-18T04:56:28.860254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:15:41.251274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060817200000000003,"stat_sum":{"num_bytes":393263,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"25'9","reported_seq":29,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.835143+0000","last_change":"2024-09-18T04:56:23.136791+0000","last_active":"2024-09-18T04:56:28.835143+0000","last_peered":"2024-09-18T04:56:28.835143+0000","last_clean":"2024-09-18T04:56:28.835143+0000","last_became_active":"2024-09-18T04:56:21.131676+0000","last_became_peered":"2024-09-18T04:56:21.131676+0000","last_unstale":"2024-09-18T04:56:28.835143+0000","last_undegraded":"2024-09-18T04:56:28.835143+0000","last_fullsized":"2024-09-18T04:56:28.835143+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:34:36.563239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000302343,"stat_sum":{"num_bytes":196608,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"26'50","reported_seq":330,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T04:56:29.231220+0000","last_change":"2024-09-18T04:56:23.143279+0000","last_active":"2024-09-18T04:56:29.231220+0000","last_peered":"2024-09-18T04:56:29.231220+0000","last_clean":"2024-09-18T04:56:29.231220+0000","last_became_active":"2024-09-18T04:56:21.133686+0000","last_became_peered":"2024-09-18T04:56:21.133686+0000","last_unstale":"2024-09-18T04:56:29.231220+0000","last_undegraded":"2024-09-18T04:56:29.231220+0000","last_fullsized":"2024-09-18T04:56:29.231220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:50:51.011660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087309599999999996,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":746,"num_read_kb":642,"num_write":52,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"25'10","reported_seq":30,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.856903+0000","last_change":"2024-09-18T04:56:23.143532+0000","last_active":"2024-09-18T04:56:28.856903+0000","last_peered":"2024-09-18T04:56:28.856903+0000","last_clean":"2024-09-18T04:56:28.856903+0000","last_became_active":"2024-09-18T04:56:21.132364+0000","last_became_peered":"2024-09-18T04:56:21.132364+0000","last_unstale":"2024-09-18T04:56:28.856903+0000","last_undegraded":"2024-09-18T04:56:28.856903+0000","last_fullsized":"2024-09-18T04:56:28.856903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:25:44.430438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010190329999999999,"stat_sum":{"num_bytes":364403,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":3,"num_read_kb":0,"num_write":9,"num_write_kb":29,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"25'57","reported_seq":326,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T04:56:29.190879+0000","last_change":"2024-09-18T04:56:23.143500+0000","last_active":"2024-09-18T04:56:29.190879+0000","last_peered":"2024-09-18T04:56:29.190879+0000","last_clean":"2024-09-18T04:56:29.190879+0000","last_became_active":"2024-09-18T04:56:21.130105+0000","last_became_peered":"2024-09-18T04:56:21.130105+0000","last_unstale":"2024-09-18T04:56:29.190879+0000","last_undegraded":"2024-09-18T04:56:29.190879+0000","last_fullsized":"2024-09-18T04:56:29.190879+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:31:55.392378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010487090000000001,"stat_sum":{"num_bytes":11583496,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":731,"num_read_kb":839,"num_write":54,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"26'30","reported_seq":72,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T04:56:29.207123+0000","last_change":"2024-09-18T04:56:23.641201+0000","last_active":"2024-09-18T04:56:29.207123+0000","last_peered":"2024-09-18T04:56:29.207123+0000","last_clean":"2024-09-18T04:56:29.207123+0000","last_became_active":"2024-09-18T04:56:21.131202+0000","last_became_peered":"2024-09-18T04:56:21.131202+0000","last_unstale":"2024-09-18T04:56:29.207123+0000","last_undegraded":"2024-09-18T04:56:29.207123+0000","last_fullsized":"2024-09-18T04:56:29.207123+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:17:35.438723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000304567,"stat_sum":{"num_bytes":22256590,"num_objects":9,"num_object_clones":3,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":39,"num_read_kb":631,"num_write":32,"num_write_kb":2087,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"25'21","reported_seq":59,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.552232+0000","last_change":"2024-09-18T04:56:23.142612+0000","last_active":"2024-09-18T04:56:28.552232+0000","last_peered":"2024-09-18T04:56:28.552232+0000","last_clean":"2024-09-18T04:56:28.552232+0000","last_became_active":"2024-09-18T04:56:21.130594+0000","last_became_peered":"2024-09-18T04:56:21.130594+0000","last_unstale":"2024-09-18T04:56:28.552232+0000","last_undegraded":"2024-09-18T04:56:28.552232+0000","last_fullsized":"2024-09-18T04:56:28.552232+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:46:34.324509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030763399999999999,"stat_sum":{"num_bytes":18327001,"num_objects":7,"num_object_clones":3,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":33,"num_read_kb":469,"num_write":21,"num_write_kb":1665,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"25'12","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-18T04:56:28.834274+0000","last_change":"2024-09-18T04:56:23.129633+0000","last_active":"2024-09-18T04:56:28.834274+0000","last_peered":"2024-09-18T04:56:28.834274+0000","last_clean":"2024-09-18T04:56:28.834274+0000","last_became_active":"2024-09-18T04:56:21.129597+0000","last_became_peered":"2024-09-18T04:56:21.129597+0000","last_unstale":"2024-09-18T04:56:28.834274+0000","last_undegraded":"2024-09-18T04:56:28.834274+0000","last_fullsized":"2024-09-18T04:56:28.834274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:50.665769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059848499999999995,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3,"num_read_kb":0,"num_write":11,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"25'59","reported_seq":255,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-18T04:56:29.197051+0000","last_change":"2024-09-18T04:56:23.143256+0000","last_active":"2024-09-18T04:56:29.197051+0000","last_peered":"2024-09-18T04:56:29.197051+0000","last_clean":"2024-09-18T04:56:29.197051+0000","last_became_active":"2024-09-18T04:56:21.130061+0000","last_became_peered":"2024-09-18T04:56:21.130061+0000","last_unstale":"2024-09-18T04:56:29.197051+0000","last_undegraded":"2024-09-18T04:56:29.197051+0000","last_fullsized":"2024-09-18T04:56:29.197051+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":59,"log_dups_size":0,"ondisk_log_size":59,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:57:15.485819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085320600000000002,"stat_sum":{"num_bytes":8705036,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":629,"num_read_kb":575,"num_write":64,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":129993109,"num_objects":128,"num_object_clones":24,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":33,"num_read":5008,"num_read_kb":16930,"num_write":1129,"num_write_kb":8706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2293760,"data_stored":22426598,"data_compressed":94968,"data_compressed_allocated":729088,"data_compressed_original":21028864,"omap_allocated":0,"internal_metadata":0},"log_size":928,"ondisk_log_size":928,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17398,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":744,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":761856,"data_stored":7341660,"data_compressed":37230,"data_compressed_allocated":241664,"data_compressed_original":6979584,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":495616,"data_stored":4074155,"data_compressed":17176,"data_compressed_allocated":126976,"data_compressed_original":3858432,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":385024,"data_stored":191342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":659456,"data_stored":6812626,"data_compressed":29052,"data_compressed_allocated":212992,"data_compressed_original":6529024,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31544,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340296,"statfs":{"total":96636764160,"available":96604463104,"internally_reserved":0,"allocated":442368,"data_stored":303069,"data_compressed":299,"data_compressed_allocated":8192,"data_compressed_original":57344,"omap_allocated":1592,"internal_metadata":31848904},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38800000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80400000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":335872,"data_stored":175650,"data_compressed":99,"data_compressed_allocated":4096,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":428,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":438272,"data_stored":886187,"data_compressed":9258,"data_compressed_allocated":49152,"data_compressed_original":688128,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.374}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.435}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31640,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94340200,"statfs":{"total":96636764160,"available":96604364800,"internally_reserved":0,"allocated":532480,"data_stored":4042946,"data_compressed":16966,"data_compressed_allocated":126976,"data_compressed_original":3805184,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":319488,"data_stored":3965987,"data_compressed":17148,"data_compressed_allocated":126976,"data_compressed_original":3788800,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":277047,"data_compressed":775,"data_compressed_allocated":16384,"data_compressed_original":147456,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":148294,"data_compressed":555,"data_compressed_allocated":12288,"data_compressed_original":81920,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":306804,"data_compressed":982,"data_compressed_allocated":16384,"data_compressed_original":126976,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":528384,"data_stored":6851388,"data_compressed":29435,"data_compressed_allocated":221184,"data_compressed_original":6569984,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":83402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":290816,"data_stored":4020028,"data_compressed":17176,"data_compressed_allocated":126976,"data_compressed_original":3858432,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":6773648,"data_compressed":28897,"data_compressed_allocated":208896,"data_compressed_original":6455296,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T04:56:32.175 INFO:tasks.thrashosds.thrasher:Setting 3.13 to [3, 4] 2024-09-18T04:56:32.176 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.13', '3', '4'] 2024-09-18T04:56:32.176 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.13 3 4 2024-09-18T04:56:32.179 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:32.175+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:32.234 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone6 from 0xa287ab5 (overlap 0xa287ab5) to 0x71c55e5 2024-09-18T04:56:32.258 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:32.258 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:56:32.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:56:32.258 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:56:32.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.258 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:693 3.f e24/19 3:f0e4723c:::rbd_data.11acdc0eab99.0000000000000056:head v 24'35, mlcod=24'32)", 2024-09-18T04:56:32.258 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.215714+0000", 2024-09-18T04:56:32.258 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.038530465, 2024-09-18T04:56:32.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.004129916, 2024-09-18T04:56:32.259 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.259 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:32.259 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215714+0000", 2024-09-18T04:56:32.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215714+0000", 2024-09-18T04:56:32.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215714+0000", 2024-09-18T04:56:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215734+0000", 2024-09-18T04:56:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0386000000000001e-05 2024-09-18T04:56:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215743+0000", 2024-09-18T04:56:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2550000000000003e-06 2024-09-18T04:56:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215763+0000", 2024-09-18T04:56:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9239e-05 2024-09-18T04:56:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215816+0000", 2024-09-18T04:56:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3121999999999997e-05 2024-09-18T04:56:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215836+0000", 2024-09-18T04:56:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9811999999999999e-05 2024-09-18T04:56:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.219810+0000", 2024-09-18T04:56:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039743620000000004 2024-09-18T04:56:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.219844+0000", 2024-09-18T04:56:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4205999999999997e-05 2024-09-18T04:56:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:694 3.a e24/19 3:54bf2b96:::rbd_data.11acdc0eab99.0000000000000067:head v 24'8, mlcod=23'3)", 2024-09-18T04:56:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.215805+0000", 2024-09-18T04:56:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.0384393440000004, 2024-09-18T04:56:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040608659999999998, 2024-09-18T04:56:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215805+0000", 2024-09-18T04:56:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215805+0000", 2024-09-18T04:56:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215805+0000", 2024-09-18T04:56:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215810+0000", 2024-09-18T04:56:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3990000000000003e-06 2024-09-18T04:56:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215819+0000", 2024-09-18T04:56:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6289999999999999e-06 2024-09-18T04:56:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215825+0000", 2024-09-18T04:56:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8939999999999998e-06 2024-09-18T04:56:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215863+0000", 2024-09-18T04:56:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8313999999999999e-05 2024-09-18T04:56:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215882+0000", 2024-09-18T04:56:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8930999999999999e-05 2024-09-18T04:56:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.219834+0000", 2024-09-18T04:56:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039519170000000001 2024-09-18T04:56:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.219866+0000", 2024-09-18T04:56:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2314000000000003e-05 2024-09-18T04:56:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:691 3.13 e24/19 3:cc0a27f0:::rbd_data.11acdc0eab99.0000000000000052:head v 24'42, mlcod=23'39)", 2024-09-18T04:56:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.216010+0000", 2024-09-18T04:56:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.0382343389999997, 2024-09-18T04:56:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0038708919999999999, 2024-09-18T04:56:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.216010+0000", 2024-09-18T04:56:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.216010+0000", 2024-09-18T04:56:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.216010+0000", 2024-09-18T04:56:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.216015+0000", 2024-09-18T04:56:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6500000000000001e-06 2024-09-18T04:56:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.216025+0000", 2024-09-18T04:56:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3570000000000004e-06 2024-09-18T04:56:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.216039+0000", 2024-09-18T04:56:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4234e-05 2024-09-18T04:56:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.216078+0000", 2024-09-18T04:56:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9317999999999999e-05 2024-09-18T04:56:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.216097+0000", 2024-09-18T04:56:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8512000000000002e-05 2024-09-18T04:56:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.219850+0000", 2024-09-18T04:56:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037528330000000001 2024-09-18T04:56:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.219881+0000", 2024-09-18T04:56:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1538000000000002e-05 2024-09-18T04:56:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:865 3.11 e24/19 3:8b581e29:::rbd_data.11ac33624de9.0000000000000008:head v 24'6, mlcod=23'3)", 2024-09-18T04:56:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.384443+0000", 2024-09-18T04:56:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.869801668, 2024-09-18T04:56:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0053766509999999997, 2024-09-18T04:56:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.384443+0000", 2024-09-18T04:56:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.384443+0000", 2024-09-18T04:56:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.384442+0000", 2024-09-18T04:56:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.385656+0000", 2024-09-18T04:56:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001213626 2024-09-18T04:56:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.385675+0000", 2024-09-18T04:56:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9765000000000001e-05 2024-09-18T04:56:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.385686+0000", 2024-09-18T04:56:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0468999999999999e-05 2024-09-18T04:56:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.385738+0000", 2024-09-18T04:56:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2552000000000001e-05 2024-09-18T04:56:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.385762+0000", 2024-09-18T04:56:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3061000000000002e-05 2024-09-18T04:56:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.389782+0000", 2024-09-18T04:56:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040200749999999997 2024-09-18T04:56:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.389820+0000", 2024-09-18T04:56:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7991999999999998e-05 2024-09-18T04:56:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:867 3.13 e24/19 3:c80b79e3:::rbd_data.11ac33624de9.000000000000000a:head v 24'45, mlcod=24'43)", 2024-09-18T04:56:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.386953+0000", 2024-09-18T04:56:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8672915510000001, 2024-09-18T04:56:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0051944479999999999, 2024-09-18T04:56:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.386953+0000", 2024-09-18T04:56:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.386953+0000", 2024-09-18T04:56:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.386952+0000", 2024-09-18T04:56:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.387720+0000", 2024-09-18T04:56:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00076791999999999995 2024-09-18T04:56:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.387731+0000", 2024-09-18T04:56:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1454000000000001e-05 2024-09-18T04:56:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.387737+0000", 2024-09-18T04:56:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6919999999999999e-06 2024-09-18T04:56:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.387779+0000", 2024-09-18T04:56:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2382999999999997e-05 2024-09-18T04:56:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.387804+0000", 2024-09-18T04:56:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4742000000000001e-05 2024-09-18T04:56:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.392114+0000", 2024-09-18T04:56:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.004309846 2024-09-18T04:56:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.392148+0000", 2024-09-18T04:56:32.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3494000000000002e-05 2024-09-18T04:56:32.284 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.284 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.284 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.284 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:868 3.11 e24/19 3:888e89d2:::rbd_data.11ac33624de9.000000000000000b:head v 24'7, mlcod=23'3)", 2024-09-18T04:56:32.284 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.387746+0000", 2024-09-18T04:56:32.284 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8664986629999998, 2024-09-18T04:56:32.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0043760630000000003, 2024-09-18T04:56:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.387746+0000", 2024-09-18T04:56:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.387746+0000", 2024-09-18T04:56:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.387746+0000", 2024-09-18T04:56:32.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.387750+0000", 2024-09-18T04:56:32.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7419999999999997e-06 2024-09-18T04:56:32.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.387754+0000", 2024-09-18T04:56:32.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1169999999999997e-06 2024-09-18T04:56:32.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.387758+0000", 2024-09-18T04:56:32.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5439999999999999e-06 2024-09-18T04:56:32.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.387795+0000", 2024-09-18T04:56:32.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7239e-05 2024-09-18T04:56:32.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.387815+0000", 2024-09-18T04:56:32.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9381e-05 2024-09-18T04:56:32.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:32.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.392095+0000", 2024-09-18T04:56:32.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0042801430000000001 2024-09-18T04:56:32.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.392122+0000", 2024-09-18T04:56:32.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7317e-05 2024-09-18T04:56:32.288 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.288 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.288 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.289 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:926 3.f e24/19 3:f33a2580:::rbd_data.11ac33624de9.0000000000000006:head v 24'40, mlcod=24'38)", 2024-09-18T04:56:32.289 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.461527+0000", 2024-09-18T04:56:32.289 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.7927173339999998, 2024-09-18T04:56:32.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037370049999999998, 2024-09-18T04:56:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461527+0000", 2024-09-18T04:56:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461527+0000", 2024-09-18T04:56:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461526+0000", 2024-09-18T04:56:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461533+0000", 2024-09-18T04:56:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7070000000000003e-06 2024-09-18T04:56:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461548+0000", 2024-09-18T04:56:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4728e-05 2024-09-18T04:56:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461556+0000", 2024-09-18T04:56:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6230000000000003e-06 2024-09-18T04:56:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461742+0000", 2024-09-18T04:56:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018596699999999999 2024-09-18T04:56:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461763+0000", 2024-09-18T04:56:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1097000000000001e-05 2024-09-18T04:56:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:32.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.465226+0000", 2024-09-18T04:56:32.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0034625960000000001 2024-09-18T04:56:32.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.465264+0000", 2024-09-18T04:56:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8219000000000001e-05 2024-09-18T04:56:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:1014 3.11 e24/19 3:8b581e29:::rbd_data.11ac33624de9.0000000000000008:head v 24'13, mlcod=24'11)", 2024-09-18T04:56:32.297 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.562742+0000", 2024-09-18T04:56:32.297 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6915028430000003, 2024-09-18T04:56:32.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00567346, 2024-09-18T04:56:32.297 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.297 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:32.297 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562742+0000", 2024-09-18T04:56:32.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562742+0000", 2024-09-18T04:56:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562741+0000", 2024-09-18T04:56:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562748+0000", 2024-09-18T04:56:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.409e-06 2024-09-18T04:56:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562761+0000", 2024-09-18T04:56:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3406e-05 2024-09-18T04:56:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562768+0000", 2024-09-18T04:56:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1489999999999997e-06 2024-09-18T04:56:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562817+0000", 2024-09-18T04:56:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8714000000000001e-05 2024-09-18T04:56:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.562837+0000", 2024-09-18T04:56:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0352000000000001e-05 2024-09-18T04:56:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.568385+0000", 2024-09-18T04:56:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0055482960000000003 2024-09-18T04:56:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.568415+0000", 2024-09-18T04:56:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9838000000000001e-05 2024-09-18T04:56:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:1017 3.11 e24/19 3:888e89d2:::rbd_data.11ac33624de9.000000000000000b:head v 24'14, mlcod=24'11)", 2024-09-18T04:56:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.563208+0000", 2024-09-18T04:56:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.6910366769999996, 2024-09-18T04:56:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0052333680000000004, 2024-09-18T04:56:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.563208+0000", 2024-09-18T04:56:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.563208+0000", 2024-09-18T04:56:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.563207+0000", 2024-09-18T04:56:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.563214+0000", 2024-09-18T04:56:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1920000000000002e-06 2024-09-18T04:56:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.563223+0000", 2024-09-18T04:56:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7229999999999999e-06 2024-09-18T04:56:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.563229+0000", 2024-09-18T04:56:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7060000000000001e-06 2024-09-18T04:56:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.563261+0000", 2024-09-18T04:56:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2094000000000001e-05 2024-09-18T04:56:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.563278+0000", 2024-09-18T04:56:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7442e-05 2024-09-18T04:56:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.568418+0000", 2024-09-18T04:56:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0051392670000000003 2024-09-18T04:56:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.568441+0000", 2024-09-18T04:56:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3570999999999998e-05 2024-09-18T04:56:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:1339 3.3 3:c3fae2b0:::rbd_data.11ac99060bb1.0000000000000029:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.212068+0000", 2024-09-18T04:56:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0421767920000002, 2024-09-18T04:56:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0096840729999999996, 2024-09-18T04:56:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1339 2024-09-18T04:56:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.212068+0000", 2024-09-18T04:56:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.212068+0000", 2024-09-18T04:56:32.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.212066+0000", 2024-09-18T04:56:32.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:32.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.212080+0000", 2024-09-18T04:56:32.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3875e-05 2024-09-18T04:56:32.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.212094+0000", 2024-09-18T04:56:32.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3474e-05 2024-09-18T04:56:32.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.212100+0000", 2024-09-18T04:56:32.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2809999999999997e-06 2024-09-18T04:56:32.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.212147+0000", 2024-09-18T04:56:32.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7441999999999997e-05 2024-09-18T04:56:32.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.212284+0000", 2024-09-18T04:56:32.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013625700000000001 2024-09-18T04:56:32.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:32.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.213890+0000", 2024-09-18T04:56:32.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016066660000000001 2024-09-18T04:56:32.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:32.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.221662+0000", 2024-09-18T04:56:32.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0077715750000000002 2024-09-18T04:56:32.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:32.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.221671+0000", 2024-09-18T04:56:32.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0180000000000007e-06 2024-09-18T04:56:32.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:32.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.221709+0000", 2024-09-18T04:56:32.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8380000000000002e-05 2024-09-18T04:56:32.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.221752+0000", 2024-09-18T04:56:32.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2697999999999997e-05 2024-09-18T04:56:32.317 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.317 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.317 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.317 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop_reply(client.4524.0:1339 3.3 e25/19 ondisk, result = 0)", 2024-09-18T04:56:32.317 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.217671+0000", 2024-09-18T04:56:32.317 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0365732889999997, 2024-09-18T04:56:32.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039927449999999998, 2024-09-18T04:56:32.317 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.318 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:32.318 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.217671+0000", 2024-09-18T04:56:32.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.217671+0000", 2024-09-18T04:56:32.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.217670+0000", 2024-09-18T04:56:32.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:32.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.217676+0000", 2024-09-18T04:56:32.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8699999999999997e-06 2024-09-18T04:56:32.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.217690+0000", 2024-09-18T04:56:32.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3591000000000001e-05 2024-09-18T04:56:32.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.217698+0000", 2024-09-18T04:56:32.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4670000000000004e-06 2024-09-18T04:56:32.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.221649+0000", 2024-09-18T04:56:32.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039510559999999997 2024-09-18T04:56:32.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.221658+0000", 2024-09-18T04:56:32.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3990000000000004e-06 2024-09-18T04:56:32.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.221664+0000", 2024-09-18T04:56:32.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5910000000000001e-06 2024-09-18T04:56:32.322 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.322 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.322 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.322 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:1487 3.f e25/19 3:f43f8cc3:::rbd_data.11ac99060bb1.00000000000000cf:head v 25'47, mlcod=25'45)", 2024-09-18T04:56:32.322 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.256403+0000", 2024-09-18T04:56:32.322 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.9978418410000001, 2024-09-18T04:56:32.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0038258939999999998, 2024-09-18T04:56:32.322 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.322 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:32.323 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.256403+0000", 2024-09-18T04:56:32.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.256403+0000", 2024-09-18T04:56:32.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.256402+0000", 2024-09-18T04:56:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.256408+0000", 2024-09-18T04:56:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3230000000000003e-06 2024-09-18T04:56:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.256418+0000", 2024-09-18T04:56:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7399999999999999e-06 2024-09-18T04:56:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.256425+0000", 2024-09-18T04:56:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0620000000000003e-06 2024-09-18T04:56:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.256480+0000", 2024-09-18T04:56:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5197e-05 2024-09-18T04:56:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.256512+0000", 2024-09-18T04:56:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1133000000000002e-05 2024-09-18T04:56:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.260205+0000", 2024-09-18T04:56:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003693034 2024-09-18T04:56:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.260229+0000", 2024-09-18T04:56:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4141000000000001e-05 2024-09-18T04:56:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:2090 3.f e25/19 3:f1bdb4ea:::rbd_data.11ac99060bb1.000000000000006e:head v 25'52, mlcod=25'50)", 2024-09-18T04:56:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.658965+0000", 2024-09-18T04:56:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.595279541, 2024-09-18T04:56:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0060974200000000001, 2024-09-18T04:56:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.658965+0000", 2024-09-18T04:56:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.658965+0000", 2024-09-18T04:56:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.658965+0000", 2024-09-18T04:56:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.658971+0000", 2024-09-18T04:56:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2570000000000004e-06 2024-09-18T04:56:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.658978+0000", 2024-09-18T04:56:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1330000000000001e-06 2024-09-18T04:56:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.658983+0000", 2024-09-18T04:56:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6410000000000001e-06 2024-09-18T04:56:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.659272+0000", 2024-09-18T04:56:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000289356 2024-09-18T04:56:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.659290+0000", 2024-09-18T04:56:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8216e-05 2024-09-18T04:56:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.665036+0000", 2024-09-18T04:56:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0057457840000000003 2024-09-18T04:56:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.665063+0000", 2024-09-18T04:56:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6427e-05 2024-09-18T04:56:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:2096 3.f e25/19 3:f1946dbd:::rbd_data.11ac99060bb1.000000000000029e:head v 25'53, mlcod=25'50)", 2024-09-18T04:56:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.659512+0000", 2024-09-18T04:56:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.5947329099999998, 2024-09-18T04:56:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0055654550000000004, 2024-09-18T04:56:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.659512+0000", 2024-09-18T04:56:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.659512+0000", 2024-09-18T04:56:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.659511+0000", 2024-09-18T04:56:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.659517+0000", 2024-09-18T04:56:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4250000000000003e-06 2024-09-18T04:56:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.659526+0000", 2024-09-18T04:56:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9060000000000008e-06 2024-09-18T04:56:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.659532+0000", 2024-09-18T04:56:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0090000000000004e-06 2024-09-18T04:56:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.659559+0000", 2024-09-18T04:56:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7078999999999999e-05 2024-09-18T04:56:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.659577+0000", 2024-09-18T04:56:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7527999999999999e-05 2024-09-18T04:56:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.665065+0000", 2024-09-18T04:56:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0054880079999999999 2024-09-18T04:56:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.665077+0000", 2024-09-18T04:56:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2191999999999999e-05 2024-09-18T04:56:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:2097 3.f e25/19 3:f2ca76b4:::rbd_data.11ac99060bb1.00000000000002a2:head v 25'54, mlcod=25'50)", 2024-09-18T04:56:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.660127+0000", 2024-09-18T04:56:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.59411719, 2024-09-18T04:56:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0049596120000000004, 2024-09-18T04:56:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.660127+0000", 2024-09-18T04:56:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.660127+0000", 2024-09-18T04:56:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.660127+0000", 2024-09-18T04:56:32.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.660132+0000", 2024-09-18T04:56:32.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5960000000000003e-06 2024-09-18T04:56:32.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.660144+0000", 2024-09-18T04:56:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1968e-05 2024-09-18T04:56:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.660150+0000", 2024-09-18T04:56:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0249999999999999e-06 2024-09-18T04:56:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.660185+0000", 2024-09-18T04:56:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4400000000000003e-05 2024-09-18T04:56:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.660204+0000", 2024-09-18T04:56:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8902999999999998e-05 2024-09-18T04:56:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.665078+0000", 2024-09-18T04:56:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0048742739999999996 2024-09-18T04:56:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.665087+0000", 2024-09-18T04:56:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0410000000000003e-06 2024-09-18T04:56:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:2129 3.1c e25/19 3:39a75482:::rbd_data.11ac99060bb1.0000000000000116:head v 25'12, mlcod=25'10)", 2024-09-18T04:56:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.706367+0000", 2024-09-18T04:56:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.5478777770000001, 2024-09-18T04:56:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047833540000000001, 2024-09-18T04:56:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.706367+0000", 2024-09-18T04:56:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.706367+0000", 2024-09-18T04:56:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.706366+0000", 2024-09-18T04:56:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.706376+0000", 2024-09-18T04:56:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0098999999999999e-05 2024-09-18T04:56:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.706386+0000", 2024-09-18T04:56:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3999999999999998e-06 2024-09-18T04:56:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.706392+0000", 2024-09-18T04:56:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8980000000000001e-06 2024-09-18T04:56:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.706431+0000", 2024-09-18T04:56:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9032000000000002e-05 2024-09-18T04:56:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.706445+0000", 2024-09-18T04:56:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4324999999999999e-05 2024-09-18T04:56:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:32.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.711122+0000", 2024-09-18T04:56:32.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0046767249999999996 2024-09-18T04:56:32.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.711150+0000", 2024-09-18T04:56:32.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8515000000000001e-05 2024-09-18T04:56:32.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.345 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:4368 3.1 e28/19 3:80d63125:::rbd_data.11ac7d9658bc.0000000000000014:head v 28'23, mlcod=27'20)", 2024-09-18T04:56:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:31.233705+0000", 2024-09-18T04:56:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.0205396499999999, 2024-09-18T04:56:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0046426310000000004, 2024-09-18T04:56:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.233705+0000", 2024-09-18T04:56:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.233705+0000", 2024-09-18T04:56:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.233705+0000", 2024-09-18T04:56:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.233710+0000", 2024-09-18T04:56:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7910000000000004e-06 2024-09-18T04:56:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.233720+0000", 2024-09-18T04:56:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0258000000000001e-05 2024-09-18T04:56:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.233726+0000", 2024-09-18T04:56:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.37e-06 2024-09-18T04:56:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.233760+0000", 2024-09-18T04:56:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3560999999999999e-05 2024-09-18T04:56:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.233774+0000", 2024-09-18T04:56:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3847e-05 2024-09-18T04:56:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.238319+0000", 2024-09-18T04:56:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0045455499999999998 2024-09-18T04:56:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.238348+0000", 2024-09-18T04:56:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8487999999999999e-05 2024-09-18T04:56:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:4376 3.14 3:2b3ed61b:::rbd_object_map.11ac7d9658bc:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T04:56:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:31.239961+0000", 2024-09-18T04:56:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.0142832509999999, 2024-09-18T04:56:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0041105289999999999, 2024-09-18T04:56:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4376 2024-09-18T04:56:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.239961+0000", 2024-09-18T04:56:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.239961+0000", 2024-09-18T04:56:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.239961+0000", 2024-09-18T04:56:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.239964+0000", 2024-09-18T04:56:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.982e-06 2024-09-18T04:56:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.239970+0000", 2024-09-18T04:56:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9159999999999998e-06 2024-09-18T04:56:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.239973+0000", 2024-09-18T04:56:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7389999999999999e-06 2024-09-18T04:56:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.241094+0000", 2024-09-18T04:56:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001121227 2024-09-18T04:56:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.241147+0000", 2024-09-18T04:56:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3377000000000001e-05 2024-09-18T04:56:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 7", 2024-09-18T04:56:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.241408+0000", 2024-09-18T04:56:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000260855 2024-09-18T04:56:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242024+0000", 2024-09-18T04:56:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00061582 2024-09-18T04:56:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.244024+0000", 2024-09-18T04:56:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020000840000000001 2024-09-18T04:56:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.244053+0000", 2024-09-18T04:56:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8887e-05 2024-09-18T04:56:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.244072+0000", 2024-09-18T04:56:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9143999999999998e-05 2024-09-18T04:56:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:4378 3.a e28/19 3:52eeb3ed:::rbd_data.11ac7d9658bc.0000000000000001:head v 28'28, mlcod=28'25)", 2024-09-18T04:56:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:31.242403+0000", 2024-09-18T04:56:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.01184162, 2024-09-18T04:56:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0051746179999999998, 2024-09-18T04:56:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242403+0000", 2024-09-18T04:56:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242403+0000", 2024-09-18T04:56:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242403+0000", 2024-09-18T04:56:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242410+0000", 2024-09-18T04:56:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7349999999999996e-06 2024-09-18T04:56:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242418+0000", 2024-09-18T04:56:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6590000000000007e-06 2024-09-18T04:56:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242422+0000", 2024-09-18T04:56:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9380000000000002e-06 2024-09-18T04:56:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242462+0000", 2024-09-18T04:56:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0225000000000002e-05 2024-09-18T04:56:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242485+0000", 2024-09-18T04:56:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3067000000000001e-05 2024-09-18T04:56:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.247550+0000", 2024-09-18T04:56:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0050648159999999998 2024-09-18T04:56:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.247578+0000", 2024-09-18T04:56:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7676999999999999e-05 2024-09-18T04:56:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4524.0:4752 3.a e29/19 3:55916b62:::rbd_object_map.11aca1181477:head v 29'40, mlcod=29'33)", 2024-09-18T04:56:32.363 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.243955+0000", 2024-09-18T04:56:32.363 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.010289672999999999, 2024-09-18T04:56:32.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0053597929999999998, 2024-09-18T04:56:32.363 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.363 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:32.363 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.243955+0000", 2024-09-18T04:56:32.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.243955+0000", 2024-09-18T04:56:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.243954+0000", 2024-09-18T04:56:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.243970+0000", 2024-09-18T04:56:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5259000000000001e-05 2024-09-18T04:56:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.243980+0000", 2024-09-18T04:56:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0859999999999999e-05 2024-09-18T04:56:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.243987+0000", 2024-09-18T04:56:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7320000000000001e-06 2024-09-18T04:56:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.244029+0000", 2024-09-18T04:56:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1462999999999999e-05 2024-09-18T04:56:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.244047+0000", 2024-09-18T04:56:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8374999999999999e-05 2024-09-18T04:56:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.249286+0000", 2024-09-18T04:56:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0052391829999999997 2024-09-18T04:56:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.249315+0000", 2024-09-18T04:56:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8577000000000001e-05 2024-09-18T04:56:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.367 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:32.369 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:56:32.369 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:32.365+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:32.380 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-18T04:56:32.396 INFO:teuthology.orchestra.run.smithi002.stdout:566 write 0xbabf540 thru 0xbacca91 (0xd552 bytes) 2024-09-18T04:56:32.403 INFO:teuthology.orchestra.run.smithi002.stdout:567 read 0x2d9f6d1 thru 0x2da9f22 (0xa852 bytes) 2024-09-18T04:56:32.404 INFO:teuthology.orchestra.run.smithi002.stdout:568 read 0x3585c7c thru 0x358699d (0xd22 bytes) 2024-09-18T04:56:32.404 INFO:teuthology.orchestra.run.smithi002.stdout:569 write 0x8f3e2bc thru 0x8f4a5a1 (0xc2e6 bytes) 2024-09-18T04:56:32.417 INFO:teuthology.orchestra.run.smithi002.stdout:570 write 0xc449d9f thru 0xc44d3e0 (0x3642 bytes) 2024-09-18T04:56:32.421 INFO:teuthology.orchestra.run.smithi002.stdout:572 punch from 0x7388e31 to 0x738ad73, (0x1f42 bytes) 2024-09-18T04:56:32.421 INFO:teuthology.orchestra.run.smithi002.stdout:573 read 0xbbb6ffc thru 0xbbc3df7 (0xcdfc bytes) 2024-09-18T04:56:32.424 INFO:teuthology.orchestra.run.smithi002.stdout:574 write 0xd2def3a thru 0xd2e005d (0x1124 bytes) 2024-09-18T04:56:32.431 INFO:teuthology.orchestra.run.smithi002.stdout:575 write 0xa8744f9 thru 0xa874527 (0x2f bytes) 2024-09-18T04:56:32.439 INFO:teuthology.orchestra.run.smithi002.stdout:577 writesame 0xd98e4aa thru 0xd993c9b data_size 0x1(0x57f2 bytes) 2024-09-18T04:56:32.470 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:32.467+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:32.485 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:32.485 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:32.485 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:56:32.486 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:32.494 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:56:32.495 INFO:teuthology.orchestra.run.smithi002.stdout:578 punch from 0x47485b8 to 0x4755e23, (0xd86b bytes) 2024-09-18T04:56:32.495 INFO:teuthology.orchestra.run.smithi002.stdout:581 read 0xb71b0dc thru 0xb71be26 (0xd4b bytes) 2024-09-18T04:56:32.495 INFO:teuthology.orchestra.run.smithi002.stdout:582 writesame 0x94d9dc5 thru 0x94d9e2a data_size 0x3(0x66 bytes) 2024-09-18T04:56:32.501 INFO:teuthology.orchestra.run.smithi002.stdout:584 write 0x5c5c280 thru 0x5c6616a (0x9eeb bytes) 2024-09-18T04:56:32.534 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:32.534 INFO:teuthology.orchestra.run.smithi002.stdout:587 read 0xcd48a0b thru 0xcd55394 (0xc98a bytes) 2024-09-18T04:56:32.534 INFO:teuthology.orchestra.run.smithi002.stdout:588 punch from 0x8aa327d to 0x8ab1e03, (0xeb86 bytes) 2024-09-18T04:56:32.534 INFO:teuthology.orchestra.run.smithi002.stdout:590 punch from 0xab22bb to 0xab8e7a, (0x6bbf bytes) 2024-09-18T04:56:32.535 INFO:teuthology.orchestra.run.smithi002.stdout:592 write 0x9a86207 thru 0x9a95858 (0xf652 bytes) 2024-09-18T04:56:32.569 INFO:teuthology.orchestra.run.smithi002.stdout:593 writesame 0x53590a1 thru 0x5360dff data_size 0x31(0x7d5f bytes) 2024-09-18T04:56:32.570 INFO:teuthology.orchestra.run.smithi002.stdout:594 writesame 0x8fadb36 thru 0x8fb684c data_size 0x1(0x8d17 bytes) 2024-09-18T04:56:32.578 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:32.576+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:32.590 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:32.590 INFO:teuthology.orchestra.run.smithi002.stdout:596 writesame 0x537bb11 thru 0x538702c data_size 0x4(0xb51c bytes) 2024-09-18T04:56:32.596 INFO:teuthology.orchestra.run.smithi002.stdout:597 write 0x263e638 thru 0x263f6d9 (0x10a2 bytes) 2024-09-18T04:56:32.598 INFO:teuthology.orchestra.run.smithi002.stdout:598 read 0x385bf26 thru 0x385e38f (0x246a bytes) 2024-09-18T04:56:32.598 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:32.610 INFO:teuthology.orchestra.run.smithi002.stdout:600 trunc from 0xd993c9c to 0x3099e2b 2024-09-18T04:56:32.648 INFO:teuthology.orchestra.run.smithi002.stdout:601 write 0xce0954b thru 0xce0a0d3 (0xb89 bytes) 2024-09-18T04:56:32.649 INFO:teuthology.orchestra.run.smithi002.stdout:602 punch from 0xc0e2520 to 0xc0e82aa, (0x5d8a bytes) 2024-09-18T04:56:32.649 INFO:teuthology.orchestra.run.smithi002.stdout:604 write 0x697caa thru 0x6a3e4c (0xc1a3 bytes) 2024-09-18T04:56:32.651 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:32.651 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:32.651 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:56:32.651 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:56:32.651 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:32.655 INFO:teuthology.orchestra.run.smithi002.stdout:605 write 0xd01c75e thru 0xd02598d (0x9230 bytes) 2024-09-18T04:56:32.660 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:56:32.670 INFO:teuthology.orchestra.run.smithi002.stdout:606 trunc from 0xd02598e to 0x987cd6 2024-09-18T04:56:32.685 INFO:teuthology.orchestra.run.smithi002.stdout:607 read 0x9404cc thru 0x940c79 (0x7ae bytes) 2024-09-18T04:56:32.685 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:32.685 INFO:teuthology.orchestra.run.smithi002.stdout:609 read 0x3b2c97 thru 0x3bfb3a (0xcea4 bytes) 2024-09-18T04:56:32.685 INFO:teuthology.orchestra.run.smithi002.stdout:610 punch from 0x634957 to 0x643339, (0xe9e2 bytes) 2024-09-18T04:56:32.685 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:32.685 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:32.686 INFO:teuthology.orchestra.run.smithi002.stdout:614 punch from 0x6ca631 to 0x6cec00, (0x45cf bytes) 2024-09-18T04:56:32.686 INFO:teuthology.orchestra.run.smithi002.stdout:615 read 0x3f4b40 thru 0x3f96fb (0x4bbc bytes) 2024-09-18T04:56:32.686 INFO:teuthology.orchestra.run.smithi002.stdout:617 read 0x70b08c thru 0x712ac5 (0x7a3a bytes) 2024-09-18T04:56:32.697 INFO:teuthology.orchestra.run.smithi002.stdout:618 writesame 0x3c1dc04 thru 0x3c29555 data_size 0x3(0xb952 bytes) 2024-09-18T04:56:32.698 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:32.698 INFO:teuthology.orchestra.run.smithi002.stdout:621 read 0x272907f thru 0x272d86c (0x47ee bytes) 2024-09-18T04:56:32.698 INFO:teuthology.orchestra.run.smithi002.stdout:622 punch from 0x156865f to 0x15735a4, (0xaf45 bytes) 2024-09-18T04:56:32.702 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:32.699+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:32.707 INFO:teuthology.orchestra.run.smithi002.stdout:623 writesame 0x5f8a1cd thru 0x5f99d27 data_size 0x3(0xfb5b bytes) 2024-09-18T04:56:32.712 INFO:teuthology.orchestra.run.smithi002.stdout:624 trunc from 0x5f99d28 to 0x3481eed 2024-09-18T04:56:32.721 INFO:teuthology.orchestra.run.smithi002.stdout:625 trunc from 0x3481eed to 0x55b755e 2024-09-18T04:56:32.737 INFO:teuthology.orchestra.run.smithi002.stdout:626 write 0x7b80400 thru 0x7b83b57 (0x3758 bytes) 2024-09-18T04:56:32.740 INFO:teuthology.orchestra.run.smithi002.stdout:627 punch from 0x1b34c1e to 0x1b4067d, (0xba5f bytes) 2024-09-18T04:56:32.740 INFO:teuthology.orchestra.run.smithi002.stdout:628 trunc from 0x7b83b58 to 0x773de2c 2024-09-18T04:56:32.758 INFO:teuthology.orchestra.run.smithi002.stdout:629 write 0xb402ccf thru 0xb405220 (0x2552 bytes) 2024-09-18T04:56:32.760 INFO:teuthology.orchestra.run.smithi002.stdout:630 read 0x987740a thru 0x9880ad4 (0x96cb bytes) 2024-09-18T04:56:32.760 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:32.760 INFO:teuthology.orchestra.run.smithi002.stdout:634 write 0xdaad02 thru 0xdafab1 (0x4db0 bytes) 2024-09-18T04:56:32.764 INFO:teuthology.orchestra.run.smithi002.stdout:635 read 0x22cbdcd thru 0x22d9fda (0xe20e bytes) 2024-09-18T04:56:32.772 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:32.770+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:32.774 INFO:teuthology.orchestra.run.smithi002.stdout:636 write 0xe179de5 thru 0xe1839d6 (0x9bf2 bytes) 2024-09-18T04:56:32.776 INFO:teuthology.orchestra.run.smithi002.stdout:638 write 0x5526230 thru 0x552674f (0x520 bytes) 2024-09-18T04:56:32.783 INFO:teuthology.orchestra.run.smithi002.stdout:639 trunc from 0xe1839d7 to 0x848eb04 2024-09-18T04:56:32.792 INFO:teuthology.orchestra.run.smithi002.stdout:641 read 0x834b1d thru 0x836e48 (0x232c bytes) 2024-09-18T04:56:32.799 INFO:teuthology.orchestra.run.smithi002.stdout:642 trunc from 0x848eb04 to 0x1f5102e 2024-09-18T04:56:32.809 INFO:teuthology.orchestra.run.smithi002.stdout:643 read 0x1941988 thru 0x1944fbb (0x3634 bytes) 2024-09-18T04:56:32.810 INFO:teuthology.orchestra.run.smithi002.stdout:644 read 0x19983c5 thru 0x19a40cd (0xbd09 bytes) 2024-09-18T04:56:32.810 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:32.810 INFO:teuthology.orchestra.run.smithi002.stdout:646 punch from 0x1c757f0 to 0x1c81945, (0xc155 bytes) 2024-09-18T04:56:32.813 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:32.814 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:56:32.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:56:32.814 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:56:32.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.814 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:1966 3.1d 3:bc541e89:::rbd_data.11ac33624de9.0000000000000000:6 [read 1671168~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:32.815 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.545051+0000", 2024-09-18T04:56:32.815 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.2641750920000003, 2024-09-18T04:56:32.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0073151220000000003, 2024-09-18T04:56:32.815 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.815 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:32.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:32.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:32.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:32.816 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1966 2024-09-18T04:56:32.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.817 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545051+0000", 2024-09-18T04:56:32.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545051+0000", 2024-09-18T04:56:32.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545050+0000", 2024-09-18T04:56:32.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:32.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545052+0000", 2024-09-18T04:56:32.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.11e-06 2024-09-18T04:56:32.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545055+0000", 2024-09-18T04:56:32.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.737e-06 2024-09-18T04:56:32.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545060+0000", 2024-09-18T04:56:32.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2810000000000002e-06 2024-09-18T04:56:32.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.552207+0000", 2024-09-18T04:56:32.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0071474700000000004 2024-09-18T04:56:32.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.552285+0000", 2024-09-18T04:56:32.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8319999999999996e-05 2024-09-18T04:56:32.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.552366+0000", 2024-09-18T04:56:32.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0407000000000005e-05 2024-09-18T04:56:32.830 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.830 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.830 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.831 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:1971 3.1d 3:bc541e89:::rbd_data.11ac33624de9.0000000000000000:6 [read 327680~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:32.831 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.545068+0000", 2024-09-18T04:56:32.832 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.2641576880000001, 2024-09-18T04:56:32.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0075104760000000003, 2024-09-18T04:56:32.832 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.833 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:32.833 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:32.833 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:32.833 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:32.834 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1971 2024-09-18T04:56:32.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.834 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545068+0000", 2024-09-18T04:56:32.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545068+0000", 2024-09-18T04:56:32.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545068+0000", 2024-09-18T04:56:32.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545070+0000", 2024-09-18T04:56:32.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.278e-06 2024-09-18T04:56:32.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545078+0000", 2024-09-18T04:56:32.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1140000000000006e-06 2024-09-18T04:56:32.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545083+0000", 2024-09-18T04:56:32.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.69e-06 2024-09-18T04:56:32.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.552402+0000", 2024-09-18T04:56:32.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0073194530000000001 2024-09-18T04:56:32.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.552486+0000", 2024-09-18T04:56:32.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4232000000000003e-05 2024-09-18T04:56:32.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.552578+0000", 2024-09-18T04:56:32.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2076000000000004e-05 2024-09-18T04:56:32.841 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.841 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.841 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.841 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:1972 3.1d 3:bc541e89:::rbd_data.11ac33624de9.0000000000000000:6 [read 786432~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:32.841 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.545137+0000", 2024-09-18T04:56:32.841 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.2640889289999997, 2024-09-18T04:56:32.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0076299109999999996, 2024-09-18T04:56:32.841 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.841 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:32.841 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:32.842 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:32.842 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:32.842 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1972 2024-09-18T04:56:32.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.842 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545137+0000", 2024-09-18T04:56:32.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545137+0000", 2024-09-18T04:56:32.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545136+0000", 2024-09-18T04:56:32.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545140+0000", 2024-09-18T04:56:32.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3100000000000002e-06 2024-09-18T04:56:32.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545152+0000", 2024-09-18T04:56:32.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1440999999999999e-05 2024-09-18T04:56:32.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545160+0000", 2024-09-18T04:56:32.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7479999999999993e-06 2024-09-18T04:56:32.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.552605+0000", 2024-09-18T04:56:32.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0074449249999999998 2024-09-18T04:56:32.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.552681+0000", 2024-09-18T04:56:32.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6500000000000003e-05 2024-09-18T04:56:32.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.552767+0000", 2024-09-18T04:56:32.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5686999999999995e-05 2024-09-18T04:56:32.846 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.846 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.846 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.846 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:1973 3.1d 3:bc541e89:::rbd_data.11ac33624de9.0000000000000000:6 [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:32.846 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.545168+0000", 2024-09-18T04:56:32.846 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.264057437, 2024-09-18T04:56:32.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0077976089999999996, 2024-09-18T04:56:32.847 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.847 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:32.847 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:32.847 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:32.847 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:32.847 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1973 2024-09-18T04:56:32.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.847 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545168+0000", 2024-09-18T04:56:32.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545168+0000", 2024-09-18T04:56:32.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545168+0000", 2024-09-18T04:56:32.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545172+0000", 2024-09-18T04:56:32.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1760000000000003e-06 2024-09-18T04:56:32.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545179+0000", 2024-09-18T04:56:32.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2359999999999998e-06 2024-09-18T04:56:32.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545185+0000", 2024-09-18T04:56:32.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6890000000000001e-06 2024-09-18T04:56:32.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.552793+0000", 2024-09-18T04:56:32.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0076078889999999996 2024-09-18T04:56:32.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.552869+0000", 2024-09-18T04:56:32.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6749999999999995e-05 2024-09-18T04:56:32.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.552966+0000", 2024-09-18T04:56:32.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6403000000000005e-05 2024-09-18T04:56:32.850 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.851 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.851 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.851 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:1974 3.1d 3:bc541e89:::rbd_data.11ac33624de9.0000000000000000:6 [read 1703936~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:32.851 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.545190+0000", 2024-09-18T04:56:32.851 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.2640355489999999, 2024-09-18T04:56:32.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.007943294, 2024-09-18T04:56:32.851 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.851 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:32.851 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:32.851 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:32.851 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:32.852 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1974 2024-09-18T04:56:32.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.852 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545190+0000", 2024-09-18T04:56:32.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545190+0000", 2024-09-18T04:56:32.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545190+0000", 2024-09-18T04:56:32.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545192+0000", 2024-09-18T04:56:32.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3949999999999999e-06 2024-09-18T04:56:32.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545200+0000", 2024-09-18T04:56:32.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.225e-06 2024-09-18T04:56:32.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545206+0000", 2024-09-18T04:56:32.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1440000000000002e-06 2024-09-18T04:56:32.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.552999+0000", 2024-09-18T04:56:32.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0077933780000000001 2024-09-18T04:56:32.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.553061+0000", 2024-09-18T04:56:32.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2440000000000005e-05 2024-09-18T04:56:32.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.553133+0000", 2024-09-18T04:56:32.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2108999999999997e-05 2024-09-18T04:56:32.855 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.855 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.855 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.855 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:1979 3.1d 3:bc541e89:::rbd_data.11ac33624de9.0000000000000000:6 [read 360448~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:32.855 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.545210+0000", 2024-09-18T04:56:32.856 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.2640153649999997, 2024-09-18T04:56:32.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0081485619999999998, 2024-09-18T04:56:32.856 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.856 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:32.856 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:32.856 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:32.856 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:32.856 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1979 2024-09-18T04:56:32.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.856 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545210+0000", 2024-09-18T04:56:32.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545210+0000", 2024-09-18T04:56:32.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545210+0000", 2024-09-18T04:56:32.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545212+0000", 2024-09-18T04:56:32.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1679999999999998e-06 2024-09-18T04:56:32.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545219+0000", 2024-09-18T04:56:32.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4350000000000002e-06 2024-09-18T04:56:32.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545224+0000", 2024-09-18T04:56:32.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1789999999999999e-06 2024-09-18T04:56:32.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.553176+0000", 2024-09-18T04:56:32.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0079525499999999992 2024-09-18T04:56:32.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.553260+0000", 2024-09-18T04:56:32.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3639000000000003e-05 2024-09-18T04:56:32.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.553359+0000", 2024-09-18T04:56:32.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8914999999999997e-05 2024-09-18T04:56:32.860 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.860 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.860 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.860 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:1980 3.1d 3:bc541e89:::rbd_data.11ac33624de9.0000000000000000:6 [read 819200~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:32.861 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.545228+0000", 2024-09-18T04:56:32.861 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.2639973170000003, 2024-09-18T04:56:32.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.008392811, 2024-09-18T04:56:32.861 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.861 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:32.861 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:32.861 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:32.861 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:32.861 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1980 2024-09-18T04:56:32.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.861 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545228+0000", 2024-09-18T04:56:32.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545228+0000", 2024-09-18T04:56:32.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545228+0000", 2024-09-18T04:56:32.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545230+0000", 2024-09-18T04:56:32.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0099999999999998e-06 2024-09-18T04:56:32.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545236+0000", 2024-09-18T04:56:32.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3010000000000003e-06 2024-09-18T04:56:32.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545241+0000", 2024-09-18T04:56:32.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4909999999999999e-06 2024-09-18T04:56:32.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.553423+0000", 2024-09-18T04:56:32.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.008181799 2024-09-18T04:56:32.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.553520+0000", 2024-09-18T04:56:32.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7533000000000003e-05 2024-09-18T04:56:32.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.553621+0000", 2024-09-18T04:56:32.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101001 2024-09-18T04:56:32.865 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.865 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.865 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.865 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:1981 3.1d 3:bc541e89:::rbd_data.11ac33624de9.0000000000000000:6 [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:32.865 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.545246+0000", 2024-09-18T04:56:32.865 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.2639800479999996, 2024-09-18T04:56:32.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0084969759999999998, 2024-09-18T04:56:32.865 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.865 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:32.865 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:32.866 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:32.866 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:32.866 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1981 2024-09-18T04:56:32.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.866 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545246+0000", 2024-09-18T04:56:32.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545246+0000", 2024-09-18T04:56:32.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545245+0000", 2024-09-18T04:56:32.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545247+0000", 2024-09-18T04:56:32.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0990000000000001e-06 2024-09-18T04:56:32.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545261+0000", 2024-09-18T04:56:32.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3322000000000001e-05 2024-09-18T04:56:32.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545266+0000", 2024-09-18T04:56:32.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2259999999999996e-06 2024-09-18T04:56:32.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.553645+0000", 2024-09-18T04:56:32.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0083786980000000004 2024-09-18T04:56:32.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.553689+0000", 2024-09-18T04:56:32.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4345000000000003e-05 2024-09-18T04:56:32.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.553743+0000", 2024-09-18T04:56:32.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3659000000000002e-05 2024-09-18T04:56:32.870 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.870 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.870 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.870 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:1982 3.1d 3:bc541e89:::rbd_data.11ac33624de9.0000000000000000:6 [read 1736704~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:32.870 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.545273+0000", 2024-09-18T04:56:32.870 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.2639530519999997, 2024-09-18T04:56:32.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0086472489999999992, 2024-09-18T04:56:32.870 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.870 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:32.870 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:32.871 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:32.871 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:32.871 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1982 2024-09-18T04:56:32.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.871 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545273+0000", 2024-09-18T04:56:32.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545273+0000", 2024-09-18T04:56:32.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545272+0000", 2024-09-18T04:56:32.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545275+0000", 2024-09-18T04:56:32.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4779999999999998e-06 2024-09-18T04:56:32.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545284+0000", 2024-09-18T04:56:32.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3919999999999992e-06 2024-09-18T04:56:32.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545289+0000", 2024-09-18T04:56:32.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0989999999999998e-06 2024-09-18T04:56:32.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.553784+0000", 2024-09-18T04:56:32.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0084949080000000007 2024-09-18T04:56:32.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.553858+0000", 2024-09-18T04:56:32.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4283999999999996e-05 2024-09-18T04:56:32.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.553920+0000", 2024-09-18T04:56:32.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1486000000000001e-05 2024-09-18T04:56:32.875 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.875 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.876 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.876 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:1987 3.1d 3:bc541e89:::rbd_data.11ac33624de9.0000000000000000:6 [read 393216~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:32.876 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.545302+0000", 2024-09-18T04:56:32.876 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.2639235229999999, 2024-09-18T04:56:32.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0087850389999999997, 2024-09-18T04:56:32.876 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.876 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:32.876 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:32.876 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:32.876 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:32.876 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1987 2024-09-18T04:56:32.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.877 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545302+0000", 2024-09-18T04:56:32.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545302+0000", 2024-09-18T04:56:32.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545302+0000", 2024-09-18T04:56:32.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545304+0000", 2024-09-18T04:56:32.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6170000000000001e-06 2024-09-18T04:56:32.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545312+0000", 2024-09-18T04:56:32.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2239999999999998e-06 2024-09-18T04:56:32.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545318+0000", 2024-09-18T04:56:32.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3559999999999998e-06 2024-09-18T04:56:32.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.553949+0000", 2024-09-18T04:56:32.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0086311280000000001 2024-09-18T04:56:32.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.554003+0000", 2024-09-18T04:56:32.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4484000000000002e-05 2024-09-18T04:56:32.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.554087+0000", 2024-09-18T04:56:32.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3727999999999995e-05 2024-09-18T04:56:32.880 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.880 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.880 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.880 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:1988 3.1d 3:bc541e89:::rbd_data.11ac33624de9.0000000000000000:6 [read 851968~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:32.880 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.545375+0000", 2024-09-18T04:56:32.881 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.2638504839999998, 2024-09-18T04:56:32.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0088537570000000003, 2024-09-18T04:56:32.881 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.881 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:32.881 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:32.881 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:32.881 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:32.881 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1988 2024-09-18T04:56:32.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.881 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545375+0000", 2024-09-18T04:56:32.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545375+0000", 2024-09-18T04:56:32.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545375+0000", 2024-09-18T04:56:32.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545379+0000", 2024-09-18T04:56:32.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5739999999999997e-06 2024-09-18T04:56:32.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545394+0000", 2024-09-18T04:56:32.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4616e-05 2024-09-18T04:56:32.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545402+0000", 2024-09-18T04:56:32.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1380000000000007e-06 2024-09-18T04:56:32.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.554115+0000", 2024-09-18T04:56:32.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00871305 2024-09-18T04:56:32.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.554170+0000", 2024-09-18T04:56:32.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5111999999999997e-05 2024-09-18T04:56:32.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.554229+0000", 2024-09-18T04:56:32.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8888000000000003e-05 2024-09-18T04:56:32.885 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.885 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.885 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.885 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:1989 3.1d 3:bc541e89:::rbd_data.11ac33624de9.0000000000000000:6 [read 1310720~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:32.885 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.545408+0000", 2024-09-18T04:56:32.885 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.2638181660000001, 2024-09-18T04:56:32.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0089682730000000006, 2024-09-18T04:56:32.885 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.885 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:32.885 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:32.885 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:32.885 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:32.886 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1989 2024-09-18T04:56:32.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.886 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545408+0000", 2024-09-18T04:56:32.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545408+0000", 2024-09-18T04:56:32.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545407+0000", 2024-09-18T04:56:32.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545411+0000", 2024-09-18T04:56:32.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4960000000000001e-06 2024-09-18T04:56:32.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545420+0000", 2024-09-18T04:56:32.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2399999999999996e-06 2024-09-18T04:56:32.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545425+0000", 2024-09-18T04:56:32.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9119999999999997e-06 2024-09-18T04:56:32.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.554255+0000", 2024-09-18T04:56:32.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0088299850000000003 2024-09-18T04:56:32.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.554305+0000", 2024-09-18T04:56:32.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0096000000000003e-05 2024-09-18T04:56:32.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.554376+0000", 2024-09-18T04:56:32.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0980000000000001e-05 2024-09-18T04:56:32.889 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.889 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.889 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.889 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:1990 3.1d 3:bc541e89:::rbd_data.11ac33624de9.0000000000000000:6 [read 1769472~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:32.889 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.545429+0000", 2024-09-18T04:56:32.890 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.2637965839999996, 2024-09-18T04:56:32.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0090762080000000005, 2024-09-18T04:56:32.890 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.890 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:32.890 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:32.890 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:32.890 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:32.890 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1990 2024-09-18T04:56:32.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.890 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545429+0000", 2024-09-18T04:56:32.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545429+0000", 2024-09-18T04:56:32.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545429+0000", 2024-09-18T04:56:32.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545431+0000", 2024-09-18T04:56:32.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9319999999999999e-06 2024-09-18T04:56:32.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545436+0000", 2024-09-18T04:56:32.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7089999999999999e-06 2024-09-18T04:56:32.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545441+0000", 2024-09-18T04:56:32.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6430000000000003e-06 2024-09-18T04:56:32.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.554402+0000", 2024-09-18T04:56:32.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.008960506 2024-09-18T04:56:32.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.554449+0000", 2024-09-18T04:56:32.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7620999999999997e-05 2024-09-18T04:56:32.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.554505+0000", 2024-09-18T04:56:32.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6119999999999998e-05 2024-09-18T04:56:32.894 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.894 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.894 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.894 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:1995 3.1d 3:bc541e89:::rbd_data.11ac33624de9.0000000000000000:6 [read 425984~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:32.895 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.545445+0000", 2024-09-18T04:56:32.895 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.2637803539999997, 2024-09-18T04:56:32.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.009228511, 2024-09-18T04:56:32.895 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.895 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:32.895 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:32.895 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:32.895 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:32.895 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1995 2024-09-18T04:56:32.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.895 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545445+0000", 2024-09-18T04:56:32.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545445+0000", 2024-09-18T04:56:32.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545445+0000", 2024-09-18T04:56:32.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545447+0000", 2024-09-18T04:56:32.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0030000000000001e-06 2024-09-18T04:56:32.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545453+0000", 2024-09-18T04:56:32.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8039999999999998e-06 2024-09-18T04:56:32.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545457+0000", 2024-09-18T04:56:32.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.42e-06 2024-09-18T04:56:32.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.554541+0000", 2024-09-18T04:56:32.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0090840420000000005 2024-09-18T04:56:32.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.554614+0000", 2024-09-18T04:56:32.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2696999999999999e-05 2024-09-18T04:56:32.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.554674+0000", 2024-09-18T04:56:32.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9935000000000001e-05 2024-09-18T04:56:32.899 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.899 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.899 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.899 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:1996 3.1d 3:bc541e89:::rbd_data.11ac33624de9.0000000000000000:6 [read 884736~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:32.899 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.545461+0000", 2024-09-18T04:56:32.899 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.2637644809999999, 2024-09-18T04:56:32.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0093838589999999996, 2024-09-18T04:56:32.899 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.899 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:32.899 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:32.899 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:32.900 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:32.900 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1996 2024-09-18T04:56:32.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.900 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545461+0000", 2024-09-18T04:56:32.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545461+0000", 2024-09-18T04:56:32.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545461+0000", 2024-09-18T04:56:32.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545463+0000", 2024-09-18T04:56:32.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8640000000000001e-06 2024-09-18T04:56:32.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545468+0000", 2024-09-18T04:56:32.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4979999999999997e-06 2024-09-18T04:56:32.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545472+0000", 2024-09-18T04:56:32.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.031e-06 2024-09-18T04:56:32.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.554703+0000", 2024-09-18T04:56:32.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0092303200000000002 2024-09-18T04:56:32.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.554760+0000", 2024-09-18T04:56:32.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7633999999999997e-05 2024-09-18T04:56:32.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.554845+0000", 2024-09-18T04:56:32.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4847999999999998e-05 2024-09-18T04:56:32.903 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.903 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.903 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.903 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:1997 3.1d 3:bc541e89:::rbd_data.11ac33624de9.0000000000000000:6 [read 1343488~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:32.903 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.545478+0000", 2024-09-18T04:56:32.904 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.2637480669999999, 2024-09-18T04:56:32.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0094827890000000001, 2024-09-18T04:56:32.904 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.904 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:32.904 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:32.904 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:32.904 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:32.904 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1997 2024-09-18T04:56:32.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.904 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545478+0000", 2024-09-18T04:56:32.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545478+0000", 2024-09-18T04:56:32.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545477+0000", 2024-09-18T04:56:32.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545480+0000", 2024-09-18T04:56:32.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8169999999999999e-06 2024-09-18T04:56:32.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545488+0000", 2024-09-18T04:56:32.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4930000000000001e-06 2024-09-18T04:56:32.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545492+0000", 2024-09-18T04:56:32.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5190000000000003e-06 2024-09-18T04:56:32.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.554864+0000", 2024-09-18T04:56:32.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0093718650000000001 2024-09-18T04:56:32.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.554905+0000", 2024-09-18T04:56:32.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0960000000000001e-05 2024-09-18T04:56:32.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.554960+0000", 2024-09-18T04:56:32.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5476000000000002e-05 2024-09-18T04:56:32.908 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.908 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.908 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.908 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:1998 3.1d 3:bc541e89:::rbd_data.11ac33624de9.0000000000000000:6 [read 1802240~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:32.908 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.545496+0000", 2024-09-18T04:56:32.908 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.2637296659999997, 2024-09-18T04:56:32.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0095937599999999998, 2024-09-18T04:56:32.909 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.909 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:32.909 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:32.909 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:32.909 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:32.909 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1998 2024-09-18T04:56:32.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.910 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545496+0000", 2024-09-18T04:56:32.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545496+0000", 2024-09-18T04:56:32.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545496+0000", 2024-09-18T04:56:32.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:32.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545498+0000", 2024-09-18T04:56:32.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9069999999999999e-06 2024-09-18T04:56:32.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545506+0000", 2024-09-18T04:56:32.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2900000000000002e-06 2024-09-18T04:56:32.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.545510+0000", 2024-09-18T04:56:32.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3599999999999998e-06 2024-09-18T04:56:32.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.554977+0000", 2024-09-18T04:56:32.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0094663009999999999 2024-09-18T04:56:32.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.555034+0000", 2024-09-18T04:56:32.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7117999999999998e-05 2024-09-18T04:56:32.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.555090+0000", 2024-09-18T04:56:32.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6101999999999999e-05 2024-09-18T04:56:32.913 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.913 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.913 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.913 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:2145 3.c 3:320d0be6:::rbd_data.11ac99060bb1.00000000000001a0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:32.913 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.745048+0000", 2024-09-18T04:56:32.913 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.064177484, 2024-09-18T04:56:32.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015933319000000001, 2024-09-18T04:56:32.914 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.914 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:32.914 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:32.914 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:32.914 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:32.914 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2145 2024-09-18T04:56:32.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.914 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.745048+0000", 2024-09-18T04:56:32.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.745048+0000", 2024-09-18T04:56:32.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.745047+0000", 2024-09-18T04:56:32.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:32.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.745055+0000", 2024-09-18T04:56:32.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4200000000000007e-06 2024-09-18T04:56:32.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.745072+0000", 2024-09-18T04:56:32.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6592e-05 2024-09-18T04:56:32.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.745087+0000", 2024-09-18T04:56:32.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4834000000000001e-05 2024-09-18T04:56:32.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.745135+0000", 2024-09-18T04:56:32.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8636e-05 2024-09-18T04:56:32.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.745274+0000", 2024-09-18T04:56:32.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013843500000000001 2024-09-18T04:56:32.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-18T04:56:32.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.748269+0000", 2024-09-18T04:56:32.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0029947860000000001 2024-09-18T04:56:32.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:32.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.760882+0000", 2024-09-18T04:56:32.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012613818000000001 2024-09-18T04:56:32.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:32.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.760895+0000", 2024-09-18T04:56:32.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2508e-05 2024-09-18T04:56:32.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:32.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.760941+0000", 2024-09-18T04:56:32.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6220000000000001e-05 2024-09-18T04:56:32.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.760982+0000", 2024-09-18T04:56:32.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0436999999999999e-05 2024-09-18T04:56:32.919 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.919 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.919 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.919 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:5206 3.1d 3:bc9b8049:::rbd_data.11ac14fc1d07.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 17326599~63058] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:32.919 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.539333+0000", 2024-09-18T04:56:32.919 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.26989266299999998, 2024-09-18T04:56:32.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.027944470999999999, 2024-09-18T04:56:32.920 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.920 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:32.920 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:32.920 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:32.920 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:32.920 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 5206 2024-09-18T04:56:32.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.920 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.539333+0000", 2024-09-18T04:56:32.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.539333+0000", 2024-09-18T04:56:32.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.539332+0000", 2024-09-18T04:56:32.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:32.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.551229+0000", 2024-09-18T04:56:32.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011896814 2024-09-18T04:56:32.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.551256+0000", 2024-09-18T04:56:32.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7208000000000001e-05 2024-09-18T04:56:32.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.551266+0000", 2024-09-18T04:56:32.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0051000000000001e-05 2024-09-18T04:56:32.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.551322+0000", 2024-09-18T04:56:32.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6317999999999998e-05 2024-09-18T04:56:32.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.551437+0000", 2024-09-18T04:56:32.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115159 2024-09-18T04:56:32.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:32.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.551798+0000", 2024-09-18T04:56:32.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000360584 2024-09-18T04:56:32.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:32.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.566830+0000", 2024-09-18T04:56:32.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015031991999999999 2024-09-18T04:56:32.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:32.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.567213+0000", 2024-09-18T04:56:32.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038325799999999998 2024-09-18T04:56:32.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:32.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.567250+0000", 2024-09-18T04:56:32.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6259e-05 2024-09-18T04:56:32.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.567278+0000", 2024-09-18T04:56:32.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7988999999999999e-05 2024-09-18T04:56:32.925 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.925 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.925 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.925 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:5208 3.c 3:33b9f7ac:::rbd_data.11ac14fc1d07.0000000000000008:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:32.925 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.569315+0000", 2024-09-18T04:56:32.925 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.23991063600000001, 2024-09-18T04:56:32.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019527049000000001, 2024-09-18T04:56:32.926 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:32.926 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:32.926 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:32.926 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:32.926 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:32.926 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 5208 2024-09-18T04:56:32.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.926 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:32.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:32.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.569315+0000", 2024-09-18T04:56:32.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:32.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.569315+0000", 2024-09-18T04:56:32.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:32.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:32.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.569314+0000", 2024-09-18T04:56:32.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:32.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:32.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.569320+0000", 2024-09-18T04:56:32.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8499999999999999e-06 2024-09-18T04:56:32.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:32.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.569334+0000", 2024-09-18T04:56:32.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.448e-05 2024-09-18T04:56:32.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:32.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.569344+0000", 2024-09-18T04:56:32.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1919999999999994e-06 2024-09-18T04:56:32.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:32.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.569380+0000", 2024-09-18T04:56:32.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6424000000000002e-05 2024-09-18T04:56:32.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:32.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.569437+0000", 2024-09-18T04:56:32.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6839999999999998e-05 2024-09-18T04:56:32.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-18T04:56:32.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.573059+0000", 2024-09-18T04:56:32.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036227270000000001 2024-09-18T04:56:32.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:32.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.588702+0000", 2024-09-18T04:56:32.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015642442999999999 2024-09-18T04:56:32.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:32.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.588748+0000", 2024-09-18T04:56:32.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6099e-05 2024-09-18T04:56:32.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:32.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.588799+0000", 2024-09-18T04:56:32.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1079999999999999e-05 2024-09-18T04:56:32.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:32.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:32.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:32.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.588842+0000", 2024-09-18T04:56:32.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3001999999999999e-05 2024-09-18T04:56:32.931 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.931 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.931 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.931 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:32.931 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:32.931 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:32.933 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:56:32.935 INFO:teuthology.orchestra.run.smithi002.stdout:647 writesame 0xce3b551 thru 0xce40b1b data_size 0x1(0x55cb bytes) 2024-09-18T04:56:32.935 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:32.935 INFO:teuthology.orchestra.run.smithi002.stdout:650 write 0xd6ce7fa thru 0xd6d8d2b (0xa532 bytes) 2024-09-18T04:56:32.935 INFO:teuthology.orchestra.run.smithi002.stdout:652 trunc from 0xd6d8d2c to 0x9a06a30 2024-09-18T04:56:32.935 INFO:teuthology.orchestra.run.smithi002.stdout:653 writesame 0xdcc23a0 thru 0xdcd087b data_size 0x61(0xe4dc bytes) 2024-09-18T04:56:32.935 INFO:teuthology.orchestra.run.smithi002.stdout:654 write 0x3552a1c thru 0x355c3dd (0x99c2 bytes) 2024-09-18T04:56:32.936 INFO:teuthology.orchestra.run.smithi002.stdout:655 writesame 0xf08291 thru 0xf105ef data_size 0xa9(0x835f bytes) 2024-09-18T04:56:32.936 INFO:teuthology.orchestra.run.smithi002.stdout:656 punch from 0xc3e8a37 to 0xc3ee71e, (0x5ce7 bytes) 2024-09-18T04:56:32.936 INFO:teuthology.orchestra.run.smithi002.stdout:657 read 0x13b027e thru 0x13b06c8 (0x44b bytes) 2024-09-18T04:56:32.936 INFO:teuthology.orchestra.run.smithi002.stdout:658 trunc from 0xdcd087c to 0x332e4f6 2024-09-18T04:56:32.936 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:32.936 INFO:teuthology.orchestra.run.smithi002.stdout:661 writesame 0x4e222e8 thru 0x4e27b22 data_size 0x1(0x583b bytes) 2024-09-18T04:56:32.936 INFO:teuthology.orchestra.run.smithi002.stdout:662 read 0x30c23d3 thru 0x30d060b (0xe239 bytes) 2024-09-18T04:56:32.936 INFO:teuthology.orchestra.run.smithi002.stdout:663 trunc from 0x4e27b23 to 0xd533c95 2024-09-18T04:56:32.936 INFO:teuthology.orchestra.run.smithi002.stdout:664 write 0x2aa6f7a thru 0x2ab0fa7 (0xa02e bytes) 2024-09-18T04:56:32.936 INFO:teuthology.orchestra.run.smithi002.stdout:665 punch from 0x11c1e23 to 0x11ca9e4, (0x8bc1 bytes) 2024-09-18T04:56:32.936 INFO:teuthology.orchestra.run.smithi002.stdout:666 trunc from 0xd533c95 to 0xa4d065f 2024-09-18T04:56:32.936 INFO:teuthology.orchestra.run.smithi002.stdout:667 read 0x3f4f8f5 thru 0x3f5b1a4 (0xb8b0 bytes) 2024-09-18T04:56:32.936 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:32.937 INFO:teuthology.orchestra.run.smithi002.stdout:669 punch from 0x9008414 to 0x9011d0f, (0x98fb bytes) 2024-09-18T04:56:32.937 INFO:teuthology.orchestra.run.smithi002.stdout:671 read 0x64f91a7 thru 0x6504c78 (0xbad2 bytes) 2024-09-18T04:56:32.937 INFO:teuthology.orchestra.run.smithi002.stdout:674 punch from 0x324b53f to 0x32571da, (0xbc9b bytes) 2024-09-18T04:56:32.937 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:32.929+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:32.938 INFO:teuthology.orchestra.run.smithi002.stdout:675 writesame 0xd60fc98 thru 0xd614127 data_size 0x8(0x4490 bytes) 2024-09-18T04:56:32.939 INFO:teuthology.orchestra.run.smithi002.stdout:676 write 0xcb87aa5 thru 0xcb8a7ce (0x2d2a bytes) 2024-09-18T04:56:32.941 INFO:teuthology.orchestra.run.smithi002.stdout:677 read 0x2d67db4 thru 0x2d74069 (0xc2b6 bytes) 2024-09-18T04:56:32.941 INFO:teuthology.orchestra.run.smithi002.stdout:678 read 0xa933750 thru 0xa94043d (0xccee bytes) 2024-09-18T04:56:32.941 INFO:teuthology.orchestra.run.smithi002.stdout:679 writesame 0x682dab7 thru 0x6831d65 data_size 0x1(0x42af bytes) 2024-09-18T04:56:32.961 INFO:teuthology.orchestra.run.smithi002.stdout:680 trunc from 0xd614128 to 0x2f01d9b 2024-09-18T04:56:32.989 INFO:teuthology.orchestra.run.smithi002.stdout:683 write 0xea91cee thru 0xea94d02 (0x3015 bytes) 2024-09-18T04:56:32.991 INFO:teuthology.orchestra.run.smithi002.stdout:684 read 0x986d633 thru 0x9878068 (0xaa36 bytes) 2024-09-18T04:56:32.991 INFO:teuthology.orchestra.run.smithi002.stdout:685 read 0x8489265 thru 0x84964d7 (0xd273 bytes) 2024-09-18T04:56:32.991 INFO:teuthology.orchestra.run.smithi002.stdout:688 read 0xe38f96f thru 0xe391c84 (0x2316 bytes) 2024-09-18T04:56:32.992 INFO:teuthology.orchestra.run.smithi002.stdout:689 read 0x5e41a96 thru 0x5e4380a (0x1d75 bytes) 2024-09-18T04:56:32.992 INFO:teuthology.orchestra.run.smithi002.stdout:690 punch from 0x6d20ea3 to 0x6d223eb, (0x1548 bytes) 2024-09-18T04:56:32.992 INFO:teuthology.orchestra.run.smithi002.stdout:694 writesame 0x981f577 thru 0x9821e37 data_size 0x1(0x28c1 bytes) 2024-09-18T04:56:32.997 INFO:teuthology.orchestra.run.smithi002.stdout:695 write 0x1f35863 thru 0x1f3fe50 (0xa5ee bytes) 2024-09-18T04:56:32.998 INFO:teuthology.orchestra.run.smithi002.stdout:696 punch from 0xea94df to 0xeb10c5, (0x7be6 bytes) 2024-09-18T04:56:32.998 INFO:teuthology.orchestra.run.smithi002.stdout:698 write 0xbc85ab6 thru 0xbc95009 (0xf554 bytes) 2024-09-18T04:56:33.000 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:33.000 INFO:teuthology.orchestra.run.smithi002.stdout:700 write 0x7fe6c5e thru 0x7fef69b (0x8a3e bytes) 2024-09-18T04:56:33.017 INFO:teuthology.orchestra.run.smithi002.stdout:701 trunc from 0xea94d03 to 0xdafe42 2024-09-18T04:56:33.034 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:33.031+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:33.044 INFO:teuthology.orchestra.run.smithi002.stdout:702 trunc from 0xdafe42 to 0xe81143c 2024-09-18T04:56:33.049 INFO:teuthology.orchestra.run.smithi002.stdout:705 clone 8 order 24 su 1048576 sc 7 2024-09-18T04:56:33.081 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:33.082 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:56:33.082 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:33.090 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:56:33.166 INFO:teuthology.orchestra.run.smithi002.stderr:set 3.13 pg_upmap mapping to [3,4] 2024-09-18T04:56:33.174 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:33.172+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:33.236 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:33.233+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:33.239 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone7 from 0xe81143c (overlap 0x987cd6) to 0x4c3e6b 2024-09-18T04:56:33.242 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:33.242 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:33.243 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:56:33.243 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:56:33.243 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:33.253 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:56:33.297 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:56:33.338 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:33.336+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:33.342 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-18T04:56:33.399 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:33.400 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:56:33.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:56:33.400 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:56:33.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.400 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4223.0:1 1.3 e15/14 1:c4c92e5a:::rbd_trash:head v 15'1, mlcod=0'0)", 2024-09-18T04:56:33.400 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:10.402353+0000", 2024-09-18T04:56:33.400 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.993891679000001, 2024-09-18T04:56:33.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047646060000000002, 2024-09-18T04:56:33.400 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:33.400 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:33.400 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:33.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:33.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.402353+0000", 2024-09-18T04:56:33.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:33.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.402353+0000", 2024-09-18T04:56:33.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:33.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.402353+0000", 2024-09-18T04:56:33.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:33.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:33.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.402357+0000", 2024-09-18T04:56:33.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3209999999999998e-06 2024-09-18T04:56:33.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:33.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.402372+0000", 2024-09-18T04:56:33.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5201999999999999e-05 2024-09-18T04:56:33.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:33.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.402382+0000", 2024-09-18T04:56:33.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8430000000000005e-06 2024-09-18T04:56:33.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:33.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.402417+0000", 2024-09-18T04:56:33.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4811999999999998e-05 2024-09-18T04:56:33.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:33.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.402434+0000", 2024-09-18T04:56:33.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6877000000000001e-05 2024-09-18T04:56:33.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:33.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.407087+0000", 2024-09-18T04:56:33.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.004653122 2024-09-18T04:56:33.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:33.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.407117+0000", 2024-09-18T04:56:33.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0738999999999998e-05 2024-09-18T04:56:33.404 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.404 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:33.404 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.405 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:694 3.a 3:54bf2b96:::rbd_data.11acdc0eab99.0000000000000067:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:33.405 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.214638+0000", 2024-09-18T04:56:33.405 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.1816068040000003, 2024-09-18T04:56:33.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0054213300000000002, 2024-09-18T04:56:33.405 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:33.405 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:33.405 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:33.405 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:33.405 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:33.405 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 694 2024-09-18T04:56:33.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.405 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:33.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:33.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.214638+0000", 2024-09-18T04:56:33.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:33.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.214638+0000", 2024-09-18T04:56:33.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:33.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.214637+0000", 2024-09-18T04:56:33.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:33.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:33.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.214640+0000", 2024-09-18T04:56:33.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2930000000000001e-06 2024-09-18T04:56:33.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:33.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.214647+0000", 2024-09-18T04:56:33.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0219999999999999e-06 2024-09-18T04:56:33.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:33.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.214653+0000", 2024-09-18T04:56:33.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3190000000000002e-06 2024-09-18T04:56:33.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:33.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215473+0000", 2024-09-18T04:56:33.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00082061799999999996 2024-09-18T04:56:33.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:33.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215545+0000", 2024-09-18T04:56:33.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1282e-05 2024-09-18T04:56:33.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:33.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215699+0000", 2024-09-18T04:56:33.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015379799999999999 2024-09-18T04:56:33.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:33.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.219619+0000", 2024-09-18T04:56:33.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039203600000000003 2024-09-18T04:56:33.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:33.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.219968+0000", 2024-09-18T04:56:33.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000348678 2024-09-18T04:56:33.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:33.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.220022+0000", 2024-09-18T04:56:33.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4049999999999999e-05 2024-09-18T04:56:33.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:33.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.220059+0000", 2024-09-18T04:56:33.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7496000000000001e-05 2024-09-18T04:56:33.410 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.410 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:33.410 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.411 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:789 3.10 3:0a366e40:::rbd_data.11ac33624de9.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 3930015~17006] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:33.411 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.310522+0000", 2024-09-18T04:56:33.411 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.0857224829999996, 2024-09-18T04:56:33.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0081018370000000006, 2024-09-18T04:56:33.411 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:33.411 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:33.411 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:33.411 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:33.411 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:33.411 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 789 2024-09-18T04:56:33.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.411 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:33.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:33.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.310522+0000", 2024-09-18T04:56:33.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:33.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.310522+0000", 2024-09-18T04:56:33.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:33.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.310520+0000", 2024-09-18T04:56:33.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:56:33.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:33.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.313893+0000", 2024-09-18T04:56:33.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033724950000000001 2024-09-18T04:56:33.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:33.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.313907+0000", 2024-09-18T04:56:33.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4185999999999999e-05 2024-09-18T04:56:33.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:33.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.313913+0000", 2024-09-18T04:56:33.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0920000000000003e-06 2024-09-18T04:56:33.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:33.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.313953+0000", 2024-09-18T04:56:33.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0593000000000003e-05 2024-09-18T04:56:33.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:33.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.314062+0000", 2024-09-18T04:56:33.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108621 2024-09-18T04:56:33.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:33.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.314262+0000", 2024-09-18T04:56:33.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019943900000000001 2024-09-18T04:56:33.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:33.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.316850+0000", 2024-09-18T04:56:33.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0025879890000000002 2024-09-18T04:56:33.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:33.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.318554+0000", 2024-09-18T04:56:33.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001704175 2024-09-18T04:56:33.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:33.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.318586+0000", 2024-09-18T04:56:33.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2638999999999997e-05 2024-09-18T04:56:33.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:33.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.318624+0000", 2024-09-18T04:56:33.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7577000000000003e-05 2024-09-18T04:56:33.417 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.417 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:33.417 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.417 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:927 3.10 3:0c01f541:::rbd_data.11ac33624de9.0000000000000007:head [stat,truncate 6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:33.417 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.461258+0000", 2024-09-18T04:56:33.417 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.934986737, 2024-09-18T04:56:33.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035903939999999998, 2024-09-18T04:56:33.417 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:33.417 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:33.417 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:33.417 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:33.418 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:33.418 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 927 2024-09-18T04:56:33.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.418 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:33.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:33.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461258+0000", 2024-09-18T04:56:33.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:33.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461258+0000", 2024-09-18T04:56:33.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:33.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461257+0000", 2024-09-18T04:56:33.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:33.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:33.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461261+0000", 2024-09-18T04:56:33.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8369999999999999e-06 2024-09-18T04:56:33.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:33.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461270+0000", 2024-09-18T04:56:33.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7329999999999992e-06 2024-09-18T04:56:33.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:33.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461276+0000", 2024-09-18T04:56:33.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.229e-06 2024-09-18T04:56:33.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:33.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461664+0000", 2024-09-18T04:56:33.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038783900000000001 2024-09-18T04:56:33.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:33.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.461953+0000", 2024-09-18T04:56:33.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00028891700000000003 2024-09-18T04:56:33.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:33.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.462121+0000", 2024-09-18T04:56:33.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016815600000000001 2024-09-18T04:56:33.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:33.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.463120+0000", 2024-09-18T04:56:33.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00099874300000000007 2024-09-18T04:56:33.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:33.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.464798+0000", 2024-09-18T04:56:33.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016781420000000001 2024-09-18T04:56:33.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:33.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.464828+0000", 2024-09-18T04:56:33.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0190999999999999e-05 2024-09-18T04:56:33.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:33.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.464848+0000", 2024-09-18T04:56:33.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0233000000000001e-05 2024-09-18T04:56:33.423 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.423 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:33.423 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.423 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:963 3.10 3:0c01f541:::rbd_data.11ac33624de9.0000000000000007:head [stat,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:33.423 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.507773+0000", 2024-09-18T04:56:33.423 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.8884711330000004, 2024-09-18T04:56:33.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040043889999999997, 2024-09-18T04:56:33.423 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:33.423 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:33.423 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:33.423 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:33.424 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:33.424 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 963 2024-09-18T04:56:33.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.424 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:33.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:33.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.507773+0000", 2024-09-18T04:56:33.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:33.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.507773+0000", 2024-09-18T04:56:33.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:33.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.507772+0000", 2024-09-18T04:56:33.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:33.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:33.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.507779+0000", 2024-09-18T04:56:33.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1890000000000001e-06 2024-09-18T04:56:33.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:33.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.507822+0000", 2024-09-18T04:56:33.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3133000000000002e-05 2024-09-18T04:56:33.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:33.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.507839+0000", 2024-09-18T04:56:33.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6840000000000001e-05 2024-09-18T04:56:33.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:33.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.508388+0000", 2024-09-18T04:56:33.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00054853699999999998 2024-09-18T04:56:33.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:33.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.508466+0000", 2024-09-18T04:56:33.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8589000000000003e-05 2024-09-18T04:56:33.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:33.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.508591+0000", 2024-09-18T04:56:33.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000124962 2024-09-18T04:56:33.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:33.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.509905+0000", 2024-09-18T04:56:33.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013138799999999999 2024-09-18T04:56:33.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:33.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.511662+0000", 2024-09-18T04:56:33.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0017570999999999999 2024-09-18T04:56:33.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:33.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.511757+0000", 2024-09-18T04:56:33.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4135999999999995e-05 2024-09-18T04:56:33.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:33.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.511778+0000", 2024-09-18T04:56:33.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1299000000000001e-05 2024-09-18T04:56:33.429 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.429 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:33.429 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.429 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:1338 3.10 3:0ee59dfa:::rbd_data.11ac99060bb1.0000000000000028:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:33.429 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.211527+0000", 2024-09-18T04:56:33.429 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.1847170829999998, 2024-09-18T04:56:33.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014563883999999999, 2024-09-18T04:56:33.429 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:33.429 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:33.429 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:33.429 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:33.430 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:33.430 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1338 2024-09-18T04:56:33.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.430 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:33.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:33.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.211527+0000", 2024-09-18T04:56:33.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:33.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.211527+0000", 2024-09-18T04:56:33.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:33.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.211527+0000", 2024-09-18T04:56:33.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:33.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:33.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.211539+0000", 2024-09-18T04:56:33.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.276e-05 2024-09-18T04:56:33.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:33.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.211550+0000", 2024-09-18T04:56:33.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0781999999999999e-05 2024-09-18T04:56:33.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:33.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.211557+0000", 2024-09-18T04:56:33.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6039999999999998e-06 2024-09-18T04:56:33.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:33.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.211589+0000", 2024-09-18T04:56:33.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2650000000000001e-05 2024-09-18T04:56:33.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:33.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.211689+0000", 2024-09-18T04:56:33.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.993e-05 2024-09-18T04:56:33.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:33.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.214589+0000", 2024-09-18T04:56:33.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028995520000000001 2024-09-18T04:56:33.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:33.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.225932+0000", 2024-09-18T04:56:33.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011343463 2024-09-18T04:56:33.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:33.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.225981+0000", 2024-09-18T04:56:33.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8198000000000002e-05 2024-09-18T04:56:33.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:33.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.226047+0000", 2024-09-18T04:56:33.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6039999999999998e-05 2024-09-18T04:56:33.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:33.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.226091+0000", 2024-09-18T04:56:33.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4795000000000001e-05 2024-09-18T04:56:33.435 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.435 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:33.435 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.435 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop_reply(client.4524.0:1338 3.10 e25/19 ondisk, result = 0)", 2024-09-18T04:56:33.435 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.221299+0000", 2024-09-18T04:56:33.435 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.1749458119999998, 2024-09-18T04:56:33.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0048018840000000002, 2024-09-18T04:56:33.435 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:33.435 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:33.435 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:33.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:33.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.221299+0000", 2024-09-18T04:56:33.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:33.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.221299+0000", 2024-09-18T04:56:33.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:33.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.221298+0000", 2024-09-18T04:56:33.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:33.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:33.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.221303+0000", 2024-09-18T04:56:33.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1839999999999998e-06 2024-09-18T04:56:33.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:33.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.221317+0000", 2024-09-18T04:56:33.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3954999999999999e-05 2024-09-18T04:56:33.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:33.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.221324+0000", 2024-09-18T04:56:33.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2740000000000002e-06 2024-09-18T04:56:33.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:33.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.225960+0000", 2024-09-18T04:56:33.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0046359829999999998 2024-09-18T04:56:33.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:33.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.225972+0000", 2024-09-18T04:56:33.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2012e-05 2024-09-18T04:56:33.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:33.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.226101+0000", 2024-09-18T04:56:33.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000128536 2024-09-18T04:56:33.439 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.439 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:33.439 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.439 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:2137 3.a 3:52b2781a:::rbd_data.11ac99060bb1.00000000000001d0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:33.439 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.726272+0000", 2024-09-18T04:56:33.439 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.669973025, 2024-09-18T04:56:33.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012668725, 2024-09-18T04:56:33.439 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:33.440 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:33.440 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:33.440 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:33.440 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:33.440 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2137 2024-09-18T04:56:33.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.440 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:33.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:33.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.726272+0000", 2024-09-18T04:56:33.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:33.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.726272+0000", 2024-09-18T04:56:33.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:33.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.726270+0000", 2024-09-18T04:56:33.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:56:33.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:33.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.726280+0000", 2024-09-18T04:56:33.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0393e-05 2024-09-18T04:56:33.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:33.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.726298+0000", 2024-09-18T04:56:33.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8371000000000001e-05 2024-09-18T04:56:33.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:33.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.726308+0000", 2024-09-18T04:56:33.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2909999999999996e-06 2024-09-18T04:56:33.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:33.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.726348+0000", 2024-09-18T04:56:33.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0824000000000001e-05 2024-09-18T04:56:33.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:33.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.726443+0000", 2024-09-18T04:56:33.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4795000000000003e-05 2024-09-18T04:56:33.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:33.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.728645+0000", 2024-09-18T04:56:33.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022017999999999999 2024-09-18T04:56:33.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:33.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.738874+0000", 2024-09-18T04:56:33.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010228832 2024-09-18T04:56:33.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:33.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.738882+0000", 2024-09-18T04:56:33.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2220000000000002e-06 2024-09-18T04:56:33.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:33.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.738915+0000", 2024-09-18T04:56:33.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3213999999999997e-05 2024-09-18T04:56:33.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:33.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.738940+0000", 2024-09-18T04:56:33.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5021999999999999e-05 2024-09-18T04:56:33.448 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.448 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:33.448 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.448 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop_reply(client.4524.0:2137 3.a e25/19 ondisk, result = 0)", 2024-09-18T04:56:33.448 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.734876+0000", 2024-09-18T04:56:33.448 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.661368242, 2024-09-18T04:56:33.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039996169999999996, 2024-09-18T04:56:33.448 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:33.448 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:33.449 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:33.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:33.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.734876+0000", 2024-09-18T04:56:33.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:33.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.734876+0000", 2024-09-18T04:56:33.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:33.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.734875+0000", 2024-09-18T04:56:33.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:33.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:33.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.734885+0000", 2024-09-18T04:56:33.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7000000000000003e-06 2024-09-18T04:56:33.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:33.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.734898+0000", 2024-09-18T04:56:33.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3481e-05 2024-09-18T04:56:33.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:33.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.734907+0000", 2024-09-18T04:56:33.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3650000000000004e-06 2024-09-18T04:56:33.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:33.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.738860+0000", 2024-09-18T04:56:33.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039534399999999999 2024-09-18T04:56:33.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:33.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.738868+0000", 2024-09-18T04:56:33.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1249999999999993e-06 2024-09-18T04:56:33.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:33.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.738876+0000", 2024-09-18T04:56:33.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6159999999999996e-06 2024-09-18T04:56:33.452 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.452 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:33.452 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.452 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:4378 3.a 3:52eeb3ed:::rbd_data.11ac7d9658bc.0000000000000001:head [stat,truncate 84775] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T04:56:33.453 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:31.242021+0000", 2024-09-18T04:56:33.453 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 2.154223172, 2024-09-18T04:56:33.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0057347800000000001, 2024-09-18T04:56:33.453 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:33.453 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:33.453 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:33.453 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:33.453 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:33.453 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4378 2024-09-18T04:56:33.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.453 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:33.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:33.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242021+0000", 2024-09-18T04:56:33.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:33.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242021+0000", 2024-09-18T04:56:33.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:33.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242020+0000", 2024-09-18T04:56:33.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:56:33.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:33.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242027+0000", 2024-09-18T04:56:33.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1110000000000002e-06 2024-09-18T04:56:33.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:33.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242040+0000", 2024-09-18T04:56:33.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3281e-05 2024-09-18T04:56:33.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:33.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242049+0000", 2024-09-18T04:56:33.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7940000000000008e-06 2024-09-18T04:56:33.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:33.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242093+0000", 2024-09-18T04:56:33.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3832999999999999e-05 2024-09-18T04:56:33.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:33.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242176+0000", 2024-09-18T04:56:33.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3184999999999997e-05 2024-09-18T04:56:33.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:33.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242339+0000", 2024-09-18T04:56:33.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016288500000000001 2024-09-18T04:56:33.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:33.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.244745+0000", 2024-09-18T04:56:33.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024063909999999999 2024-09-18T04:56:33.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:33.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.247692+0000", 2024-09-18T04:56:33.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002947149 2024-09-18T04:56:33.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:33.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.247720+0000", 2024-09-18T04:56:33.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7645e-05 2024-09-18T04:56:33.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:33.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.247756+0000", 2024-09-18T04:56:33.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.625e-05 2024-09-18T04:56:33.458 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.458 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:33.458 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.458 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:4742 3.a 3:55916b62:::rbd_object_map.11aca1181477:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:33.458 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.238085+0000", 2024-09-18T04:56:33.459 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.1581594669999999, 2024-09-18T04:56:33.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0038238590000000002, 2024-09-18T04:56:33.459 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:33.459 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:33.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:33.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:33.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:33.459 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4742 2024-09-18T04:56:33.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.459 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:33.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:33.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238085+0000", 2024-09-18T04:56:33.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:33.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238085+0000", 2024-09-18T04:56:33.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:33.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238085+0000", 2024-09-18T04:56:33.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:33.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:33.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238087+0000", 2024-09-18T04:56:33.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8310000000000001e-06 2024-09-18T04:56:33.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:33.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238093+0000", 2024-09-18T04:56:33.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7089999999999999e-06 2024-09-18T04:56:33.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:33.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238098+0000", 2024-09-18T04:56:33.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.515e-06 2024-09-18T04:56:33.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:33.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.239147+0000", 2024-09-18T04:56:33.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010497740000000001 2024-09-18T04:56:33.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:33.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.239220+0000", 2024-09-18T04:56:33.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2616999999999999e-05 2024-09-18T04:56:33.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:33.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.239536+0000", 2024-09-18T04:56:33.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031586500000000003 2024-09-18T04:56:33.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:33.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241207+0000", 2024-09-18T04:56:33.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001671442 2024-09-18T04:56:33.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:33.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241822+0000", 2024-09-18T04:56:33.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00061487599999999999 2024-09-18T04:56:33.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:33.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241871+0000", 2024-09-18T04:56:33.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8971000000000003e-05 2024-09-18T04:56:33.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:33.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241909+0000", 2024-09-18T04:56:33.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7688000000000002e-05 2024-09-18T04:56:33.464 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.464 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:33.464 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.464 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:4744 3.a 3:55916b62:::rbd_object_map.11aca1181477:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:33.464 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.238285+0000", 2024-09-18T04:56:33.465 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.1579598719999999, 2024-09-18T04:56:33.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0069044099999999997, 2024-09-18T04:56:33.465 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:33.465 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:33.465 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:33.465 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:33.465 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:33.465 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4744 2024-09-18T04:56:33.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.466 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:33.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:33.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238285+0000", 2024-09-18T04:56:33.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:33.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238285+0000", 2024-09-18T04:56:33.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:33.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238284+0000", 2024-09-18T04:56:33.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:33.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:33.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238289+0000", 2024-09-18T04:56:33.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.096e-06 2024-09-18T04:56:33.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:33.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238300+0000", 2024-09-18T04:56:33.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1161999999999999e-05 2024-09-18T04:56:33.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:33.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238308+0000", 2024-09-18T04:56:33.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.267e-06 2024-09-18T04:56:33.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:33.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.239837+0000", 2024-09-18T04:56:33.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015297990000000001 2024-09-18T04:56:33.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:33.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.239911+0000", 2024-09-18T04:56:33.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3356999999999996e-05 2024-09-18T04:56:33.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:33.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.240235+0000", 2024-09-18T04:56:33.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00032462600000000001 2024-09-18T04:56:33.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:33.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.243412+0000", 2024-09-18T04:56:33.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031762240000000001 2024-09-18T04:56:33.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:33.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245106+0000", 2024-09-18T04:56:33.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001694128 2024-09-18T04:56:33.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:33.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245151+0000", 2024-09-18T04:56:33.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5225999999999997e-05 2024-09-18T04:56:33.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:33.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245189+0000", 2024-09-18T04:56:33.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8229000000000003e-05 2024-09-18T04:56:33.471 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.471 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:33.471 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.471 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:4746 3.a 3:55916b62:::rbd_object_map.11aca1181477:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:33.471 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.238313+0000", 2024-09-18T04:56:33.471 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.157931711, 2024-09-18T04:56:33.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0069673240000000004, 2024-09-18T04:56:33.471 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:33.471 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:33.471 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:33.472 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:33.472 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:33.472 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4746 2024-09-18T04:56:33.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.472 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:33.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:33.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238313+0000", 2024-09-18T04:56:33.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:33.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238313+0000", 2024-09-18T04:56:33.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:33.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238312+0000", 2024-09-18T04:56:33.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:33.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:33.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238317+0000", 2024-09-18T04:56:33.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3189999999999996e-06 2024-09-18T04:56:33.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:33.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238323+0000", 2024-09-18T04:56:33.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.28e-06 2024-09-18T04:56:33.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:33.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238328+0000", 2024-09-18T04:56:33.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7990000000000001e-06 2024-09-18T04:56:33.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:33.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.240597+0000", 2024-09-18T04:56:33.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022687229999999998 2024-09-18T04:56:33.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:33.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.240672+0000", 2024-09-18T04:56:33.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5142000000000007e-05 2024-09-18T04:56:33.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:33.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.240973+0000", 2024-09-18T04:56:33.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030111999999999999 2024-09-18T04:56:33.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:33.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.243446+0000", 2024-09-18T04:56:33.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024727479999999999 2024-09-18T04:56:33.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:33.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245199+0000", 2024-09-18T04:56:33.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0017535809999999999 2024-09-18T04:56:33.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:33.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245244+0000", 2024-09-18T04:56:33.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4458e-05 2024-09-18T04:56:33.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:33.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245280+0000", 2024-09-18T04:56:33.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6613999999999999e-05 2024-09-18T04:56:33.477 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.477 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:33.477 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.477 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:4747 3.a 3:55916b62:::rbd_object_map.11aca1181477:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:33.477 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.238332+0000", 2024-09-18T04:56:33.477 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.1579125610000001, 2024-09-18T04:56:33.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0070349059999999996, 2024-09-18T04:56:33.477 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:33.478 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:33.478 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:33.478 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:33.478 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:33.478 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4747 2024-09-18T04:56:33.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.478 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:33.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:33.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238332+0000", 2024-09-18T04:56:33.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:33.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238332+0000", 2024-09-18T04:56:33.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:33.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238332+0000", 2024-09-18T04:56:33.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:33.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:33.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238334+0000", 2024-09-18T04:56:33.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5780000000000001e-06 2024-09-18T04:56:33.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:33.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238340+0000", 2024-09-18T04:56:33.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4399999999999996e-06 2024-09-18T04:56:33.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:33.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238345+0000", 2024-09-18T04:56:33.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4099999999999999e-06 2024-09-18T04:56:33.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:33.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241231+0000", 2024-09-18T04:56:33.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028862369999999998 2024-09-18T04:56:33.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:33.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241274+0000", 2024-09-18T04:56:33.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2308000000000002e-05 2024-09-18T04:56:33.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:33.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241450+0000", 2024-09-18T04:56:33.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000176276 2024-09-18T04:56:33.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:33.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.243482+0000", 2024-09-18T04:56:33.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020318689999999999 2024-09-18T04:56:33.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:33.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245289+0000", 2024-09-18T04:56:33.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001807016 2024-09-18T04:56:33.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:33.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245331+0000", 2024-09-18T04:56:33.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2218999999999997e-05 2024-09-18T04:56:33.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:33.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245367+0000", 2024-09-18T04:56:33.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5920999999999997e-05 2024-09-18T04:56:33.483 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.483 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:33.483 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.483 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:4749 3.a 3:55916b62:::rbd_object_map.11aca1181477:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:33.483 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.238349+0000", 2024-09-18T04:56:33.484 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.1578950960000001, 2024-09-18T04:56:33.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0071690089999999996, 2024-09-18T04:56:33.484 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:33.484 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:33.484 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:33.484 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:33.484 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:33.484 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4749 2024-09-18T04:56:33.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.484 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:33.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:33.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238349+0000", 2024-09-18T04:56:33.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:33.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238349+0000", 2024-09-18T04:56:33.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:33.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238349+0000", 2024-09-18T04:56:33.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:33.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:33.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238362+0000", 2024-09-18T04:56:33.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2802999999999999e-05 2024-09-18T04:56:33.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:33.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238370+0000", 2024-09-18T04:56:33.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6520000000000006e-06 2024-09-18T04:56:33.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:33.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238375+0000", 2024-09-18T04:56:33.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1930000000000001e-06 2024-09-18T04:56:33.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:33.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241943+0000", 2024-09-18T04:56:33.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035684710000000001 2024-09-18T04:56:33.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:33.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.242012+0000", 2024-09-18T04:56:33.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9282999999999994e-05 2024-09-18T04:56:33.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:33.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.242328+0000", 2024-09-18T04:56:33.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031586899999999998 2024-09-18T04:56:33.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:33.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245375+0000", 2024-09-18T04:56:33.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030469120000000001 2024-09-18T04:56:33.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:33.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245425+0000", 2024-09-18T04:56:33.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9391999999999997e-05 2024-09-18T04:56:33.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:33.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245484+0000", 2024-09-18T04:56:33.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9169999999999998e-05 2024-09-18T04:56:33.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:33.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245518+0000", 2024-09-18T04:56:33.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4653999999999997e-05 2024-09-18T04:56:33.489 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.489 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:33.489 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.490 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:4750 3.a 3:55916b62:::rbd_object_map.11aca1181477:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:33.490 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.238701+0000", 2024-09-18T04:56:33.490 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.1575432779999999, 2024-09-18T04:56:33.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0069625720000000002, 2024-09-18T04:56:33.490 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:33.490 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:33.490 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:33.490 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:33.490 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:33.490 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4750 2024-09-18T04:56:33.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.490 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:33.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:33.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238701+0000", 2024-09-18T04:56:33.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:33.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238701+0000", 2024-09-18T04:56:33.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:33.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238700+0000", 2024-09-18T04:56:33.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238706+0000", 2024-09-18T04:56:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8869999999999997e-06 2024-09-18T04:56:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238718+0000", 2024-09-18T04:56:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1640999999999999e-05 2024-09-18T04:56:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:33.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238725+0000", 2024-09-18T04:56:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.221e-06 2024-09-18T04:56:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.242659+0000", 2024-09-18T04:56:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039341649999999999 2024-09-18T04:56:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.242735+0000", 2024-09-18T04:56:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6190000000000001e-05 2024-09-18T04:56:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:33.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.243087+0000", 2024-09-18T04:56:33.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035155499999999998 2024-09-18T04:56:33.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245578+0000", 2024-09-18T04:56:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002490618 2024-09-18T04:56:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245586+0000", 2024-09-18T04:56:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2619999999999998e-06 2024-09-18T04:56:33.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245630+0000", 2024-09-18T04:56:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.405e-05 2024-09-18T04:56:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245664+0000", 2024-09-18T04:56:33.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3983000000000003e-05 2024-09-18T04:56:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:4752 3.a 3:55916b62:::rbd_object_map.11aca1181477:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.240989+0000", 2024-09-18T04:56:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.155255881, 2024-09-18T04:56:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0085146609999999998, 2024-09-18T04:56:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:33.497 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4752 2024-09-18T04:56:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.240989+0000", 2024-09-18T04:56:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.240989+0000", 2024-09-18T04:56:33.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.240988+0000", 2024-09-18T04:56:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241002+0000", 2024-09-18T04:56:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4042999999999999e-05 2024-09-18T04:56:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241014+0000", 2024-09-18T04:56:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2469e-05 2024-09-18T04:56:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241021+0000", 2024-09-18T04:56:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1489999999999997e-06 2024-09-18T04:56:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.243506+0000", 2024-09-18T04:56:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024841199999999998 2024-09-18T04:56:33.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.243572+0000", 2024-09-18T04:56:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6873999999999997e-05 2024-09-18T04:56:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.243899+0000", 2024-09-18T04:56:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000326161 2024-09-18T04:56:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:33.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.247781+0000", 2024-09-18T04:56:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0038827359999999999 2024-09-18T04:56:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.249442+0000", 2024-09-18T04:56:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016604460000000001 2024-09-18T04:56:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.249478+0000", 2024-09-18T04:56:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6695e-05 2024-09-18T04:56:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.249503+0000", 2024-09-18T04:56:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4905999999999998e-05 2024-09-18T04:56:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:4753 3.a 3:55916b62:::rbd_object_map.11aca1181477:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.241030+0000", 2024-09-18T04:56:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.15521469, 2024-09-18T04:56:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011505873999999999, 2024-09-18T04:56:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:33.503 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4753 2024-09-18T04:56:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241030+0000", 2024-09-18T04:56:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241030+0000", 2024-09-18T04:56:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241029+0000", 2024-09-18T04:56:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241033+0000", 2024-09-18T04:56:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5410000000000001e-06 2024-09-18T04:56:33.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241040+0000", 2024-09-18T04:56:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1019999999999999e-06 2024-09-18T04:56:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241045+0000", 2024-09-18T04:56:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1449999999999999e-06 2024-09-18T04:56:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.244262+0000", 2024-09-18T04:56:33.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003216413 2024-09-18T04:56:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.244353+0000", 2024-09-18T04:56:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1048e-05 2024-09-18T04:56:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.244728+0000", 2024-09-18T04:56:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000375516 2024-09-18T04:56:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:33.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.247786+0000", 2024-09-18T04:56:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030578609999999998 2024-09-18T04:56:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.252457+0000", 2024-09-18T04:56:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0046708319999999998 2024-09-18T04:56:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.252502+0000", 2024-09-18T04:56:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4694000000000003e-05 2024-09-18T04:56:33.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.252536+0000", 2024-09-18T04:56:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4181999999999999e-05 2024-09-18T04:56:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:4754 3.a 3:55916b62:::rbd_object_map.11aca1181477:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.241193+0000", 2024-09-18T04:56:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.1550512399999999, 2024-09-18T04:56:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011431838, 2024-09-18T04:56:33.509 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4754 2024-09-18T04:56:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241193+0000", 2024-09-18T04:56:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241193+0000", 2024-09-18T04:56:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241192+0000", 2024-09-18T04:56:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241198+0000", 2024-09-18T04:56:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1419999999999998e-06 2024-09-18T04:56:33.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241210+0000", 2024-09-18T04:56:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.187e-05 2024-09-18T04:56:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241218+0000", 2024-09-18T04:56:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5000000000000002e-06 2024-09-18T04:56:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:33.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245710+0000", 2024-09-18T04:56:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0044919039999999997 2024-09-18T04:56:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245780+0000", 2024-09-18T04:56:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0445999999999996e-05 2024-09-18T04:56:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.246139+0000", 2024-09-18T04:56:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035877800000000003 2024-09-18T04:56:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.247790+0000", 2024-09-18T04:56:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016513980000000001 2024-09-18T04:56:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.252569+0000", 2024-09-18T04:56:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047786149999999999 2024-09-18T04:56:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.252595+0000", 2024-09-18T04:56:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6347999999999999e-05 2024-09-18T04:56:33.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.252625+0000", 2024-09-18T04:56:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9884000000000001e-05 2024-09-18T04:56:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:4755 3.a 3:55916b62:::rbd_object_map.11aca1181477:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.241239+0000", 2024-09-18T04:56:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.1550053330000001, 2024-09-18T04:56:33.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011505253, 2024-09-18T04:56:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4755 2024-09-18T04:56:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.516 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241239+0000", 2024-09-18T04:56:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241239+0000", 2024-09-18T04:56:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:33.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241239+0000", 2024-09-18T04:56:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241242+0000", 2024-09-18T04:56:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7120000000000002e-06 2024-09-18T04:56:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241253+0000", 2024-09-18T04:56:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0308e-05 2024-09-18T04:56:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241258+0000", 2024-09-18T04:56:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2460000000000003e-06 2024-09-18T04:56:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.246891+0000", 2024-09-18T04:56:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0056336399999999997 2024-09-18T04:56:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.246977+0000", 2024-09-18T04:56:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.577e-05 2024-09-18T04:56:33.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.247393+0000", 2024-09-18T04:56:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041592899999999999 2024-09-18T04:56:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.251987+0000", 2024-09-18T04:56:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0045936450000000004 2024-09-18T04:56:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:33.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.252670+0000", 2024-09-18T04:56:33.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00068347000000000004 2024-09-18T04:56:33.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:33.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.252712+0000", 2024-09-18T04:56:33.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1730000000000002e-05 2024-09-18T04:56:33.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:33.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:33.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:33.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.252744+0000", 2024-09-18T04:56:33.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2477000000000001e-05 2024-09-18T04:56:33.521 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.521 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:33.521 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.522 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:33.522 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:33.522 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:33.522 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:56:33.523 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:56:33.524 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:56:33.524 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:56:33.524 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:56:33.525 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:56:33.525 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:56:33.526 INFO:teuthology.orchestra.run.smithi002.stdout:707 punch from 0x52f7de1 to 0x5300e0c, (0x902b bytes) 2024-09-18T04:56:33.526 INFO:teuthology.orchestra.run.smithi002.stdout:708 write 0x8dcb68f thru 0x8dcf4eb (0x3e5d bytes) 2024-09-18T04:56:33.526 INFO:teuthology.orchestra.run.smithi002.stdout:709 write 0x8548086 thru 0x8554ded (0xcd68 bytes) 2024-09-18T04:56:33.526 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:33.526 INFO:teuthology.orchestra.run.smithi002.stdout:712 writesame 0xba03a48 thru 0xba1058f data_size 0x14(0xcb48 bytes) 2024-09-18T04:56:33.526 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:56:33.527 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:56:33.527 INFO:teuthology.orchestra.run.smithi002.stdout:713 trunc from 0xe81143c to 0x1db2246 2024-09-18T04:56:33.527 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:56:33.527 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:56:33.528 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:56:33.528 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:56:33.528 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:33.520+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:33.529 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:33.529 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:33.529 INFO:teuthology.orchestra.run.smithi002.stdout:716 read 0x1d372cd thru 0x1d3d1ce (0x5f02 bytes) 2024-09-18T04:56:33.529 INFO:teuthology.orchestra.run.smithi002.stdout:717 read 0x1288820 thru 0x1288862 (0x43 bytes) 2024-09-18T04:56:33.529 INFO:teuthology.orchestra.run.smithi002.stdout:719 read 0xda3ff1 thru 0xda447e (0x48e bytes) 2024-09-18T04:56:33.538 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:56:33.538 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:56:33.541 INFO:teuthology.orchestra.run.smithi002.stdout:721 writesame 0x93a742a thru 0x93ae8e6 data_size 0x2b(0x74bd bytes) 2024-09-18T04:56:33.541 INFO:teuthology.orchestra.run.smithi002.stdout:722 read 0x1092178 thru 0x1093e4c (0x1cd5 bytes) 2024-09-18T04:56:33.551 INFO:teuthology.orchestra.run.smithi002.stdout:724 writesame 0xb89fa42 thru 0xb89ff46 data_size 0x1(0x505 bytes) 2024-09-18T04:56:33.551 INFO:teuthology.orchestra.run.smithi002.stdout:725 read 0x381c994 thru 0x3821e36 (0x54a3 bytes) 2024-09-18T04:56:33.551 INFO:teuthology.orchestra.run.smithi002.stdout:728 write 0x78b39ec thru 0x78bede8 (0xb3fd bytes) 2024-09-18T04:56:33.556 INFO:teuthology.orchestra.run.smithi002.stdout:729 trunc from 0xb89ff47 to 0xdd72338 2024-09-18T04:56:33.556 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:56:33.556 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:56:33.561 INFO:teuthology.orchestra.run.smithi002.stdout:730 writesame 0xa90d9e thru 0xa92aee data_size 0x13(0x1d51 bytes) 2024-09-18T04:56:33.562 INFO:teuthology.orchestra.run.smithi002.stdout:732 read 0xcf81d36 thru 0xcf87fed (0x62b8 bytes) 2024-09-18T04:56:33.562 INFO:teuthology.orchestra.run.smithi002.stdout:733 write 0x35e8e6d thru 0x35f2a76 (0x9c0a bytes) 2024-09-18T04:56:33.571 INFO:teuthology.orchestra.run.smithi002.stdout:735 write 0xe5a24e0 thru 0xe5a270b (0x22c bytes) 2024-09-18T04:56:33.574 INFO:teuthology.orchestra.run.smithi002.stdout:737 punch from 0x44eaadc to 0x44f6ebc, (0xc3e0 bytes) 2024-09-18T04:56:33.574 INFO:teuthology.orchestra.run.smithi002.stdout:738 punch from 0xb8f1e1f to 0xb8f31e7, (0x13c8 bytes) 2024-09-18T04:56:33.574 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:33.574 INFO:teuthology.orchestra.run.smithi002.stdout:741 write 0xe19719a thru 0xe19d001 (0x5e68 bytes) 2024-09-18T04:56:33.577 INFO:teuthology.orchestra.run.smithi002.stdout:742 read 0x3d53b0e thru 0x3d5ddb4 (0xa2a7 bytes) 2024-09-18T04:56:33.577 INFO:teuthology.orchestra.run.smithi002.stdout:744 writesame 0xa59bf6b thru 0xa5a3964 data_size 0xd(0x79fa bytes) 2024-09-18T04:56:33.580 INFO:teuthology.orchestra.run.smithi002.stdout:745 writesame 0x17b0cef thru 0x17b0e1b data_size 0x1(0x12d bytes) 2024-09-18T04:56:33.594 INFO:teuthology.orchestra.run.smithi002.stdout:746 trunc from 0xe5a270c to 0x3df1b99 2024-09-18T04:56:33.623 INFO:teuthology.orchestra.run.smithi002.stdout:747 writesame 0xcd34ad3 thru 0xcd39e94 data_size 0x2(0x53c2 bytes) 2024-09-18T04:56:33.623 INFO:teuthology.orchestra.run.smithi002.stdout:749 write 0x640953c thru 0x640cf46 (0x3a0b bytes) 2024-09-18T04:56:33.624 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:33.621+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:33.624 INFO:teuthology.orchestra.run.smithi002.stdout:750 read 0x9b22e07 thru 0x9b26722 (0x391c bytes) 2024-09-18T04:56:33.624 INFO:teuthology.orchestra.run.smithi002.stdout:751 read 0x438fed8 thru 0x43967f2 (0x691b bytes) 2024-09-18T04:56:33.625 INFO:teuthology.orchestra.run.smithi002.stdout:752 read 0x99e0790 thru 0x99e8799 (0x800a bytes) 2024-09-18T04:56:33.625 INFO:teuthology.orchestra.run.smithi002.stdout:753 punch from 0x4fc99b3 to 0x4fd2f35, (0x9582 bytes) 2024-09-18T04:56:33.625 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:33.625 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:33.625 INFO:teuthology.orchestra.run.smithi002.stdout:757 read 0x7da60a9 thru 0x7daff95 (0x9eed bytes) 2024-09-18T04:56:33.626 INFO:teuthology.orchestra.run.smithi002.stdout:758 write 0xb8dedd thru 0xb9220e (0x4332 bytes) 2024-09-18T04:56:33.626 INFO:teuthology.orchestra.run.smithi002.stdout:759 write 0x67bfd64 thru 0x67cec68 (0xef05 bytes) 2024-09-18T04:56:33.634 INFO:teuthology.orchestra.run.smithi002.stdout:760 write 0xd2ec1e2 thru 0xd2f4f24 (0x8d43 bytes) 2024-09-18T04:56:33.637 INFO:teuthology.orchestra.run.smithi002.stdout:762 write 0xaa336c5 thru 0xaa368c7 (0x3203 bytes) 2024-09-18T04:56:33.639 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:33.639 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:33.640 INFO:teuthology.orchestra.run.smithi002.stdout:765 write 0xad76536 thru 0xad7cd3d (0x6808 bytes) 2024-09-18T04:56:33.644 INFO:teuthology.orchestra.run.smithi002.stdout:766 trunc from 0xd2f4f25 to 0xeaf6420 2024-09-18T04:56:33.650 INFO:teuthology.orchestra.run.smithi002.stdout:767 write 0x88e5f0 thru 0x8928a8 (0x42b9 bytes) 2024-09-18T04:56:33.652 INFO:teuthology.orchestra.run.smithi002.stdout:769 punch from 0xc487d6 to 0xc53108, (0xa932 bytes) 2024-09-18T04:56:33.652 INFO:teuthology.orchestra.run.smithi002.stdout:770 write 0xac9d7e2 thru 0xaca705f (0x987e bytes) 2024-09-18T04:56:33.655 INFO:teuthology.orchestra.run.smithi002.stdout:773 punch from 0x1e9c619 to 0x1ea09f7, (0x43de bytes) 2024-09-18T04:56:33.655 INFO:teuthology.orchestra.run.smithi002.stdout:774 writesame 0xacec210 thru 0xacf7107 data_size 0x2c(0xaef8 bytes) 2024-09-18T04:56:33.657 INFO:teuthology.orchestra.run.smithi002.stdout:775 write 0xd86d252 thru 0xd87cad4 (0xf883 bytes) 2024-09-18T04:56:33.662 INFO:teuthology.orchestra.run.smithi002.stdout:776 write 0x59700c6 thru 0x5979940 (0x987b bytes) 2024-09-18T04:56:33.663 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:33.664 INFO:teuthology.orchestra.run.smithi002.stdout:778 trunc from 0xeaf6420 to 0xe0928f8 2024-09-18T04:56:33.669 INFO:teuthology.orchestra.run.smithi002.stdout:779 write 0x4606ae4 thru 0x46099e1 (0x2efe bytes) 2024-09-18T04:56:33.673 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:33.674 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:33.682 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:56:33.684 INFO:teuthology.orchestra.run.smithi002.stdout:781 trunc from 0xe0928f8 to 0x228fdc4 2024-09-18T04:56:33.703 INFO:teuthology.orchestra.run.smithi002.stdout:782 read 0x499477 thru 0x49ae7d (0x1a07 bytes) 2024-09-18T04:56:33.703 INFO:teuthology.orchestra.run.smithi002.stdout:784 read 0x175869c thru 0x175b3d4 (0x2d39 bytes) 2024-09-18T04:56:33.703 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:33.703 INFO:teuthology.orchestra.run.smithi002.stdout:786 read 0x6f5c44 thru 0x6f5f15 (0x2d2 bytes) 2024-09-18T04:56:33.705 INFO:teuthology.orchestra.run.smithi002.stdout:788 trunc from 0x228fdc4 to 0xb29f36 2024-09-18T04:56:33.724 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:33.721+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:33.729 INFO:teuthology.orchestra.run.smithi002.stdout:789 write 0x8d47e06 thru 0x8d53019 (0xb214 bytes) 2024-09-18T04:56:33.741 INFO:teuthology.orchestra.run.smithi002.stdout:790 writesame 0xb0ecb95 thru 0xb0f5250 data_size 0x2(0x86bc bytes) 2024-09-18T04:56:33.745 INFO:teuthology.orchestra.run.smithi002.stdout:791 trunc from 0xb0f5251 to 0xdddbdfa 2024-09-18T04:56:33.752 INFO:teuthology.orchestra.run.smithi002.stdout:792 writesame 0x51d96ab thru 0x51e042c data_size 0x2(0x6d82 bytes) 2024-09-18T04:56:33.759 INFO:teuthology.orchestra.run.smithi002.stdout:793 write 0x84a1476 thru 0x84abadd (0xa668 bytes) 2024-09-18T04:56:33.762 INFO:teuthology.orchestra.run.smithi002.stdout:795 writesame 0x4e6bf39 thru 0x4e75f7a data_size 0x49(0xa042 bytes) 2024-09-18T04:56:33.764 INFO:teuthology.orchestra.run.smithi002.stdout:796 read 0x90327ef thru 0x9038446 (0x5c58 bytes) 2024-09-18T04:56:33.764 INFO:teuthology.orchestra.run.smithi002.stdout:797 read 0x71cfd1c thru 0x71d2679 (0x295e bytes) 2024-09-18T04:56:33.765 INFO:teuthology.orchestra.run.smithi002.stdout:798 write 0xa217210 thru 0xa2258db (0xe6cc bytes) 2024-09-18T04:56:33.766 INFO:teuthology.orchestra.run.smithi002.stdout:799 writesame 0xba88d97 thru 0xba8929f data_size 0x1(0x509 bytes) 2024-09-18T04:56:33.768 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:33.779 INFO:teuthology.orchestra.run.smithi002.stdout:802 trunc from 0xdddbdfa to 0x428c136 2024-09-18T04:56:33.791 INFO:teuthology.orchestra.run.smithi002.stdout:808 read 0x23e085a thru 0x23eeead (0xe654 bytes) 2024-09-18T04:56:33.808 INFO:teuthology.orchestra.run.smithi002.stdout:809 writesame 0xcb35f1b thru 0xcb3f788 data_size 0x2(0x986e bytes) 2024-09-18T04:56:33.808 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:33.808 INFO:teuthology.orchestra.run.smithi002.stdout:813 read 0xb6be5b6 thru 0xb6cab13 (0xc55e bytes) 2024-09-18T04:56:33.808 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:33.808 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:33.809 INFO:teuthology.orchestra.run.smithi002.stdout:817 punch from 0x6fdd8ca to 0x6fe2553, (0x4c89 bytes) 2024-09-18T04:56:33.809 INFO:teuthology.orchestra.run.smithi002.stdout:818 punch from 0x3daf70 to 0x3de784, (0x3814 bytes) 2024-09-18T04:56:33.809 INFO:teuthology.orchestra.run.smithi002.stdout:819 write 0x60b3f3c thru 0x60bb9d8 (0x7a9d bytes) 2024-09-18T04:56:33.810 INFO:teuthology.orchestra.run.smithi002.stdout:820 read 0xc4f95b5 thru 0xc5046a3 (0xb0ef bytes) 2024-09-18T04:56:33.810 INFO:teuthology.orchestra.run.smithi002.stdout:821 punch from 0xc9ad3e4 to 0xc9b62c1, (0x8edd bytes) 2024-09-18T04:56:33.810 INFO:teuthology.orchestra.run.smithi002.stdout:822 punch from 0x5360dbd to 0x536f848, (0xea8b bytes) 2024-09-18T04:56:33.810 INFO:teuthology.orchestra.run.smithi002.stdout:823 read 0x76d6584 thru 0x76e28ec (0xc369 bytes) 2024-09-18T04:56:33.810 INFO:teuthology.orchestra.run.smithi002.stdout:824 write 0xc082d21 thru 0xc08f043 (0xc323 bytes) 2024-09-18T04:56:33.821 INFO:teuthology.orchestra.run.smithi002.stdout:826 writesame 0xdf0e59b thru 0xdf177c7 data_size 0x1(0x922d bytes) 2024-09-18T04:56:33.822 INFO:teuthology.orchestra.run.smithi002.stdout:827 read 0xc92b239 thru 0xc92bee4 (0xcac bytes) 2024-09-18T04:56:33.822 INFO:teuthology.orchestra.run.smithi002.stdout:828 write 0x7022f1f thru 0x7032c87 (0xfd69 bytes) 2024-09-18T04:56:33.824 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:33.822+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:33.825 INFO:teuthology.orchestra.run.smithi002.stdout:829 trunc from 0xdf177c8 to 0xd064c08 2024-09-18T04:56:33.830 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:33.830 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:33.832 INFO:teuthology.orchestra.run.smithi002.stdout:830 punch from 0x4d1001c to 0x4d1a357, (0xa33b bytes) 2024-09-18T04:56:33.832 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:33.832 INFO:teuthology.orchestra.run.smithi002.stdout:832 read 0xb3decd5 thru 0xb3ea287 (0xb5b3 bytes) 2024-09-18T04:56:33.833 INFO:teuthology.orchestra.run.smithi002.stdout:833 write 0xfb7ae0 thru 0xfc2278 (0xa799 bytes) 2024-09-18T04:56:33.834 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:33.834 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:33.834 INFO:teuthology.orchestra.run.smithi002.stdout:836 read 0xa0b2486 thru 0xa0bb5a8 (0x9123 bytes) 2024-09-18T04:56:33.835 INFO:teuthology.orchestra.run.smithi002.stdout:837 trunc from 0xd064c08 to 0xdc732dc 2024-09-18T04:56:33.840 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:56:33.847 INFO:teuthology.orchestra.run.smithi002.stdout:838 trunc from 0xdc732dc to 0x8730f87 2024-09-18T04:56:33.853 INFO:teuthology.orchestra.run.smithi002.stdout:839 read 0x5a8ca23 thru 0x5a9322f (0x680d bytes) 2024-09-18T04:56:33.854 INFO:teuthology.orchestra.run.smithi002.stdout:840 punch from 0x6cfcb2c to 0x6d0a172, (0xd646 bytes) 2024-09-18T04:56:33.854 INFO:teuthology.orchestra.run.smithi002.stdout:843 punch from 0x4db5ba9 to 0x4db8a9e, (0x2ef5 bytes) 2024-09-18T04:56:33.854 INFO:teuthology.orchestra.run.smithi002.stdout:845 write 0x2852105 thru 0x2852b84 (0xa80 bytes) 2024-09-18T04:56:33.855 INFO:teuthology.orchestra.run.smithi002.stdout:846 read 0x5775ea7 thru 0x577af00 (0x505a bytes) 2024-09-18T04:56:33.862 INFO:teuthology.orchestra.run.smithi002.stdout:847 trunc from 0x8730f87 to 0x29178c0 2024-09-18T04:56:33.886 INFO:teuthology.orchestra.run.smithi002.stdout:848 writesame 0x83125de thru 0x8315c66 data_size 0x1(0x3689 bytes) 2024-09-18T04:56:33.886 INFO:teuthology.orchestra.run.smithi002.stdout:849 read 0x5cfe5a3 thru 0x5d09363 (0xadc1 bytes) 2024-09-18T04:56:33.886 INFO:teuthology.orchestra.run.smithi002.stdout:850 read 0xf1118f thru 0xf17c5c (0x6ace bytes) 2024-09-18T04:56:33.887 INFO:teuthology.orchestra.run.smithi002.stdout:851 write 0x51c6908 thru 0x51d13b7 (0xaab0 bytes) 2024-09-18T04:56:33.888 INFO:teuthology.orchestra.run.smithi002.stdout:852 punch from 0x107e939 to 0x10822c0, (0x3987 bytes) 2024-09-18T04:56:33.889 INFO:teuthology.orchestra.run.smithi002.stdout:853 writesame 0x3e80ee9 thru 0x3e90535 data_size 0x1(0xf64d bytes) 2024-09-18T04:56:33.926 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:33.924+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:33.932 INFO:teuthology.orchestra.run.smithi002.stdout:855 write 0xe454f54 thru 0xe45689d (0x194a bytes) 2024-09-18T04:56:33.944 INFO:teuthology.orchestra.run.smithi002.stdout:856 trunc from 0xe45689e to 0x5a1fe38 2024-09-18T04:56:33.971 INFO:teuthology.orchestra.run.smithi002.stdout:858 write 0xb03a668 thru 0xb046c84 (0xc61d bytes) 2024-09-18T04:56:33.973 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:33.982 INFO:teuthology.orchestra.run.smithi002.stdout:862 write 0xdbbfd04 thru 0xdbcc02d (0xc32a bytes) 2024-09-18T04:56:33.984 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:33.985 INFO:teuthology.orchestra.run.smithi002.stdout:864 read 0x7d82547 thru 0x7d8c952 (0xa40c bytes) 2024-09-18T04:56:33.985 INFO:teuthology.orchestra.run.smithi002.stdout:865 read 0x582be2 thru 0x589b98 (0x6fb7 bytes) 2024-09-18T04:56:33.990 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:33.990 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:33.991 INFO:teuthology.orchestra.run.smithi002.stdout:866 write 0xe3eca12 thru 0xe3fc63e (0xfc2d bytes) 2024-09-18T04:56:33.994 INFO:teuthology.orchestra.run.smithi002.stdout:867 read 0x4fc32d5 thru 0x4fc5336 (0x2062 bytes) 2024-09-18T04:56:33.994 INFO:teuthology.orchestra.run.smithi002.stdout:868 read 0x4ccf66 thru 0x4d84f3 (0xb58e bytes) 2024-09-18T04:56:33.995 INFO:teuthology.orchestra.run.smithi002.stdout:869 write 0x4c41bc7 thru 0x4c4a368 (0x87a2 bytes) 2024-09-18T04:56:34.000 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:56:34.004 INFO:teuthology.orchestra.run.smithi002.stdout:870 trunc from 0xe3fc63f to 0x7725414 2024-09-18T04:56:34.015 INFO:teuthology.orchestra.run.smithi002.stdout:871 write 0x1ed43c1 thru 0x1eda984 (0x65c4 bytes) 2024-09-18T04:56:34.017 INFO:teuthology.orchestra.run.smithi002.stdout:875 read 0x78f9ac thru 0x7908a6 (0xefb bytes) 2024-09-18T04:56:34.018 INFO:teuthology.orchestra.run.smithi002.stdout:876 trunc from 0x7725414 to 0x6727f95 2024-09-18T04:56:34.027 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:34.024+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:34.029 INFO:teuthology.orchestra.run.smithi002.stdout:877 writesame 0x82493f thru 0x82cdd3 data_size 0x1(0x8495 bytes) 2024-09-18T04:56:34.045 INFO:teuthology.orchestra.run.smithi002.stdout:878 clone 9 order 22 su 262144 sc 6 2024-09-18T04:56:34.110 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:34.110 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:34.119 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:56:34.128 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:34.126+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:34.180 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T04:56:34.248 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:34.245+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:34.260 INFO:teuthology.orchestra.run.smithi002.stdout:leaving image image_client.0-clone8 intact 2024-09-18T04:56:34.277 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:34.277 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:34.288 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:56:34.330 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:34.327+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:34.416 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-09-18T04:56:34.430 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:34.430 INFO:teuthology.orchestra.run.smithi002.stdout:881 read 0x55846a5 thru 0x558e7bd (0xa119 bytes) 2024-09-18T04:56:34.430 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:34.428+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:34.431 INFO:teuthology.orchestra.run.smithi002.stdout:882 read 0x5401000 thru 0x540efb9 (0xdfba bytes) 2024-09-18T04:56:34.432 INFO:teuthology.orchestra.run.smithi002.stdout:883 write 0x534edc2 thru 0x535579e (0x69dd bytes) 2024-09-18T04:56:34.440 INFO:teuthology.orchestra.run.smithi002.stdout:885 writesame 0x1421354 thru 0x142cc92 data_size 0x1(0xb93f bytes) 2024-09-18T04:56:34.445 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:34.446 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:34.455 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:56:34.467 INFO:teuthology.orchestra.run.smithi002.stdout:887 punch from 0x31fb9b9 to 0x320327b, (0x78c2 bytes) 2024-09-18T04:56:34.470 INFO:teuthology.orchestra.run.smithi002.stdout:888 trunc from 0x6727f95 to 0x95c7354 2024-09-18T04:56:34.482 INFO:teuthology.orchestra.run.smithi002.stdout:889 write 0xa2a4aba thru 0xa2afec9 (0xb410 bytes) 2024-09-18T04:56:34.485 INFO:teuthology.orchestra.run.smithi002.stdout:891 read 0x17bab16 thru 0x17c393e (0x8e29 bytes) 2024-09-18T04:56:34.490 INFO:teuthology.orchestra.run.smithi002.stdout:893 trunc from 0xa2afeca to 0x6fe0c23 2024-09-18T04:56:34.510 INFO:teuthology.orchestra.run.smithi002.stdout:894 read 0x5b37f6 thru 0x5bc79e (0x8fa9 bytes) 2024-09-18T04:56:34.516 INFO:teuthology.orchestra.run.smithi002.stdout:896 write 0x7336e2f thru 0x733e102 (0x72d4 bytes) 2024-09-18T04:56:34.518 INFO:teuthology.orchestra.run.smithi002.stdout:898 read 0x10e312 thru 0x113905 (0x55f4 bytes) 2024-09-18T04:56:34.518 INFO:teuthology.orchestra.run.smithi002.stdout:899 read 0x4830128 thru 0x483e4cd (0xe3a6 bytes) 2024-09-18T04:56:34.528 INFO:teuthology.orchestra.run.smithi002.stdout:901 write 0xa22281d thru 0xa22675e (0x3f42 bytes) 2024-09-18T04:56:34.530 INFO:teuthology.orchestra.run.smithi002.stdout:902 writesame 0x86b360c thru 0x86b8e9d data_size 0x3(0x5892 bytes) 2024-09-18T04:56:34.531 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:34.528+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:34.535 INFO:teuthology.orchestra.run.smithi002.stdout:903 punch from 0x3b5063e to 0x3b5a7c6, (0xa188 bytes) 2024-09-18T04:56:34.540 INFO:teuthology.orchestra.run.smithi002.stdout:904 trunc from 0xa22675f to 0xddfe43a 2024-09-18T04:56:34.547 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:34.560 INFO:teuthology.orchestra.run.smithi002.stdout:907 trunc from 0xddfe43a to 0x2675f70 2024-09-18T04:56:34.583 INFO:teuthology.orchestra.run.smithi002.stdout:908 read 0xc9cb6 thru 0xd17d8 (0x7b23 bytes) 2024-09-18T04:56:34.596 INFO:teuthology.orchestra.run.smithi002.stdout:909 trunc from 0x2675f70 to 0xe1d226a 2024-09-18T04:56:34.598 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:34.598 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:34.602 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:34.602 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:34.602 INFO:teuthology.orchestra.run.smithi002.stdout:912 read 0xde54727 thru 0xde63492 (0xed6c bytes) 2024-09-18T04:56:34.602 INFO:teuthology.orchestra.run.smithi002.stdout:913 punch from 0x49f262 to 0x49faae, (0x84c bytes) 2024-09-18T04:56:34.602 INFO:teuthology.orchestra.run.smithi002.stdout:917 punch from 0x8cf67ef to 0x8d05c6b, (0xf47c bytes) 2024-09-18T04:56:34.602 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:34.602 INFO:teuthology.orchestra.run.smithi002.stdout:922 write 0x2730795 thru 0x273c881 (0xc0ed bytes) 2024-09-18T04:56:34.603 INFO:teuthology.orchestra.run.smithi002.stdout:924 write 0x90f71d4 thru 0x9101af5 (0xa922 bytes) 2024-09-18T04:56:34.607 INFO:teuthology.orchestra.run.smithi002.stdout:925 punch from 0x829344e to 0x8298001, (0x4bb3 bytes) 2024-09-18T04:56:34.608 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:56:34.608 INFO:teuthology.orchestra.run.smithi002.stdout:927 writesame 0x240da51 thru 0x241ca88 data_size 0x4(0xf038 bytes) 2024-09-18T04:56:34.615 INFO:teuthology.orchestra.run.smithi002.stdout:928 punch from 0x172559c to 0x1727661, (0x20c5 bytes) 2024-09-18T04:56:34.615 INFO:teuthology.orchestra.run.smithi002.stdout:931 write 0x17bb8a4 thru 0x17ca6a6 (0xee03 bytes) 2024-09-18T04:56:34.620 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:34.620 INFO:teuthology.orchestra.run.smithi002.stdout:933 read 0xf49de8 thru 0xf59d2d (0xff46 bytes) 2024-09-18T04:56:34.632 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:34.629+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:34.632 INFO:teuthology.orchestra.run.smithi002.stdout:934 trunc from 0xe1d226a to 0x480170f 2024-09-18T04:56:34.641 INFO:teuthology.orchestra.run.smithi002.stdout:935 read 0x109b4be thru 0x10a5a92 (0xa5d5 bytes) 2024-09-18T04:56:34.642 INFO:teuthology.orchestra.run.smithi002.stdout:937 read 0x2baea26 thru 0x2bb59ee (0x6fc9 bytes) 2024-09-18T04:56:34.642 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:34.652 INFO:teuthology.orchestra.run.smithi002.stdout:939 trunc from 0x480170f to 0xd0e2a62 2024-09-18T04:56:34.657 INFO:teuthology.orchestra.run.smithi002.stdout:940 read 0x3b93ebc thru 0x3b96bf2 (0x2d37 bytes) 2024-09-18T04:56:34.657 INFO:teuthology.orchestra.run.smithi002.stdout:941 write 0xb583684 thru 0xb591ff3 (0xe970 bytes) 2024-09-18T04:56:34.666 INFO:teuthology.orchestra.run.smithi002.stdout:942 writesame 0xda312e8 thru 0xda3e187 data_size 0x3a(0xcea0 bytes) 2024-09-18T04:56:34.666 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:34.670 INFO:teuthology.orchestra.run.smithi002.stdout:945 write 0x9da29d0 thru 0x9dab46d (0x8a9e bytes) 2024-09-18T04:56:34.670 INFO:teuthology.orchestra.run.smithi002.stdout:946 read 0x8c0c377 thru 0x8c14d29 (0x89b3 bytes) 2024-09-18T04:56:34.670 INFO:teuthology.orchestra.run.smithi002.stdout:948 read 0xb68450c thru 0xb68bd2d (0x7822 bytes) 2024-09-18T04:56:34.670 INFO:teuthology.orchestra.run.smithi002.stdout:949 writesame 0x38057c0 thru 0x38119db data_size 0x65(0xc21c bytes) 2024-09-18T04:56:34.671 INFO:teuthology.orchestra.run.smithi002.stdout:950 writesame 0x5151743 thru 0x5153092 data_size 0x3c(0x1950 bytes) 2024-09-18T04:56:34.675 INFO:teuthology.orchestra.run.smithi002.stdout:951 write 0x96c598a thru 0x96d2ece (0xd545 bytes) 2024-09-18T04:56:34.677 INFO:teuthology.orchestra.run.smithi002.stdout:952 read 0xd906d27 thru 0xd910691 (0x996b bytes) 2024-09-18T04:56:34.677 INFO:teuthology.orchestra.run.smithi002.stdout:953 read 0xa1bc779 thru 0xa1c74a3 (0xad2b bytes) 2024-09-18T04:56:34.678 INFO:teuthology.orchestra.run.smithi002.stdout:954 punch from 0xcae6ddd to 0xcaf4321, (0xd544 bytes) 2024-09-18T04:56:34.678 INFO:teuthology.orchestra.run.smithi002.stdout:956 write 0x144b76 thru 0x14dfef (0x947a bytes) 2024-09-18T04:56:34.679 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:34.679 INFO:teuthology.orchestra.run.smithi002.stdout:958 write 0x68480ea thru 0x685523d (0xd154 bytes) 2024-09-18T04:56:34.682 INFO:teuthology.orchestra.run.smithi002.stdout:959 write 0x88c6269 thru 0x88c9c22 (0x39ba bytes) 2024-09-18T04:56:34.685 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:34.685 INFO:teuthology.orchestra.run.smithi002.stdout:961 trunc from 0xda3e188 to 0xd2ceb8c 2024-09-18T04:56:34.690 INFO:teuthology.orchestra.run.smithi002.stdout:964 writesame 0xc9bd929 thru 0xc9c9613 data_size 0x8d(0xbceb bytes) 2024-09-18T04:56:34.698 INFO:teuthology.orchestra.run.smithi002.stdout:965 trunc from 0xd2ceb8c to 0x97d118e 2024-09-18T04:56:34.709 INFO:teuthology.orchestra.run.smithi002.stdout:966 trunc from 0x97d118e to 0xa0a468c 2024-09-18T04:56:34.714 INFO:teuthology.orchestra.run.smithi002.stdout:967 read 0x9ecdfff thru 0x9edc43a (0xe43c bytes) 2024-09-18T04:56:34.714 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:34.714 INFO:teuthology.orchestra.run.smithi002.stdout:969 write 0x94b657e thru 0x94c305c (0xcadf bytes) 2024-09-18T04:56:34.717 INFO:teuthology.orchestra.run.smithi002.stdout:970 clone 10 order 20 su 4096 sc 12 2024-09-18T04:56:34.733 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:34.730+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:34.738 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:34.738 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:34.746 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:56:34.834 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:34.832+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:34.894 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:34.894 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:34.902 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:56:34.944 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:34.941+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:35.036 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:35.034+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:35.050 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:35.050 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:35.059 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:56:35.136 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:35.134+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:35.165 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:35.165 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:35.174 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:56:35.175 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-18T04:56:35.272 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:35.270+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:35.273 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone9 from 0xa0a468c (overlap 0x2675f70) to 0x0 2024-09-18T04:56:35.338 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:35.336+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:35.352 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:35.352 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:35.361 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:56:35.397 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-09-18T04:56:35.441 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:35.439+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:35.512 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:35.512 INFO:teuthology.orchestra.run.smithi002.stdout:972 punch from 0x7c65719 to 0x7c6ec1c, (0x9503 bytes) 2024-09-18T04:56:35.512 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:35.513 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:35.516 INFO:teuthology.orchestra.run.smithi002.stdout:974 punch from 0x7f7acf7 to 0x7f83d6e, (0x9077 bytes) 2024-09-18T04:56:35.516 INFO:teuthology.orchestra.run.smithi002.stdout:975 read 0x6586604 thru 0x659059a (0x9f97 bytes) 2024-09-18T04:56:35.516 INFO:teuthology.orchestra.run.smithi002.stdout:976 read 0x7963afc thru 0x7967cfa (0x41ff bytes) 2024-09-18T04:56:35.523 INFO:teuthology.orchestra.run.smithi002.stdout:977 write 0xb622604 thru 0xb622869 (0x266 bytes) 2024-09-18T04:56:35.524 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:56:35.526 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:35.526 INFO:teuthology.orchestra.run.smithi002.stdout:980 write 0x6c3e4ec thru 0x6c4d9d3 (0xf4e8 bytes) 2024-09-18T04:56:35.568 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:56:35.569 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:35.567+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:35.602 INFO:teuthology.orchestra.run.smithi002.stdout:982 write 0xde1e6b8 thru 0xde25e1c (0x7765 bytes) 2024-09-18T04:56:35.618 INFO:teuthology.orchestra.run.smithi002.stdout:983 trunc from 0xde25e1d to 0x83bda02 2024-09-18T04:56:35.641 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:35.638+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:35.653 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:35.653 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:35.663 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:56:35.717 INFO:teuthology.orchestra.run.smithi002.stdout:984 trunc from 0x83bda02 to 0x32f5aa7 2024-09-18T04:56:35.719 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:56:35.719 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:56:35.741 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:35.739+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:35.743 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:56:35.743 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:56:35.766 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:56:35.766 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:56:35.799 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:56:35.799 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:56:35.824 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:56:35.825 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:56:35.828 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:35.828 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:35.841 INFO:teuthology.orchestra.run.smithi002.stdout:987 write 0xa1a0f79 thru 0xa1a3fa4 (0x302c bytes) 2024-09-18T04:56:35.842 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:35.839+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:35.851 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:35.851 INFO:teuthology.orchestra.run.smithi002.stdout:990 read 0x48f7499 thru 0x4901151 (0x9cb9 bytes) 2024-09-18T04:56:35.852 INFO:teuthology.orchestra.run.smithi002.stdout:991 writesame 0x6c76a54 thru 0x6c7da9d data_size 0x9(0x704a bytes) 2024-09-18T04:56:35.857 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:56:35.857 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:56:35.860 INFO:teuthology.orchestra.run.smithi002.stdout:992 read 0x2946681 thru 0x2952bea (0xc56a bytes) 2024-09-18T04:56:35.863 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:35.864 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:56:35.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:56:35.864 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:56:35.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.864 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:855 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:35.864 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.375028+0000", 2024-09-18T04:56:35.864 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.4843245379999992, 2024-09-18T04:56:35.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0063269789999999999, 2024-09-18T04:56:35.864 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:35.864 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:35.864 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:35.864 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:35.865 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:35.865 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 855 2024-09-18T04:56:35.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.865 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:35.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:35.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375028+0000", 2024-09-18T04:56:35.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:35.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375028+0000", 2024-09-18T04:56:35.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:35.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375027+0000", 2024-09-18T04:56:35.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:35.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:35.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375033+0000", 2024-09-18T04:56:35.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9869999999999996e-06 2024-09-18T04:56:35.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:35.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375042+0000", 2024-09-18T04:56:35.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3899999999999999e-06 2024-09-18T04:56:35.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:35.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375051+0000", 2024-09-18T04:56:35.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2839999999999995e-06 2024-09-18T04:56:35.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:35.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.377760+0000", 2024-09-18T04:56:35.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027089869999999999 2024-09-18T04:56:35.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:35.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.377862+0000", 2024-09-18T04:56:35.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010184200000000001 2024-09-18T04:56:35.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:35.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.378261+0000", 2024-09-18T04:56:35.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00039838900000000002 2024-09-18T04:56:35.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:35.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.380468+0000", 2024-09-18T04:56:35.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002207156 2024-09-18T04:56:35.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:35.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381284+0000", 2024-09-18T04:56:35.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00081592199999999996 2024-09-18T04:56:35.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:35.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381332+0000", 2024-09-18T04:56:35.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8047000000000003e-05 2024-09-18T04:56:35.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:35.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381355+0000", 2024-09-18T04:56:35.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2802999999999998e-05 2024-09-18T04:56:35.870 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.870 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:35.870 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.870 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:856 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:35.870 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.375180+0000", 2024-09-18T04:56:35.870 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.484171688, 2024-09-18T04:56:35.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0063488080000000001, 2024-09-18T04:56:35.871 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:35.871 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:35.871 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:35.871 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:35.871 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:35.871 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 856 2024-09-18T04:56:35.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.871 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:35.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:35.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375180+0000", 2024-09-18T04:56:35.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:35.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375180+0000", 2024-09-18T04:56:35.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:35.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375180+0000", 2024-09-18T04:56:35.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:35.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:35.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375184+0000", 2024-09-18T04:56:35.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6530000000000002e-06 2024-09-18T04:56:35.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:35.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375202+0000", 2024-09-18T04:56:35.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8099000000000001e-05 2024-09-18T04:56:35.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:35.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375209+0000", 2024-09-18T04:56:35.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7469999999999999e-06 2024-09-18T04:56:35.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:35.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.378708+0000", 2024-09-18T04:56:35.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0034990329999999999 2024-09-18T04:56:35.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:35.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.378801+0000", 2024-09-18T04:56:35.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2967999999999995e-05 2024-09-18T04:56:35.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:35.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.379196+0000", 2024-09-18T04:56:35.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00039495699999999998 2024-09-18T04:56:35.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:35.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381369+0000", 2024-09-18T04:56:35.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021726559999999998 2024-09-18T04:56:35.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:35.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381422+0000", 2024-09-18T04:56:35.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3523000000000002e-05 2024-09-18T04:56:35.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:35.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381469+0000", 2024-09-18T04:56:35.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7107999999999997e-05 2024-09-18T04:56:35.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:35.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381529+0000", 2024-09-18T04:56:35.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9846000000000003e-05 2024-09-18T04:56:35.877 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.877 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:35.877 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.877 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:857 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:35.878 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.375218+0000", 2024-09-18T04:56:35.878 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.4841336409999997, 2024-09-18T04:56:35.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006463809, 2024-09-18T04:56:35.878 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:35.878 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:35.878 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:35.878 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:35.878 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:35.878 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 857 2024-09-18T04:56:35.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.878 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:35.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:35.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375218+0000", 2024-09-18T04:56:35.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:35.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375218+0000", 2024-09-18T04:56:35.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:35.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375218+0000", 2024-09-18T04:56:35.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:35.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:35.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375222+0000", 2024-09-18T04:56:35.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.704e-06 2024-09-18T04:56:35.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:35.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375231+0000", 2024-09-18T04:56:35.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3999999999999998e-06 2024-09-18T04:56:35.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:35.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375237+0000", 2024-09-18T04:56:35.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5169999999999999e-06 2024-09-18T04:56:35.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:35.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.379666+0000", 2024-09-18T04:56:35.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0044297390000000002 2024-09-18T04:56:35.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:35.882 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.379747+0000", 2024-09-18T04:56:35.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0316999999999997e-05 2024-09-18T04:56:35.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:35.882 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.380138+0000", 2024-09-18T04:56:35.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00039156599999999998 2024-09-18T04:56:35.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:35.882 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381578+0000", 2024-09-18T04:56:35.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001439423 2024-09-18T04:56:35.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:35.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381587+0000", 2024-09-18T04:56:35.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2499999999999995e-06 2024-09-18T04:56:35.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:35.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381641+0000", 2024-09-18T04:56:35.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3961000000000001e-05 2024-09-18T04:56:35.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:35.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381682+0000", 2024-09-18T04:56:35.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1454e-05 2024-09-18T04:56:35.884 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.884 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:35.885 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.885 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:860 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:35.885 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.375377+0000", 2024-09-18T04:56:35.885 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.4839753560000002, 2024-09-18T04:56:35.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0066062229999999996, 2024-09-18T04:56:35.885 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:35.885 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:35.886 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:35.886 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:35.886 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:35.886 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 860 2024-09-18T04:56:35.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.886 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:35.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:35.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375377+0000", 2024-09-18T04:56:35.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:35.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375377+0000", 2024-09-18T04:56:35.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:35.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375375+0000", 2024-09-18T04:56:35.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:35.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:35.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375382+0000", 2024-09-18T04:56:35.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0280000000000003e-06 2024-09-18T04:56:35.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:35.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375393+0000", 2024-09-18T04:56:35.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0518e-05 2024-09-18T04:56:35.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:35.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.375401+0000", 2024-09-18T04:56:35.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1680000000000004e-06 2024-09-18T04:56:35.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:35.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.380505+0000", 2024-09-18T04:56:35.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0051037920000000002 2024-09-18T04:56:35.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:35.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.380589+0000", 2024-09-18T04:56:35.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4259999999999996e-05 2024-09-18T04:56:35.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:35.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.380912+0000", 2024-09-18T04:56:35.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000322763 2024-09-18T04:56:35.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:35.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381694+0000", 2024-09-18T04:56:35.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00078214099999999998 2024-09-18T04:56:35.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:35.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381927+0000", 2024-09-18T04:56:35.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000233248 2024-09-18T04:56:35.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:35.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381961+0000", 2024-09-18T04:56:35.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3691e-05 2024-09-18T04:56:35.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:35.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.381983+0000", 2024-09-18T04:56:35.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1968999999999999e-05 2024-09-18T04:56:35.894 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.894 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:35.894 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.895 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:1132 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:35.895 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.074574+0000", 2024-09-18T04:56:35.895 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.7847779969999999, 2024-09-18T04:56:35.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0059714329999999999, 2024-09-18T04:56:35.895 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:35.895 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:35.895 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:35.895 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:35.895 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:35.895 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1132 2024-09-18T04:56:35.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.895 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:35.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:35.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074574+0000", 2024-09-18T04:56:35.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:35.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074574+0000", 2024-09-18T04:56:35.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:35.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074574+0000", 2024-09-18T04:56:35.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:56:35.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:35.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074575+0000", 2024-09-18T04:56:35.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4869999999999999e-06 2024-09-18T04:56:35.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:35.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074579+0000", 2024-09-18T04:56:35.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.14e-06 2024-09-18T04:56:35.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:35.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074581+0000", 2024-09-18T04:56:35.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4549999999999998e-06 2024-09-18T04:56:35.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:35.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.077782+0000", 2024-09-18T04:56:35.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0032005779999999999 2024-09-18T04:56:35.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:35.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.077867+0000", 2024-09-18T04:56:35.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5494999999999994e-05 2024-09-18T04:56:35.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:35.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.078233+0000", 2024-09-18T04:56:35.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00036551800000000001 2024-09-18T04:56:35.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:35.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.080412+0000", 2024-09-18T04:56:35.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021793630000000001 2024-09-18T04:56:35.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:35.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.080464+0000", 2024-09-18T04:56:35.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2024000000000002e-05 2024-09-18T04:56:35.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:35.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.080510+0000", 2024-09-18T04:56:35.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6393000000000001e-05 2024-09-18T04:56:35.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:35.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.080546+0000", 2024-09-18T04:56:35.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5206000000000001e-05 2024-09-18T04:56:35.901 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.901 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:35.901 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.901 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:1133 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:35.901 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.074583+0000", 2024-09-18T04:56:35.901 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.7847688579999996, 2024-09-18T04:56:35.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0060199470000000003, 2024-09-18T04:56:35.901 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:35.902 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:35.902 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:35.902 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:35.902 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:35.902 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1133 2024-09-18T04:56:35.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.903 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:35.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:35.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074583+0000", 2024-09-18T04:56:35.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:35.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074583+0000", 2024-09-18T04:56:35.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:35.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074583+0000", 2024-09-18T04:56:35.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:56:35.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:35.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074585+0000", 2024-09-18T04:56:35.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7209999999999999e-06 2024-09-18T04:56:35.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:35.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074588+0000", 2024-09-18T04:56:35.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0060000000000001e-06 2024-09-18T04:56:35.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:35.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074591+0000", 2024-09-18T04:56:35.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1259999999999998e-06 2024-09-18T04:56:35.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:35.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.078883+0000", 2024-09-18T04:56:35.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004291614 2024-09-18T04:56:35.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:35.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.078949+0000", 2024-09-18T04:56:35.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6367999999999999e-05 2024-09-18T04:56:35.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:35.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.079191+0000", 2024-09-18T04:56:35.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000242112 2024-09-18T04:56:35.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:35.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.080456+0000", 2024-09-18T04:56:35.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001265033 2024-09-18T04:56:35.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:35.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.080553+0000", 2024-09-18T04:56:35.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7374999999999995e-05 2024-09-18T04:56:35.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:35.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.080583+0000", 2024-09-18T04:56:35.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9677999999999999e-05 2024-09-18T04:56:35.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:35.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.080603+0000", 2024-09-18T04:56:35.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.014e-05 2024-09-18T04:56:35.909 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.909 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:35.909 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.909 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:1134 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:35.909 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.074594+0000", 2024-09-18T04:56:35.909 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.7847579869999999, 2024-09-18T04:56:35.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0076433650000000001, 2024-09-18T04:56:35.910 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:35.910 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:35.910 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:35.910 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:35.910 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:35.910 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1134 2024-09-18T04:56:35.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.910 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:35.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:35.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074594+0000", 2024-09-18T04:56:35.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:35.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074594+0000", 2024-09-18T04:56:35.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:35.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074594+0000", 2024-09-18T04:56:35.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:56:35.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:35.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074596+0000", 2024-09-18T04:56:35.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.86e-06 2024-09-18T04:56:35.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:35.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074599+0000", 2024-09-18T04:56:35.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1240000000000001e-06 2024-09-18T04:56:35.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:35.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074601+0000", 2024-09-18T04:56:35.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.407e-06 2024-09-18T04:56:35.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:35.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.079611+0000", 2024-09-18T04:56:35.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0050095080000000002 2024-09-18T04:56:35.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:35.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.079702+0000", 2024-09-18T04:56:35.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0915000000000006e-05 2024-09-18T04:56:35.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:35.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.080046+0000", 2024-09-18T04:56:35.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00034459399999999999 2024-09-18T04:56:35.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:35.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.082150+0000", 2024-09-18T04:56:35.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00210392 2024-09-18T04:56:35.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:35.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.082161+0000", 2024-09-18T04:56:35.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0383999999999999e-05 2024-09-18T04:56:35.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:35.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.082203+0000", 2024-09-18T04:56:35.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2542e-05 2024-09-18T04:56:35.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:35.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.082238+0000", 2024-09-18T04:56:35.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4345999999999999e-05 2024-09-18T04:56:35.916 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.916 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:35.916 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.917 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:1135 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:35.917 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.074614+0000", 2024-09-18T04:56:35.917 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.7847386079999996, 2024-09-18T04:56:35.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0093099629999999992, 2024-09-18T04:56:35.917 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:35.917 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:35.917 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:35.917 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:35.917 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:35.918 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1135 2024-09-18T04:56:35.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.918 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:35.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:35.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074614+0000", 2024-09-18T04:56:35.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:35.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074614+0000", 2024-09-18T04:56:35.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:35.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074613+0000", 2024-09-18T04:56:35.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:35.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:35.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074615+0000", 2024-09-18T04:56:35.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0030000000000001e-06 2024-09-18T04:56:35.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:35.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074619+0000", 2024-09-18T04:56:35.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8870000000000002e-06 2024-09-18T04:56:35.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:35.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074624+0000", 2024-09-18T04:56:35.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8339999999999998e-06 2024-09-18T04:56:35.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:35.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.080635+0000", 2024-09-18T04:56:35.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0060110900000000002 2024-09-18T04:56:35.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:35.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.080721+0000", 2024-09-18T04:56:35.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5717000000000006e-05 2024-09-18T04:56:35.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:35.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.080956+0000", 2024-09-18T04:56:35.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000234859 2024-09-18T04:56:35.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:35.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.083773+0000", 2024-09-18T04:56:35.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028177580000000001 2024-09-18T04:56:35.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:35.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.083831+0000", 2024-09-18T04:56:35.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8118999999999997e-05 2024-09-18T04:56:35.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:35.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.083881+0000", 2024-09-18T04:56:35.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9845000000000001e-05 2024-09-18T04:56:35.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:35.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.083923+0000", 2024-09-18T04:56:35.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2160999999999998e-05 2024-09-18T04:56:35.923 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.923 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:35.923 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.923 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:1136 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:35.923 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.074627+0000", 2024-09-18T04:56:35.923 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.7847254660000003, 2024-09-18T04:56:35.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0094502679999999995, 2024-09-18T04:56:35.924 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:35.924 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:35.924 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:35.924 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:35.924 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:35.924 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1136 2024-09-18T04:56:35.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.924 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:35.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:35.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074627+0000", 2024-09-18T04:56:35.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:35.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074627+0000", 2024-09-18T04:56:35.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:35.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074626+0000", 2024-09-18T04:56:35.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:56:35.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:35.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074628+0000", 2024-09-18T04:56:35.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.951e-06 2024-09-18T04:56:35.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:35.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074632+0000", 2024-09-18T04:56:35.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2150000000000001e-06 2024-09-18T04:56:35.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:35.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074634+0000", 2024-09-18T04:56:35.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4159999999999998e-06 2024-09-18T04:56:35.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:35.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.081336+0000", 2024-09-18T04:56:35.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067015340000000003 2024-09-18T04:56:35.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:35.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.081411+0000", 2024-09-18T04:56:35.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5067999999999993e-05 2024-09-18T04:56:35.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:35.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.081740+0000", 2024-09-18T04:56:35.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00032918800000000001 2024-09-18T04:56:35.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:35.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.083985+0000", 2024-09-18T04:56:35.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022451189999999998 2024-09-18T04:56:35.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:35.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.083990+0000", 2024-09-18T04:56:35.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.356e-06 2024-09-18T04:56:35.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:35.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084040+0000", 2024-09-18T04:56:35.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9492e-05 2024-09-18T04:56:35.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:35.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084077+0000", 2024-09-18T04:56:35.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7156999999999997e-05 2024-09-18T04:56:35.929 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.929 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:35.929 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.929 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:1137 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:35.929 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.074636+0000", 2024-09-18T04:56:35.930 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.784715963, 2024-09-18T04:56:35.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0095487100000000002, 2024-09-18T04:56:35.930 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:35.930 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:35.930 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:35.930 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:35.930 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:35.930 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1137 2024-09-18T04:56:35.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.930 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:35.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:35.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074636+0000", 2024-09-18T04:56:35.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:35.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074636+0000", 2024-09-18T04:56:35.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:35.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074636+0000", 2024-09-18T04:56:35.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:56:35.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:35.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074639+0000", 2024-09-18T04:56:35.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7010000000000001e-06 2024-09-18T04:56:35.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:35.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074642+0000", 2024-09-18T04:56:35.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0410000000000002e-06 2024-09-18T04:56:35.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:35.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074644+0000", 2024-09-18T04:56:35.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4099999999999998e-06 2024-09-18T04:56:35.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:35.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.082287+0000", 2024-09-18T04:56:35.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0076424140000000002 2024-09-18T04:56:35.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:35.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.082372+0000", 2024-09-18T04:56:35.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5289000000000003e-05 2024-09-18T04:56:35.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:35.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.082705+0000", 2024-09-18T04:56:35.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000332951 2024-09-18T04:56:35.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:35.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084087+0000", 2024-09-18T04:56:35.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013827030000000001 2024-09-18T04:56:35.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:35.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084119+0000", 2024-09-18T04:56:35.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1139000000000001e-05 2024-09-18T04:56:35.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:35.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084158+0000", 2024-09-18T04:56:35.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9421000000000001e-05 2024-09-18T04:56:35.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:35.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084185+0000", 2024-09-18T04:56:35.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6855999999999998e-05 2024-09-18T04:56:35.937 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.937 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:35.937 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.937 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:1138 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:35.937 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.074647+0000", 2024-09-18T04:56:35.937 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.7847055019999996, 2024-09-18T04:56:35.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.010428664000000001, 2024-09-18T04:56:35.937 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:35.937 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:35.937 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:35.938 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:35.938 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:35.938 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1138 2024-09-18T04:56:35.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.938 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:35.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:35.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074647+0000", 2024-09-18T04:56:35.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:35.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074647+0000", 2024-09-18T04:56:35.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:35.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074646+0000", 2024-09-18T04:56:35.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:35.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:35.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074648+0000", 2024-09-18T04:56:35.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6479999999999999e-06 2024-09-18T04:56:35.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:35.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074655+0000", 2024-09-18T04:56:35.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8410000000000002e-06 2024-09-18T04:56:35.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:35.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074658+0000", 2024-09-18T04:56:35.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.678e-06 2024-09-18T04:56:35.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:35.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.083112+0000", 2024-09-18T04:56:35.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0084543150000000004 2024-09-18T04:56:35.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:35.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.083182+0000", 2024-09-18T04:56:35.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9782000000000004e-05 2024-09-18T04:56:35.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:35.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.083469+0000", 2024-09-18T04:56:35.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028766099999999998 2024-09-18T04:56:35.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:35.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084836+0000", 2024-09-18T04:56:35.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001366826 2024-09-18T04:56:35.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:35.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084997+0000", 2024-09-18T04:56:35.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016116500000000001 2024-09-18T04:56:35.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:35.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.085043+0000", 2024-09-18T04:56:35.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5234000000000001e-05 2024-09-18T04:56:35.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:35.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.085075+0000", 2024-09-18T04:56:35.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2753999999999998e-05 2024-09-18T04:56:35.943 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.943 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:35.943 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.944 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:1139 3.18 3:19eb27e0:::rbd_object_map.11ac33624de9:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:35.944 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.074660+0000", 2024-09-18T04:56:35.944 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.7846920060000002, 2024-09-18T04:56:35.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011003634999999999, 2024-09-18T04:56:35.944 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:35.944 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:35.944 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:35.944 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:35.944 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:35.945 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1139 2024-09-18T04:56:35.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.945 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:35.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:35.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074660+0000", 2024-09-18T04:56:35.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:35.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074660+0000", 2024-09-18T04:56:35.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:35.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074660+0000", 2024-09-18T04:56:35.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:56:35.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:35.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074661+0000", 2024-09-18T04:56:35.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.528e-06 2024-09-18T04:56:35.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:35.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074664+0000", 2024-09-18T04:56:35.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.039e-06 2024-09-18T04:56:35.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:35.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.074667+0000", 2024-09-18T04:56:35.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6029999999999999e-06 2024-09-18T04:56:35.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:35.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084216+0000", 2024-09-18T04:56:35.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0095484650000000008 2024-09-18T04:56:35.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:35.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084281+0000", 2024-09-18T04:56:35.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5046e-05 2024-09-18T04:56:35.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:35.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.084527+0000", 2024-09-18T04:56:35.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00024690600000000002 2024-09-18T04:56:35.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:35.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.085206+0000", 2024-09-18T04:56:35.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067853800000000001 2024-09-18T04:56:35.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:35.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.085581+0000", 2024-09-18T04:56:35.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00037512 2024-09-18T04:56:35.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:35.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.085629+0000", 2024-09-18T04:56:35.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7620000000000001e-05 2024-09-18T04:56:35.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:35.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.085664+0000", 2024-09-18T04:56:35.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4987999999999998e-05 2024-09-18T04:56:35.951 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.951 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:35.952 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.952 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:1279 3.18 3:1b790e41:::rbd_data.11ac99060bb1.0000000000000101:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:35.952 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.174431+0000", 2024-09-18T04:56:35.952 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.6849207159999997, 2024-09-18T04:56:35.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0053931070000000003, 2024-09-18T04:56:35.952 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:35.952 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:35.952 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:35.952 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:35.952 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:35.952 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1279 2024-09-18T04:56:35.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.953 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:35.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:35.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.174431+0000", 2024-09-18T04:56:35.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:35.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.174431+0000", 2024-09-18T04:56:35.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:35.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.174431+0000", 2024-09-18T04:56:35.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:35.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:35.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.174437+0000", 2024-09-18T04:56:35.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2580000000000001e-06 2024-09-18T04:56:35.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:35.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.174446+0000", 2024-09-18T04:56:35.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.448e-06 2024-09-18T04:56:35.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:35.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.174451+0000", 2024-09-18T04:56:35.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3800000000000004e-06 2024-09-18T04:56:35.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:35.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.174483+0000", 2024-09-18T04:56:35.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2190999999999997e-05 2024-09-18T04:56:35.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:35.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.174584+0000", 2024-09-18T04:56:35.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101208 2024-09-18T04:56:35.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:35.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.175626+0000", 2024-09-18T04:56:35.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001041859 2024-09-18T04:56:35.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:35.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.179711+0000", 2024-09-18T04:56:35.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00408545 2024-09-18T04:56:35.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:35.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.179723+0000", 2024-09-18T04:56:35.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1258e-05 2024-09-18T04:56:35.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:35.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.179785+0000", 2024-09-18T04:56:35.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2150999999999995e-05 2024-09-18T04:56:35.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:35.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.179825+0000", 2024-09-18T04:56:35.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9647000000000002e-05 2024-09-18T04:56:35.958 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.958 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:35.958 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.958 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:1467 3.7 3:e6f33ce6:::rbd_object_map.11ac99060bb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:35.959 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.247138+0000", 2024-09-18T04:56:35.959 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.6122144929999997, 2024-09-18T04:56:35.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0058561139999999999, 2024-09-18T04:56:35.959 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:35.959 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:35.959 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:35.959 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:35.959 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:35.959 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1467 2024-09-18T04:56:35.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.959 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:35.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:35.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.247138+0000", 2024-09-18T04:56:35.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:35.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.247138+0000", 2024-09-18T04:56:35.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:35.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.247137+0000", 2024-09-18T04:56:35.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:35.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:35.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.247142+0000", 2024-09-18T04:56:35.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9010000000000002e-06 2024-09-18T04:56:35.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:35.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.247153+0000", 2024-09-18T04:56:35.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1032e-05 2024-09-18T04:56:35.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:35.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.247159+0000", 2024-09-18T04:56:35.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8370000000000002e-06 2024-09-18T04:56:35.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:35.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.248305+0000", 2024-09-18T04:56:35.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001146453 2024-09-18T04:56:35.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:35.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.248375+0000", 2024-09-18T04:56:35.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9748999999999999e-05 2024-09-18T04:56:35.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-18T04:56:35.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.248728+0000", 2024-09-18T04:56:35.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00035267600000000001 2024-09-18T04:56:35.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:35.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.250710+0000", 2024-09-18T04:56:35.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019824869999999998 2024-09-18T04:56:35.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:35.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.252938+0000", 2024-09-18T04:56:35.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022282439999999999 2024-09-18T04:56:35.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:35.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.252968+0000", 2024-09-18T04:56:35.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9386999999999998e-05 2024-09-18T04:56:35.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:35.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.252994+0000", 2024-09-18T04:56:35.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6092e-05 2024-09-18T04:56:35.964 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.964 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:35.964 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.964 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:1611 3.7 3:e6f33ce6:::rbd_object_map.11ac99060bb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:35.964 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.388383+0000", 2024-09-18T04:56:35.965 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.4709694520000003, 2024-09-18T04:56:35.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.006597538, 2024-09-18T04:56:35.965 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:35.965 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:35.965 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:35.965 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:35.965 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:35.965 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 1611 2024-09-18T04:56:35.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.965 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:35.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:35.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.388383+0000", 2024-09-18T04:56:35.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:35.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.388383+0000", 2024-09-18T04:56:35.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:35.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.388382+0000", 2024-09-18T04:56:35.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:35.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:35.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.388387+0000", 2024-09-18T04:56:35.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6569999999999996e-06 2024-09-18T04:56:35.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:35.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.388392+0000", 2024-09-18T04:56:35.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.673e-06 2024-09-18T04:56:35.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:35.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.388395+0000", 2024-09-18T04:56:35.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9220000000000001e-06 2024-09-18T04:56:35.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:35.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.389502+0000", 2024-09-18T04:56:35.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011077400000000001 2024-09-18T04:56:35.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:35.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.389552+0000", 2024-09-18T04:56:35.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9861999999999998e-05 2024-09-18T04:56:35.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-18T04:56:35.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.389775+0000", 2024-09-18T04:56:35.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00022317699999999999 2024-09-18T04:56:35.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:35.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.390312+0000", 2024-09-18T04:56:35.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00053678000000000005 2024-09-18T04:56:35.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:35.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.394931+0000", 2024-09-18T04:56:35.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0046185100000000001 2024-09-18T04:56:35.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:35.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.394959+0000", 2024-09-18T04:56:35.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7994999999999999e-05 2024-09-18T04:56:35.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:35.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.394980+0000", 2024-09-18T04:56:35.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1472999999999999e-05 2024-09-18T04:56:35.971 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.971 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:35.971 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.971 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:2020 3.7 3:e6f33ce6:::rbd_object_map.11ac99060bb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:35.971 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.554716+0000", 2024-09-18T04:56:35.971 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.3046362250000003, 2024-09-18T04:56:35.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0069705610000000001, 2024-09-18T04:56:35.972 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:35.972 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:35.972 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:35.972 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:35.972 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:35.972 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2020 2024-09-18T04:56:35.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.972 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:35.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:35.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.554716+0000", 2024-09-18T04:56:35.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:35.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.554716+0000", 2024-09-18T04:56:35.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:35.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.554715+0000", 2024-09-18T04:56:35.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:35.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:35.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.554721+0000", 2024-09-18T04:56:35.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8379999999999998e-06 2024-09-18T04:56:35.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:35.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.554751+0000", 2024-09-18T04:56:35.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9896e-05 2024-09-18T04:56:35.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:35.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.554758+0000", 2024-09-18T04:56:35.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3599999999999998e-06 2024-09-18T04:56:35.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:35.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.555664+0000", 2024-09-18T04:56:35.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00090563299999999998 2024-09-18T04:56:35.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:35.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.555752+0000", 2024-09-18T04:56:35.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8358999999999999e-05 2024-09-18T04:56:35.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-18T04:56:35.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.556088+0000", 2024-09-18T04:56:35.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00033591099999999999 2024-09-18T04:56:35.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:35.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.557300+0000", 2024-09-18T04:56:35.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012117969999999999 2024-09-18T04:56:35.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:35.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.561500+0000", 2024-09-18T04:56:35.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0042000639999999999 2024-09-18T04:56:35.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:35.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.561605+0000", 2024-09-18T04:56:35.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010563100000000001 2024-09-18T04:56:35.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:35.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.561686+0000", 2024-09-18T04:56:35.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0965999999999997e-05 2024-09-18T04:56:35.977 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.977 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:35.977 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.978 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:2021 3.7 3:e6f33ce6:::rbd_object_map.11ac99060bb1:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:35.978 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.555332+0000", 2024-09-18T04:56:35.978 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.3040204439999998, 2024-09-18T04:56:35.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0064605660000000001, 2024-09-18T04:56:35.978 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:35.978 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:35.978 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:35.978 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:35.978 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:35.978 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2021 2024-09-18T04:56:35.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.978 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:35.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:35.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.555332+0000", 2024-09-18T04:56:35.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:35.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.555332+0000", 2024-09-18T04:56:35.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:35.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.555331+0000", 2024-09-18T04:56:35.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:35.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:35.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.555336+0000", 2024-09-18T04:56:35.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0150000000000003e-06 2024-09-18T04:56:35.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:35.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.555345+0000", 2024-09-18T04:56:35.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0310000000000004e-06 2024-09-18T04:56:35.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:35.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.555350+0000", 2024-09-18T04:56:35.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5280000000000003e-06 2024-09-18T04:56:35.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:35.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.556568+0000", 2024-09-18T04:56:35.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012179339999999999 2024-09-18T04:56:35.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:35.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.556642+0000", 2024-09-18T04:56:35.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3280000000000003e-05 2024-09-18T04:56:35.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-18T04:56:35.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.556926+0000", 2024-09-18T04:56:35.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000284739 2024-09-18T04:56:35.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:35.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.557667+0000", 2024-09-18T04:56:35.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00074095300000000003 2024-09-18T04:56:35.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:35.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.561722+0000", 2024-09-18T04:56:35.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0040548270000000004 2024-09-18T04:56:35.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:35.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.561750+0000", 2024-09-18T04:56:35.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7440000000000002e-05 2024-09-18T04:56:35.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:35.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.561792+0000", 2024-09-18T04:56:35.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2592999999999997e-05 2024-09-18T04:56:35.983 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.983 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:35.983 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.983 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:2138 3.7 3:e50557b3:::rbd_data.11ac99060bb1.00000000000001d1:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:35.983 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.728591+0000", 2024-09-18T04:56:35.984 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.1307607160000002, 2024-09-18T04:56:35.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0083980019999999999, 2024-09-18T04:56:35.984 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:35.984 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:35.984 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:35.984 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:35.984 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:35.984 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2138 2024-09-18T04:56:35.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.984 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:35.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:35.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.728591+0000", 2024-09-18T04:56:35.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:35.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.728591+0000", 2024-09-18T04:56:35.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:35.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.728591+0000", 2024-09-18T04:56:35.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:35.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:35.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.728595+0000", 2024-09-18T04:56:35.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.746e-06 2024-09-18T04:56:35.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:35.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.728606+0000", 2024-09-18T04:56:35.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0448999999999999e-05 2024-09-18T04:56:35.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:35.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.728613+0000", 2024-09-18T04:56:35.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4789999999999999e-06 2024-09-18T04:56:35.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:35.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.728642+0000", 2024-09-18T04:56:35.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9141000000000001e-05 2024-09-18T04:56:35.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:35.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.728722+0000", 2024-09-18T04:56:35.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9268000000000001e-05 2024-09-18T04:56:35.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-18T04:56:35.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.730157+0000", 2024-09-18T04:56:35.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014354179999999999 2024-09-18T04:56:35.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:35.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.736871+0000", 2024-09-18T04:56:35.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0067139469999999996 2024-09-18T04:56:35.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:35.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.736886+0000", 2024-09-18T04:56:35.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5056e-05 2024-09-18T04:56:35.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:35.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.736942+0000", 2024-09-18T04:56:35.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6153999999999998e-05 2024-09-18T04:56:35.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:35.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.736989+0000", 2024-09-18T04:56:35.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7184000000000001e-05 2024-09-18T04:56:35.989 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.989 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:35.989 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.989 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:2144 3.18 3:1d491e03:::rbd_data.11ac99060bb1.000000000000019f:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:35.989 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:28.744894+0000", 2024-09-18T04:56:35.989 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.114457883, 2024-09-18T04:56:35.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0073157459999999997, 2024-09-18T04:56:35.990 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:35.990 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:35.990 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:35.990 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:35.990 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:35.990 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 2144 2024-09-18T04:56:35.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.990 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:35.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:35.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.744894+0000", 2024-09-18T04:56:35.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:35.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.744894+0000", 2024-09-18T04:56:35.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:35.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.744893+0000", 2024-09-18T04:56:35.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:35.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:35.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.744898+0000", 2024-09-18T04:56:35.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9609999999999996e-06 2024-09-18T04:56:35.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:35.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.744908+0000", 2024-09-18T04:56:35.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2560000000000008e-06 2024-09-18T04:56:35.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:35.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.744912+0000", 2024-09-18T04:56:35.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3970000000000004e-06 2024-09-18T04:56:35.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:35.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.744957+0000", 2024-09-18T04:56:35.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5241999999999998e-05 2024-09-18T04:56:35.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:35.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.745068+0000", 2024-09-18T04:56:35.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000111118 2024-09-18T04:56:35.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:35.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.746476+0000", 2024-09-18T04:56:35.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014079489999999999 2024-09-18T04:56:35.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:35.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.752138+0000", 2024-09-18T04:56:35.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0056614259999999998 2024-09-18T04:56:35.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:35.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.752146+0000", 2024-09-18T04:56:35.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7520000000000002e-06 2024-09-18T04:56:35.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:35.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.752182+0000", 2024-09-18T04:56:35.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6003000000000001e-05 2024-09-18T04:56:35.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:35.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:28.752210+0000", 2024-09-18T04:56:35.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7529e-05 2024-09-18T04:56:35.995 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.995 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:35.995 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:35.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.996 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:4763 3.7 3:e2524419:::rbd_data.11aca1181477.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 10747904~4718592] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:35.996 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:32.245778+0000", 2024-09-18T04:56:35.996 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.6135738100000001, 2024-09-18T04:56:35.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0065135210000000004, 2024-09-18T04:56:35.996 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:35.996 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:35.996 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:35.996 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:35.996 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:35.996 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 4763 2024-09-18T04:56:35.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.996 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:35.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:35.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:32.245778+0000", 2024-09-18T04:56:35.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:35.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:32.245778+0000", 2024-09-18T04:56:35.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:35.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:35.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:32.245776+0000", 2024-09-18T04:56:35.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999981 2024-09-18T04:56:35.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:35.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:32.245786+0000", 2024-09-18T04:56:35.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8859999999999999e-06 2024-09-18T04:56:35.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:35.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:32.245803+0000", 2024-09-18T04:56:35.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6773e-05 2024-09-18T04:56:35.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:35.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:32.245812+0000", 2024-09-18T04:56:35.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6160000000000002e-06 2024-09-18T04:56:35.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:35.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:32.245863+0000", 2024-09-18T04:56:35.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0862000000000002e-05 2024-09-18T04:56:35.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:35.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:35.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:35.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:32.246011+0000", 2024-09-18T04:56:35.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014823399999999999 2024-09-18T04:56:35.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:36.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:36.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 7", 2024-09-18T04:56:36.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:32.246215+0000", 2024-09-18T04:56:36.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000204117 2024-09-18T04:56:36.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:36.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:36.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:56:36.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:32.246969+0000", 2024-09-18T04:56:36.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00075440700000000002 2024-09-18T04:56:36.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:36.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:36.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:36.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:32.252239+0000", 2024-09-18T04:56:36.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0052694320000000001 2024-09-18T04:56:36.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:36.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:36.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:36.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:32.252271+0000", 2024-09-18T04:56:36.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1869999999999998e-05 2024-09-18T04:56:36.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:36.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:36.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:36.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:32.252292+0000", 2024-09-18T04:56:36.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1168000000000001e-05 2024-09-18T04:56:36.001 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:36.001 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:36.001 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:36.001 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:36.002 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:36.002 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:36.002 INFO:teuthology.orchestra.run.smithi002.stdout:993 write 0x1f6dd68 thru 0x1f76b36 (0x8dcf bytes) 2024-09-18T04:56:36.003 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:56:36.003 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:56:36.004 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:56:36.004 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:56:36.005 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:56:36.006 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:36.003+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:36.007 INFO:teuthology.orchestra.run.smithi002.stdout:995 write 0xe0528be thru 0xe055667 (0x2daa bytes) 2024-09-18T04:56:36.007 INFO:teuthology.orchestra.run.smithi002.stdout:998 punch from 0x9ed4d9b to 0x9ee13ec, (0xc651 bytes) 2024-09-18T04:56:36.007 INFO:teuthology.orchestra.run.smithi002.stdout:999 punch from 0x11f1234 to 0x11f8757, (0x7523 bytes) 2024-09-18T04:56:36.007 INFO:teuthology.orchestra.run.smithi002.stdout:1000 write 0xcfce91a thru 0xcfdcf51 (0xe638 bytes) 2024-09-18T04:56:36.007 INFO:teuthology.orchestra.run.smithi002.stdout:1002 read 0x20c91d0 thru 0x20d65e5 (0xd416 bytes) 2024-09-18T04:56:36.011 INFO:teuthology.orchestra.run.smithi002.stdout:1003 trunc from 0xe055668 to 0x5809ff1 2024-09-18T04:56:36.024 INFO:teuthology.orchestra.run.smithi002.stdout:1004 read 0x37b5f47 thru 0x37be4b6 (0x8570 bytes) 2024-09-18T04:56:36.025 INFO:teuthology.orchestra.run.smithi002.stdout:1005 punch from 0x142a210 to 0x142dba5, (0x3995 bytes) 2024-09-18T04:56:36.025 INFO:teuthology.orchestra.run.smithi002.stdout:1006 flatten 2024-09-18T04:56:36.108 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:36.105+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:36.112 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:36.112 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:36.112 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:56:36.112 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:36.121 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:56:36.208 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:36.206+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:36.280 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:56:36.281 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:56:36.281 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:36.289 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:56:36.333 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:36.331+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:36.407 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:36.407 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:56:36.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:56:36.408 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:56:36.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.408 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:5082 3.1a 3:5d34c0ec:::rbd_data.11ac14fc1d07.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 26797226~22514] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:36.408 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.444662+0000", 2024-09-18T04:56:36.408 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.9597060329999998, 2024-09-18T04:56:36.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048360470000000003, 2024-09-18T04:56:36.408 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.408 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:36.408 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.409 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.409 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.409 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 5082 2024-09-18T04:56:36.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.409 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.444662+0000", 2024-09-18T04:56:36.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.444662+0000", 2024-09-18T04:56:36.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.444661+0000", 2024-09-18T04:56:36.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:36.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.469347+0000", 2024-09-18T04:56:36.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024685830999999998 2024-09-18T04:56:36.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.469368+0000", 2024-09-18T04:56:36.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1166999999999999e-05 2024-09-18T04:56:36.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.469377+0000", 2024-09-18T04:56:36.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5790000000000004e-06 2024-09-18T04:56:36.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.469420+0000", 2024-09-18T04:56:36.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3238999999999997e-05 2024-09-18T04:56:36.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.469526+0000", 2024-09-18T04:56:36.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010607699999999999 2024-09-18T04:56:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.469895+0000", 2024-09-18T04:56:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036904 2024-09-18T04:56:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.481413+0000", 2024-09-18T04:56:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011517583 2024-09-18T04:56:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:36.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.492856+0000", 2024-09-18T04:56:36.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011443511999999999 2024-09-18T04:56:36.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:36.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.492944+0000", 2024-09-18T04:56:36.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7835000000000002e-05 2024-09-18T04:56:36.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.493023+0000", 2024-09-18T04:56:36.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8617999999999998e-05 2024-09-18T04:56:36.414 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.415 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.415 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.415 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11063 3.9 3:940e577e:::rbd_data.11acf292c51a.0000000000000007:d [read 1785856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.415 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.886557+0000", 2024-09-18T04:56:36.415 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.51781136900000002, 2024-09-18T04:56:36.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032320949000000002, 2024-09-18T04:56:36.415 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.415 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.415 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.415 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.415 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.415 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11063 2024-09-18T04:56:36.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.416 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886557+0000", 2024-09-18T04:56:36.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886557+0000", 2024-09-18T04:56:36.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886556+0000", 2024-09-18T04:56:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886565+0000", 2024-09-18T04:56:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2390000000000002e-06 2024-09-18T04:56:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886571+0000", 2024-09-18T04:56:36.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6200000000000001e-06 2024-09-18T04:56:36.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886584+0000", 2024-09-18T04:56:36.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2656999999999999e-05 2024-09-18T04:56:36.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.918677+0000", 2024-09-18T04:56:36.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032092729 2024-09-18T04:56:36.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.918779+0000", 2024-09-18T04:56:36.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101878 2024-09-18T04:56:36.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.918878+0000", 2024-09-18T04:56:36.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9122000000000004e-05 2024-09-18T04:56:36.419 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.419 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.419 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.419 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11064 3.9 3:940e577e:::rbd_data.11acf292c51a.0000000000000007:d [read 1851392~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.886604+0000", 2024-09-18T04:56:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.51776460599999996, 2024-09-18T04:56:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032441968000000002, 2024-09-18T04:56:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11064 2024-09-18T04:56:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886604+0000", 2024-09-18T04:56:36.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886604+0000", 2024-09-18T04:56:36.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886603+0000", 2024-09-18T04:56:36.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886606+0000", 2024-09-18T04:56:36.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4540000000000001e-06 2024-09-18T04:56:36.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886612+0000", 2024-09-18T04:56:36.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.105e-06 2024-09-18T04:56:36.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886621+0000", 2024-09-18T04:56:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1609999999999992e-06 2024-09-18T04:56:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.918916+0000", 2024-09-18T04:56:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032295088 2024-09-18T04:56:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.918976+0000", 2024-09-18T04:56:36.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.033e-05 2024-09-18T04:56:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.919045+0000", 2024-09-18T04:56:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9189000000000005e-05 2024-09-18T04:56:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11065 3.9 3:940e577e:::rbd_data.11acf292c51a.0000000000000007:d [read 1900544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.886625+0000", 2024-09-18T04:56:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.51774306400000003, 2024-09-18T04:56:36.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032630876000000003, 2024-09-18T04:56:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11065 2024-09-18T04:56:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886625+0000", 2024-09-18T04:56:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886625+0000", 2024-09-18T04:56:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886625+0000", 2024-09-18T04:56:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886630+0000", 2024-09-18T04:56:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5300000000000004e-06 2024-09-18T04:56:36.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886636+0000", 2024-09-18T04:56:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6659999999999996e-06 2024-09-18T04:56:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886640+0000", 2024-09-18T04:56:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1999999999999996e-06 2024-09-18T04:56:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.919083+0000", 2024-09-18T04:56:36.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032442963999999998 2024-09-18T04:56:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.919172+0000", 2024-09-18T04:56:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9048000000000005e-05 2024-09-18T04:56:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.919256+0000", 2024-09-18T04:56:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3914999999999999e-05 2024-09-18T04:56:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.428 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.429 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11066 3.9 3:940e577e:::rbd_data.11acf292c51a.0000000000000007:d [read 1949696~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.429 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.886646+0000", 2024-09-18T04:56:36.429 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.51772206700000001, 2024-09-18T04:56:36.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032739868999999998, 2024-09-18T04:56:36.429 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.429 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.429 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.429 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.429 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.429 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11066 2024-09-18T04:56:36.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886646+0000", 2024-09-18T04:56:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886646+0000", 2024-09-18T04:56:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886646+0000", 2024-09-18T04:56:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886651+0000", 2024-09-18T04:56:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.383e-06 2024-09-18T04:56:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886657+0000", 2024-09-18T04:56:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.75e-06 2024-09-18T04:56:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886661+0000", 2024-09-18T04:56:36.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2169999999999996e-06 2024-09-18T04:56:36.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.919283+0000", 2024-09-18T04:56:36.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032621775999999998 2024-09-18T04:56:36.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.919325+0000", 2024-09-18T04:56:36.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2144000000000002e-05 2024-09-18T04:56:36.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.919386+0000", 2024-09-18T04:56:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0927000000000001e-05 2024-09-18T04:56:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11067 3.9 3:940e577e:::rbd_data.11acf292c51a.0000000000000007:d [read 1998848~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.886665+0000", 2024-09-18T04:56:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.51770329599999998, 2024-09-18T04:56:36.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03284372, 2024-09-18T04:56:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11067 2024-09-18T04:56:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886665+0000", 2024-09-18T04:56:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886665+0000", 2024-09-18T04:56:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886665+0000", 2024-09-18T04:56:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886666+0000", 2024-09-18T04:56:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7859999999999999e-06 2024-09-18T04:56:36.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886672+0000", 2024-09-18T04:56:36.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3870000000000003e-06 2024-09-18T04:56:36.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886676+0000", 2024-09-18T04:56:36.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1649999999999999e-06 2024-09-18T04:56:36.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.919411+0000", 2024-09-18T04:56:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032735080999999999 2024-09-18T04:56:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.919457+0000", 2024-09-18T04:56:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6029000000000003e-05 2024-09-18T04:56:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.919509+0000", 2024-09-18T04:56:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1576999999999998e-05 2024-09-18T04:56:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.437 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11068 3.9 3:940e577e:::rbd_data.11acf292c51a.0000000000000007:d [read 2048000~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.886679+0000", 2024-09-18T04:56:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.51768864599999997, 2024-09-18T04:56:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032982946999999999, 2024-09-18T04:56:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11068 2024-09-18T04:56:36.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886679+0000", 2024-09-18T04:56:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886679+0000", 2024-09-18T04:56:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886679+0000", 2024-09-18T04:56:36.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886681+0000", 2024-09-18T04:56:36.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6819999999999999e-06 2024-09-18T04:56:36.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886686+0000", 2024-09-18T04:56:36.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2700000000000004e-06 2024-09-18T04:56:36.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.886691+0000", 2024-09-18T04:56:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1760000000000001e-06 2024-09-18T04:56:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.919536+0000", 2024-09-18T04:56:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032845036000000001 2024-09-18T04:56:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.919578+0000", 2024-09-18T04:56:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2103e-05 2024-09-18T04:56:36.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.919662+0000", 2024-09-18T04:56:36.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3978000000000001e-05 2024-09-18T04:56:36.442 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.442 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.442 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.442 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11316 3.9 3:940e577e:::rbd_data.11acf292c51a.0000000000000007:d [read 1593344~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.442 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.893024+0000", 2024-09-18T04:56:36.442 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.51134423699999998, 2024-09-18T04:56:36.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032413116999999998, 2024-09-18T04:56:36.443 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.443 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.443 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.443 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.443 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.443 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11316 2024-09-18T04:56:36.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.443 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893024+0000", 2024-09-18T04:56:36.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893024+0000", 2024-09-18T04:56:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893024+0000", 2024-09-18T04:56:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893025+0000", 2024-09-18T04:56:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.759e-06 2024-09-18T04:56:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893031+0000", 2024-09-18T04:56:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.071e-06 2024-09-18T04:56:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893036+0000", 2024-09-18T04:56:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5449999999999997e-06 2024-09-18T04:56:36.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.925282+0000", 2024-09-18T04:56:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032245957999999998 2024-09-18T04:56:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.925333+0000", 2024-09-18T04:56:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1199000000000002e-05 2024-09-18T04:56:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.925437+0000", 2024-09-18T04:56:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000103886 2024-09-18T04:56:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11317 3.9 3:940e577e:::rbd_data.11acf292c51a.0000000000000007:d [read 1642496~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.893040+0000", 2024-09-18T04:56:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.51132777399999996, 2024-09-18T04:56:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032545393999999998, 2024-09-18T04:56:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.447 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11317 2024-09-18T04:56:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893040+0000", 2024-09-18T04:56:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893040+0000", 2024-09-18T04:56:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893040+0000", 2024-09-18T04:56:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893042+0000", 2024-09-18T04:56:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0020000000000001e-06 2024-09-18T04:56:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893047+0000", 2024-09-18T04:56:36.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5049999999999998e-06 2024-09-18T04:56:36.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893052+0000", 2024-09-18T04:56:36.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.639e-06 2024-09-18T04:56:36.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.925471+0000", 2024-09-18T04:56:36.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032418466999999999 2024-09-18T04:56:36.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.925533+0000", 2024-09-18T04:56:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2408000000000003e-05 2024-09-18T04:56:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.925586+0000", 2024-09-18T04:56:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2735000000000002e-05 2024-09-18T04:56:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.451 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11318 3.9 3:940e577e:::rbd_data.11acf292c51a.0000000000000007:d [read 1691648~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.893056+0000", 2024-09-18T04:56:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.51131210999999999, 2024-09-18T04:56:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032654845000000002, 2024-09-18T04:56:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11318 2024-09-18T04:56:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893056+0000", 2024-09-18T04:56:36.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893056+0000", 2024-09-18T04:56:36.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893056+0000", 2024-09-18T04:56:36.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893058+0000", 2024-09-18T04:56:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9810000000000002e-06 2024-09-18T04:56:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893065+0000", 2024-09-18T04:56:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8759999999999999e-06 2024-09-18T04:56:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893070+0000", 2024-09-18T04:56:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9910000000000002e-06 2024-09-18T04:56:36.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.925612+0000", 2024-09-18T04:56:36.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032542804000000002 2024-09-18T04:56:36.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.925655+0000", 2024-09-18T04:56:36.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2877000000000003e-05 2024-09-18T04:56:36.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.925711+0000", 2024-09-18T04:56:36.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5643000000000002e-05 2024-09-18T04:56:36.456 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.456 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.456 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.456 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11319 3.9 3:940e577e:::rbd_data.11acf292c51a.0000000000000007:d [read 1740800~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.456 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.893074+0000", 2024-09-18T04:56:36.456 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.51129425100000003, 2024-09-18T04:56:36.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032855608000000001, 2024-09-18T04:56:36.456 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.456 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11319 2024-09-18T04:56:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893074+0000", 2024-09-18T04:56:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893074+0000", 2024-09-18T04:56:36.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893074+0000", 2024-09-18T04:56:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893076+0000", 2024-09-18T04:56:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0250000000000001e-06 2024-09-18T04:56:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893082+0000", 2024-09-18T04:56:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0730000000000001e-06 2024-09-18T04:56:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893087+0000", 2024-09-18T04:56:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7869999999999998e-06 2024-09-18T04:56:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.925752+0000", 2024-09-18T04:56:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032664436999999998 2024-09-18T04:56:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.925849+0000", 2024-09-18T04:56:36.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6973999999999997e-05 2024-09-18T04:56:36.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.925929+0000", 2024-09-18T04:56:36.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0636000000000005e-05 2024-09-18T04:56:36.460 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.460 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.460 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11320 3.9 3:940e577e:::rbd_data.11acf292c51a.0000000000000007:d [read 1789952~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.893092+0000", 2024-09-18T04:56:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.511276179, 2024-09-18T04:56:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033056205999999998, 2024-09-18T04:56:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11320 2024-09-18T04:56:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893092+0000", 2024-09-18T04:56:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893092+0000", 2024-09-18T04:56:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893092+0000", 2024-09-18T04:56:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893094+0000", 2024-09-18T04:56:36.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0059999999999999e-06 2024-09-18T04:56:36.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893100+0000", 2024-09-18T04:56:36.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0299999999999999e-06 2024-09-18T04:56:36.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893104+0000", 2024-09-18T04:56:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7419999999999997e-06 2024-09-18T04:56:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.925965+0000", 2024-09-18T04:56:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032860458000000002 2024-09-18T04:56:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.926060+0000", 2024-09-18T04:56:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4926999999999994e-05 2024-09-18T04:56:36.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.926148+0000", 2024-09-18T04:56:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8400999999999996e-05 2024-09-18T04:56:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11321 3.9 3:940e577e:::rbd_data.11acf292c51a.0000000000000007:d [read 1855488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.893109+0000", 2024-09-18T04:56:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.51125959899999995, 2024-09-18T04:56:36.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033250490000000001, 2024-09-18T04:56:36.466 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.466 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.466 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.466 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.466 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.466 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11321 2024-09-18T04:56:36.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.466 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893109+0000", 2024-09-18T04:56:36.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893109+0000", 2024-09-18T04:56:36.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893108+0000", 2024-09-18T04:56:36.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893114+0000", 2024-09-18T04:56:36.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3709999999999996e-06 2024-09-18T04:56:36.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893122+0000", 2024-09-18T04:56:36.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1899999999999998e-06 2024-09-18T04:56:36.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893133+0000", 2024-09-18T04:56:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1877e-05 2024-09-18T04:56:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.926189+0000", 2024-09-18T04:56:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033055231999999997 2024-09-18T04:56:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.926284+0000", 2024-09-18T04:56:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5254999999999995e-05 2024-09-18T04:56:36.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.926359+0000", 2024-09-18T04:56:36.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5073000000000004e-05 2024-09-18T04:56:36.470 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.470 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.470 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.470 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11322 3.9 3:940e577e:::rbd_data.11acf292c51a.0000000000000007:d [read 1904640~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.470 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.893140+0000", 2024-09-18T04:56:36.470 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.51122859899999995, 2024-09-18T04:56:36.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033403225000000002, 2024-09-18T04:56:36.470 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11322 2024-09-18T04:56:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893140+0000", 2024-09-18T04:56:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893140+0000", 2024-09-18T04:56:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893139+0000", 2024-09-18T04:56:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893141+0000", 2024-09-18T04:56:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1449999999999998e-06 2024-09-18T04:56:36.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893153+0000", 2024-09-18T04:56:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1979e-05 2024-09-18T04:56:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893158+0000", 2024-09-18T04:56:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2850000000000002e-06 2024-09-18T04:56:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.926396+0000", 2024-09-18T04:56:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033237558 2024-09-18T04:56:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.926484+0000", 2024-09-18T04:56:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7489000000000002e-05 2024-09-18T04:56:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.926543+0000", 2024-09-18T04:56:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9187000000000001e-05 2024-09-18T04:56:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.474 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11323 3.9 3:940e577e:::rbd_data.11acf292c51a.0000000000000007:d [read 1953792~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.893163+0000", 2024-09-18T04:56:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.51120484600000005, 2024-09-18T04:56:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033586346000000003, 2024-09-18T04:56:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11323 2024-09-18T04:56:36.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.476 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893163+0000", 2024-09-18T04:56:36.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893163+0000", 2024-09-18T04:56:36.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893163+0000", 2024-09-18T04:56:36.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893170+0000", 2024-09-18T04:56:36.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3930000000000002e-06 2024-09-18T04:56:36.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893177+0000", 2024-09-18T04:56:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7830000000000001e-06 2024-09-18T04:56:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893182+0000", 2024-09-18T04:56:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6099999999999999e-06 2024-09-18T04:56:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.926576+0000", 2024-09-18T04:56:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033394803000000001 2024-09-18T04:56:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.926672+0000", 2024-09-18T04:56:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5513999999999994e-05 2024-09-18T04:56:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.926750+0000", 2024-09-18T04:56:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7730000000000003e-05 2024-09-18T04:56:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.479 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11324 3.9 3:940e577e:::rbd_data.11acf292c51a.0000000000000007:d [read 2002944~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.893186+0000", 2024-09-18T04:56:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.51118204700000003, 2024-09-18T04:56:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033751982, 2024-09-18T04:56:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11324 2024-09-18T04:56:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893186+0000", 2024-09-18T04:56:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893186+0000", 2024-09-18T04:56:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893186+0000", 2024-09-18T04:56:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893188+0000", 2024-09-18T04:56:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1299999999999999e-06 2024-09-18T04:56:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893194+0000", 2024-09-18T04:56:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1229999999999997e-06 2024-09-18T04:56:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893198+0000", 2024-09-18T04:56:36.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4819999999999996e-06 2024-09-18T04:56:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.926785+0000", 2024-09-18T04:56:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033587028999999997 2024-09-18T04:56:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.926876+0000", 2024-09-18T04:56:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0744999999999999e-05 2024-09-18T04:56:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.926938+0000", 2024-09-18T04:56:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1892999999999998e-05 2024-09-18T04:56:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11325 3.9 3:940e577e:::rbd_data.11acf292c51a.0000000000000007:d [read 2052096~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.893203+0000", 2024-09-18T04:56:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.51116541800000004, 2024-09-18T04:56:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033935780999999998, 2024-09-18T04:56:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.484 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.485 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.485 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.485 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11325 2024-09-18T04:56:36.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.485 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893203+0000", 2024-09-18T04:56:36.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893203+0000", 2024-09-18T04:56:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893202+0000", 2024-09-18T04:56:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893204+0000", 2024-09-18T04:56:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0320000000000002e-06 2024-09-18T04:56:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893211+0000", 2024-09-18T04:56:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9999999999999999e-06 2024-09-18T04:56:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.893216+0000", 2024-09-18T04:56:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6199999999999998e-06 2024-09-18T04:56:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.926971+0000", 2024-09-18T04:56:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.033754974 2024-09-18T04:56:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927071+0000", 2024-09-18T04:56:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9995999999999996e-05 2024-09-18T04:56:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927138+0000", 2024-09-18T04:56:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7500000000000001e-05 2024-09-18T04:56:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.488 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11579 3.9 3:940e577e:::rbd_data.11acf292c51a.0000000000000007:d [read 1908736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.489 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.901471+0000", 2024-09-18T04:56:36.489 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.50289749800000005, 2024-09-18T04:56:36.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032451309999999997, 2024-09-18T04:56:36.489 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.489 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.489 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.489 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.489 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.489 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11579 2024-09-18T04:56:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.901471+0000", 2024-09-18T04:56:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.901471+0000", 2024-09-18T04:56:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.901470+0000", 2024-09-18T04:56:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.901472+0000", 2024-09-18T04:56:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.232e-06 2024-09-18T04:56:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.901478+0000", 2024-09-18T04:56:36.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9070000000000004e-06 2024-09-18T04:56:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.901484+0000", 2024-09-18T04:56:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8900000000000004e-06 2024-09-18T04:56:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.933741+0000", 2024-09-18T04:56:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032256764 2024-09-18T04:56:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.933831+0000", 2024-09-18T04:56:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9623000000000006e-05 2024-09-18T04:56:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.933922+0000", 2024-09-18T04:56:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1290000000000002e-05 2024-09-18T04:56:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11581 3.9 3:940e577e:::rbd_data.11acf292c51a.0000000000000007:d [read 2007040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.493 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.901802+0000", 2024-09-18T04:56:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.50256650000000003, 2024-09-18T04:56:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032416215999999998, 2024-09-18T04:56:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11581 2024-09-18T04:56:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.901802+0000", 2024-09-18T04:56:36.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.901802+0000", 2024-09-18T04:56:36.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.901801+0000", 2024-09-18T04:56:36.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.901803+0000", 2024-09-18T04:56:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.511e-06 2024-09-18T04:56:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.901806+0000", 2024-09-18T04:56:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2200000000000001e-06 2024-09-18T04:56:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.901809+0000", 2024-09-18T04:56:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5150000000000001e-06 2024-09-18T04:56:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.934093+0000", 2024-09-18T04:56:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032284581 2024-09-18T04:56:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.934157+0000", 2024-09-18T04:56:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3361000000000005e-05 2024-09-18T04:56:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.934218+0000", 2024-09-18T04:56:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1309000000000005e-05 2024-09-18T04:56:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11582 3.9 3:940e577e:::rbd_data.11acf292c51a.0000000000000007:d [read 2056192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.901811+0000", 2024-09-18T04:56:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.50255740599999998, 2024-09-18T04:56:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032626699000000002, 2024-09-18T04:56:36.498 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11582 2024-09-18T04:56:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.901811+0000", 2024-09-18T04:56:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.901811+0000", 2024-09-18T04:56:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.901810+0000", 2024-09-18T04:56:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.901817+0000", 2024-09-18T04:56:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9059999999999996e-06 2024-09-18T04:56:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.901821+0000", 2024-09-18T04:56:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3569999999999998e-06 2024-09-18T04:56:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.901823+0000", 2024-09-18T04:56:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.554e-06 2024-09-18T04:56:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.934259+0000", 2024-09-18T04:56:36.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.032435881 2024-09-18T04:56:36.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.934344+0000", 2024-09-18T04:56:36.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4833000000000006e-05 2024-09-18T04:56:36.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.934437+0000", 2024-09-18T04:56:36.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3398000000000003e-05 2024-09-18T04:56:36.506 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.506 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.506 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.506 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.506 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.506 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:36.507 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:56:36.508 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:36.505+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:36.608 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:36.606+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:36.650 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:36.650 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:36.650 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:56:36.650 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:36.658 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:56:36.709 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:36.706+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:36.766 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:36.766 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:36.766 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:56:36.766 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:56:36.766 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:36.775 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:56:36.819 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:36.818+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:36.889 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:56:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:56:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:56:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:5206 3.1d 3:bc9b8049:::rbd_data.11ac14fc1d07.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 17326599~63058] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.539333+0000", 2024-09-18T04:56:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.3459702900000003, 2024-09-18T04:56:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.027944470999999999, 2024-09-18T04:56:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.889 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 5206 2024-09-18T04:56:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.539333+0000", 2024-09-18T04:56:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.539333+0000", 2024-09-18T04:56:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.539332+0000", 2024-09-18T04:56:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.551229+0000", 2024-09-18T04:56:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011896814 2024-09-18T04:56:36.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.551256+0000", 2024-09-18T04:56:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7208000000000001e-05 2024-09-18T04:56:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.551266+0000", 2024-09-18T04:56:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0051000000000001e-05 2024-09-18T04:56:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.551322+0000", 2024-09-18T04:56:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6317999999999998e-05 2024-09-18T04:56:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.551437+0000", 2024-09-18T04:56:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115159 2024-09-18T04:56:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.551798+0000", 2024-09-18T04:56:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000360584 2024-09-18T04:56:36.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.566830+0000", 2024-09-18T04:56:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015031991999999999 2024-09-18T04:56:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.567213+0000", 2024-09-18T04:56:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038325799999999998 2024-09-18T04:56:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:36.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.567250+0000", 2024-09-18T04:56:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6259e-05 2024-09-18T04:56:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.567278+0000", 2024-09-18T04:56:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7988999999999999e-05 2024-09-18T04:56:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11020 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 1654784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.889307+0000", 2024-09-18T04:56:36.895 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.99599583599999997, 2024-09-18T04:56:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023721609000000001, 2024-09-18T04:56:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11020 2024-09-18T04:56:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.889307+0000", 2024-09-18T04:56:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.889307+0000", 2024-09-18T04:56:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.889307+0000", 2024-09-18T04:56:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.889309+0000", 2024-09-18T04:56:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2730000000000001e-06 2024-09-18T04:56:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.889318+0000", 2024-09-18T04:56:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6119999999999999e-06 2024-09-18T04:56:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.889322+0000", 2024-09-18T04:56:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6509999999999999e-06 2024-09-18T04:56:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.912900+0000", 2024-09-18T04:56:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023578154 2024-09-18T04:56:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.912968+0000", 2024-09-18T04:56:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8757999999999995e-05 2024-09-18T04:56:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.913029+0000", 2024-09-18T04:56:36.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0616999999999999e-05 2024-09-18T04:56:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11283 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 1970176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.897179+0000", 2024-09-18T04:56:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.98812446899999995, 2024-09-18T04:56:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023627705999999998, 2024-09-18T04:56:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.900 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11283 2024-09-18T04:56:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897179+0000", 2024-09-18T04:56:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897179+0000", 2024-09-18T04:56:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897178+0000", 2024-09-18T04:56:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897181+0000", 2024-09-18T04:56:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.187e-06 2024-09-18T04:56:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897187+0000", 2024-09-18T04:56:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8769999999999998e-06 2024-09-18T04:56:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897191+0000", 2024-09-18T04:56:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3470000000000001e-06 2024-09-18T04:56:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.920639+0000", 2024-09-18T04:56:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023447835 2024-09-18T04:56:36.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.920715+0000", 2024-09-18T04:56:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6476000000000004e-05 2024-09-18T04:56:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.920807+0000", 2024-09-18T04:56:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1388e-05 2024-09-18T04:56:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.905 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11284 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.897195+0000", 2024-09-18T04:56:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.98810873200000005, 2024-09-18T04:56:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023843923, 2024-09-18T04:56:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11284 2024-09-18T04:56:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897195+0000", 2024-09-18T04:56:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897195+0000", 2024-09-18T04:56:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897194+0000", 2024-09-18T04:56:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897196+0000", 2024-09-18T04:56:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.838e-06 2024-09-18T04:56:36.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897202+0000", 2024-09-18T04:56:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.677e-06 2024-09-18T04:56:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897206+0000", 2024-09-18T04:56:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1699999999999999e-06 2024-09-18T04:56:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.920838+0000", 2024-09-18T04:56:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023632162000000002 2024-09-18T04:56:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.920916+0000", 2024-09-18T04:56:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7762000000000005e-05 2024-09-18T04:56:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.921039+0000", 2024-09-18T04:56:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012267499999999999 2024-09-18T04:56:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11285 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 2068480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.897210+0000", 2024-09-18T04:56:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.98809345699999995, 2024-09-18T04:56:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024026921999999999, 2024-09-18T04:56:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11285 2024-09-18T04:56:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.911 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897210+0000", 2024-09-18T04:56:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897210+0000", 2024-09-18T04:56:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897210+0000", 2024-09-18T04:56:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897211+0000", 2024-09-18T04:56:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9439999999999999e-06 2024-09-18T04:56:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897217+0000", 2024-09-18T04:56:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4009999999999996e-06 2024-09-18T04:56:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897221+0000", 2024-09-18T04:56:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3170000000000003e-06 2024-09-18T04:56:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.921075+0000", 2024-09-18T04:56:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023853760000000002 2024-09-18T04:56:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.921152+0000", 2024-09-18T04:56:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7012000000000001e-05 2024-09-18T04:56:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.921237+0000", 2024-09-18T04:56:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4818e-05 2024-09-18T04:56:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11533 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 1613824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.903270+0000", 2024-09-18T04:56:36.915 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.98203301300000001, 2024-09-18T04:56:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023848581000000001, 2024-09-18T04:56:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11533 2024-09-18T04:56:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903270+0000", 2024-09-18T04:56:36.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903270+0000", 2024-09-18T04:56:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903270+0000", 2024-09-18T04:56:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903271+0000", 2024-09-18T04:56:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.08e-06 2024-09-18T04:56:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903274+0000", 2024-09-18T04:56:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8059999999999999e-06 2024-09-18T04:56:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903276+0000", 2024-09-18T04:56:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2450000000000001e-06 2024-09-18T04:56:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.926915+0000", 2024-09-18T04:56:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023639191 2024-09-18T04:56:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927008+0000", 2024-09-18T04:56:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2682999999999993e-05 2024-09-18T04:56:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927119+0000", 2024-09-18T04:56:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110777 2024-09-18T04:56:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.919 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11534 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 1662976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.903279+0000", 2024-09-18T04:56:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.98202434900000002, 2024-09-18T04:56:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024055705, 2024-09-18T04:56:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.920 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11534 2024-09-18T04:56:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903279+0000", 2024-09-18T04:56:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903279+0000", 2024-09-18T04:56:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903279+0000", 2024-09-18T04:56:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903280+0000", 2024-09-18T04:56:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1510000000000001e-06 2024-09-18T04:56:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903283+0000", 2024-09-18T04:56:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.83e-06 2024-09-18T04:56:36.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903286+0000", 2024-09-18T04:56:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0290000000000001e-06 2024-09-18T04:56:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927172+0000", 2024-09-18T04:56:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023886627000000001 2024-09-18T04:56:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927252+0000", 2024-09-18T04:56:36.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9424000000000004e-05 2024-09-18T04:56:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927335+0000", 2024-09-18T04:56:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2844999999999997e-05 2024-09-18T04:56:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11535 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 1712128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.924 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.903288+0000", 2024-09-18T04:56:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.98201539800000004, 2024-09-18T04:56:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024184574, 2024-09-18T04:56:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11535 2024-09-18T04:56:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903288+0000", 2024-09-18T04:56:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903288+0000", 2024-09-18T04:56:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903288+0000", 2024-09-18T04:56:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903289+0000", 2024-09-18T04:56:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.048e-06 2024-09-18T04:56:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903292+0000", 2024-09-18T04:56:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8839999999999998e-06 2024-09-18T04:56:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903294+0000", 2024-09-18T04:56:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3099999999999999e-06 2024-09-18T04:56:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927363+0000", 2024-09-18T04:56:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024068984000000002 2024-09-18T04:56:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927407+0000", 2024-09-18T04:56:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3608e-05 2024-09-18T04:56:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927473+0000", 2024-09-18T04:56:36.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5955000000000001e-05 2024-09-18T04:56:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11536 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 1761280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.903296+0000", 2024-09-18T04:56:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.98200720200000002, 2024-09-18T04:56:36.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024304868, 2024-09-18T04:56:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11536 2024-09-18T04:56:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903296+0000", 2024-09-18T04:56:36.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903296+0000", 2024-09-18T04:56:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903296+0000", 2024-09-18T04:56:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903297+0000", 2024-09-18T04:56:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0139999999999999e-06 2024-09-18T04:56:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903300+0000", 2024-09-18T04:56:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8590000000000001e-06 2024-09-18T04:56:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903302+0000", 2024-09-18T04:56:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2900000000000001e-06 2024-09-18T04:56:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927500+0000", 2024-09-18T04:56:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024198130000000002 2024-09-18T04:56:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927545+0000", 2024-09-18T04:56:36.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5176000000000003e-05 2024-09-18T04:56:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927601+0000", 2024-09-18T04:56:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5618999999999997e-05 2024-09-18T04:56:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11537 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 1810432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.903305+0000", 2024-09-18T04:56:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.98199837899999998, 2024-09-18T04:56:36.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024484314, 2024-09-18T04:56:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11537 2024-09-18T04:56:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903305+0000", 2024-09-18T04:56:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903305+0000", 2024-09-18T04:56:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903305+0000", 2024-09-18T04:56:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:36.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903306+0000", 2024-09-18T04:56:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0449999999999999e-06 2024-09-18T04:56:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903309+0000", 2024-09-18T04:56:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7920000000000001e-06 2024-09-18T04:56:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903311+0000", 2024-09-18T04:56:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3159999999999999e-06 2024-09-18T04:56:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927631+0000", 2024-09-18T04:56:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024320235999999999 2024-09-18T04:56:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927722+0000", 2024-09-18T04:56:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1020000000000006e-05 2024-09-18T04:56:36.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927789+0000", 2024-09-18T04:56:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7131000000000005e-05 2024-09-18T04:56:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11541 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 2023424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.904815+0000", 2024-09-18T04:56:36.940 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.98048798500000001, 2024-09-18T04:56:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023644554000000002, 2024-09-18T04:56:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11541 2024-09-18T04:56:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.904815+0000", 2024-09-18T04:56:36.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.904815+0000", 2024-09-18T04:56:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.904815+0000", 2024-09-18T04:56:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.904821+0000", 2024-09-18T04:56:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.19e-06 2024-09-18T04:56:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.904827+0000", 2024-09-18T04:56:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2780000000000001e-06 2024-09-18T04:56:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.904829+0000", 2024-09-18T04:56:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6369999999999999e-06 2024-09-18T04:56:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.928326+0000", 2024-09-18T04:56:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023496435999999999 2024-09-18T04:56:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.928373+0000", 2024-09-18T04:56:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7851e-05 2024-09-18T04:56:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.928460+0000", 2024-09-18T04:56:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6420000000000003e-05 2024-09-18T04:56:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11542 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 2072576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.904832+0000", 2024-09-18T04:56:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.98047123899999999, 2024-09-18T04:56:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.02378951, 2024-09-18T04:56:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.945 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11542 2024-09-18T04:56:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.904832+0000", 2024-09-18T04:56:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.904832+0000", 2024-09-18T04:56:36.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.904832+0000", 2024-09-18T04:56:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.904833+0000", 2024-09-18T04:56:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8139999999999999e-06 2024-09-18T04:56:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.904839+0000", 2024-09-18T04:56:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4410000000000001e-06 2024-09-18T04:56:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.904843+0000", 2024-09-18T04:56:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3089999999999997e-06 2024-09-18T04:56:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.928475+0000", 2024-09-18T04:56:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023632057000000001 2024-09-18T04:56:36.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.928531+0000", 2024-09-18T04:56:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6177000000000001e-05 2024-09-18T04:56:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.928622+0000", 2024-09-18T04:56:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0155999999999995e-05 2024-09-18T04:56:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11791 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 1716224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.910762+0000", 2024-09-18T04:56:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.97454169800000001, 2024-09-18T04:56:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023797966, 2024-09-18T04:56:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11791 2024-09-18T04:56:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910762+0000", 2024-09-18T04:56:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910762+0000", 2024-09-18T04:56:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910761+0000", 2024-09-18T04:56:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:36.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910763+0000", 2024-09-18T04:56:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.119e-06 2024-09-18T04:56:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910767+0000", 2024-09-18T04:56:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5589999999999999e-06 2024-09-18T04:56:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910771+0000", 2024-09-18T04:56:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3989999999999997e-06 2024-09-18T04:56:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.934393+0000", 2024-09-18T04:56:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023621782000000001 2024-09-18T04:56:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.934456+0000", 2024-09-18T04:56:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2572999999999999e-05 2024-09-18T04:56:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.934560+0000", 2024-09-18T04:56:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000103754 2024-09-18T04:56:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11792 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 1765376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.910776+0000", 2024-09-18T04:56:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.974526907, 2024-09-18T04:56:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024009782, 2024-09-18T04:56:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.954 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11792 2024-09-18T04:56:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910776+0000", 2024-09-18T04:56:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910776+0000", 2024-09-18T04:56:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910776+0000", 2024-09-18T04:56:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910778+0000", 2024-09-18T04:56:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0789999999999999e-06 2024-09-18T04:56:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910784+0000", 2024-09-18T04:56:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.942e-06 2024-09-18T04:56:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910788+0000", 2024-09-18T04:56:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3309999999999997e-06 2024-09-18T04:56:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.934612+0000", 2024-09-18T04:56:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023823205 2024-09-18T04:56:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.934694+0000", 2024-09-18T04:56:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2588000000000003e-05 2024-09-18T04:56:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.934786+0000", 2024-09-18T04:56:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1988000000000001e-05 2024-09-18T04:56:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11793 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 1814528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.910795+0000", 2024-09-18T04:56:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.97450801499999995, 2024-09-18T04:56:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024155223999999999, 2024-09-18T04:56:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.959 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11793 2024-09-18T04:56:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910795+0000", 2024-09-18T04:56:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910795+0000", 2024-09-18T04:56:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910795+0000", 2024-09-18T04:56:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910797+0000", 2024-09-18T04:56:36.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.626e-06 2024-09-18T04:56:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910800+0000", 2024-09-18T04:56:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3249999999999999e-06 2024-09-18T04:56:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910803+0000", 2024-09-18T04:56:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.819e-06 2024-09-18T04:56:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.934831+0000", 2024-09-18T04:56:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024028040000000001 2024-09-18T04:56:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.934891+0000", 2024-09-18T04:56:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0405000000000002e-05 2024-09-18T04:56:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.934951+0000", 2024-09-18T04:56:36.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9406e-05 2024-09-18T04:56:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11794 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 1880064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.910805+0000", 2024-09-18T04:56:36.964 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.97449853600000003, 2024-09-18T04:56:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024296933, 2024-09-18T04:56:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11794 2024-09-18T04:56:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910805+0000", 2024-09-18T04:56:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910805+0000", 2024-09-18T04:56:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910805+0000", 2024-09-18T04:56:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910806+0000", 2024-09-18T04:56:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.155e-06 2024-09-18T04:56:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910810+0000", 2024-09-18T04:56:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7859999999999998e-06 2024-09-18T04:56:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910812+0000", 2024-09-18T04:56:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5380000000000001e-06 2024-09-18T04:56:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.934983+0000", 2024-09-18T04:56:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024170555999999999 2024-09-18T04:56:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.935046+0000", 2024-09-18T04:56:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3274000000000004e-05 2024-09-18T04:56:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.935102+0000", 2024-09-18T04:56:36.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5829999999999999e-05 2024-09-18T04:56:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11795 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 1929216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.910814+0000", 2024-09-18T04:56:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.97448917800000001, 2024-09-18T04:56:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024598887, 2024-09-18T04:56:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.969 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11795 2024-09-18T04:56:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910814+0000", 2024-09-18T04:56:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910814+0000", 2024-09-18T04:56:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910814+0000", 2024-09-18T04:56:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910815+0000", 2024-09-18T04:56:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.362e-06 2024-09-18T04:56:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910818+0000", 2024-09-18T04:56:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7690000000000001e-06 2024-09-18T04:56:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910821+0000", 2024-09-18T04:56:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.447e-06 2024-09-18T04:56:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.935190+0000", 2024-09-18T04:56:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024369289999999998 2024-09-18T04:56:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.935231+0000", 2024-09-18T04:56:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1551999999999998e-05 2024-09-18T04:56:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.935413+0000", 2024-09-18T04:56:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018169 2024-09-18T04:56:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.973 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11796 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 1978368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.910823+0000", 2024-09-18T04:56:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.974480609, 2024-09-18T04:56:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024780176000000001, 2024-09-18T04:56:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.974 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11796 2024-09-18T04:56:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910823+0000", 2024-09-18T04:56:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910823+0000", 2024-09-18T04:56:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910822+0000", 2024-09-18T04:56:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910824+0000", 2024-09-18T04:56:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.269e-06 2024-09-18T04:56:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910827+0000", 2024-09-18T04:56:36.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9110000000000002e-06 2024-09-18T04:56:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910829+0000", 2024-09-18T04:56:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4339999999999999e-06 2024-09-18T04:56:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.935452+0000", 2024-09-18T04:56:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024622733000000001 2024-09-18T04:56:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.935528+0000", 2024-09-18T04:56:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6087000000000005e-05 2024-09-18T04:56:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.935603+0000", 2024-09-18T04:56:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5059000000000001e-05 2024-09-18T04:56:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11797 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 2027520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.978 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.910831+0000", 2024-09-18T04:56:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.97447226799999997, 2024-09-18T04:56:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025033228000000001, 2024-09-18T04:56:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11797 2024-09-18T04:56:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910831+0000", 2024-09-18T04:56:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910831+0000", 2024-09-18T04:56:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910831+0000", 2024-09-18T04:56:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910835+0000", 2024-09-18T04:56:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9149999999999998e-06 2024-09-18T04:56:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910838+0000", 2024-09-18T04:56:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1659999999999998e-06 2024-09-18T04:56:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910840+0000", 2024-09-18T04:56:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4430000000000002e-06 2024-09-18T04:56:36.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.935669+0000", 2024-09-18T04:56:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024828894000000001 2024-09-18T04:56:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.935777+0000", 2024-09-18T04:56:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107308 2024-09-18T04:56:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.935864+0000", 2024-09-18T04:56:36.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7717000000000001e-05 2024-09-18T04:56:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11798 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 2076672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.910843+0000", 2024-09-18T04:56:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.97446081699999998, 2024-09-18T04:56:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025237124, 2024-09-18T04:56:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:36.983 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11798 2024-09-18T04:56:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910843+0000", 2024-09-18T04:56:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910843+0000", 2024-09-18T04:56:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:36.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910842+0000", 2024-09-18T04:56:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910843+0000", 2024-09-18T04:56:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.186e-06 2024-09-18T04:56:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:36.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910847+0000", 2024-09-18T04:56:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5250000000000001e-06 2024-09-18T04:56:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910849+0000", 2024-09-18T04:56:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4600000000000002e-06 2024-09-18T04:56:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.935892+0000", 2024-09-18T04:56:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025042792000000001 2024-09-18T04:56:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.935977+0000", 2024-09-18T04:56:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4606999999999998e-05 2024-09-18T04:56:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.936080+0000", 2024-09-18T04:56:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010279 2024-09-18T04:56:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:36.987 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:36.987 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:36.989 INFO:teuthology.orchestra.run.smithi002.stdout:1007 read 0x46cd80a thru 0x46d8929 (0xb120 bytes) 2024-09-18T04:56:36.989 INFO:teuthology.orchestra.run.smithi002.stdout:1008 trunc from 0x5809ff1 to 0x8e76a07 2024-09-18T04:56:36.989 INFO:teuthology.orchestra.run.smithi002.stdout:1009 punch from 0x1c5a293 to 0x1c6154c, (0x72b9 bytes) 2024-09-18T04:56:36.989 INFO:teuthology.orchestra.run.smithi002.stdout:1010 trunc from 0x8e76a07 to 0xd49fd35 2024-09-18T04:56:36.989 INFO:teuthology.orchestra.run.smithi002.stdout:1011 trunc from 0xd49fd35 to 0xa51ae29 2024-09-18T04:56:36.990 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:36.986+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:36.990 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:56:36.991 INFO:teuthology.orchestra.run.smithi002.stdout:1013 read 0xb3caa thru 0xb8b3e (0x4e95 bytes) 2024-09-18T04:56:36.998 INFO:teuthology.orchestra.run.smithi002.stdout:1015 write 0xc65a9cd thru 0xc65e35f (0x3993 bytes) 2024-09-18T04:56:37.004 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:37.004 INFO:teuthology.orchestra.run.smithi002.stdout:1017 punch from 0xad43fa7 to 0xad49045, (0x509e bytes) 2024-09-18T04:56:37.004 INFO:teuthology.orchestra.run.smithi002.stdout:1018 write 0x1d31553 thru 0x1d3a343 (0x8df1 bytes) 2024-09-18T04:56:37.008 INFO:teuthology.orchestra.run.smithi002.stdout:1022 read 0x90d79e8 thru 0x90e7163 (0xf77c bytes) 2024-09-18T04:56:37.008 INFO:teuthology.orchestra.run.smithi002.stdout:1024 write 0x20b18cb thru 0x20b4a7f (0x31b5 bytes) 2024-09-18T04:56:37.011 INFO:teuthology.orchestra.run.smithi002.stdout:1025 write 0x8b9d6f8 thru 0x8ba7d86 (0xa68f bytes) 2024-09-18T04:56:37.023 INFO:teuthology.orchestra.run.smithi002.stdout:1027 punch from 0xc226258 to 0xc22c93f, (0x66e7 bytes) 2024-09-18T04:56:37.023 INFO:teuthology.orchestra.run.smithi002.stdout:1028 punch from 0x8be12d5 to 0x8bea59e, (0x92c9 bytes) 2024-09-18T04:56:37.024 INFO:teuthology.orchestra.run.smithi002.stdout:1029 write 0x136e07d thru 0x1371b39 (0x3abd bytes) 2024-09-18T04:56:37.025 INFO:teuthology.orchestra.run.smithi002.stdout:1030 read 0x57fb10b thru 0x57fd58a (0x2480 bytes) 2024-09-18T04:56:37.025 INFO:teuthology.orchestra.run.smithi002.stdout:1031 write 0x62d6e84 thru 0x62d7e84 (0x1001 bytes) 2024-09-18T04:56:37.028 INFO:teuthology.orchestra.run.smithi002.stdout:1033 trunc from 0xc65e360 to 0xc4bc3bd 2024-09-18T04:56:37.037 INFO:teuthology.orchestra.run.smithi002.stdout:1035 punch from 0x4016cb2 to 0x401fe94, (0x91e2 bytes) 2024-09-18T04:56:37.445 INFO:teuthology.orchestra.run.smithi002.stdout:1036 read 0x9667a8b thru 0x967410a (0xc680 bytes) 2024-09-18T04:56:37.445 INFO:teuthology.orchestra.run.smithi002.stdout:1037 punch from 0x6ace470 to 0x6ad42f6, (0x5e86 bytes) 2024-09-18T04:56:37.445 INFO:teuthology.orchestra.run.smithi002.stdout:1040 read 0x9da2933 thru 0x9da959c (0x6c6a bytes) 2024-09-18T04:56:37.445 INFO:teuthology.orchestra.run.smithi002.stdout:1041 write 0x5f009d5 thru 0x5f0b07f (0xa6ab bytes) 2024-09-18T04:56:37.446 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T04:56:37.447 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:37.448 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:37.448 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:56:37.448 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:37.448 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:56:37.448 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:37.443+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:37.450 INFO:teuthology.orchestra.run.smithi002.stdout:1042 punch from 0x8101e25 to 0x8109cb9, (0x7e94 bytes) 2024-09-18T04:56:37.450 INFO:teuthology.orchestra.run.smithi002.stdout:1043 punch from 0x718757a to 0x7195017, (0xda9d bytes) 2024-09-18T04:56:37.451 INFO:teuthology.orchestra.run.smithi002.stdout:1046 read 0x57ba68f thru 0x57be445 (0x3db7 bytes) 2024-09-18T04:56:37.451 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:37.451 INFO:teuthology.orchestra.run.smithi002.stdout:1048 trunc from 0xc4bc3bd to 0xed87e56 2024-09-18T04:56:37.451 INFO:teuthology.orchestra.run.smithi002.stdout:1050 punch from 0xac03b86 to 0xac0423e, (0x6b8 bytes) 2024-09-18T04:56:37.451 INFO:teuthology.orchestra.run.smithi002.stdout:1051 punch from 0x5418d4b to 0x541eaf8, (0x5dad bytes) 2024-09-18T04:56:37.451 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:37.451 INFO:teuthology.orchestra.run.smithi002.stdout:1054 writesame 0x45006d9 thru 0x45090a4 data_size 0x2(0x89cc bytes) 2024-09-18T04:56:37.451 INFO:teuthology.orchestra.run.smithi002.stdout:1055 write 0x624847 thru 0x62a57d (0x5d37 bytes) 2024-09-18T04:56:37.451 INFO:teuthology.orchestra.run.smithi002.stdout:1056 writesame 0xd6ea676 thru 0xd6f549d data_size 0x4(0xae28 bytes) 2024-09-18T04:56:37.451 INFO:teuthology.orchestra.run.smithi002.stdout:1057 writesame 0x95f2595 thru 0x95f312b data_size 0x3(0xb97 bytes) 2024-09-18T04:56:37.451 INFO:teuthology.orchestra.run.smithi002.stdout:1058 read 0xbbc3bc2 thru 0xbbd13d5 (0xd814 bytes) 2024-09-18T04:56:37.451 INFO:teuthology.orchestra.run.smithi002.stdout:1059 clone 11 order 20 su 262144 sc 8 2024-09-18T04:56:37.451 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-09-18T04:56:37.451 INFO:teuthology.orchestra.run.smithi002.stdout:1060 read 0xdd05343 thru 0xdd0f464 (0xa122 bytes) 2024-09-18T04:56:37.452 INFO:teuthology.orchestra.run.smithi002.stdout:1061 write 0x4127cd4 thru 0x412af7f (0x32ac bytes) 2024-09-18T04:56:37.452 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:37.452 INFO:teuthology.orchestra.run.smithi002.stdout:1064 write 0xbe829fa thru 0xbe8e5d6 (0xbbdd bytes) 2024-09-18T04:56:37.452 INFO:teuthology.orchestra.run.smithi002.stdout:1065 write 0x24dfe4a thru 0x24e4354 (0x450b bytes) 2024-09-18T04:56:37.452 INFO:teuthology.orchestra.run.smithi002.stdout:1069 write 0xb7beb73 thru 0xb7cae42 (0xc2d0 bytes) 2024-09-18T04:56:37.452 INFO:teuthology.orchestra.run.smithi002.stdout:1071 writesame 0xbf777c9 thru 0xbf82efb data_size 0x51(0xb733 bytes) 2024-09-18T04:56:37.452 INFO:teuthology.orchestra.run.smithi002.stdout:1072 trunc from 0xed87e56 to 0x68514bf 2024-09-18T04:56:37.452 INFO:teuthology.orchestra.run.smithi002.stdout:1073 read 0x616a12 thru 0x619294 (0x2883 bytes) 2024-09-18T04:56:37.452 INFO:teuthology.orchestra.run.smithi002.stdout:1075 writesame 0x8dba417 thru 0x8dc3681 data_size 0x1(0x926b bytes) 2024-09-18T04:56:37.452 INFO:teuthology.orchestra.run.smithi002.stdout:1076 trunc from 0x8dc3682 to 0x853b14b 2024-09-18T04:56:37.461 INFO:teuthology.orchestra.run.smithi002.stdout:1077 write 0x9ee494a thru 0x9eeb69d (0x6d54 bytes) 2024-09-18T04:56:37.464 INFO:teuthology.orchestra.run.smithi002.stdout:1078 punch from 0x2fb5ecf to 0x2fc4884, (0xe9b5 bytes) 2024-09-18T04:56:37.464 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:37.464 INFO:teuthology.orchestra.run.smithi002.stdout:1080 read 0x4dccb77 thru 0x4dda712 (0xdb9c bytes) 2024-09-18T04:56:37.471 INFO:teuthology.orchestra.run.smithi002.stdout:1081 write 0xba5be8d thru 0xba69668 (0xd7dc bytes) 2024-09-18T04:56:37.481 INFO:teuthology.orchestra.run.smithi002.stdout:1082 write 0xdd27406 thru 0xdd2d3e0 (0x5fdb bytes) 2024-09-18T04:56:37.484 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:37.484 INFO:teuthology.orchestra.run.smithi002.stdout:1086 read 0xbd9da52 thru 0xbda97b6 (0xbd65 bytes) 2024-09-18T04:56:37.484 INFO:teuthology.orchestra.run.smithi002.stdout:1087 writesame 0xc65abe5 thru 0xc6668fd data_size 0x1(0xbd19 bytes) 2024-09-18T04:56:37.509 INFO:teuthology.orchestra.run.smithi002.stdout:1088 writesame 0x2a3e33 thru 0x2ae932 data_size 0x4(0xab00 bytes) 2024-09-18T04:56:37.521 INFO:teuthology.orchestra.run.smithi002.stdout:1089 writesame 0xd0e37f0 thru 0xd0ecfcc data_size 0x1(0x97dd bytes) 2024-09-18T04:56:37.538 INFO:teuthology.orchestra.run.smithi002.stdout:1090 punch from 0xc5256a9 to 0xc5351cd, (0xfb24 bytes) 2024-09-18T04:56:37.538 INFO:teuthology.orchestra.run.smithi002.stdout:1091 read 0x1624b66 thru 0x16331da (0xe675 bytes) 2024-09-18T04:56:37.539 INFO:teuthology.orchestra.run.smithi002.stdout:1092 read 0x22d7afb thru 0x22e684e (0xed54 bytes) 2024-09-18T04:56:37.540 INFO:teuthology.orchestra.run.smithi002.stdout:1093 read 0x838b72a thru 0x838f6a1 (0x3f78 bytes) 2024-09-18T04:56:37.546 INFO:teuthology.orchestra.run.smithi002.stdout:1095 trunc from 0xdd2d3e1 to 0x834d15e 2024-09-18T04:56:37.546 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:37.543+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:37.560 INFO:teuthology.orchestra.run.smithi002.stdout:1097 trunc from 0x834d15e to 0xc80d0d8 2024-09-18T04:56:37.573 INFO:teuthology.orchestra.run.smithi002.stdout:1101 write 0xec715a5 thru 0xec7ad1f (0x977b bytes) 2024-09-18T04:56:37.575 INFO:teuthology.orchestra.run.smithi002.stdout:1102 write 0x8dc7dcc thru 0x8dd3a64 (0xbc99 bytes) 2024-09-18T04:56:37.578 INFO:teuthology.orchestra.run.smithi002.stdout:1103 writesame 0x6ca1b21 thru 0x6ca2d0e data_size 0x22(0x11ee bytes) 2024-09-18T04:56:37.581 INFO:teuthology.orchestra.run.smithi002.stdout:1105 write 0x7fd435f thru 0x7fdc87a (0x851c bytes) 2024-09-18T04:56:37.593 INFO:teuthology.orchestra.run.smithi002.stdout:1107 trunc from 0xec7ad20 to 0x6c190cb 2024-09-18T04:56:37.600 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:37.600 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:37.600 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:56:37.600 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:56:37.600 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:37.609 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:56:37.610 INFO:teuthology.orchestra.run.smithi002.stdout:1108 trunc from 0x6c190cb to 0x1820aa4 2024-09-18T04:56:37.642 INFO:teuthology.orchestra.run.smithi002.stdout:1109 write 0xcec1619 thru 0xcecefda (0xd9c2 bytes) 2024-09-18T04:56:37.645 INFO:teuthology.orchestra.run.smithi002.stdout:1110 write 0x6373d93 thru 0x63781b3 (0x4421 bytes) 2024-09-18T04:56:37.647 INFO:teuthology.orchestra.run.smithi002.stdout:1112 write 0xb5ea48f thru 0xb5ed1db (0x2d4d bytes) 2024-09-18T04:56:37.649 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:37.647+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:37.651 INFO:teuthology.orchestra.run.smithi002.stdout:1113 read 0x3286ed6 thru 0x328ddd0 (0x6efb bytes) 2024-09-18T04:56:37.651 INFO:teuthology.orchestra.run.smithi002.stdout:1114 writesame 0x4c1c65f thru 0x4c1d7a5 data_size 0x1(0x1147 bytes) 2024-09-18T04:56:37.654 INFO:teuthology.orchestra.run.smithi002.stdout:1116 writesame 0xb5f0c64 thru 0xb5fd91e data_size 0x1(0xccbb bytes) 2024-09-18T04:56:37.679 INFO:teuthology.orchestra.run.smithi002.stdout:1117 punch from 0x73705d5 to 0x7374ca7, (0x46d2 bytes) 2024-09-18T04:56:37.680 INFO:teuthology.orchestra.run.smithi002.stdout:1119 read 0x1485fd0 thru 0x1491d1e (0xbd4f bytes) 2024-09-18T04:56:37.681 INFO:teuthology.orchestra.run.smithi002.stdout:1122 read 0x496921f thru 0x4970ea9 (0x7c8b bytes) 2024-09-18T04:56:37.681 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:37.681 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:37.681 INFO:teuthology.orchestra.run.smithi002.stdout:1125 read 0x7c813dc thru 0x7c85a46 (0x466b bytes) 2024-09-18T04:56:37.681 INFO:teuthology.orchestra.run.smithi002.stdout:1127 read 0xbd989a thru 0xbdd5d7 (0x3d3e bytes) 2024-09-18T04:56:37.681 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:37.681 INFO:teuthology.orchestra.run.smithi002.stdout:1129 writesame 0x229b28a thru 0x229b488 data_size 0x1(0x1ff bytes) 2024-09-18T04:56:37.686 INFO:teuthology.orchestra.run.smithi002.stdout:1130 read 0x8f13d3e thru 0x8f1a3a7 (0x666a bytes) 2024-09-18T04:56:37.686 INFO:teuthology.orchestra.run.smithi002.stdout:1131 write 0x1373323 thru 0x137cf3e (0x9c1c bytes) 2024-09-18T04:56:37.708 INFO:teuthology.orchestra.run.smithi002.stdout:1132 trunc from 0xcecefdb to 0x2c60c8 2024-09-18T04:56:37.742 INFO:teuthology.orchestra.run.smithi002.stdout:1134 punch from 0x25d19a to 0x267523, (0xa389 bytes) 2024-09-18T04:56:37.742 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:37.742 INFO:teuthology.orchestra.run.smithi002.stdout:1137 read 0x10a495 thru 0x10b091 (0xbfd bytes) 2024-09-18T04:56:37.749 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:37.746+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:37.758 INFO:teuthology.orchestra.run.smithi002.stdout:1139 write 0x87d0c48 thru 0x87d10a0 (0x459 bytes) 2024-09-18T04:56:37.761 INFO:teuthology.orchestra.run.smithi002.stdout:1140 read 0x2bd007f thru 0x2bd01ea (0x16c bytes) 2024-09-18T04:56:37.761 INFO:teuthology.orchestra.run.smithi002.stdout:1141 read 0x7f8ae9 thru 0x801644 (0x8b5c bytes) 2024-09-18T04:56:37.761 INFO:teuthology.orchestra.run.smithi002.stdout:1142 read 0x4c2126a thru 0x4c26a3c (0x57d3 bytes) 2024-09-18T04:56:37.761 INFO:teuthology.orchestra.run.smithi002.stdout:1143 read 0x395241a thru 0x395c2d9 (0x9ec0 bytes) 2024-09-18T04:56:37.762 INFO:teuthology.orchestra.run.smithi002.stdout:1145 write 0x342608e thru 0x3426bd5 (0xb48 bytes) 2024-09-18T04:56:37.764 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:37.764 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:56:37.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:56:37.764 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:56:37.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.764 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop(client.4223.0:1 1.3 e15/14 1:c4c92e5a:::rbd_trash:head v 15'1, mlcod=0'0)", 2024-09-18T04:56:37.764 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:10.402353+0000", 2024-09-18T04:56:37.765 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.358521044, 2024-09-18T04:56:37.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047646060000000002, 2024-09-18T04:56:37.765 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:37.765 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:37.765 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:37.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:37.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.402353+0000", 2024-09-18T04:56:37.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:37.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.402353+0000", 2024-09-18T04:56:37.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:37.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.402353+0000", 2024-09-18T04:56:37.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:37.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:37.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.402357+0000", 2024-09-18T04:56:37.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3209999999999998e-06 2024-09-18T04:56:37.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:37.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.402372+0000", 2024-09-18T04:56:37.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5201999999999999e-05 2024-09-18T04:56:37.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:37.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.402382+0000", 2024-09-18T04:56:37.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8430000000000005e-06 2024-09-18T04:56:37.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:37.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.402417+0000", 2024-09-18T04:56:37.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4811999999999998e-05 2024-09-18T04:56:37.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:37.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.402434+0000", 2024-09-18T04:56:37.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6877000000000001e-05 2024-09-18T04:56:37.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:37.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.407087+0000", 2024-09-18T04:56:37.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.004653122 2024-09-18T04:56:37.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:37.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:10.407117+0000", 2024-09-18T04:56:37.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0738999999999998e-05 2024-09-18T04:56:37.769 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.769 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:37.769 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.769 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:694 3.a 3:54bf2b96:::rbd_data.11acdc0eab99.0000000000000067:head [delete] snapc 5=[5] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:37.770 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.214638+0000", 2024-09-18T04:56:37.770 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.546236169, 2024-09-18T04:56:37.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0054213300000000002, 2024-09-18T04:56:37.770 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:37.770 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:37.770 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:37.770 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:37.770 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:37.771 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 694 2024-09-18T04:56:37.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.771 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:37.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:37.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.214638+0000", 2024-09-18T04:56:37.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:37.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.214638+0000", 2024-09-18T04:56:37.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:37.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.214637+0000", 2024-09-18T04:56:37.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:37.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:37.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.214640+0000", 2024-09-18T04:56:37.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2930000000000001e-06 2024-09-18T04:56:37.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:37.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.214647+0000", 2024-09-18T04:56:37.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0219999999999999e-06 2024-09-18T04:56:37.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:37.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.214653+0000", 2024-09-18T04:56:37.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3190000000000002e-06 2024-09-18T04:56:37.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:37.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215473+0000", 2024-09-18T04:56:37.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00082061799999999996 2024-09-18T04:56:37.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:37.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215545+0000", 2024-09-18T04:56:37.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1282e-05 2024-09-18T04:56:37.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:37.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.215699+0000", 2024-09-18T04:56:37.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015379799999999999 2024-09-18T04:56:37.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:37.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.219619+0000", 2024-09-18T04:56:37.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039203600000000003 2024-09-18T04:56:37.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:37.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.219968+0000", 2024-09-18T04:56:37.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000348678 2024-09-18T04:56:37.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:37.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.220022+0000", 2024-09-18T04:56:37.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4049999999999999e-05 2024-09-18T04:56:37.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:37.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.220059+0000", 2024-09-18T04:56:37.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7496000000000001e-05 2024-09-18T04:56:37.776 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.776 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:37.776 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.776 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:789 3.10 3:0a366e40:::rbd_data.11ac33624de9.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write 3930015~17006] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:37.776 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.310522+0000", 2024-09-18T04:56:37.776 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.450351848, 2024-09-18T04:56:37.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0081018370000000006, 2024-09-18T04:56:37.777 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:37.777 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:37.777 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:37.777 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:37.777 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:37.777 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 789 2024-09-18T04:56:37.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.777 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:37.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:37.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.310522+0000", 2024-09-18T04:56:37.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:37.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.310522+0000", 2024-09-18T04:56:37.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:37.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.310520+0000", 2024-09-18T04:56:37.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:56:37.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:37.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.313893+0000", 2024-09-18T04:56:37.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0033724950000000001 2024-09-18T04:56:37.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:37.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.313907+0000", 2024-09-18T04:56:37.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4185999999999999e-05 2024-09-18T04:56:37.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:37.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.313913+0000", 2024-09-18T04:56:37.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0920000000000003e-06 2024-09-18T04:56:37.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:37.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.313953+0000", 2024-09-18T04:56:37.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0593000000000003e-05 2024-09-18T04:56:37.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:37.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.314062+0000", 2024-09-18T04:56:37.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108621 2024-09-18T04:56:37.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:37.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.314262+0000", 2024-09-18T04:56:37.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00019943900000000001 2024-09-18T04:56:37.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:37.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.316850+0000", 2024-09-18T04:56:37.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0025879890000000002 2024-09-18T04:56:37.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:37.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.318554+0000", 2024-09-18T04:56:37.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001704175 2024-09-18T04:56:37.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:37.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.318586+0000", 2024-09-18T04:56:37.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2638999999999997e-05 2024-09-18T04:56:37.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:37.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.318624+0000", 2024-09-18T04:56:37.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7577000000000003e-05 2024-09-18T04:56:37.782 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.782 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:37.782 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.783 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:963 3.10 3:0c01f541:::rbd_data.11ac33624de9.0000000000000007:head [stat,truncate 4194304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:37.783 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.507773+0000", 2024-09-18T04:56:37.783 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.253100498, 2024-09-18T04:56:37.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0040043889999999997, 2024-09-18T04:56:37.783 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:37.783 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:37.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:37.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:37.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:37.783 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 963 2024-09-18T04:56:37.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.783 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:37.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:37.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.507773+0000", 2024-09-18T04:56:37.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:37.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.507773+0000", 2024-09-18T04:56:37.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:37.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.507772+0000", 2024-09-18T04:56:37.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:37.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:37.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.507779+0000", 2024-09-18T04:56:37.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1890000000000001e-06 2024-09-18T04:56:37.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:37.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.507822+0000", 2024-09-18T04:56:37.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3133000000000002e-05 2024-09-18T04:56:37.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:37.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.507839+0000", 2024-09-18T04:56:37.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6840000000000001e-05 2024-09-18T04:56:37.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:37.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.508388+0000", 2024-09-18T04:56:37.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00054853699999999998 2024-09-18T04:56:37.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:37.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.508466+0000", 2024-09-18T04:56:37.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8589000000000003e-05 2024-09-18T04:56:37.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:37.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.508591+0000", 2024-09-18T04:56:37.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000124962 2024-09-18T04:56:37.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:37.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.509905+0000", 2024-09-18T04:56:37.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0013138799999999999 2024-09-18T04:56:37.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:37.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.511662+0000", 2024-09-18T04:56:37.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0017570999999999999 2024-09-18T04:56:37.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:37.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.511757+0000", 2024-09-18T04:56:37.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4135999999999995e-05 2024-09-18T04:56:37.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:37.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.511778+0000", 2024-09-18T04:56:37.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1299000000000001e-05 2024-09-18T04:56:37.788 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.789 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:37.789 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.789 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:1338 3.10 3:0ee59dfa:::rbd_data.11ac99060bb1.0000000000000028:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:37.789 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.211527+0000", 2024-09-18T04:56:37.789 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.5493464479999997, 2024-09-18T04:56:37.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014563883999999999, 2024-09-18T04:56:37.789 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:37.789 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:37.789 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:37.789 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:37.789 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:37.790 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1338 2024-09-18T04:56:37.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.790 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:37.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:37.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.211527+0000", 2024-09-18T04:56:37.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:37.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.211527+0000", 2024-09-18T04:56:37.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:37.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.211527+0000", 2024-09-18T04:56:37.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:37.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:37.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.211539+0000", 2024-09-18T04:56:37.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.276e-05 2024-09-18T04:56:37.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:37.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.211550+0000", 2024-09-18T04:56:37.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0781999999999999e-05 2024-09-18T04:56:37.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:37.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.211557+0000", 2024-09-18T04:56:37.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6039999999999998e-06 2024-09-18T04:56:37.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:37.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.211589+0000", 2024-09-18T04:56:37.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2650000000000001e-05 2024-09-18T04:56:37.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:37.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.211689+0000", 2024-09-18T04:56:37.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.993e-05 2024-09-18T04:56:37.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:56:37.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.214589+0000", 2024-09-18T04:56:37.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028995520000000001 2024-09-18T04:56:37.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:37.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.225932+0000", 2024-09-18T04:56:37.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011343463 2024-09-18T04:56:37.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:37.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.225981+0000", 2024-09-18T04:56:37.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8198000000000002e-05 2024-09-18T04:56:37.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:37.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.226047+0000", 2024-09-18T04:56:37.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6039999999999998e-05 2024-09-18T04:56:37.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:37.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.226091+0000", 2024-09-18T04:56:37.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4795000000000001e-05 2024-09-18T04:56:37.795 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.795 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:37.795 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.795 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop_reply(client.4524.0:1338 3.10 e25/19 ondisk, result = 0)", 2024-09-18T04:56:37.795 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.221299+0000", 2024-09-18T04:56:37.795 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.5395751769999997, 2024-09-18T04:56:37.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0048018840000000002, 2024-09-18T04:56:37.795 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:37.795 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:37.795 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:37.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:37.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.221299+0000", 2024-09-18T04:56:37.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:37.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.221299+0000", 2024-09-18T04:56:37.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:37.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.221298+0000", 2024-09-18T04:56:37.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:37.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:37.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.221303+0000", 2024-09-18T04:56:37.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1839999999999998e-06 2024-09-18T04:56:37.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:37.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.221317+0000", 2024-09-18T04:56:37.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3954999999999999e-05 2024-09-18T04:56:37.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:37.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.221324+0000", 2024-09-18T04:56:37.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2740000000000002e-06 2024-09-18T04:56:37.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:37.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.225960+0000", 2024-09-18T04:56:37.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0046359829999999998 2024-09-18T04:56:37.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:37.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.225972+0000", 2024-09-18T04:56:37.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2012e-05 2024-09-18T04:56:37.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:37.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.226101+0000", 2024-09-18T04:56:37.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000128536 2024-09-18T04:56:37.799 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.799 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:37.799 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.800 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:2137 3.a 3:52b2781a:::rbd_data.11ac99060bb1.00000000000001d0:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25)", 2024-09-18T04:56:37.800 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.726272+0000", 2024-09-18T04:56:37.800 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0346023899999999, 2024-09-18T04:56:37.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012668725, 2024-09-18T04:56:37.800 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:37.800 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:37.800 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:37.800 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:37.801 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:37.801 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 2137 2024-09-18T04:56:37.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.801 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:37.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:37.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.726272+0000", 2024-09-18T04:56:37.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:37.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.726272+0000", 2024-09-18T04:56:37.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:37.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.726270+0000", 2024-09-18T04:56:37.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:56:37.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:37.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.726280+0000", 2024-09-18T04:56:37.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0393e-05 2024-09-18T04:56:37.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:37.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.726298+0000", 2024-09-18T04:56:37.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8371000000000001e-05 2024-09-18T04:56:37.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:37.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.726308+0000", 2024-09-18T04:56:37.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2909999999999996e-06 2024-09-18T04:56:37.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:37.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.726348+0000", 2024-09-18T04:56:37.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0824000000000001e-05 2024-09-18T04:56:37.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:37.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.726443+0000", 2024-09-18T04:56:37.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4795000000000003e-05 2024-09-18T04:56:37.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:37.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.728645+0000", 2024-09-18T04:56:37.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022017999999999999 2024-09-18T04:56:37.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:37.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.738874+0000", 2024-09-18T04:56:37.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.010228832 2024-09-18T04:56:37.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:37.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.738882+0000", 2024-09-18T04:56:37.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2220000000000002e-06 2024-09-18T04:56:37.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:37.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.738915+0000", 2024-09-18T04:56:37.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3213999999999997e-05 2024-09-18T04:56:37.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:37.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.738940+0000", 2024-09-18T04:56:37.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5021999999999999e-05 2024-09-18T04:56:37.806 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.806 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:37.806 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.806 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_repop_reply(client.4524.0:2137 3.a e25/19 ondisk, result = 0)", 2024-09-18T04:56:37.806 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:28.734876+0000", 2024-09-18T04:56:37.806 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0259976070000008, 2024-09-18T04:56:37.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039996169999999996, 2024-09-18T04:56:37.806 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:37.806 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:37.806 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:37.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:37.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.734876+0000", 2024-09-18T04:56:37.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:37.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.734876+0000", 2024-09-18T04:56:37.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:37.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.734875+0000", 2024-09-18T04:56:37.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:37.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:37.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.734885+0000", 2024-09-18T04:56:37.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7000000000000003e-06 2024-09-18T04:56:37.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:37.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.734898+0000", 2024-09-18T04:56:37.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3481e-05 2024-09-18T04:56:37.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:37.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.734907+0000", 2024-09-18T04:56:37.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3650000000000004e-06 2024-09-18T04:56:37.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:37.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.738860+0000", 2024-09-18T04:56:37.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039534399999999999 2024-09-18T04:56:37.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:37.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.738868+0000", 2024-09-18T04:56:37.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1249999999999993e-06 2024-09-18T04:56:37.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:37.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:28.738876+0000", 2024-09-18T04:56:37.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6159999999999996e-06 2024-09-18T04:56:37.810 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.810 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:37.810 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.810 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:4378 3.a 3:52eeb3ed:::rbd_data.11ac7d9658bc.0000000000000001:head [stat,truncate 84775] snapc 9=[9] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-09-18T04:56:37.810 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:31.242021+0000", 2024-09-18T04:56:37.810 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.5188525369999999, 2024-09-18T04:56:37.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0057347800000000001, 2024-09-18T04:56:37.811 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:37.811 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:37.811 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:37.811 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:37.811 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:37.811 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4378 2024-09-18T04:56:37.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.811 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:37.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:37.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242021+0000", 2024-09-18T04:56:37.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:37.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242021+0000", 2024-09-18T04:56:37.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:37.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242020+0000", 2024-09-18T04:56:37.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:56:37.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:37.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242027+0000", 2024-09-18T04:56:37.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1110000000000002e-06 2024-09-18T04:56:37.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:37.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242040+0000", 2024-09-18T04:56:37.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3281e-05 2024-09-18T04:56:37.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:37.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242049+0000", 2024-09-18T04:56:37.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7940000000000008e-06 2024-09-18T04:56:37.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:37.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242093+0000", 2024-09-18T04:56:37.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3832999999999999e-05 2024-09-18T04:56:37.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:37.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242176+0000", 2024-09-18T04:56:37.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3184999999999997e-05 2024-09-18T04:56:37.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:37.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.242339+0000", 2024-09-18T04:56:37.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00016288500000000001 2024-09-18T04:56:37.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:37.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.244745+0000", 2024-09-18T04:56:37.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024063909999999999 2024-09-18T04:56:37.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:37.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.247692+0000", 2024-09-18T04:56:37.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002947149 2024-09-18T04:56:37.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:37.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.247720+0000", 2024-09-18T04:56:37.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7645e-05 2024-09-18T04:56:37.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:37.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:31.247756+0000", 2024-09-18T04:56:37.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.625e-05 2024-09-18T04:56:37.816 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.816 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:37.816 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.817 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:4742 3.a 3:55916b62:::rbd_object_map.11aca1181477:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:37.817 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.238085+0000", 2024-09-18T04:56:37.817 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.5227888319999998, 2024-09-18T04:56:37.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0038238590000000002, 2024-09-18T04:56:37.817 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:37.817 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:37.817 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:37.817 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:37.817 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:37.817 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4742 2024-09-18T04:56:37.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.817 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:37.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:37.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238085+0000", 2024-09-18T04:56:37.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:37.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238085+0000", 2024-09-18T04:56:37.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:37.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238085+0000", 2024-09-18T04:56:37.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:37.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:37.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238087+0000", 2024-09-18T04:56:37.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8310000000000001e-06 2024-09-18T04:56:37.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:37.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238093+0000", 2024-09-18T04:56:37.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7089999999999999e-06 2024-09-18T04:56:37.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:37.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238098+0000", 2024-09-18T04:56:37.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.515e-06 2024-09-18T04:56:37.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:37.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.239147+0000", 2024-09-18T04:56:37.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0010497740000000001 2024-09-18T04:56:37.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:37.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.239220+0000", 2024-09-18T04:56:37.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2616999999999999e-05 2024-09-18T04:56:37.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:37.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.239536+0000", 2024-09-18T04:56:37.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031586500000000003 2024-09-18T04:56:37.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:37.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241207+0000", 2024-09-18T04:56:37.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001671442 2024-09-18T04:56:37.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:37.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241822+0000", 2024-09-18T04:56:37.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00061487599999999999 2024-09-18T04:56:37.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:37.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241871+0000", 2024-09-18T04:56:37.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8971000000000003e-05 2024-09-18T04:56:37.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:37.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241909+0000", 2024-09-18T04:56:37.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7688000000000002e-05 2024-09-18T04:56:37.822 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.822 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:37.822 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.823 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:4744 3.a 3:55916b62:::rbd_object_map.11aca1181477:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:37.823 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.238285+0000", 2024-09-18T04:56:37.823 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.522589237, 2024-09-18T04:56:37.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0069044099999999997, 2024-09-18T04:56:37.823 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:37.823 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:37.823 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:37.823 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:37.823 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:37.823 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4744 2024-09-18T04:56:37.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.823 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:37.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:37.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238285+0000", 2024-09-18T04:56:37.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:37.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238285+0000", 2024-09-18T04:56:37.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:37.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238284+0000", 2024-09-18T04:56:37.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:37.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:37.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238289+0000", 2024-09-18T04:56:37.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.096e-06 2024-09-18T04:56:37.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:37.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238300+0000", 2024-09-18T04:56:37.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1161999999999999e-05 2024-09-18T04:56:37.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:37.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238308+0000", 2024-09-18T04:56:37.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.267e-06 2024-09-18T04:56:37.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:37.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.239837+0000", 2024-09-18T04:56:37.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0015297990000000001 2024-09-18T04:56:37.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:37.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.239911+0000", 2024-09-18T04:56:37.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3356999999999996e-05 2024-09-18T04:56:37.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:37.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.240235+0000", 2024-09-18T04:56:37.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00032462600000000001 2024-09-18T04:56:37.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:37.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.243412+0000", 2024-09-18T04:56:37.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0031762240000000001 2024-09-18T04:56:37.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:37.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245106+0000", 2024-09-18T04:56:37.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001694128 2024-09-18T04:56:37.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:37.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245151+0000", 2024-09-18T04:56:37.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5225999999999997e-05 2024-09-18T04:56:37.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:37.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245189+0000", 2024-09-18T04:56:37.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8229000000000003e-05 2024-09-18T04:56:37.828 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.828 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:37.828 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.828 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:4746 3.a 3:55916b62:::rbd_object_map.11aca1181477:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:37.828 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.238313+0000", 2024-09-18T04:56:37.829 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.5225610759999997, 2024-09-18T04:56:37.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0069673240000000004, 2024-09-18T04:56:37.829 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:37.829 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:37.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:37.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:37.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:37.829 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4746 2024-09-18T04:56:37.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.829 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:37.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:37.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238313+0000", 2024-09-18T04:56:37.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:37.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238313+0000", 2024-09-18T04:56:37.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:37.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238312+0000", 2024-09-18T04:56:37.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:37.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:37.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238317+0000", 2024-09-18T04:56:37.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3189999999999996e-06 2024-09-18T04:56:37.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:37.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238323+0000", 2024-09-18T04:56:37.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.28e-06 2024-09-18T04:56:37.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:37.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238328+0000", 2024-09-18T04:56:37.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7990000000000001e-06 2024-09-18T04:56:37.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:37.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.240597+0000", 2024-09-18T04:56:37.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022687229999999998 2024-09-18T04:56:37.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:37.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.240672+0000", 2024-09-18T04:56:37.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5142000000000007e-05 2024-09-18T04:56:37.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:37.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.240973+0000", 2024-09-18T04:56:37.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030111999999999999 2024-09-18T04:56:37.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:37.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.243446+0000", 2024-09-18T04:56:37.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024727479999999999 2024-09-18T04:56:37.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:37.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245199+0000", 2024-09-18T04:56:37.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0017535809999999999 2024-09-18T04:56:37.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:37.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245244+0000", 2024-09-18T04:56:37.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4458e-05 2024-09-18T04:56:37.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:37.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245280+0000", 2024-09-18T04:56:37.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6613999999999999e-05 2024-09-18T04:56:37.834 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.835 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:37.835 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.835 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:4747 3.a 3:55916b62:::rbd_object_map.11aca1181477:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:37.835 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.238332+0000", 2024-09-18T04:56:37.835 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.5225419259999997, 2024-09-18T04:56:37.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0070349059999999996, 2024-09-18T04:56:37.835 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:37.835 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:37.835 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:37.835 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:37.835 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:37.835 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4747 2024-09-18T04:56:37.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.836 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:37.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:37.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238332+0000", 2024-09-18T04:56:37.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:37.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238332+0000", 2024-09-18T04:56:37.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:37.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238332+0000", 2024-09-18T04:56:37.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:37.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:37.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238334+0000", 2024-09-18T04:56:37.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5780000000000001e-06 2024-09-18T04:56:37.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:37.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238340+0000", 2024-09-18T04:56:37.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4399999999999996e-06 2024-09-18T04:56:37.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:37.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238345+0000", 2024-09-18T04:56:37.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4099999999999999e-06 2024-09-18T04:56:37.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:37.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241231+0000", 2024-09-18T04:56:37.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028862369999999998 2024-09-18T04:56:37.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:37.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241274+0000", 2024-09-18T04:56:37.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2308000000000002e-05 2024-09-18T04:56:37.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:37.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241450+0000", 2024-09-18T04:56:37.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000176276 2024-09-18T04:56:37.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:37.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.243482+0000", 2024-09-18T04:56:37.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0020318689999999999 2024-09-18T04:56:37.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:37.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245289+0000", 2024-09-18T04:56:37.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001807016 2024-09-18T04:56:37.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:37.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245331+0000", 2024-09-18T04:56:37.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2218999999999997e-05 2024-09-18T04:56:37.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:37.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245367+0000", 2024-09-18T04:56:37.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5920999999999997e-05 2024-09-18T04:56:37.841 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.841 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:37.841 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.841 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:4749 3.a 3:55916b62:::rbd_object_map.11aca1181477:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:37.841 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.238349+0000", 2024-09-18T04:56:37.841 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.5225244609999997, 2024-09-18T04:56:37.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0071690089999999996, 2024-09-18T04:56:37.841 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:37.841 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:37.841 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:37.841 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:37.841 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:37.842 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4749 2024-09-18T04:56:37.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.842 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:37.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:37.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238349+0000", 2024-09-18T04:56:37.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:37.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238349+0000", 2024-09-18T04:56:37.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:37.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238349+0000", 2024-09-18T04:56:37.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:37.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:37.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238362+0000", 2024-09-18T04:56:37.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2802999999999999e-05 2024-09-18T04:56:37.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:37.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238370+0000", 2024-09-18T04:56:37.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6520000000000006e-06 2024-09-18T04:56:37.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:37.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238375+0000", 2024-09-18T04:56:37.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1930000000000001e-06 2024-09-18T04:56:37.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:37.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241943+0000", 2024-09-18T04:56:37.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0035684710000000001 2024-09-18T04:56:37.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:37.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.242012+0000", 2024-09-18T04:56:37.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9282999999999994e-05 2024-09-18T04:56:37.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:37.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.242328+0000", 2024-09-18T04:56:37.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00031586899999999998 2024-09-18T04:56:37.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:37.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245375+0000", 2024-09-18T04:56:37.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030469120000000001 2024-09-18T04:56:37.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:37.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245425+0000", 2024-09-18T04:56:37.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9391999999999997e-05 2024-09-18T04:56:37.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:37.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245484+0000", 2024-09-18T04:56:37.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9169999999999998e-05 2024-09-18T04:56:37.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:37.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245518+0000", 2024-09-18T04:56:37.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4653999999999997e-05 2024-09-18T04:56:37.848 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.848 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:37.848 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.848 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:4750 3.a 3:55916b62:::rbd_object_map.11aca1181477:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:37.848 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.238701+0000", 2024-09-18T04:56:37.848 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.5221726430000002, 2024-09-18T04:56:37.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0069625720000000002, 2024-09-18T04:56:37.848 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:37.848 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:37.848 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:37.849 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:37.849 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:37.849 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4750 2024-09-18T04:56:37.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.849 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:37.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:37.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238701+0000", 2024-09-18T04:56:37.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:37.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238701+0000", 2024-09-18T04:56:37.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:37.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238700+0000", 2024-09-18T04:56:37.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:37.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:37.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238706+0000", 2024-09-18T04:56:37.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8869999999999997e-06 2024-09-18T04:56:37.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:37.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238718+0000", 2024-09-18T04:56:37.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1640999999999999e-05 2024-09-18T04:56:37.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:37.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.238725+0000", 2024-09-18T04:56:37.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.221e-06 2024-09-18T04:56:37.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:37.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.242659+0000", 2024-09-18T04:56:37.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039341649999999999 2024-09-18T04:56:37.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:37.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.242735+0000", 2024-09-18T04:56:37.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6190000000000001e-05 2024-09-18T04:56:37.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:37.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.243087+0000", 2024-09-18T04:56:37.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035155499999999998 2024-09-18T04:56:37.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:37.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245578+0000", 2024-09-18T04:56:37.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.002490618 2024-09-18T04:56:37.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:37.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245586+0000", 2024-09-18T04:56:37.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2619999999999998e-06 2024-09-18T04:56:37.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:37.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245630+0000", 2024-09-18T04:56:37.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.405e-05 2024-09-18T04:56:37.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:37.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245664+0000", 2024-09-18T04:56:37.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3983000000000003e-05 2024-09-18T04:56:37.854 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.854 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:37.854 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.854 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:4752 3.a 3:55916b62:::rbd_object_map.11aca1181477:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:37.854 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.240989+0000", 2024-09-18T04:56:37.854 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.5198852460000003, 2024-09-18T04:56:37.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0085146609999999998, 2024-09-18T04:56:37.854 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:37.855 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:37.855 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:37.855 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:37.855 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:37.855 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4752 2024-09-18T04:56:37.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.855 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:37.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:37.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.240989+0000", 2024-09-18T04:56:37.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:37.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.240989+0000", 2024-09-18T04:56:37.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:37.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.240988+0000", 2024-09-18T04:56:37.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:37.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:37.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241002+0000", 2024-09-18T04:56:37.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4042999999999999e-05 2024-09-18T04:56:37.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:37.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241014+0000", 2024-09-18T04:56:37.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2469e-05 2024-09-18T04:56:37.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:37.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241021+0000", 2024-09-18T04:56:37.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1489999999999997e-06 2024-09-18T04:56:37.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:37.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.243506+0000", 2024-09-18T04:56:37.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024841199999999998 2024-09-18T04:56:37.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:37.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.243572+0000", 2024-09-18T04:56:37.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6873999999999997e-05 2024-09-18T04:56:37.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:37.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.243899+0000", 2024-09-18T04:56:37.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000326161 2024-09-18T04:56:37.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:37.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.247781+0000", 2024-09-18T04:56:37.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0038827359999999999 2024-09-18T04:56:37.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:37.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.249442+0000", 2024-09-18T04:56:37.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016604460000000001 2024-09-18T04:56:37.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:37.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.249478+0000", 2024-09-18T04:56:37.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6695e-05 2024-09-18T04:56:37.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:37.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.249503+0000", 2024-09-18T04:56:37.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4905999999999998e-05 2024-09-18T04:56:37.860 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.860 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:37.860 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.861 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:4753 3.a 3:55916b62:::rbd_object_map.11aca1181477:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:37.861 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.241030+0000", 2024-09-18T04:56:37.861 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.5198440550000001, 2024-09-18T04:56:37.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011505873999999999, 2024-09-18T04:56:37.861 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:37.861 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:37.861 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:37.861 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:37.861 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:37.861 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4753 2024-09-18T04:56:37.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.861 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:37.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:37.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241030+0000", 2024-09-18T04:56:37.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:37.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241030+0000", 2024-09-18T04:56:37.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:37.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241029+0000", 2024-09-18T04:56:37.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:37.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:37.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241033+0000", 2024-09-18T04:56:37.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5410000000000001e-06 2024-09-18T04:56:37.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:37.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241040+0000", 2024-09-18T04:56:37.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1019999999999999e-06 2024-09-18T04:56:37.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:37.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241045+0000", 2024-09-18T04:56:37.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1449999999999999e-06 2024-09-18T04:56:37.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:37.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.244262+0000", 2024-09-18T04:56:37.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003216413 2024-09-18T04:56:37.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:37.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.244353+0000", 2024-09-18T04:56:37.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1048e-05 2024-09-18T04:56:37.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:37.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.244728+0000", 2024-09-18T04:56:37.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000375516 2024-09-18T04:56:37.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:37.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.247786+0000", 2024-09-18T04:56:37.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030578609999999998 2024-09-18T04:56:37.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:37.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.252457+0000", 2024-09-18T04:56:37.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0046708319999999998 2024-09-18T04:56:37.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:37.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.252502+0000", 2024-09-18T04:56:37.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4694000000000003e-05 2024-09-18T04:56:37.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:37.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.252536+0000", 2024-09-18T04:56:37.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4181999999999999e-05 2024-09-18T04:56:37.866 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.866 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:37.866 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.867 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:4754 3.a 3:55916b62:::rbd_object_map.11aca1181477:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:37.867 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.241193+0000", 2024-09-18T04:56:37.867 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.5196806049999996, 2024-09-18T04:56:37.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011431838, 2024-09-18T04:56:37.867 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:37.867 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:37.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:37.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:37.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:37.867 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4754 2024-09-18T04:56:37.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.868 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:37.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:37.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241193+0000", 2024-09-18T04:56:37.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:37.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241193+0000", 2024-09-18T04:56:37.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:37.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241192+0000", 2024-09-18T04:56:37.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:37.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:37.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241198+0000", 2024-09-18T04:56:37.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1419999999999998e-06 2024-09-18T04:56:37.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:37.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241210+0000", 2024-09-18T04:56:37.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.187e-05 2024-09-18T04:56:37.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:37.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241218+0000", 2024-09-18T04:56:37.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5000000000000002e-06 2024-09-18T04:56:37.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:37.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245710+0000", 2024-09-18T04:56:37.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0044919039999999997 2024-09-18T04:56:37.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:37.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.245780+0000", 2024-09-18T04:56:37.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0445999999999996e-05 2024-09-18T04:56:37.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:37.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.246139+0000", 2024-09-18T04:56:37.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035877800000000003 2024-09-18T04:56:37.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:37.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.247790+0000", 2024-09-18T04:56:37.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0016513980000000001 2024-09-18T04:56:37.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:37.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.252569+0000", 2024-09-18T04:56:37.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0047786149999999999 2024-09-18T04:56:37.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:37.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.252595+0000", 2024-09-18T04:56:37.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6347999999999999e-05 2024-09-18T04:56:37.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:37.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.252625+0000", 2024-09-18T04:56:37.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9884000000000001e-05 2024-09-18T04:56:37.873 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.873 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:37.873 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.873 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:4755 3.a 3:55916b62:::rbd_object_map.11aca1181477:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:37.873 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.241239+0000", 2024-09-18T04:56:37.873 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.519634698, 2024-09-18T04:56:37.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011505253, 2024-09-18T04:56:37.873 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:37.873 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:37.873 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:37.874 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:37.874 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:37.874 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 4755 2024-09-18T04:56:37.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.874 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:37.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:37.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241239+0000", 2024-09-18T04:56:37.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:37.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241239+0000", 2024-09-18T04:56:37.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241239+0000", 2024-09-18T04:56:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241242+0000", 2024-09-18T04:56:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7120000000000002e-06 2024-09-18T04:56:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241253+0000", 2024-09-18T04:56:37.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0308e-05 2024-09-18T04:56:37.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:37.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.241258+0000", 2024-09-18T04:56:37.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2460000000000003e-06 2024-09-18T04:56:37.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:37.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.246891+0000", 2024-09-18T04:56:37.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0056336399999999997 2024-09-18T04:56:37.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.246977+0000", 2024-09-18T04:56:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.577e-05 2024-09-18T04:56:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.247393+0000", 2024-09-18T04:56:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00041592899999999999 2024-09-18T04:56:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.251987+0000", 2024-09-18T04:56:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0045936450000000004 2024-09-18T04:56:37.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.252670+0000", 2024-09-18T04:56:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00068347000000000004 2024-09-18T04:56:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.252712+0000", 2024-09-18T04:56:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1730000000000002e-05 2024-09-18T04:56:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:37.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.252744+0000", 2024-09-18T04:56:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2477000000000001e-05 2024-09-18T04:56:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:22391 3.a 3:571d19ac:::rbd_data.11ac82b416ac.000000000000006a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-18T04:56:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:37.424203+0000", 2024-09-18T04:56:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.33667080900000002, 2024-09-18T04:56:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0039124629999999997, 2024-09-18T04:56:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:37.879 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:37.880 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:37.880 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:37.880 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 22391 2024-09-18T04:56:37.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.880 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:37.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:37.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.424203+0000", 2024-09-18T04:56:37.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:37.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.424203+0000", 2024-09-18T04:56:37.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:37.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:37.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.424202+0000", 2024-09-18T04:56:37.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:37.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:37.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.424209+0000", 2024-09-18T04:56:37.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2200000000000003e-06 2024-09-18T04:56:37.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:37.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.424225+0000", 2024-09-18T04:56:37.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5957999999999999e-05 2024-09-18T04:56:37.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:37.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.424234+0000", 2024-09-18T04:56:37.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.884e-06 2024-09-18T04:56:37.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:37.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.424284+0000", 2024-09-18T04:56:37.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0646999999999998e-05 2024-09-18T04:56:37.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:37.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.424411+0000", 2024-09-18T04:56:37.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012684600000000001 2024-09-18T04:56:37.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:37.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:37.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:37.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.428116+0000", 2024-09-18T04:56:37.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037042970000000001 2024-09-18T04:56:37.884 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.884 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:37.884 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.884 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:37.884 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:37.884 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:37.887 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:56:37.887 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:37.883+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:37.889 INFO:teuthology.orchestra.run.smithi002.stdout:1147 trunc from 0x87d10a1 to 0x1179ec8 2024-09-18T04:56:37.889 INFO:teuthology.orchestra.run.smithi002.stdout:1148 write 0xd176e60 thru 0xd17c01c (0x51bd bytes) 2024-09-18T04:56:37.889 INFO:teuthology.orchestra.run.smithi002.stdout:1149 writesame 0x58506b7 thru 0x585629e data_size 0x8(0x5be8 bytes) 2024-09-18T04:56:37.889 INFO:teuthology.orchestra.run.smithi002.stdout:1150 write 0x224f9fb thru 0x225f091 (0xf697 bytes) 2024-09-18T04:56:37.889 INFO:teuthology.orchestra.run.smithi002.stdout:1151 write 0xb55026b thru 0xb55907a (0x8e10 bytes) 2024-09-18T04:56:37.889 INFO:teuthology.orchestra.run.smithi002.stdout:1152 read 0x37d153d thru 0x37d8935 (0x73f9 bytes) 2024-09-18T04:56:37.890 INFO:teuthology.orchestra.run.smithi002.stdout:1153 write 0x3b291cb thru 0x3b360dd (0xcf13 bytes) 2024-09-18T04:56:37.890 INFO:teuthology.orchestra.run.smithi002.stdout:1154 write 0x2ed23c2 thru 0x2ed3911 (0x1550 bytes) 2024-09-18T04:56:37.890 INFO:teuthology.orchestra.run.smithi002.stdout:1155 read 0x8d5df19 thru 0x8d5ec88 (0xd70 bytes) 2024-09-18T04:56:37.890 INFO:teuthology.orchestra.run.smithi002.stdout:1157 write 0x6b70070 thru 0x6b7cd46 (0xccd7 bytes) 2024-09-18T04:56:37.890 INFO:teuthology.orchestra.run.smithi002.stdout:1158 writesame 0xd8350b7 thru 0xd837734 data_size 0xd(0x267e bytes) 2024-09-18T04:56:37.890 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:37.890 INFO:teuthology.orchestra.run.smithi002.stdout:1160 trunc from 0xd837735 to 0xa05228d 2024-09-18T04:56:37.890 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:37.890 INFO:teuthology.orchestra.run.smithi002.stdout:1162 write 0xccdcf64 thru 0xcce6687 (0x9724 bytes) 2024-09-18T04:56:37.890 INFO:teuthology.orchestra.run.smithi002.stdout:1163 read 0xa5c2cfe thru 0xa5cc75a (0x9a5d bytes) 2024-09-18T04:56:37.890 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:37.890 INFO:teuthology.orchestra.run.smithi002.stdout:1166 write 0x9095dcb thru 0x90996e8 (0x391e bytes) 2024-09-18T04:56:37.890 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:37.891 INFO:teuthology.orchestra.run.smithi002.stdout:1169 read 0xbcd3fb2 thru 0xbce2281 (0xe2d0 bytes) 2024-09-18T04:56:37.891 INFO:teuthology.orchestra.run.smithi002.stdout:1170 writesame 0xae0aaea thru 0xae0fc15 data_size 0xa(0x512c bytes) 2024-09-18T04:56:37.891 INFO:teuthology.orchestra.run.smithi002.stdout:1171 trunc from 0xcce6688 to 0xd8282a6 2024-09-18T04:56:37.891 INFO:teuthology.orchestra.run.smithi002.stdout:1172 writesame 0x849c69d thru 0x84a1f38 data_size 0x4(0x589c bytes) 2024-09-18T04:56:37.891 INFO:teuthology.orchestra.run.smithi002.stdout:1173 punch from 0x593d6ef to 0x594819c, (0xaaad bytes) 2024-09-18T04:56:37.891 INFO:teuthology.orchestra.run.smithi002.stdout:1174 punch from 0x198976a to 0x1998f0c, (0xf7a2 bytes) 2024-09-18T04:56:37.891 INFO:teuthology.orchestra.run.smithi002.stdout:1175 trunc from 0xd8282a6 to 0xa07e32e 2024-09-18T04:56:37.891 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:37.891 INFO:teuthology.orchestra.run.smithi002.stdout:1179 write 0x73f82f2 thru 0x73fe6ed (0x63fc bytes) 2024-09-18T04:56:37.891 INFO:teuthology.orchestra.run.smithi002.stdout:1181 write 0x66654b6 thru 0x666abba (0x5705 bytes) 2024-09-18T04:56:37.892 INFO:teuthology.orchestra.run.smithi002.stdout:1184 write 0x538c9c thru 0x5465d0 (0xd935 bytes) 2024-09-18T04:56:37.893 INFO:teuthology.orchestra.run.smithi002.stdout:1186 read 0x79f638e thru 0x7a049cf (0xe642 bytes) 2024-09-18T04:56:37.893 INFO:teuthology.orchestra.run.smithi002.stdout:1187 punch from 0x5043123 to 0x5052617, (0xf4f4 bytes) 2024-09-18T04:56:37.893 INFO:teuthology.orchestra.run.smithi002.stdout:1189 read 0xfbbc6d thru 0xfc2b80 (0x6f14 bytes) 2024-09-18T04:56:37.893 INFO:teuthology.orchestra.run.smithi002.stdout:1190 punch from 0x4dd2d3f to 0x4dd3e62, (0x1123 bytes) 2024-09-18T04:56:37.894 INFO:teuthology.orchestra.run.smithi002.stdout:1191 punch from 0x3c7b050 to 0x3c7dcc6, (0x2c76 bytes) 2024-09-18T04:56:37.894 INFO:teuthology.orchestra.run.smithi002.stdout:1192 writesame 0x39e02ba thru 0x39e500b data_size 0x3(0x4d52 bytes) 2024-09-18T04:56:37.898 INFO:teuthology.orchestra.run.smithi002.stdout:1193 flatten 2024-09-18T04:56:37.903 INFO:teuthology.orchestra.run.smithi002.stdout:1194 write 0x7e2381a thru 0x7e26486 (0x2c6d bytes) 2024-09-18T04:56:37.915 INFO:teuthology.orchestra.run.smithi002.stdout:1197 trunc from 0xa07e32e to 0x2b84eee 2024-09-18T04:56:37.925 INFO:teuthology.orchestra.run.smithi002.stdout:1198 read 0x2681c95 thru 0x268ab3e (0x8eaa bytes) 2024-09-18T04:56:37.925 INFO:teuthology.orchestra.run.smithi002.stdout:1200 punch from 0xd39514 to 0xd3a2d4, (0xdc0 bytes) 2024-09-18T04:56:37.925 INFO:teuthology.orchestra.run.smithi002.stdout:1201 read 0x2a98741 thru 0x2aa404c (0xb90c bytes) 2024-09-18T04:56:37.926 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:37.926 INFO:teuthology.orchestra.run.smithi002.stdout:1203 punch from 0x256ea5c to 0x257a9b1, (0xbf55 bytes) 2024-09-18T04:56:37.926 INFO:teuthology.orchestra.run.smithi002.stdout:1204 punch from 0x290b5d1 to 0x291a231, (0xec60 bytes) 2024-09-18T04:56:37.936 INFO:teuthology.orchestra.run.smithi002.stdout:1206 trunc from 0x2b84eee to 0xc9fc260 2024-09-18T04:56:37.942 INFO:teuthology.orchestra.run.smithi002.stdout:1207 punch from 0x30e64b to 0x3169b6, (0x836b bytes) 2024-09-18T04:56:37.942 INFO:teuthology.orchestra.run.smithi002.stdout:1208 trunc from 0xc9fc260 to 0xc14ae43 2024-09-18T04:56:37.948 INFO:teuthology.orchestra.run.smithi002.stdout:1209 punch from 0x6b8f2ab to 0x6b98253, (0x8fa8 bytes) 2024-09-18T04:56:37.948 INFO:teuthology.orchestra.run.smithi002.stdout:1210 read 0x82f70e2 thru 0x82f8404 (0x1323 bytes) 2024-09-18T04:56:37.948 INFO:teuthology.orchestra.run.smithi002.stdout:1211 writesame 0x5d8fc94 thru 0x5d999da data_size 0x1(0x9d47 bytes) 2024-09-18T04:56:37.970 INFO:teuthology.orchestra.run.smithi002.stdout:1212 punch from 0xbe561c7 to 0xbe5809c, (0x1ed5 bytes) 2024-09-18T04:56:37.970 INFO:teuthology.orchestra.run.smithi002.stdout:1214 punch from 0x2d5d4b6 to 0x2d63711, (0x625b bytes) 2024-09-18T04:56:37.970 INFO:teuthology.orchestra.run.smithi002.stdout:1215 writesame 0x5e25c1a thru 0x5e3002c data_size 0x27(0xa413 bytes) 2024-09-18T04:56:37.973 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:37.973 INFO:teuthology.orchestra.run.smithi002.stdout:1217 read 0x5c36a00 thru 0x5c3bb13 (0x5114 bytes) 2024-09-18T04:56:37.973 INFO:teuthology.orchestra.run.smithi002.stdout:1218 read 0xad301f9 thru 0xad3712f (0x6f37 bytes) 2024-09-18T04:56:37.974 INFO:teuthology.orchestra.run.smithi002.stdout:1219 write 0xbf15ea4 thru 0xbf214d9 (0xb636 bytes) 2024-09-18T04:56:37.979 INFO:teuthology.orchestra.run.smithi002.stdout:1220 trunc from 0xc14ae43 to 0x91a9e6c 2024-09-18T04:56:37.987 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:37.984+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:38.001 INFO:teuthology.orchestra.run.smithi002.stdout:1221 writesame 0xdb50870 thru 0xdb52212 data_size 0x1(0x19a3 bytes) 2024-09-18T04:56:38.001 INFO:teuthology.orchestra.run.smithi002.stdout:1222 read 0x9e2b5cc thru 0x9e36383 (0xadb8 bytes) 2024-09-18T04:56:38.001 INFO:teuthology.orchestra.run.smithi002.stdout:1223 read 0x291c086 thru 0x2926a14 (0xa98f bytes) 2024-09-18T04:56:38.001 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.001 INFO:teuthology.orchestra.run.smithi002.stdout:1225 read 0x380f461 thru 0x380fd64 (0x904 bytes) 2024-09-18T04:56:38.002 INFO:teuthology.orchestra.run.smithi002.stdout:1226 read 0xd9df196 thru 0xd9ebbb9 (0xca24 bytes) 2024-09-18T04:56:38.002 INFO:teuthology.orchestra.run.smithi002.stdout:1227 write 0x3c106c9 thru 0x3c12b9b (0x24d3 bytes) 2024-09-18T04:56:38.007 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:56:38.015 INFO:teuthology.orchestra.run.smithi002.stdout:1228 trunc from 0xdb52213 to 0x3654c1e 2024-09-18T04:56:38.026 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.026 INFO:teuthology.orchestra.run.smithi002.stdout:1230 read 0xaeb216 thru 0xaf53a9 (0xa194 bytes) 2024-09-18T04:56:38.038 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:38.038 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:38.039 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:56:38.039 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:38.040 INFO:teuthology.orchestra.run.smithi002.stdout:1234 writesame 0x931b054 thru 0x93212e7 data_size 0x12(0x6294 bytes) 2024-09-18T04:56:38.040 INFO:teuthology.orchestra.run.smithi002.stdout:1235 writesame 0xd42dcd thru 0xd478e6 data_size 0x1(0x4b1a bytes) 2024-09-18T04:56:38.048 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:56:38.050 INFO:teuthology.orchestra.run.smithi002.stdout:1236 trunc from 0x93212e8 to 0x951bfa9 2024-09-18T04:56:38.054 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.059 INFO:teuthology.orchestra.run.smithi002.stdout:1240 trunc from 0x951bfa9 to 0xe716900 2024-09-18T04:56:38.066 INFO:teuthology.orchestra.run.smithi002.stdout:1243 read 0x9e04e2c thru 0x9e10d52 (0xbf27 bytes) 2024-09-18T04:56:38.066 INFO:teuthology.orchestra.run.smithi002.stdout:1246 write 0x60b0f80 thru 0x60bbe3f (0xaec0 bytes) 2024-09-18T04:56:38.069 INFO:teuthology.orchestra.run.smithi002.stdout:1247 writesame 0x29c6afd thru 0x29d50c1 data_size 0x7(0xe5c5 bytes) 2024-09-18T04:56:38.087 INFO:teuthology.orchestra.run.smithi002.stdout:1248 trunc from 0xe716900 to 0x32227ed 2024-09-18T04:56:38.089 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:38.087+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:38.096 INFO:teuthology.orchestra.run.smithi002.stdout:1249 read 0x2ca91f8 thru 0x2cb4573 (0xb37c bytes) 2024-09-18T04:56:38.096 INFO:teuthology.orchestra.run.smithi002.stdout:1250 read 0xc263d0 thru 0xc2b0db (0x4d0c bytes) 2024-09-18T04:56:38.096 INFO:teuthology.orchestra.run.smithi002.stdout:1254 writesame 0xa22081 thru 0xa2e731 data_size 0x5(0xc6b1 bytes) 2024-09-18T04:56:38.103 INFO:teuthology.orchestra.run.smithi002.stdout:1255 read 0x1f544ad thru 0x1f5a575 (0x60c9 bytes) 2024-09-18T04:56:38.103 INFO:teuthology.orchestra.run.smithi002.stdout:1256 punch from 0x75af91 to 0x75e3dc, (0x344b bytes) 2024-09-18T04:56:38.103 INFO:teuthology.orchestra.run.smithi002.stdout:1257 read 0x1abb3fd thru 0x1ac6540 (0xb144 bytes) 2024-09-18T04:56:38.103 INFO:teuthology.orchestra.run.smithi002.stdout:1258 read 0xd545be thru 0xd58514 (0x3f57 bytes) 2024-09-18T04:56:38.103 INFO:teuthology.orchestra.run.smithi002.stdout:1259 read 0x1294668 thru 0x1298b29 (0x44c2 bytes) 2024-09-18T04:56:38.104 INFO:teuthology.orchestra.run.smithi002.stdout:1261 trunc from 0x32227ed to 0x20a1086 2024-09-18T04:56:38.116 INFO:teuthology.orchestra.run.smithi002.stdout:1262 writesame 0x1e109cc thru 0x1e1d44a data_size 0x1(0xca7f bytes) 2024-09-18T04:56:38.143 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:56:38.143 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:56:38.159 INFO:teuthology.orchestra.run.smithi002.stdout:1265 write 0xeb7b33f thru 0xeb828f7 (0x75b9 bytes) 2024-09-18T04:56:38.161 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:56:38.162 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:56:38.165 INFO:teuthology.orchestra.run.smithi002.stdout:1267 trunc from 0xeb828f8 to 0xbe7308c 2024-09-18T04:56:38.175 INFO:teuthology.orchestra.run.smithi002.stdout:1270 read 0xbc40d5 thru 0xbcc1bd (0x80e9 bytes) 2024-09-18T04:56:38.175 INFO:teuthology.orchestra.run.smithi002.stdout:1271 read 0x90e73a0 thru 0x90eea13 (0x7674 bytes) 2024-09-18T04:56:38.175 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.175 INFO:teuthology.orchestra.run.smithi002.stdout:1274 read 0x34a6261 thru 0x34a8a88 (0x2828 bytes) 2024-09-18T04:56:38.175 INFO:teuthology.orchestra.run.smithi002.stdout:1276 writesame 0xa6742fa thru 0xa6783ca data_size 0x1(0x40d1 bytes) 2024-09-18T04:56:38.180 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T04:56:38.180 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T04:56:38.181 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-18T04:56:38.181 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T04:56:38.181 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T04:56:38.181 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-09-18T04:56:38.182 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:56:38.182 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:56:38.182 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-18T04:56:38.185 INFO:teuthology.orchestra.run.smithi002.stdout:1277 write 0xbb88f4 thru 0xbbced7 (0x45e4 bytes) 2024-09-18T04:56:38.189 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:38.187+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:38.191 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.193 INFO:teuthology.orchestra.run.smithi002.stdout:1279 trunc from 0xbe7308c to 0xcd56961 2024-09-18T04:56:38.197 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:38.197 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:38.197 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:56:38.197 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:56:38.197 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:38.198 INFO:teuthology.orchestra.run.smithi002.stdout:1280 write 0x21df67a thru 0x21e13c9 (0x1d50 bytes) 2024-09-18T04:56:38.201 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:56:38.201 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:56:38.203 INFO:teuthology.orchestra.run.smithi002.stdout:1282 read 0x342e536 thru 0x343112a (0x2bf5 bytes) 2024-09-18T04:56:38.203 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.203 INFO:teuthology.orchestra.run.smithi002.stdout:1284 read 0x37f089d thru 0x37fc173 (0xb8d7 bytes) 2024-09-18T04:56:38.203 INFO:teuthology.orchestra.run.smithi002.stdout:1286 read 0x6343bb5 thru 0x634e9d9 (0xae25 bytes) 2024-09-18T04:56:38.203 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.204 INFO:teuthology.orchestra.run.smithi002.stdout:1290 trunc from 0xcd56961 to 0xda351cc 2024-09-18T04:56:38.207 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:56:38.209 INFO:teuthology.orchestra.run.smithi002.stdout:1293 write 0x96daa7e thru 0x96e9d65 (0xf2e8 bytes) 2024-09-18T04:56:38.214 INFO:teuthology.orchestra.run.smithi002.stdout:1294 write 0x1adc1f4 thru 0x1ae65ee (0xa3fb bytes) 2024-09-18T04:56:38.217 INFO:teuthology.orchestra.run.smithi002.stdout:1296 read 0x35ad27a thru 0x35b8ea2 (0xbc29 bytes) 2024-09-18T04:56:38.217 INFO:teuthology.orchestra.run.smithi002.stdout:1297 write 0xb8345fd thru 0xb8413e2 (0xcde6 bytes) 2024-09-18T04:56:38.219 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:56:38.219 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:56:38.222 INFO:teuthology.orchestra.run.smithi002.stdout:1298 read 0x4ace937 thru 0x4ad60cc (0x7796 bytes) 2024-09-18T04:56:38.223 INFO:teuthology.orchestra.run.smithi002.stdout:1299 write 0xd2c634d thru 0xd2cb9f5 (0x56a9 bytes) 2024-09-18T04:56:38.226 INFO:teuthology.orchestra.run.smithi002.stdout:1300 punch from 0xba49c1f to 0xba5882c, (0xec0d bytes) 2024-09-18T04:56:38.226 INFO:teuthology.orchestra.run.smithi002.stdout:1301 write 0x6dd3221 thru 0x6ddf074 (0xbe54 bytes) 2024-09-18T04:56:38.229 INFO:teuthology.orchestra.run.smithi002.stdout:1302 punch from 0xb7e792d to 0xb7e7d3b, (0x40e bytes) 2024-09-18T04:56:38.229 INFO:teuthology.orchestra.run.smithi002.stdout:1303 writesame 0xd8d4fd3 thru 0xd8d7c01 data_size 0x1(0x2c2f bytes) 2024-09-18T04:56:38.236 INFO:teuthology.orchestra.run.smithi002.stdout:1305 trunc from 0xda351cc to 0xda2ead4 2024-09-18T04:56:38.237 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:56:38.237 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:56:38.241 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.241 INFO:teuthology.orchestra.run.smithi002.stdout:1307 write 0x547c57c thru 0x5480852 (0x42d7 bytes) 2024-09-18T04:56:38.244 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.245 INFO:teuthology.orchestra.run.smithi002.stdout:1309 read 0xab2b4a3 thru 0xab2fcf4 (0x4852 bytes) 2024-09-18T04:56:38.254 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:56:38.255 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:56:38.256 INFO:teuthology.orchestra.run.smithi002.stdout:1310 trunc from 0xda2ead4 to 0x3f2d9a1 2024-09-18T04:56:38.267 INFO:teuthology.orchestra.run.smithi002.stdout:1311 punch from 0x1319329 to 0x13283b5, (0xf08c bytes) 2024-09-18T04:56:38.267 INFO:teuthology.orchestra.run.smithi002.stdout:1312 punch from 0x179e69c to 0x179fd26, (0x168a bytes) 2024-09-18T04:56:38.267 INFO:teuthology.orchestra.run.smithi002.stdout:1313 read 0x2ce4327 thru 0x2ce71c1 (0x2e9b bytes) 2024-09-18T04:56:38.273 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:56:38.274 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:56:38.276 INFO:teuthology.orchestra.run.smithi002.stdout:1314 write 0x82ddc04 thru 0x82e01b5 (0x25b2 bytes) 2024-09-18T04:56:38.279 INFO:teuthology.orchestra.run.smithi002.stdout:1315 write 0x646cbee thru 0x646e3a7 (0x17ba bytes) 2024-09-18T04:56:38.282 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.282 INFO:teuthology.orchestra.run.smithi002.stdout:1319 read 0x4ad54a4 thru 0x4adc979 (0x74d6 bytes) 2024-09-18T04:56:38.283 INFO:teuthology.orchestra.run.smithi002.stdout:1320 trunc from 0x82e01b6 to 0x7460bfd 2024-09-18T04:56:38.295 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.295 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.295 INFO:teuthology.orchestra.run.smithi002.stdout:1325 read 0x140bd60 thru 0x140bded (0x8e bytes) 2024-09-18T04:56:38.295 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:38.293+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:38.302 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:38.302 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "" 2024-09-18T04:56:38.302 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:38.308 INFO:teuthology.orchestra.run.smithi002.stdout:1326 writesame 0xe9fb9f7 thru 0xea0329f data_size 0x17(0x78a9 bytes) 2024-09-18T04:56:38.308 INFO:teuthology.orchestra.run.smithi002.stdout:1327 read 0x114e614 thru 0x114fbab (0x1598 bytes) 2024-09-18T04:56:38.308 INFO:teuthology.orchestra.run.smithi002.stdout:1328 writesame 0x237db78 thru 0x2385c2f data_size 0xc(0x80b8 bytes) 2024-09-18T04:56:38.314 INFO:teuthology.orchestra.run.smithi002.stdout:1329 punch from 0x2de08cc to 0x2de8d16, (0x844a bytes) 2024-09-18T04:56:38.320 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:38.320 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:38.324 INFO:teuthology.orchestra.run.smithi002.stdout:1331 trunc from 0xea032a0 to 0x5447f0c 2024-09-18T04:56:38.330 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:56:38.334 INFO:teuthology.orchestra.run.smithi002.stdout:1335 writesame 0x37e9003 thru 0x37ef504 data_size 0x7(0x6502 bytes) 2024-09-18T04:56:38.338 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.338 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.348 INFO:teuthology.orchestra.run.smithi002.stdout:1343 write 0x98a6990 thru 0x98b601d (0xf68e bytes) 2024-09-18T04:56:38.351 INFO:teuthology.orchestra.run.smithi002.stdout:1344 read 0x59985ef thru 0x59a53c1 (0xcdd3 bytes) 2024-09-18T04:56:38.357 INFO:teuthology.orchestra.run.smithi002.stdout:1345 write 0xa496f2b thru 0xa4a05a0 (0x9676 bytes) 2024-09-18T04:56:38.363 INFO:teuthology.orchestra.run.smithi002.stdout:1346 trunc from 0xa4a05a1 to 0xd5de399 2024-09-18T04:56:38.368 INFO:teuthology.orchestra.run.smithi002.stdout:1347 read 0x4d36703 thru 0x4d3c776 (0x6074 bytes) 2024-09-18T04:56:38.368 INFO:teuthology.orchestra.run.smithi002.stdout:1349 read 0x1bf5afa thru 0x1c007fc (0xad03 bytes) 2024-09-18T04:56:38.368 INFO:teuthology.orchestra.run.smithi002.stdout:1350 read 0xb9ae43d thru 0xb9b06ff (0x22c3 bytes) 2024-09-18T04:56:38.368 INFO:teuthology.orchestra.run.smithi002.stdout:1353 write 0x6fbccbd thru 0x6fc4f9c (0x82e0 bytes) 2024-09-18T04:56:38.373 INFO:teuthology.orchestra.run.smithi002.stdout:1354 read 0xd45eab5 thru 0xd464733 (0x5c7f bytes) 2024-09-18T04:56:38.373 INFO:teuthology.orchestra.run.smithi002.stdout:1355 write 0x3684c46 thru 0x36850d5 (0x490 bytes) 2024-09-18T04:56:38.375 INFO:teuthology.orchestra.run.smithi002.stdout:1356 write 0x27c1cb9 thru 0x27c8263 (0x65ab bytes) 2024-09-18T04:56:38.377 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.388 INFO:teuthology.orchestra.run.smithi002.stdout:1361 trunc from 0xd5de399 to 0x361c908 2024-09-18T04:56:38.390 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:38.388+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:38.401 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.411 INFO:teuthology.orchestra.run.smithi002.stdout:1363 writesame 0x721f2aa thru 0x722da17 data_size 0xb(0xe76e bytes) 2024-09-18T04:56:38.418 INFO:teuthology.orchestra.run.smithi002.stdout:1364 trunc from 0x722da18 to 0xc39874d 2024-09-18T04:56:38.431 INFO:teuthology.orchestra.run.smithi002.stdout:1365 trunc from 0xc39874d to 0x6d24cdc 2024-09-18T04:56:38.438 INFO:teuthology.orchestra.run.smithi002.stdout:1366 punch from 0x1f011d8 to 0x1f1069c, (0xf4c4 bytes) 2024-09-18T04:56:38.438 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.438 INFO:teuthology.orchestra.run.smithi002.stdout:1368 read 0x322c56a thru 0x3233320 (0x6db7 bytes) 2024-09-18T04:56:38.444 INFO:teuthology.orchestra.run.smithi002.stdout:1369 write 0x7c5f138 thru 0x7c6724a (0x8113 bytes) 2024-09-18T04:56:38.447 INFO:teuthology.orchestra.run.smithi002.stdout:1370 write 0x649c58d thru 0x64aaed4 (0xe948 bytes) 2024-09-18T04:56:38.449 INFO:teuthology.orchestra.run.smithi002.stdout:1371 write 0x6b014ec thru 0x6b10e90 (0xf9a5 bytes) 2024-09-18T04:56:38.452 INFO:teuthology.orchestra.run.smithi002.stdout:1373 read 0x57a6ccc thru 0x57b1e5d (0xb192 bytes) 2024-09-18T04:56:38.452 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.452 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.453 INFO:teuthology.orchestra.run.smithi002.stdout:1380 write 0x70eff86 thru 0x70f703d (0x70b8 bytes) 2024-09-18T04:56:38.453 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:38.453 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:38.455 INFO:teuthology.orchestra.run.smithi002.stdout:1381 read 0x7690661 thru 0x7690a94 (0x434 bytes) 2024-09-18T04:56:38.455 INFO:teuthology.orchestra.run.smithi002.stdout:1382 write 0x365f8ed thru 0x36699cb (0xa0df bytes) 2024-09-18T04:56:38.463 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:56:38.465 INFO:teuthology.orchestra.run.smithi002.stdout:1383 trunc from 0x7c6724b to 0x1298e2f 2024-09-18T04:56:38.485 INFO:teuthology.orchestra.run.smithi002.stdout:1384 trunc from 0x1298e2f to 0x75fb5ef 2024-09-18T04:56:38.490 INFO:teuthology.orchestra.run.smithi002.stdout:1386 write 0x289e8 thru 0x33488 (0xaaa1 bytes) 2024-09-18T04:56:38.491 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:38.488+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:38.501 INFO:teuthology.orchestra.run.smithi002.stdout:1387 write 0xb2231be thru 0xb22c769 (0x95ac bytes) 2024-09-18T04:56:38.515 INFO:teuthology.orchestra.run.smithi002.stdout:1389 trunc from 0xb22c76a to 0x1f77089 2024-09-18T04:56:38.524 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.524 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.524 INFO:teuthology.orchestra.run.smithi002.stdout:1394 read 0x156d490 thru 0x157b6c4 (0xe235 bytes) 2024-09-18T04:56:38.540 INFO:teuthology.orchestra.run.smithi002.stdout:1396 write 0x9e5506a thru 0x9e61327 (0xc2be bytes) 2024-09-18T04:56:38.542 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.542 INFO:teuthology.orchestra.run.smithi002.stdout:1400 read 0x4154d66 thru 0x415fa29 (0xacc4 bytes) 2024-09-18T04:56:38.544 INFO:teuthology.orchestra.run.smithi002.stdout:1404 trunc from 0x9e61328 to 0xb19add6 2024-09-18T04:56:38.557 INFO:teuthology.orchestra.run.smithi002.stdout:1405 write 0xdf0ffcc thru 0xdf1da6b (0xdaa0 bytes) 2024-09-18T04:56:38.560 INFO:teuthology.orchestra.run.smithi002.stdout:1406 read 0xcdb8c4 thru 0xce58cc (0xa009 bytes) 2024-09-18T04:56:38.560 INFO:teuthology.orchestra.run.smithi002.stdout:1408 read 0xcaf9467 thru 0xcb002ac (0x6e46 bytes) 2024-09-18T04:56:38.560 INFO:teuthology.orchestra.run.smithi002.stdout:1409 writesame 0x68898e8 thru 0x6897c35 data_size 0x73(0xe34e bytes) 2024-09-18T04:56:38.567 INFO:teuthology.orchestra.run.smithi002.stdout:1410 write 0xe81f03c thru 0xe829ea6 (0xae6b bytes) 2024-09-18T04:56:38.570 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.570 INFO:teuthology.orchestra.run.smithi002.stdout:1412 punch from 0x26677d9 to 0x266e3b0, (0x6bd7 bytes) 2024-09-18T04:56:38.571 INFO:teuthology.orchestra.run.smithi002.stdout:1414 write 0x3f8747c thru 0x3f885f8 (0x117d bytes) 2024-09-18T04:56:38.585 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:38.585 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:38.588 INFO:teuthology.orchestra.run.smithi002.stdout:1418 trunc from 0xe829ea7 to 0x158ca0e 2024-09-18T04:56:38.592 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:38.590+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:38.596 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:56:38.609 INFO:teuthology.orchestra.run.smithi002.stdout:1419 write 0x69871d1 thru 0x699183c (0xa66c bytes) 2024-09-18T04:56:38.611 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.619 INFO:teuthology.orchestra.run.smithi002.stdout:1422 write 0x96cd5fa thru 0x96d31b8 (0x5bbf bytes) 2024-09-18T04:56:38.630 INFO:teuthology.orchestra.run.smithi002.stdout:1423 write 0xc29577f thru 0xc2a23a2 (0xcc24 bytes) 2024-09-18T04:56:38.634 INFO:teuthology.orchestra.run.smithi002.stdout:1424 punch from 0x3e8ffa to 0x3f681e, (0xd824 bytes) 2024-09-18T04:56:38.634 INFO:teuthology.orchestra.run.smithi002.stdout:1425 read 0x5e5452a thru 0x5e564a0 (0x1f77 bytes) 2024-09-18T04:56:38.634 INFO:teuthology.orchestra.run.smithi002.stdout:1426 punch from 0x1fb6591 to 0x1fbe312, (0x7d81 bytes) 2024-09-18T04:56:38.634 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.634 INFO:teuthology.orchestra.run.smithi002.stdout:1428 write 0xbd292ed thru 0xbd3766e (0xe382 bytes) 2024-09-18T04:56:38.637 INFO:teuthology.orchestra.run.smithi002.stdout:1429 read 0x601db31 thru 0x6024caa (0x717a bytes) 2024-09-18T04:56:38.637 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.637 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.637 INFO:teuthology.orchestra.run.smithi002.stdout:1434 writesame 0x21e5c0d thru 0x21ec6cf data_size 0x1(0x6ac3 bytes) 2024-09-18T04:56:38.649 INFO:teuthology.orchestra.run.smithi002.stdout:1436 read 0x95cc366 thru 0x95d8705 (0xc3a0 bytes) 2024-09-18T04:56:38.649 INFO:teuthology.orchestra.run.smithi002.stdout:1437 read 0x421fc69 thru 0x42217d6 (0x1b6e bytes) 2024-09-18T04:56:38.649 INFO:teuthology.orchestra.run.smithi002.stdout:1438 write 0x51533f5 thru 0x515921f (0x5e2b bytes) 2024-09-18T04:56:38.659 INFO:teuthology.orchestra.run.smithi002.stdout:1439 write 0xdea27b5 thru 0xdea3832 (0x107e bytes) 2024-09-18T04:56:38.674 INFO:teuthology.orchestra.run.smithi002.stdout:1440 trunc from 0xdea3833 to 0x2e2b4bb 2024-09-18T04:56:38.693 INFO:teuthology.orchestra.run.smithi002.stdout:1441 trunc from 0x2e2b4bb to 0x9046825 2024-09-18T04:56:38.694 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:38.691+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:38.700 INFO:teuthology.orchestra.run.smithi002.stdout:1442 punch from 0x3f12bab to 0x3f2228e, (0xf6e3 bytes) 2024-09-18T04:56:38.700 INFO:teuthology.orchestra.run.smithi002.stdout:1444 read 0x35e722c thru 0x35ee94c (0x7721 bytes) 2024-09-18T04:56:38.700 INFO:teuthology.orchestra.run.smithi002.stdout:1445 read 0x491419d thru 0x4919144 (0x4fa8 bytes) 2024-09-18T04:56:38.700 INFO:teuthology.orchestra.run.smithi002.stdout:1446 read 0x35676fd thru 0x35691b9 (0x1abd bytes) 2024-09-18T04:56:38.700 INFO:teuthology.orchestra.run.smithi002.stdout:1447 writesame 0x614fb94 thru 0x615f525 data_size 0x5(0xf992 bytes) 2024-09-18T04:56:38.708 INFO:teuthology.orchestra.run.smithi002.stdout:1448 trunc from 0x9046825 to 0x99dca7a 2024-09-18T04:56:38.721 INFO:teuthology.orchestra.run.smithi002.stdout:1450 write 0xc198f59 thru 0xc1a22ea (0x9392 bytes) 2024-09-18T04:56:38.724 INFO:teuthology.orchestra.run.smithi002.stdout:1451 punch from 0x3b4389f to 0x3b4eb6f, (0xb2d0 bytes) 2024-09-18T04:56:38.724 INFO:teuthology.orchestra.run.smithi002.stdout:1452 punch from 0x13a6b79 to 0x13ad677, (0x6afe bytes) 2024-09-18T04:56:38.724 INFO:teuthology.orchestra.run.smithi002.stdout:1453 writesame 0x8c444aa thru 0x8c47eea data_size 0x3(0x3a41 bytes) 2024-09-18T04:56:38.725 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:38.725 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:38.736 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:56:38.737 INFO:teuthology.orchestra.run.smithi002.stdout:1455 writesame 0xdfab909 thru 0xdfbb777 data_size 0x7(0xfe6f bytes) 2024-09-18T04:56:38.737 INFO:teuthology.orchestra.run.smithi002.stdout:1457 write 0xc40cad4 thru 0xc4178e3 (0xae10 bytes) 2024-09-18T04:56:38.753 INFO:teuthology.orchestra.run.smithi002.stdout:1458 trunc from 0xdfbb778 to 0x383bc2f 2024-09-18T04:56:38.771 INFO:teuthology.orchestra.run.smithi002.stdout:1459 writesame 0x7d4f11e thru 0x7d50e0d data_size 0x8(0x1cf0 bytes) 2024-09-18T04:56:38.772 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.779 INFO:teuthology.orchestra.run.smithi002.stdout:1462 writesame 0x9cfef95 thru 0x9d04a70 data_size 0xa(0x5adc bytes) 2024-09-18T04:56:38.779 INFO:teuthology.orchestra.run.smithi002.stdout:1463 write 0x636d987 thru 0x6376509 (0x8b83 bytes) 2024-09-18T04:56:38.782 INFO:teuthology.orchestra.run.smithi002.stdout:1464 punch from 0x4904fcd to 0x490af76, (0x5fa9 bytes) 2024-09-18T04:56:38.782 INFO:teuthology.orchestra.run.smithi002.stdout:1465 read 0x7f2669c thru 0x7f26fa4 (0x909 bytes) 2024-09-18T04:56:38.792 INFO:teuthology.orchestra.run.smithi002.stdout:1469 trunc from 0x9d04a71 to 0x190ba4b 2024-09-18T04:56:38.795 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:38.792+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:38.803 INFO:teuthology.orchestra.run.smithi002.stdout:1470 trunc from 0x190ba4b to 0x144e72f 2024-09-18T04:56:38.807 INFO:teuthology.orchestra.run.smithi002.stdout:1471 writesame 0x109174b thru 0x109cff2 data_size 0x98(0xb8a8 bytes) 2024-09-18T04:56:38.810 INFO:teuthology.orchestra.run.smithi002.stdout:1472 read 0x3ed989 thru 0x3f7a7b (0xa0f3 bytes) 2024-09-18T04:56:38.811 INFO:teuthology.orchestra.run.smithi002.stdout:1474 writesame 0x13f7f3c thru 0x13fbf3a data_size 0x3(0x3fff bytes) 2024-09-18T04:56:38.816 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.816 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.816 INFO:teuthology.orchestra.run.smithi002.stdout:1477 read 0xf1407e thru 0xf21d14 (0xdc97 bytes) 2024-09-18T04:56:38.816 INFO:teuthology.orchestra.run.smithi002.stdout:1478 read 0x118faf4 thru 0x11978fc (0x7e09 bytes) 2024-09-18T04:56:38.825 INFO:teuthology.orchestra.run.smithi002.stdout:1479 writesame 0x49e26a6 thru 0x49ef74b data_size 0x11(0xd0a6 bytes) 2024-09-18T04:56:38.825 INFO:teuthology.orchestra.run.smithi002.stdout:1480 punch from 0x182d347 to 0x182d571, (0x22a bytes) 2024-09-18T04:56:38.837 INFO:teuthology.orchestra.run.smithi002.stdout:1481 write 0xa58ec62 thru 0xa599231 (0xa5d0 bytes) 2024-09-18T04:56:38.844 INFO:teuthology.orchestra.run.smithi002.stdout:1485 trunc from 0xa599232 to 0x75b5061 2024-09-18T04:56:38.856 INFO:teuthology.orchestra.run.smithi002.stdout:1486 read 0x59c07cd thru 0x59cc426 (0xbc5a bytes) 2024-09-18T04:56:38.856 INFO:teuthology.orchestra.run.smithi002.stdout:1487 punch from 0x6b20cb9 to 0x6b21b21, (0xe68 bytes) 2024-09-18T04:56:38.856 INFO:teuthology.orchestra.run.smithi002.stdout:1488 write 0x18502 thru 0x216b2 (0x91b1 bytes) 2024-09-18T04:56:38.857 INFO:teuthology.orchestra.run.smithi002.stdout:1489 read 0x4a883ab thru 0x4a8cb0d (0x4763 bytes) 2024-09-18T04:56:38.863 INFO:teuthology.orchestra.run.smithi002.stdout:1490 write 0x7b2947a thru 0x7b31c57 (0x87de bytes) 2024-09-18T04:56:38.869 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:38.869 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:38.871 INFO:teuthology.orchestra.run.smithi002.stdout:1491 write 0x83f85bd thru 0x840445b (0xbe9f bytes) 2024-09-18T04:56:38.875 INFO:teuthology.orchestra.run.smithi002.stdout:1495 read 0x4ce11b7 thru 0x4cef4cb (0xe315 bytes) 2024-09-18T04:56:38.875 INFO:teuthology.orchestra.run.smithi002.stdout:1496 punch from 0x7fdff42 to 0x7fe1d7d, (0x1e3b bytes) 2024-09-18T04:56:38.875 INFO:teuthology.orchestra.run.smithi002.stdout:1497 write 0x2dc0b4f thru 0x2dcc2fb (0xb7ad bytes) 2024-09-18T04:56:38.877 INFO:teuthology.orchestra.run.smithi002.stdout:1498 read 0x5d84dac thru 0x5d90551 (0xb7a6 bytes) 2024-09-18T04:56:38.879 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:56:38.890 INFO:teuthology.orchestra.run.smithi002.stdout:1500 writesame 0xe3d13a6 thru 0xe3e10ed data_size 0x14(0xfd48 bytes) 2024-09-18T04:56:38.890 INFO:teuthology.orchestra.run.smithi002.stdout:1501 read 0x75f24db thru 0x75f51cd (0x2cf3 bytes) 2024-09-18T04:56:38.896 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:38.893+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:38.901 INFO:teuthology.orchestra.run.smithi002.stdout:1503 trunc from 0xe3e10ee to 0x442a166 2024-09-18T04:56:38.912 INFO:teuthology.orchestra.run.smithi002.stdout:1505 read 0x40ddaae thru 0x40e0e4f (0x33a2 bytes) 2024-09-18T04:56:38.925 INFO:teuthology.orchestra.run.smithi002.stdout:1507 write 0xcbcbf79 thru 0xcbd780e (0xb896 bytes) 2024-09-18T04:56:38.928 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:38.928 INFO:teuthology.orchestra.run.smithi002.stdout:1511 read 0xb257439 thru 0xb2638f5 (0xc4bd bytes) 2024-09-18T04:56:38.928 INFO:teuthology.orchestra.run.smithi002.stdout:1512 read 0x8105198 thru 0x81058f7 (0x760 bytes) 2024-09-18T04:56:38.928 INFO:teuthology.orchestra.run.smithi002.stdout:1513 writesame 0x27baf53 thru 0x27c611f data_size 0x1(0xb1cd bytes) 2024-09-18T04:56:38.944 INFO:teuthology.orchestra.run.smithi002.stdout:1514 read 0x7883762 thru 0x788fd2a (0xc5c9 bytes) 2024-09-18T04:56:38.945 INFO:teuthology.orchestra.run.smithi002.stdout:1515 writesame 0x49455d7 thru 0x495462f data_size 0x1(0xf059 bytes) 2024-09-18T04:56:38.975 INFO:teuthology.orchestra.run.smithi002.stdout:1516 read 0x60cf4dd thru 0x60df397 (0xfebb bytes) 2024-09-18T04:56:38.975 INFO:teuthology.orchestra.run.smithi002.stdout:1517 read 0x46fbfd1 thru 0x470542c (0x945c bytes) 2024-09-18T04:56:38.989 INFO:teuthology.orchestra.run.smithi002.stdout:1518 trunc from 0xcbd780f to 0x1072a81 2024-09-18T04:56:38.996 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:38.994+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:39.012 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:39.012 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:39.015 INFO:teuthology.orchestra.run.smithi002.stdout:1519 read 0x1007fdc thru 0x1008a92 (0xab7 bytes) 2024-09-18T04:56:39.015 INFO:teuthology.orchestra.run.smithi002.stdout:1520 write 0xc9cfe thru 0xcafd3 (0x12d6 bytes) 2024-09-18T04:56:39.024 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:56:39.038 INFO:teuthology.orchestra.run.smithi002.stdout:1521 write 0xe3d1d3b thru 0xe3d4f4e (0x3214 bytes) 2024-09-18T04:56:39.040 INFO:teuthology.orchestra.run.smithi002.stdout:1524 write 0x10fc511 thru 0x11055ce (0x90be bytes) 2024-09-18T04:56:39.057 INFO:teuthology.orchestra.run.smithi002.stdout:1526 trunc from 0xe3d4f4f to 0x4893819 2024-09-18T04:56:39.068 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:39.068 INFO:teuthology.orchestra.run.smithi002.stdout:1528 writesame 0x3917e61 thru 0x391b047 data_size 0x23(0x31e7 bytes) 2024-09-18T04:56:39.070 INFO:teuthology.orchestra.run.smithi002.stdout:1530 clone 12 order 25 su 262144 sc 12 2024-09-18T04:56:39.098 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:39.095+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:39.146 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:39.146 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:39.154 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:56:39.198 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:39.196+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:39.299 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:39.297+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:39.303 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-09-18T04:56:39.310 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:39.311 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:39.322 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:56:39.400 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:39.397+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:39.479 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:39.479 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:39.488 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:56:39.531 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:39.529+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:39.586 INFO:teuthology.orchestra.run.smithi002.stdout:1531 read 0x2f80612 thru 0x2f8791c (0x730b bytes) 2024-09-18T04:56:39.597 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:39.597 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:39.599 INFO:teuthology.orchestra.run.smithi002.stdout:1532 write 0x8c80dc3 thru 0x8c8cd5f (0xbf9d bytes) 2024-09-18T04:56:39.601 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:39.599+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:39.604 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:39.604 INFO:teuthology.orchestra.run.smithi002.stdout:1534 read 0x42dde73 thru 0x42eb157 (0xd2e5 bytes) 2024-09-18T04:56:39.605 INFO:teuthology.orchestra.run.smithi002.stdout:1535 read 0x5f1a723 thru 0x5f1c56f (0x1e4d bytes) 2024-09-18T04:56:39.605 INFO:teuthology.orchestra.run.smithi002.stdout:1536 punch from 0x3cbb211 to 0x3cc372d, (0x851c bytes) 2024-09-18T04:56:39.605 INFO:teuthology.orchestra.run.smithi002.stdout:1537 punch from 0x2f46ac9 to 0x2f561f8, (0xf72f bytes) 2024-09-18T04:56:39.605 INFO:teuthology.orchestra.run.smithi002.stdout:1539 write 0x69928af thru 0x699c6b0 (0x9e02 bytes) 2024-09-18T04:56:39.607 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:56:39.609 INFO:teuthology.orchestra.run.smithi002.stdout:1540 writesame 0x291441 thru 0x2a113c data_size 0x3f(0xfcfc bytes) 2024-09-18T04:56:39.611 INFO:teuthology.orchestra.run.smithi002.stdout:1541 punch from 0x57a0289 to 0x57a73a9, (0x7120 bytes) 2024-09-18T04:56:39.611 INFO:teuthology.orchestra.run.smithi002.stdout:1542 clone 13 order 20 su 4096 sc 7 2024-09-18T04:56:39.703 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:39.700+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:39.714 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:39.714 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:39.723 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:56:39.803 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:39.801+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:39.829 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:39.829 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:39.837 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:56:39.904 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:39.902+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:39.943 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:39.944 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:39.952 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:56:40.005 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:40.003+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:40.057 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:40.058 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:40.066 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:56:40.106 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:40.104+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:40.168 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:40.168 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:40.178 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:56:40.199 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T04:56:40.247 INFO:teuthology.orchestra.run.smithi002.stdout:leaving image image_client.0-clone12 intact 2024-09-18T04:56:40.253 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:40.250+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:40.286 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:56:40.289 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:40.289 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:40.297 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:56:40.312 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:40.309+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:40.354 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-09-18T04:56:40.412 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:40.412 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:40.413 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:40.411+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:40.421 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:56:40.438 INFO:teuthology.orchestra.run.smithi002.stdout:1544 punch from 0x4bc3fad to 0x4bc77df, (0x3832 bytes) 2024-09-18T04:56:40.445 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:56:40.445 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:56:40.448 INFO:teuthology.orchestra.run.smithi002.stdout:1545 writesame 0xa53fad1 thru 0xa54051f data_size 0x1(0xa4f bytes) 2024-09-18T04:56:40.448 INFO:teuthology.orchestra.run.smithi002.stdout:1546 write 0x5b4c7ec thru 0x5b51329 (0x4b3e bytes) 2024-09-18T04:56:40.473 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:56:40.474 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:56:40.483 INFO:teuthology.orchestra.run.smithi002.stdout:1547 trunc from 0xa540520 to 0x412f138 2024-09-18T04:56:40.492 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:56:40.493 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:56:40.510 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:40.508+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:40.524 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:56:40.524 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:56:40.544 INFO:teuthology.orchestra.run.smithi002.stdout:1549 trunc from 0x412f138 to 0x6471b61 2024-09-18T04:56:40.547 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:56:40.547 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:56:40.549 INFO:teuthology.orchestra.run.smithi002.stdout:1552 read 0x40c7fbd thru 0x40cc833 (0x4877 bytes) 2024-09-18T04:56:40.550 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:40.551 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:40.558 INFO:teuthology.orchestra.run.smithi002.stdout:1553 write 0x8f85fac thru 0x8f910e4 (0xb139 bytes) 2024-09-18T04:56:40.561 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:56:40.565 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:56:40.565 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:56:40.569 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:40.579 INFO:teuthology.orchestra.run.smithi002.stdout:1556 write 0xc3b73fe thru 0xc3bd70c (0x630f bytes) 2024-09-18T04:56:40.584 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:56:40.585 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:56:40.593 INFO:teuthology.orchestra.run.smithi002.stdout:1558 read 0x6cc8702 thru 0x6cd69cf (0xe2ce bytes) 2024-09-18T04:56:40.593 INFO:teuthology.orchestra.run.smithi002.stdout:1559 write 0x77ad7f2 thru 0x77bbe32 (0xe641 bytes) 2024-09-18T04:56:40.602 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:56:40.602 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:56:40.603 INFO:teuthology.orchestra.run.smithi002.stdout:1561 punch from 0x247b315 to 0x247c604, (0x12ef bytes) 2024-09-18T04:56:40.604 INFO:teuthology.orchestra.run.smithi002.stdout:1562 trunc from 0xc3bd70d to 0xd9f4ccf 2024-09-18T04:56:40.608 INFO:teuthology.orchestra.run.smithi002.stdout:1563 punch from 0xb702593 to 0xb70fd4a, (0xd7b7 bytes) 2024-09-18T04:56:40.609 INFO:teuthology.orchestra.run.smithi002.stdout:1564 write 0x121ed2c thru 0x12291a9 (0xa47e bytes) 2024-09-18T04:56:40.628 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:40.625+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:40.670 INFO:teuthology.orchestra.run.smithi002.stdout:1566 read 0x3527be4 thru 0x35379ca (0xfde7 bytes) 2024-09-18T04:56:40.672 INFO:teuthology.orchestra.run.smithi002.stdout:1567 trunc from 0xd9f4ccf to 0xe9b2339 2024-09-18T04:56:40.677 INFO:teuthology.orchestra.run.smithi002.stdout:1568 write 0x2e7402e thru 0x2e821fb (0xe1ce bytes) 2024-09-18T04:56:40.703 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:40.703 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:56:40.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:56:40.703 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:56:40.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.704 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:5206 3.1d 3:bc9b8049:::rbd_data.11ac14fc1d07.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 17326599~63058] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:40.704 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.539333+0000", 2024-09-18T04:56:40.704 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.1604375559999998, 2024-09-18T04:56:40.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.027944470999999999, 2024-09-18T04:56:40.704 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:40.704 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:40.704 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:40.704 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:40.704 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:40.704 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 5206 2024-09-18T04:56:40.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.704 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:40.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:40.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.539333+0000", 2024-09-18T04:56:40.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:40.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.539333+0000", 2024-09-18T04:56:40.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:40.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.539332+0000", 2024-09-18T04:56:40.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:40.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:40.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.551229+0000", 2024-09-18T04:56:40.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011896814 2024-09-18T04:56:40.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:40.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.551256+0000", 2024-09-18T04:56:40.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7208000000000001e-05 2024-09-18T04:56:40.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:40.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.551266+0000", 2024-09-18T04:56:40.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0051000000000001e-05 2024-09-18T04:56:40.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:40.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.551322+0000", 2024-09-18T04:56:40.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6317999999999998e-05 2024-09-18T04:56:40.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:40.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.551437+0000", 2024-09-18T04:56:40.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000115159 2024-09-18T04:56:40.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.551798+0000", 2024-09-18T04:56:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000360584 2024-09-18T04:56:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.566830+0000", 2024-09-18T04:56:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015031991999999999 2024-09-18T04:56:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.567213+0000", 2024-09-18T04:56:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00038325799999999998 2024-09-18T04:56:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:40.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.567250+0000", 2024-09-18T04:56:40.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6259e-05 2024-09-18T04:56:40.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:40.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.567278+0000", 2024-09-18T04:56:40.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7988999999999999e-05 2024-09-18T04:56:40.709 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.709 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:40.709 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.709 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11020 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 1654784~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:40.710 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.889307+0000", 2024-09-18T04:56:40.710 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.8104631019999999, 2024-09-18T04:56:40.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023721609000000001, 2024-09-18T04:56:40.710 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:40.710 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:40.710 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:40.710 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:40.710 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:40.710 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11020 2024-09-18T04:56:40.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.710 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:40.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.889307+0000", 2024-09-18T04:56:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.889307+0000", 2024-09-18T04:56:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.889307+0000", 2024-09-18T04:56:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.889309+0000", 2024-09-18T04:56:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2730000000000001e-06 2024-09-18T04:56:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.889318+0000", 2024-09-18T04:56:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6119999999999999e-06 2024-09-18T04:56:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.889322+0000", 2024-09-18T04:56:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6509999999999999e-06 2024-09-18T04:56:40.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.912900+0000", 2024-09-18T04:56:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023578154 2024-09-18T04:56:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.912968+0000", 2024-09-18T04:56:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8757999999999995e-05 2024-09-18T04:56:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.913029+0000", 2024-09-18T04:56:40.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0616999999999999e-05 2024-09-18T04:56:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11283 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 1970176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.897179+0000", 2024-09-18T04:56:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.802591735, 2024-09-18T04:56:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023627705999999998, 2024-09-18T04:56:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:40.714 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11283 2024-09-18T04:56:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897179+0000", 2024-09-18T04:56:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897179+0000", 2024-09-18T04:56:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897178+0000", 2024-09-18T04:56:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897181+0000", 2024-09-18T04:56:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.187e-06 2024-09-18T04:56:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897187+0000", 2024-09-18T04:56:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8769999999999998e-06 2024-09-18T04:56:40.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897191+0000", 2024-09-18T04:56:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3470000000000001e-06 2024-09-18T04:56:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.920639+0000", 2024-09-18T04:56:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023447835 2024-09-18T04:56:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:40.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.920715+0000", 2024-09-18T04:56:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6476000000000004e-05 2024-09-18T04:56:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.920807+0000", 2024-09-18T04:56:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1388e-05 2024-09-18T04:56:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11284 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.897195+0000", 2024-09-18T04:56:40.718 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.802575998, 2024-09-18T04:56:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023843923, 2024-09-18T04:56:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11284 2024-09-18T04:56:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897195+0000", 2024-09-18T04:56:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897195+0000", 2024-09-18T04:56:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897194+0000", 2024-09-18T04:56:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897196+0000", 2024-09-18T04:56:40.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.838e-06 2024-09-18T04:56:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897202+0000", 2024-09-18T04:56:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.677e-06 2024-09-18T04:56:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897206+0000", 2024-09-18T04:56:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1699999999999999e-06 2024-09-18T04:56:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:40.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.920838+0000", 2024-09-18T04:56:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023632162000000002 2024-09-18T04:56:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.920916+0000", 2024-09-18T04:56:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7762000000000005e-05 2024-09-18T04:56:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.921039+0000", 2024-09-18T04:56:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012267499999999999 2024-09-18T04:56:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:40.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11285 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 2068480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.897210+0000", 2024-09-18T04:56:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.802560723, 2024-09-18T04:56:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024026921999999999, 2024-09-18T04:56:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11285 2024-09-18T04:56:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.723 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897210+0000", 2024-09-18T04:56:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897210+0000", 2024-09-18T04:56:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:40.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897210+0000", 2024-09-18T04:56:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897211+0000", 2024-09-18T04:56:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9439999999999999e-06 2024-09-18T04:56:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897217+0000", 2024-09-18T04:56:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4009999999999996e-06 2024-09-18T04:56:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.897221+0000", 2024-09-18T04:56:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3170000000000003e-06 2024-09-18T04:56:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.921075+0000", 2024-09-18T04:56:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023853760000000002 2024-09-18T04:56:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.921152+0000", 2024-09-18T04:56:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7012000000000001e-05 2024-09-18T04:56:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.921237+0000", 2024-09-18T04:56:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4818e-05 2024-09-18T04:56:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11533 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 1613824~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.903270+0000", 2024-09-18T04:56:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.796500279, 2024-09-18T04:56:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023848581000000001, 2024-09-18T04:56:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:40.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11533 2024-09-18T04:56:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903270+0000", 2024-09-18T04:56:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:40.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903270+0000", 2024-09-18T04:56:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903270+0000", 2024-09-18T04:56:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903271+0000", 2024-09-18T04:56:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.08e-06 2024-09-18T04:56:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:40.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903274+0000", 2024-09-18T04:56:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8059999999999999e-06 2024-09-18T04:56:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903276+0000", 2024-09-18T04:56:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2450000000000001e-06 2024-09-18T04:56:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.926915+0000", 2024-09-18T04:56:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023639191 2024-09-18T04:56:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927008+0000", 2024-09-18T04:56:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2682999999999993e-05 2024-09-18T04:56:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927119+0000", 2024-09-18T04:56:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000110777 2024-09-18T04:56:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.731 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11534 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 1662976~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.903279+0000", 2024-09-18T04:56:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.7964916149999999, 2024-09-18T04:56:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024055705, 2024-09-18T04:56:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11534 2024-09-18T04:56:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:40.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903279+0000", 2024-09-18T04:56:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903279+0000", 2024-09-18T04:56:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903279+0000", 2024-09-18T04:56:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903280+0000", 2024-09-18T04:56:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1510000000000001e-06 2024-09-18T04:56:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903283+0000", 2024-09-18T04:56:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.83e-06 2024-09-18T04:56:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903286+0000", 2024-09-18T04:56:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0290000000000001e-06 2024-09-18T04:56:40.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927172+0000", 2024-09-18T04:56:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023886627000000001 2024-09-18T04:56:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927252+0000", 2024-09-18T04:56:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9424000000000004e-05 2024-09-18T04:56:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927335+0000", 2024-09-18T04:56:40.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2844999999999997e-05 2024-09-18T04:56:40.736 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.736 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:40.736 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.736 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11535 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 1712128~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:40.736 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.903288+0000", 2024-09-18T04:56:40.736 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.796482664, 2024-09-18T04:56:40.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024184574, 2024-09-18T04:56:40.736 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:40.736 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:40.736 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:40.736 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11535 2024-09-18T04:56:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903288+0000", 2024-09-18T04:56:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903288+0000", 2024-09-18T04:56:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903288+0000", 2024-09-18T04:56:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903289+0000", 2024-09-18T04:56:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.048e-06 2024-09-18T04:56:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903292+0000", 2024-09-18T04:56:40.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8839999999999998e-06 2024-09-18T04:56:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903294+0000", 2024-09-18T04:56:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3099999999999999e-06 2024-09-18T04:56:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927363+0000", 2024-09-18T04:56:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024068984000000002 2024-09-18T04:56:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:40.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927407+0000", 2024-09-18T04:56:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3608e-05 2024-09-18T04:56:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927473+0000", 2024-09-18T04:56:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5955000000000001e-05 2024-09-18T04:56:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11536 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 1761280~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.903296+0000", 2024-09-18T04:56:40.740 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.7964744679999995, 2024-09-18T04:56:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024304868, 2024-09-18T04:56:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11536 2024-09-18T04:56:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903296+0000", 2024-09-18T04:56:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903296+0000", 2024-09-18T04:56:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903296+0000", 2024-09-18T04:56:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903297+0000", 2024-09-18T04:56:40.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0139999999999999e-06 2024-09-18T04:56:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903300+0000", 2024-09-18T04:56:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8590000000000001e-06 2024-09-18T04:56:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903302+0000", 2024-09-18T04:56:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2900000000000001e-06 2024-09-18T04:56:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927500+0000", 2024-09-18T04:56:40.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024198130000000002 2024-09-18T04:56:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927545+0000", 2024-09-18T04:56:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5176000000000003e-05 2024-09-18T04:56:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927601+0000", 2024-09-18T04:56:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5618999999999997e-05 2024-09-18T04:56:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.745 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11537 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 1810432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:40.745 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.903305+0000", 2024-09-18T04:56:40.745 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.7964656449999996, 2024-09-18T04:56:40.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024484314, 2024-09-18T04:56:40.745 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:40.745 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:40.745 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:40.745 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:40.745 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:40.745 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11537 2024-09-18T04:56:40.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.745 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903305+0000", 2024-09-18T04:56:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903305+0000", 2024-09-18T04:56:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903305+0000", 2024-09-18T04:56:40.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903306+0000", 2024-09-18T04:56:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0449999999999999e-06 2024-09-18T04:56:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903309+0000", 2024-09-18T04:56:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7920000000000001e-06 2024-09-18T04:56:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.903311+0000", 2024-09-18T04:56:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3159999999999999e-06 2024-09-18T04:56:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927631+0000", 2024-09-18T04:56:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024320235999999999 2024-09-18T04:56:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927722+0000", 2024-09-18T04:56:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1020000000000006e-05 2024-09-18T04:56:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.927789+0000", 2024-09-18T04:56:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7131000000000005e-05 2024-09-18T04:56:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11541 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 2023424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.904815+0000", 2024-09-18T04:56:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.7949552510000002, 2024-09-18T04:56:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023644554000000002, 2024-09-18T04:56:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:40.749 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11541 2024-09-18T04:56:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.904815+0000", 2024-09-18T04:56:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:40.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.904815+0000", 2024-09-18T04:56:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.904815+0000", 2024-09-18T04:56:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.904821+0000", 2024-09-18T04:56:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.19e-06 2024-09-18T04:56:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.904827+0000", 2024-09-18T04:56:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2780000000000001e-06 2024-09-18T04:56:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.904829+0000", 2024-09-18T04:56:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6369999999999999e-06 2024-09-18T04:56:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.928326+0000", 2024-09-18T04:56:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023496435999999999 2024-09-18T04:56:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:40.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.928373+0000", 2024-09-18T04:56:40.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7851e-05 2024-09-18T04:56:40.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:40.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.928460+0000", 2024-09-18T04:56:40.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6420000000000003e-05 2024-09-18T04:56:40.753 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.753 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:40.753 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.753 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11542 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 2072576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.904832+0000", 2024-09-18T04:56:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.7949385050000002, 2024-09-18T04:56:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.02378951, 2024-09-18T04:56:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11542 2024-09-18T04:56:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:40.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.904832+0000", 2024-09-18T04:56:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.904832+0000", 2024-09-18T04:56:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.904832+0000", 2024-09-18T04:56:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.904833+0000", 2024-09-18T04:56:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8139999999999999e-06 2024-09-18T04:56:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.904839+0000", 2024-09-18T04:56:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4410000000000001e-06 2024-09-18T04:56:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.904843+0000", 2024-09-18T04:56:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3089999999999997e-06 2024-09-18T04:56:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.928475+0000", 2024-09-18T04:56:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023632057000000001 2024-09-18T04:56:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.928531+0000", 2024-09-18T04:56:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6177000000000001e-05 2024-09-18T04:56:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.928622+0000", 2024-09-18T04:56:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0155999999999995e-05 2024-09-18T04:56:40.757 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11791 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 1716224~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.910762+0000", 2024-09-18T04:56:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.7890089639999998, 2024-09-18T04:56:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023797966, 2024-09-18T04:56:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:40.758 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11791 2024-09-18T04:56:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910762+0000", 2024-09-18T04:56:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910762+0000", 2024-09-18T04:56:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910761+0000", 2024-09-18T04:56:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910763+0000", 2024-09-18T04:56:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.119e-06 2024-09-18T04:56:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910767+0000", 2024-09-18T04:56:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5589999999999999e-06 2024-09-18T04:56:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:40.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910771+0000", 2024-09-18T04:56:40.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3989999999999997e-06 2024-09-18T04:56:40.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:40.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.934393+0000", 2024-09-18T04:56:40.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023621782000000001 2024-09-18T04:56:40.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:40.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.934456+0000", 2024-09-18T04:56:40.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2572999999999999e-05 2024-09-18T04:56:40.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.934560+0000", 2024-09-18T04:56:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000103754 2024-09-18T04:56:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11792 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 1765376~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.910776+0000", 2024-09-18T04:56:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.7889941729999999, 2024-09-18T04:56:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024009782, 2024-09-18T04:56:40.762 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11792 2024-09-18T04:56:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910776+0000", 2024-09-18T04:56:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:40.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910776+0000", 2024-09-18T04:56:40.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:40.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910776+0000", 2024-09-18T04:56:40.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:40.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:40.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910778+0000", 2024-09-18T04:56:40.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0789999999999999e-06 2024-09-18T04:56:40.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:40.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910784+0000", 2024-09-18T04:56:40.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.942e-06 2024-09-18T04:56:40.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:40.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910788+0000", 2024-09-18T04:56:40.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3309999999999997e-06 2024-09-18T04:56:40.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:40.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.934612+0000", 2024-09-18T04:56:40.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023823205 2024-09-18T04:56:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.934694+0000", 2024-09-18T04:56:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2588000000000003e-05 2024-09-18T04:56:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.934786+0000", 2024-09-18T04:56:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1988000000000001e-05 2024-09-18T04:56:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11793 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 1814528~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.910795+0000", 2024-09-18T04:56:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.7889752809999999, 2024-09-18T04:56:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024155223999999999, 2024-09-18T04:56:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11793 2024-09-18T04:56:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:40.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910795+0000", 2024-09-18T04:56:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910795+0000", 2024-09-18T04:56:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910795+0000", 2024-09-18T04:56:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:40.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910797+0000", 2024-09-18T04:56:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.626e-06 2024-09-18T04:56:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910800+0000", 2024-09-18T04:56:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3249999999999999e-06 2024-09-18T04:56:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910803+0000", 2024-09-18T04:56:40.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.819e-06 2024-09-18T04:56:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.934831+0000", 2024-09-18T04:56:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024028040000000001 2024-09-18T04:56:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.934891+0000", 2024-09-18T04:56:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0405000000000002e-05 2024-09-18T04:56:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:40.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.934951+0000", 2024-09-18T04:56:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9406e-05 2024-09-18T04:56:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11794 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 1880064~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.910805+0000", 2024-09-18T04:56:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.7889658019999999, 2024-09-18T04:56:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024296933, 2024-09-18T04:56:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:40.771 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:40.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:40.772 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11794 2024-09-18T04:56:40.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.772 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:40.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:40.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910805+0000", 2024-09-18T04:56:40.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:40.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910805+0000", 2024-09-18T04:56:40.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:40.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910805+0000", 2024-09-18T04:56:40.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:40.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:40.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910806+0000", 2024-09-18T04:56:40.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.155e-06 2024-09-18T04:56:40.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:40.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910810+0000", 2024-09-18T04:56:40.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7859999999999998e-06 2024-09-18T04:56:40.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:40.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910812+0000", 2024-09-18T04:56:40.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5380000000000001e-06 2024-09-18T04:56:40.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:40.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.934983+0000", 2024-09-18T04:56:40.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024170555999999999 2024-09-18T04:56:40.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:40.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.935046+0000", 2024-09-18T04:56:40.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3274000000000004e-05 2024-09-18T04:56:40.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:40.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.935102+0000", 2024-09-18T04:56:40.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5829999999999999e-05 2024-09-18T04:56:40.775 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.775 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:40.775 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.776 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11795 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 1929216~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:40.776 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.910814+0000", 2024-09-18T04:56:40.776 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.7889564440000001, 2024-09-18T04:56:40.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024598887, 2024-09-18T04:56:40.776 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:40.776 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:40.776 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:40.776 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:40.776 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:40.776 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11795 2024-09-18T04:56:40.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.777 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:40.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:40.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910814+0000", 2024-09-18T04:56:40.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:40.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910814+0000", 2024-09-18T04:56:40.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:40.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910814+0000", 2024-09-18T04:56:40.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:40.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:40.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910815+0000", 2024-09-18T04:56:40.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.362e-06 2024-09-18T04:56:40.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:40.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910818+0000", 2024-09-18T04:56:40.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7690000000000001e-06 2024-09-18T04:56:40.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:40.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910821+0000", 2024-09-18T04:56:40.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.447e-06 2024-09-18T04:56:40.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:40.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.935190+0000", 2024-09-18T04:56:40.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024369289999999998 2024-09-18T04:56:40.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:40.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.935231+0000", 2024-09-18T04:56:40.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1551999999999998e-05 2024-09-18T04:56:40.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:40.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.935413+0000", 2024-09-18T04:56:40.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00018169 2024-09-18T04:56:40.780 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.780 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:40.780 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.780 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11796 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 1978368~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:40.780 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.910823+0000", 2024-09-18T04:56:40.780 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.7889478749999999, 2024-09-18T04:56:40.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024780176000000001, 2024-09-18T04:56:40.780 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:40.781 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:40.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:40.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:40.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:40.781 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11796 2024-09-18T04:56:40.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.781 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:40.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:40.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910823+0000", 2024-09-18T04:56:40.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:40.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910823+0000", 2024-09-18T04:56:40.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:40.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910822+0000", 2024-09-18T04:56:40.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:40.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:40.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910824+0000", 2024-09-18T04:56:40.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.269e-06 2024-09-18T04:56:40.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:40.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910827+0000", 2024-09-18T04:56:40.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9110000000000002e-06 2024-09-18T04:56:40.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:40.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910829+0000", 2024-09-18T04:56:40.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4339999999999999e-06 2024-09-18T04:56:40.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:40.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.935452+0000", 2024-09-18T04:56:40.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024622733000000001 2024-09-18T04:56:40.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:40.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.935528+0000", 2024-09-18T04:56:40.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6087000000000005e-05 2024-09-18T04:56:40.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:40.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.935603+0000", 2024-09-18T04:56:40.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5059000000000001e-05 2024-09-18T04:56:40.784 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.784 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:40.784 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.785 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11797 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 2027520~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:40.785 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.910831+0000", 2024-09-18T04:56:40.785 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.7889395339999998, 2024-09-18T04:56:40.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025033228000000001, 2024-09-18T04:56:40.785 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:40.785 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:40.785 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:40.785 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:40.785 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:40.785 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11797 2024-09-18T04:56:40.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.785 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:40.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:40.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910831+0000", 2024-09-18T04:56:40.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:40.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910831+0000", 2024-09-18T04:56:40.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:40.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910831+0000", 2024-09-18T04:56:40.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:40.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:40.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910835+0000", 2024-09-18T04:56:40.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9149999999999998e-06 2024-09-18T04:56:40.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:40.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910838+0000", 2024-09-18T04:56:40.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1659999999999998e-06 2024-09-18T04:56:40.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:40.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910840+0000", 2024-09-18T04:56:40.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4430000000000002e-06 2024-09-18T04:56:40.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:40.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.935669+0000", 2024-09-18T04:56:40.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024828894000000001 2024-09-18T04:56:40.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:40.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.935777+0000", 2024-09-18T04:56:40.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107308 2024-09-18T04:56:40.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:40.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.935864+0000", 2024-09-18T04:56:40.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7717000000000001e-05 2024-09-18T04:56:40.789 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.789 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:40.789 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.789 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:11798 3.5 3:a12cd555:::rbd_data.11acf292c51a.0000000000000006:d [read 2076672~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:40.789 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:35.910843+0000", 2024-09-18T04:56:40.789 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.7889280830000001, 2024-09-18T04:56:40.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025237124, 2024-09-18T04:56:40.789 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:40.789 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:40.789 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:40.789 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:40.790 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:40.790 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 11798 2024-09-18T04:56:40.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.790 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:40.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:40.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910843+0000", 2024-09-18T04:56:40.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:40.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910843+0000", 2024-09-18T04:56:40.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:40.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:40.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910842+0000", 2024-09-18T04:56:40.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:40.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:40.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910843+0000", 2024-09-18T04:56:40.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.186e-06 2024-09-18T04:56:40.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:40.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910847+0000", 2024-09-18T04:56:40.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5250000000000001e-06 2024-09-18T04:56:40.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:40.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.910849+0000", 2024-09-18T04:56:40.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4600000000000002e-06 2024-09-18T04:56:40.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:40.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.935892+0000", 2024-09-18T04:56:40.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.025042792000000001 2024-09-18T04:56:40.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:40.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.935977+0000", 2024-09-18T04:56:40.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4606999999999998e-05 2024-09-18T04:56:40.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:40.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:40.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:40.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:35.936080+0000", 2024-09-18T04:56:40.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010279 2024-09-18T04:56:40.793 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.793 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:40.793 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.793 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:40.793 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:40.793 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:40.796 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:56:40.797 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:40.797 INFO:teuthology.orchestra.run.smithi002.stdout:1570 writesame 0x8a76f2e thru 0x8a7c25a data_size 0x1(0x532d bytes) 2024-09-18T04:56:40.797 INFO:teuthology.orchestra.run.smithi002.stdout:1571 read 0x5afb157 thru 0x5b02dc6 (0x7c70 bytes) 2024-09-18T04:56:40.797 INFO:teuthology.orchestra.run.smithi002.stdout:1572 read 0xa4f135d thru 0xa4f6655 (0x52f9 bytes) 2024-09-18T04:56:40.797 INFO:teuthology.orchestra.run.smithi002.stdout:1574 read 0x72da655 thru 0x72e37c8 (0x9174 bytes) 2024-09-18T04:56:40.798 INFO:teuthology.orchestra.run.smithi002.stdout:1575 read 0xb2e7337 thru 0xb2ec109 (0x4dd3 bytes) 2024-09-18T04:56:40.798 INFO:teuthology.orchestra.run.smithi002.stdout:1576 write 0x4913ec5 thru 0x4915dca (0x1f06 bytes) 2024-09-18T04:56:40.798 INFO:teuthology.orchestra.run.smithi002.stdout:1578 read 0xcd2872e thru 0xcd35232 (0xcb05 bytes) 2024-09-18T04:56:40.798 INFO:teuthology.orchestra.run.smithi002.stdout:1579 trunc from 0xe9b2339 to 0xc02a6fd 2024-09-18T04:56:40.798 INFO:teuthology.orchestra.run.smithi002.stdout:1581 write 0xe4c8c2 thru 0xe51e17 (0x5556 bytes) 2024-09-18T04:56:40.798 INFO:teuthology.orchestra.run.smithi002.stdout:1582 write 0x90c742 thru 0x90f3fb (0x2cba bytes) 2024-09-18T04:56:40.798 INFO:teuthology.orchestra.run.smithi002.stdout:1583 write 0xa3ce8ab thru 0xa3d2eff (0x4655 bytes) 2024-09-18T04:56:40.798 INFO:teuthology.orchestra.run.smithi002.stdout:1584 punch from 0x36474cc to 0x364abc9, (0x36fd bytes) 2024-09-18T04:56:40.798 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:40.798 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:40.798 INFO:teuthology.orchestra.run.smithi002.stdout:1590 write 0xb0bf3a5 thru 0xb0c5e53 (0x6aaf bytes) 2024-09-18T04:56:40.798 INFO:teuthology.orchestra.run.smithi002.stdout:1592 read 0xaebaadb thru 0xaec207f (0x75a5 bytes) 2024-09-18T04:56:40.799 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:40.793+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:40.799 INFO:teuthology.orchestra.run.smithi002.stdout:1594 trunc from 0xc02a6fd to 0x21f843 2024-09-18T04:56:40.896 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:40.894+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:40.904 INFO:teuthology.orchestra.run.smithi002.stdout:1595 writesame 0x4927920 thru 0x4928689 data_size 0x11(0xd6a bytes) 2024-09-18T04:56:40.915 INFO:teuthology.orchestra.run.smithi002.stdout:1596 writesame 0x81956d7 thru 0x81a041c data_size 0x3(0xad46 bytes) 2024-09-18T04:56:40.916 INFO:teuthology.orchestra.run.smithi002.stdout:1599 read 0x668598d thru 0x668a025 (0x4699 bytes) 2024-09-18T04:56:40.916 INFO:teuthology.orchestra.run.smithi002.stdout:1600 read 0x3fb4111 thru 0x3fb43b9 (0x2a9 bytes) 2024-09-18T04:56:40.922 INFO:teuthology.orchestra.run.smithi002.stdout:1601 writesame 0x9ae40a8 thru 0x9ae5950 data_size 0x1(0x18a9 bytes) 2024-09-18T04:56:40.932 INFO:teuthology.orchestra.run.smithi002.stdout:1602 write 0xd6e825b thru 0xd6ebd73 (0x3b19 bytes) 2024-09-18T04:56:40.942 INFO:teuthology.orchestra.run.smithi002.stdout:1603 trunc from 0xd6ebd74 to 0x901a8a3 2024-09-18T04:56:40.953 INFO:teuthology.orchestra.run.smithi002.stdout:1604 write 0x5c2e9ac thru 0x5c35b24 (0x7179 bytes) 2024-09-18T04:56:40.959 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:40.960 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:40.960 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:56:40.960 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:40.969 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:56:40.971 INFO:teuthology.orchestra.run.smithi002.stdout:1605 trunc from 0x901a8a3 to 0x2ea8ed5 2024-09-18T04:56:41.001 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:40.995+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:41.001 INFO:teuthology.orchestra.run.smithi002.stdout:1606 writesame 0xa126230 thru 0xa12b936 data_size 0x1(0x5707 bytes) 2024-09-18T04:56:41.001 INFO:teuthology.orchestra.run.smithi002.stdout:1608 write 0x4e6b118 thru 0x4e74e65 (0x9d4e bytes) 2024-09-18T04:56:41.014 INFO:teuthology.orchestra.run.smithi002.stdout:1609 write 0x579d01c thru 0x57ac54b (0xf530 bytes) 2024-09-18T04:56:41.026 INFO:teuthology.orchestra.run.smithi002.stdout:1610 read 0x7b11764 thru 0x7b14ebd (0x375a bytes) 2024-09-18T04:56:41.026 INFO:teuthology.orchestra.run.smithi002.stdout:1611 read 0x6ac7953 thru 0x6acb827 (0x3ed5 bytes) 2024-09-18T04:56:41.028 INFO:teuthology.orchestra.run.smithi002.stdout:1613 trunc from 0xa12b937 to 0xbe092c2 2024-09-18T04:56:41.039 INFO:teuthology.orchestra.run.smithi002.stdout:1614 write 0xbfd1c13 thru 0xbfe0c18 (0xf006 bytes) 2024-09-18T04:56:41.051 INFO:teuthology.orchestra.run.smithi002.stdout:1616 write 0xa47a818 thru 0xa48653f (0xbd28 bytes) 2024-09-18T04:56:41.063 INFO:teuthology.orchestra.run.smithi002.stdout:1617 write 0x1453645 thru 0x1456939 (0x32f5 bytes) 2024-09-18T04:56:41.069 INFO:teuthology.orchestra.run.smithi002.stdout:1619 write 0x52b6b79 thru 0x52c3ed6 (0xd35e bytes) 2024-09-18T04:56:41.075 INFO:teuthology.orchestra.run.smithi002.stdout:1620 write 0xc14d3ce thru 0xc152ce0 (0x5913 bytes) 2024-09-18T04:56:41.077 INFO:teuthology.orchestra.run.smithi002.stdout:1621 punch from 0x67d4698 to 0x67d856a, (0x3ed2 bytes) 2024-09-18T04:56:41.078 INFO:teuthology.orchestra.run.smithi002.stdout:1624 punch from 0x77600dd to 0x77635bf, (0x34e2 bytes) 2024-09-18T04:56:41.078 INFO:teuthology.orchestra.run.smithi002.stdout:1625 read 0xa7c5524 thru 0xa7d2e27 (0xd904 bytes) 2024-09-18T04:56:41.085 INFO:teuthology.orchestra.run.smithi002.stdout:1630 write 0xd7568f2 thru 0xd75c423 (0x5b32 bytes) 2024-09-18T04:56:41.091 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:41.092 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:41.092 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:56:41.092 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:56:41.092 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:41.095 INFO:teuthology.orchestra.run.smithi002.stdout:1631 punch from 0x418111c to 0x418cfc3, (0xbea7 bytes) 2024-09-18T04:56:41.095 INFO:teuthology.orchestra.run.smithi002.stdout:1632 read 0x570309e thru 0x570b741 (0x86a4 bytes) 2024-09-18T04:56:41.095 INFO:teuthology.orchestra.run.smithi002.stdout:1634 write 0x258d103 thru 0x2597f36 (0xae34 bytes) 2024-09-18T04:56:41.098 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:41.096+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:41.101 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:56:41.108 INFO:teuthology.orchestra.run.smithi002.stdout:1635 writesame 0x5e1a57 thru 0x5f0b84 data_size 0x1(0xf12e bytes) 2024-09-18T04:56:41.116 INFO:teuthology.orchestra.run.smithi002.stdout:1636 write 0xa9f8177 thru 0xa9f8389 (0x213 bytes) 2024-09-18T04:56:41.119 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:41.119 INFO:teuthology.orchestra.run.smithi002.stdout:1639 writesame 0x33ca539 thru 0x33d2241 data_size 0x1(0x7d09 bytes) 2024-09-18T04:56:41.134 INFO:teuthology.orchestra.run.smithi002.stdout:1640 write 0x1feb861 thru 0x1fedeff (0x269f bytes) 2024-09-18T04:56:41.144 INFO:teuthology.orchestra.run.smithi002.stdout:1641 writesame 0xbfbfab2 thru 0xbfc94fa data_size 0x1(0x9a49 bytes) 2024-09-18T04:56:41.150 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:41.151 INFO:teuthology.orchestra.run.smithi002.stdout:1645 write 0xa0103c1 thru 0xa014066 (0x3ca6 bytes) 2024-09-18T04:56:41.166 INFO:teuthology.orchestra.run.smithi002.stdout:1646 trunc from 0xd75c424 to 0x82e57d4 2024-09-18T04:56:41.187 INFO:teuthology.orchestra.run.smithi002.stdout:1647 read 0x2a87850 thru 0x2a8c759 (0x4f0a bytes) 2024-09-18T04:56:41.187 INFO:teuthology.orchestra.run.smithi002.stdout:1648 punch from 0x693b27 to 0x69fda9, (0xc282 bytes) 2024-09-18T04:56:41.187 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:41.187 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:41.187 INFO:teuthology.orchestra.run.smithi002.stdout:1653 writesame 0x34562f thru 0x349b48 data_size 0x3d(0x451a bytes) 2024-09-18T04:56:41.188 INFO:teuthology.orchestra.run.smithi002.stdout:1654 read 0x7304536 thru 0x730892e (0x43f9 bytes) 2024-09-18T04:56:41.189 INFO:teuthology.orchestra.run.smithi002.stdout:1655 read 0x2b4e4b6 thru 0x2b50d68 (0x28b3 bytes) 2024-09-18T04:56:41.189 INFO:teuthology.orchestra.run.smithi002.stdout:1656 write 0x40bcc93 thru 0x40c5848 (0x8bb6 bytes) 2024-09-18T04:56:41.199 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:41.197+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:41.200 INFO:teuthology.orchestra.run.smithi002.stdout:1658 trunc from 0x82e57d4 to 0x9296ffe 2024-09-18T04:56:41.201 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-18T04:56:41.206 INFO:teuthology.orchestra.run.smithi002.stdout:1660 writesame 0x497822c thru 0x49825e2 data_size 0x1(0xa3b7 bytes) 2024-09-18T04:56:41.219 INFO:teuthology.orchestra.run.smithi002.stdout:1661 write 0x2987404 thru 0x2995eb8 (0xeab5 bytes) 2024-09-18T04:56:41.224 INFO:teuthology.orchestra.run.smithi002.stdout:1662 trunc from 0x9296ffe to 0xa54bdef 2024-09-18T04:56:41.233 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:41.233 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:56:41.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:56:41.234 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:56:41.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.234 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26189 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 237568~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.234 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:56:41.234 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.58921878299999997, 2024-09-18T04:56:41.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022263431, 2024-09-18T04:56:41.235 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.235 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:41.235 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.235 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.235 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.236 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26189 2024-09-18T04:56:41.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.236 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:56:41.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:56:41.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:56:41.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639787+0000", 2024-09-18T04:56:41.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9759999999999998e-06 2024-09-18T04:56:41.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639792+0000", 2024-09-18T04:56:41.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5829999999999997e-06 2024-09-18T04:56:41.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639796+0000", 2024-09-18T04:56:41.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0690000000000003e-06 2024-09-18T04:56:41.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.661944+0000", 2024-09-18T04:56:41.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022147463999999999 2024-09-18T04:56:41.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:41.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.661985+0000", 2024-09-18T04:56:41.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1360999999999999e-05 2024-09-18T04:56:41.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662048+0000", 2024-09-18T04:56:41.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3298000000000003e-05 2024-09-18T04:56:41.239 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.239 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.239 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.239 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26199 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 4096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.239 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:56:41.239 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.58920349699999996, 2024-09-18T04:56:41.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022348776000000001, 2024-09-18T04:56:41.240 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.240 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:41.240 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.240 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.240 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.240 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26199 2024-09-18T04:56:41.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.240 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:56:41.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:56:41.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:56:41.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639806+0000", 2024-09-18T04:56:41.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1539999999999999e-06 2024-09-18T04:56:41.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639812+0000", 2024-09-18T04:56:41.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5520000000000004e-06 2024-09-18T04:56:41.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639816+0000", 2024-09-18T04:56:41.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1840000000000001e-06 2024-09-18T04:56:41.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662063+0000", 2024-09-18T04:56:41.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022247125999999999 2024-09-18T04:56:41.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:41.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662098+0000", 2024-09-18T04:56:41.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4968000000000001e-05 2024-09-18T04:56:41.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662149+0000", 2024-09-18T04:56:41.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1122000000000003e-05 2024-09-18T04:56:41.244 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.244 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.244 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.244 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26200 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 32768~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.244 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:56:41.244 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.58918397199999994, 2024-09-18T04:56:41.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022501118000000001, 2024-09-18T04:56:41.245 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.245 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:41.245 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.245 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.245 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.245 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26200 2024-09-18T04:56:41.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.245 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:56:41.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:56:41.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:56:41.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639822+0000", 2024-09-18T04:56:41.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.007e-06 2024-09-18T04:56:41.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639827+0000", 2024-09-18T04:56:41.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5709999999999997e-06 2024-09-18T04:56:41.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639831+0000", 2024-09-18T04:56:41.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2819999999999998e-06 2024-09-18T04:56:41.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662181+0000", 2024-09-18T04:56:41.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022349878 2024-09-18T04:56:41.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:41.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662252+0000", 2024-09-18T04:56:41.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0542000000000003e-05 2024-09-18T04:56:41.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662321+0000", 2024-09-18T04:56:41.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9167999999999999e-05 2024-09-18T04:56:41.249 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.250 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.250 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.250 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26201 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 61440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.250 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:56:41.250 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.58916868099999997, 2024-09-18T04:56:41.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022569783, 2024-09-18T04:56:41.250 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.250 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:41.251 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.251 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.251 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.251 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26201 2024-09-18T04:56:41.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.251 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:56:41.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:56:41.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:56:41.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639837+0000", 2024-09-18T04:56:41.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.928e-06 2024-09-18T04:56:41.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639843+0000", 2024-09-18T04:56:41.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8130000000000001e-06 2024-09-18T04:56:41.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639847+0000", 2024-09-18T04:56:41.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0999999999999997e-06 2024-09-18T04:56:41.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662335+0000", 2024-09-18T04:56:41.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022488207999999999 2024-09-18T04:56:41.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:41.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662370+0000", 2024-09-18T04:56:41.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5292e-05 2024-09-18T04:56:41.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662405+0000", 2024-09-18T04:56:41.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4765999999999999e-05 2024-09-18T04:56:41.254 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.254 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.254 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.255 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26202 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 90112~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.255 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:56:41.255 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.58915326400000001, 2024-09-18T04:56:41.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022655937000000001, 2024-09-18T04:56:41.255 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.255 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:41.255 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.255 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.255 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.255 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26202 2024-09-18T04:56:41.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.255 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:56:41.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:56:41.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639850+0000", 2024-09-18T04:56:41.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639852+0000", 2024-09-18T04:56:41.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9700000000000002e-06 2024-09-18T04:56:41.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639858+0000", 2024-09-18T04:56:41.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.434e-06 2024-09-18T04:56:41.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639862+0000", 2024-09-18T04:56:41.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3499999999999999e-06 2024-09-18T04:56:41.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662429+0000", 2024-09-18T04:56:41.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022567409 2024-09-18T04:56:41.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:41.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662469+0000", 2024-09-18T04:56:41.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9801000000000001e-05 2024-09-18T04:56:41.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662507+0000", 2024-09-18T04:56:41.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7311000000000002e-05 2024-09-18T04:56:41.260 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.260 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.260 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.261 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26203 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 118784~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.261 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:56:41.261 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.58913810899999997, 2024-09-18T04:56:41.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022730358999999999, 2024-09-18T04:56:41.261 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.261 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:41.262 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.262 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.262 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.262 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26203 2024-09-18T04:56:41.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.262 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:56:41.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:56:41.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639865+0000", 2024-09-18T04:56:41.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639867+0000", 2024-09-18T04:56:41.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0889999999999998e-06 2024-09-18T04:56:41.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639873+0000", 2024-09-18T04:56:41.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4979999999999997e-06 2024-09-18T04:56:41.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639877+0000", 2024-09-18T04:56:41.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4510000000000002e-06 2024-09-18T04:56:41.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662528+0000", 2024-09-18T04:56:41.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022650791999999999 2024-09-18T04:56:41.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:41.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662563+0000", 2024-09-18T04:56:41.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4530000000000003e-05 2024-09-18T04:56:41.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662596+0000", 2024-09-18T04:56:41.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3463000000000001e-05 2024-09-18T04:56:41.266 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.266 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.266 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.266 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26204 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 147456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.266 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:56:41.266 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.58912271699999996, 2024-09-18T04:56:41.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022803233999999999, 2024-09-18T04:56:41.266 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.266 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:41.266 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.266 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.266 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.267 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26204 2024-09-18T04:56:41.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.267 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:56:41.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:56:41.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:56:41.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639883+0000", 2024-09-18T04:56:41.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6520000000000002e-06 2024-09-18T04:56:41.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639888+0000", 2024-09-18T04:56:41.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2870000000000004e-06 2024-09-18T04:56:41.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639893+0000", 2024-09-18T04:56:41.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3839999999999999e-06 2024-09-18T04:56:41.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662617+0000", 2024-09-18T04:56:41.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022723876 2024-09-18T04:56:41.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:41.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662651+0000", 2024-09-18T04:56:41.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3797000000000002e-05 2024-09-18T04:56:41.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662684+0000", 2024-09-18T04:56:41.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3828999999999997e-05 2024-09-18T04:56:41.273 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.273 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.273 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.273 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26205 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 176128~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.273 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:56:41.273 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.58909975599999997, 2024-09-18T04:56:41.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022919108000000001, 2024-09-18T04:56:41.273 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.274 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:41.274 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.274 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.274 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.274 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26205 2024-09-18T04:56:41.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.274 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:56:41.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:56:41.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:56:41.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639906+0000", 2024-09-18T04:56:41.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.531e-06 2024-09-18T04:56:41.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639913+0000", 2024-09-18T04:56:41.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.331e-06 2024-09-18T04:56:41.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639920+0000", 2024-09-18T04:56:41.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5399999999999998e-06 2024-09-18T04:56:41.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662706+0000", 2024-09-18T04:56:41.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.02278587 2024-09-18T04:56:41.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:41.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662764+0000", 2024-09-18T04:56:41.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7630000000000002e-05 2024-09-18T04:56:41.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662823+0000", 2024-09-18T04:56:41.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9648000000000002e-05 2024-09-18T04:56:41.278 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.278 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.278 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.278 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26206 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 204800~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.278 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:56:41.278 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.58907935700000003, 2024-09-18T04:56:41.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023047517, 2024-09-18T04:56:41.278 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.278 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:41.278 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.278 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.279 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.279 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26206 2024-09-18T04:56:41.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.279 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:56:41.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:56:41.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:56:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639926+0000", 2024-09-18T04:56:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3120000000000001e-06 2024-09-18T04:56:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639932+0000", 2024-09-18T04:56:41.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4890000000000003e-06 2024-09-18T04:56:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639936+0000", 2024-09-18T04:56:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.07e-06 2024-09-18T04:56:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662850+0000", 2024-09-18T04:56:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022913579999999999 2024-09-18T04:56:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662903+0000", 2024-09-18T04:56:41.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3176e-05 2024-09-18T04:56:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662972+0000", 2024-09-18T04:56:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9270999999999995e-05 2024-09-18T04:56:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26207 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 233472~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:56:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.589064213, 2024-09-18T04:56:41.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023139541999999999, 2024-09-18T04:56:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26207 2024-09-18T04:56:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:56:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:56:41.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639939+0000", 2024-09-18T04:56:41.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639941+0000", 2024-09-18T04:56:41.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0080000000000001e-06 2024-09-18T04:56:41.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639947+0000", 2024-09-18T04:56:41.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.485e-06 2024-09-18T04:56:41.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639951+0000", 2024-09-18T04:56:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4549999999999997e-06 2024-09-18T04:56:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663005+0000", 2024-09-18T04:56:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023053569999999999 2024-09-18T04:56:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663042+0000", 2024-09-18T04:56:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7245e-05 2024-09-18T04:56:41.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663079+0000", 2024-09-18T04:56:41.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7119000000000002e-05 2024-09-18T04:56:41.287 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.287 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.287 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.287 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26208 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 524288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.287 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:56:41.287 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.58904746500000005, 2024-09-18T04:56:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023283564999999999, 2024-09-18T04:56:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26208 2024-09-18T04:56:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:56:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:56:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:56:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639958+0000", 2024-09-18T04:56:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0899999999999999e-06 2024-09-18T04:56:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639963+0000", 2024-09-18T04:56:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2209999999999996e-06 2024-09-18T04:56:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639967+0000", 2024-09-18T04:56:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1169999999999997e-06 2024-09-18T04:56:41.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663114+0000", 2024-09-18T04:56:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023146102000000002 2024-09-18T04:56:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663183+0000", 2024-09-18T04:56:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9686999999999999e-05 2024-09-18T04:56:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663240+0000", 2024-09-18T04:56:41.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6728000000000003e-05 2024-09-18T04:56:41.292 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.292 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.292 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.292 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26209 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 552960~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.292 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:56:41.292 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.58902968899999997, 2024-09-18T04:56:41.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023440966000000001, 2024-09-18T04:56:41.292 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.292 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:41.292 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.292 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26209 2024-09-18T04:56:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:56:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:56:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:56:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639976+0000", 2024-09-18T04:56:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4940000000000002e-06 2024-09-18T04:56:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639982+0000", 2024-09-18T04:56:41.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0560000000000001e-06 2024-09-18T04:56:41.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639987+0000", 2024-09-18T04:56:41.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-18T04:56:41.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663278+0000", 2024-09-18T04:56:41.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023290789999999999 2024-09-18T04:56:41.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:41.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663348+0000", 2024-09-18T04:56:41.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0837e-05 2024-09-18T04:56:41.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663415+0000", 2024-09-18T04:56:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6736000000000006e-05 2024-09-18T04:56:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26210 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 581632~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:56:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.58901268100000004, 2024-09-18T04:56:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023616437000000001, 2024-09-18T04:56:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:41.297 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26210 2024-09-18T04:56:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:56:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:56:41.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:56:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639993+0000", 2024-09-18T04:56:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.419e-06 2024-09-18T04:56:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640006+0000", 2024-09-18T04:56:41.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2918000000000001e-05 2024-09-18T04:56:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640011+0000", 2024-09-18T04:56:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9320000000000004e-06 2024-09-18T04:56:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663454+0000", 2024-09-18T04:56:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023443035000000001 2024-09-18T04:56:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663532+0000", 2024-09-18T04:56:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7443999999999999e-05 2024-09-18T04:56:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663608+0000", 2024-09-18T04:56:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6093000000000005e-05 2024-09-18T04:56:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26211 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 610304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:56:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.58898812899999997, 2024-09-18T04:56:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023774337999999999, 2024-09-18T04:56:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26211 2024-09-18T04:56:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:56:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:56:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640015+0000", 2024-09-18T04:56:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640023+0000", 2024-09-18T04:56:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3900000000000004e-06 2024-09-18T04:56:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640036+0000", 2024-09-18T04:56:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3091999999999999e-05 2024-09-18T04:56:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640041+0000", 2024-09-18T04:56:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6219999999999997e-06 2024-09-18T04:56:41.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663644+0000", 2024-09-18T04:56:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023602643999999999 2024-09-18T04:56:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663718+0000", 2024-09-18T04:56:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4481999999999996e-05 2024-09-18T04:56:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663790+0000", 2024-09-18T04:56:41.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1589999999999997e-05 2024-09-18T04:56:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26212 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 638976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:56:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.58895508900000004, 2024-09-18T04:56:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023924748999999999, 2024-09-18T04:56:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.306 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26212 2024-09-18T04:56:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:56:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:56:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640048+0000", 2024-09-18T04:56:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640051+0000", 2024-09-18T04:56:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.095e-06 2024-09-18T04:56:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640060+0000", 2024-09-18T04:56:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8929999999999994e-06 2024-09-18T04:56:41.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640065+0000", 2024-09-18T04:56:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0270000000000003e-06 2024-09-18T04:56:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663824+0000", 2024-09-18T04:56:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023758873999999999 2024-09-18T04:56:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663904+0000", 2024-09-18T04:56:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9856000000000003e-05 2024-09-18T04:56:41.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663974+0000", 2024-09-18T04:56:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9450999999999996e-05 2024-09-18T04:56:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26213 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 667648~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:56:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.58893391799999995, 2024-09-18T04:56:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024059812999999999, 2024-09-18T04:56:41.310 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26213 2024-09-18T04:56:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:56:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:56:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:56:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640072+0000", 2024-09-18T04:56:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4200000000000001e-06 2024-09-18T04:56:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640078+0000", 2024-09-18T04:56:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0179999999999998e-06 2024-09-18T04:56:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640083+0000", 2024-09-18T04:56:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:56:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664008+0000", 2024-09-18T04:56:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023925287 2024-09-18T04:56:41.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664059+0000", 2024-09-18T04:56:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0730999999999999e-05 2024-09-18T04:56:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664130+0000", 2024-09-18T04:56:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0801000000000002e-05 2024-09-18T04:56:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26214 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 696320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:56:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.58891694800000005, 2024-09-18T04:56:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024223738000000002, 2024-09-18T04:56:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26214 2024-09-18T04:56:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:56:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:56:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:56:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640089+0000", 2024-09-18T04:56:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0849999999999999e-06 2024-09-18T04:56:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640095+0000", 2024-09-18T04:56:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9279999999999998e-06 2024-09-18T04:56:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640099+0000", 2024-09-18T04:56:41.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:56:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664166+0000", 2024-09-18T04:56:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024066513000000001 2024-09-18T04:56:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664239+0000", 2024-09-18T04:56:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3201999999999995e-05 2024-09-18T04:56:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664311+0000", 2024-09-18T04:56:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1434999999999996e-05 2024-09-18T04:56:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26215 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 724992~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:56:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.58890018700000002, 2024-09-18T04:56:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024302284, 2024-09-18T04:56:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.319 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26215 2024-09-18T04:56:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:56:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:56:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640103+0000", 2024-09-18T04:56:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640106+0000", 2024-09-18T04:56:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2120000000000002e-06 2024-09-18T04:56:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640111+0000", 2024-09-18T04:56:41.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7880000000000003e-06 2024-09-18T04:56:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640116+0000", 2024-09-18T04:56:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6099999999999999e-06 2024-09-18T04:56:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664327+0000", 2024-09-18T04:56:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024210941999999999 2024-09-18T04:56:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664367+0000", 2024-09-18T04:56:41.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9666000000000003e-05 2024-09-18T04:56:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664406+0000", 2024-09-18T04:56:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9402e-05 2024-09-18T04:56:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26216 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 753664~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:56:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.58888369299999999, 2024-09-18T04:56:41.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024452854999999999, 2024-09-18T04:56:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26216 2024-09-18T04:56:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:56:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:56:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:56:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640122+0000", 2024-09-18T04:56:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3599999999999999e-06 2024-09-18T04:56:41.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640128+0000", 2024-09-18T04:56:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6339999999999998e-06 2024-09-18T04:56:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640132+0000", 2024-09-18T04:56:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:56:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664436+0000", 2024-09-18T04:56:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024303585999999999 2024-09-18T04:56:41.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:41.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664507+0000", 2024-09-18T04:56:41.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1046999999999999e-05 2024-09-18T04:56:41.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664573+0000", 2024-09-18T04:56:41.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6217000000000007e-05 2024-09-18T04:56:41.327 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.327 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.327 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26217 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 782336~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:56:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.58886745900000004, 2024-09-18T04:56:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024526287000000001, 2024-09-18T04:56:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26217 2024-09-18T04:56:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:56:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:56:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:56:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640140+0000", 2024-09-18T04:56:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2989999999999998e-06 2024-09-18T04:56:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640146+0000", 2024-09-18T04:56:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9569999999999999e-06 2024-09-18T04:56:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640151+0000", 2024-09-18T04:56:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:56:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664590+0000", 2024-09-18T04:56:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0244389 2024-09-18T04:56:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664628+0000", 2024-09-18T04:56:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7910999999999997e-05 2024-09-18T04:56:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664663+0000", 2024-09-18T04:56:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5024999999999998e-05 2024-09-18T04:56:41.331 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.332 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.332 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.332 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.332 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.332 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:41.334 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:56:41.335 INFO:teuthology.orchestra.run.smithi002.stdout:1663 read 0x425f9e9 thru 0x4263b41 (0x4159 bytes) 2024-09-18T04:56:41.335 INFO:teuthology.orchestra.run.smithi002.stdout:1664 writesame 0xe5b8255 thru 0xe5c67de data_size 0xb(0xe58a bytes) 2024-09-18T04:56:41.335 INFO:teuthology.orchestra.run.smithi002.stdout:1665 read 0x4395ad0 thru 0x43a548f (0xf9c0 bytes) 2024-09-18T04:56:41.335 INFO:teuthology.orchestra.run.smithi002.stdout:1666 write 0x269cce6 thru 0x26a80a3 (0xb3be bytes) 2024-09-18T04:56:41.335 INFO:teuthology.orchestra.run.smithi002.stdout:1667 flatten 2024-09-18T04:56:41.335 INFO:teuthology.orchestra.run.smithi002.stdout:1668 read 0x910494d thru 0x9111b49 (0xd1fd bytes) 2024-09-18T04:56:41.336 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:41.336 INFO:teuthology.orchestra.run.smithi002.stdout:1670 write 0xb494181 thru 0xb49edfd (0xac7d bytes) 2024-09-18T04:56:41.336 INFO:teuthology.orchestra.run.smithi002.stdout:1671 punch from 0x3d7511e to 0x3d842df, (0xf1c1 bytes) 2024-09-18T04:56:41.336 INFO:teuthology.orchestra.run.smithi002.stdout:1672 read 0x8bfb6b thru 0x8cf529 (0xf9bf bytes) 2024-09-18T04:56:41.336 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:41.336 INFO:teuthology.orchestra.run.smithi002.stdout:1674 write 0xa690c6e thru 0xa696071 (0x5404 bytes) 2024-09-18T04:56:41.336 INFO:teuthology.orchestra.run.smithi002.stdout:1675 write 0x8615446 thru 0x8622c38 (0xd7f3 bytes) 2024-09-18T04:56:41.336 INFO:teuthology.orchestra.run.smithi002.stdout:1676 read 0xde6b5e0 thru 0xde6c4ce (0xeef bytes) 2024-09-18T04:56:41.336 INFO:teuthology.orchestra.run.smithi002.stdout:1677 read 0xa0f8a09 thru 0xa1060e8 (0xd6e0 bytes) 2024-09-18T04:56:41.336 INFO:teuthology.orchestra.run.smithi002.stdout:1678 writesame 0x8146227 thru 0x814dc28 data_size 0x17(0x7a02 bytes) 2024-09-18T04:56:41.336 INFO:teuthology.orchestra.run.smithi002.stdout:1679 writesame 0x98204c1 thru 0x982b3ef data_size 0x63(0xaf2f bytes) 2024-09-18T04:56:41.336 INFO:teuthology.orchestra.run.smithi002.stdout:1680 trunc from 0xe5c67df to 0x305ed73 2024-09-18T04:56:41.337 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:41.331+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:41.342 INFO:teuthology.orchestra.run.smithi002.stdout:1682 writesame 0x124a401 thru 0x125916b data_size 0x1(0xed6b bytes) 2024-09-18T04:56:41.359 INFO:teuthology.orchestra.run.smithi002.stdout:1685 write 0xc030fcd thru 0xc03ecb1 (0xdce5 bytes) 2024-09-18T04:56:41.371 INFO:teuthology.orchestra.run.smithi002.stdout:1686 read 0xc2a4 thru 0xe124 (0x1e81 bytes) 2024-09-18T04:56:41.372 INFO:teuthology.orchestra.run.smithi002.stdout:1687 read 0x92314f4 thru 0x9234dcd (0x38da bytes) 2024-09-18T04:56:41.372 INFO:teuthology.orchestra.run.smithi002.stdout:1688 read 0x910fd51 thru 0x9110447 (0x6f7 bytes) 2024-09-18T04:56:41.374 INFO:teuthology.orchestra.run.smithi002.stdout:1689 trunc from 0xc03ecb2 to 0xe1893a7 2024-09-18T04:56:41.393 INFO:teuthology.orchestra.run.smithi002.stdout:1691 trunc from 0xe1893a7 to 0x41b0f89 2024-09-18T04:56:41.416 INFO:teuthology.orchestra.run.smithi002.stdout:1693 write 0x9f0b7e1 thru 0x9f18a59 (0xd279 bytes) 2024-09-18T04:56:41.435 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:41.432+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:41.440 INFO:teuthology.orchestra.run.smithi002.stdout:1694 write 0xde92a3e thru 0xde9c642 (0x9c05 bytes) 2024-09-18T04:56:41.448 INFO:teuthology.orchestra.run.smithi002.stdout:1695 write 0xb002ba0 thru 0xb00c453 (0x98b4 bytes) 2024-09-18T04:56:41.460 INFO:teuthology.orchestra.run.smithi002.stdout:1696 trunc from 0xde9c643 to 0x9525459 2024-09-18T04:56:41.475 INFO:teuthology.orchestra.run.smithi002.stdout:1697 write 0x9338b1b thru 0x9338e2e (0x314 bytes) 2024-09-18T04:56:41.477 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:41.484 INFO:teuthology.orchestra.run.smithi002.stdout:1700 writesame 0xa8d08c9 thru 0xa8d47da data_size 0x4e(0x3f12 bytes) 2024-09-18T04:56:41.485 INFO:teuthology.orchestra.run.smithi002.stdout:1702 write 0x80807e2 thru 0x808b578 (0xad97 bytes) 2024-09-18T04:56:41.497 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:41.497 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:56:41.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.497 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:29848 3.f 3:f61077a9:::rbd_data.11ac138032e7.0000000000000083:head [set-alloc-hint object_size 1048576 write_size 1048576,write 372736~8192 in=8192b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-09-18T04:56:41.497 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:41.489646+0000", 2024-09-18T04:56:41.498 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.0059611530000000003, 2024-09-18T04:56:41.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059783930000000002, 2024-09-18T04:56:41.498 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.498 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "waiting for sub ops", 2024-09-18T04:56:41.498 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.498 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.498 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.498 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 29848 2024-09-18T04:56:41.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.498 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:41.489646+0000", 2024-09-18T04:56:41.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:41.489646+0000", 2024-09-18T04:56:41.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:41.489645+0000", 2024-09-18T04:56:41.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:41.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:41.489656+0000", 2024-09-18T04:56:41.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0379e-05 2024-09-18T04:56:41.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:41.489667+0000", 2024-09-18T04:56:41.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1596e-05 2024-09-18T04:56:41.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:41.489674+0000", 2024-09-18T04:56:41.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6780000000000003e-06 2024-09-18T04:56:41.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:41.489702+0000", 2024-09-18T04:56:41.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8456e-05 2024-09-18T04:56:41.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:41.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:41.489750+0000", 2024-09-18T04:56:41.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7404000000000002e-05 2024-09-18T04:56:41.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:41.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:41.489826+0000", 2024-09-18T04:56:41.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6038000000000006e-05 2024-09-18T04:56:41.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:41.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:41.492840+0000", 2024-09-18T04:56:41.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0030146650000000001 2024-09-18T04:56:41.502 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.502 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.502 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.502 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.502 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-18T04:56:41.502 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 1 2024-09-18T04:56:41.502 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:41.503 INFO:teuthology.orchestra.run.smithi002.stdout:1703 punch from 0xa8c43d7 to 0xa8c83ff, (0x4028 bytes) 2024-09-18T04:56:41.503 INFO:teuthology.orchestra.run.smithi002.stdout:1704 write 0xa1fab9d thru 0xa1fca0b (0x1e6f bytes) 2024-09-18T04:56:41.507 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:56:41.509 INFO:teuthology.orchestra.run.smithi002.stdout:1705 write 0x6b9dca2 thru 0x6ba05d5 (0x2934 bytes) 2024-09-18T04:56:41.521 INFO:teuthology.orchestra.run.smithi002.stdout:1706 writesame 0x54da9fc thru 0x54e4dc6 data_size 0x9(0xa3cb bytes) 2024-09-18T04:56:41.535 INFO:teuthology.orchestra.run.smithi002.stdout:1707 punch from 0x2c7ced2 to 0x2c88dc8, (0xbef6 bytes) 2024-09-18T04:56:41.536 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:41.534+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:41.543 INFO:teuthology.orchestra.run.smithi002.stdout:1709 write 0xd3d821c thru 0xd3ddad3 (0x58b8 bytes) 2024-09-18T04:56:41.553 INFO:teuthology.orchestra.run.smithi002.stdout:1710 punch from 0x18ab99a to 0x18b9161, (0xd7c7 bytes) 2024-09-18T04:56:41.554 INFO:teuthology.orchestra.run.smithi002.stdout:1711 write 0x7502438 thru 0x750517d (0x2d46 bytes) 2024-09-18T04:56:41.558 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:41.558 INFO:teuthology.orchestra.run.smithi002.stdout:1713 read 0x26d27e3 thru 0x26d9214 (0x6a32 bytes) 2024-09-18T04:56:41.559 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:41.559 INFO:teuthology.orchestra.run.smithi002.stdout:1716 punch from 0x78a4b31 to 0x78b178d, (0xcc5c bytes) 2024-09-18T04:56:41.559 INFO:teuthology.orchestra.run.smithi002.stdout:1717 punch from 0x808d16a to 0x8099aa0, (0xc936 bytes) 2024-09-18T04:56:41.559 INFO:teuthology.orchestra.run.smithi002.stdout:1718 punch from 0x7ec17c8 to 0x7ed0915, (0xf14d bytes) 2024-09-18T04:56:41.559 INFO:teuthology.orchestra.run.smithi002.stdout:1719 punch from 0x9f3884d to 0x9f43160, (0xa913 bytes) 2024-09-18T04:56:41.565 INFO:teuthology.orchestra.run.smithi002.stdout:1720 trunc from 0xd3ddad4 to 0x7cc1153 2024-09-18T04:56:41.579 INFO:teuthology.orchestra.run.smithi002.stdout:1721 write 0x77e421 thru 0x780af3 (0x26d3 bytes) 2024-09-18T04:56:41.588 INFO:teuthology.orchestra.run.smithi002.stdout:1723 trunc from 0x7cc1153 to 0x98cad6e 2024-09-18T04:56:41.595 INFO:teuthology.orchestra.run.smithi002.stdout:1724 trunc from 0x98cad6e to 0x826fc07 2024-09-18T04:56:41.601 INFO:teuthology.orchestra.run.smithi002.stdout:1726 writesame 0x5384f66 thru 0x5393e13 data_size 0x2(0xeeae bytes) 2024-09-18T04:56:41.614 INFO:teuthology.orchestra.run.smithi002.stdout:1727 read 0x4a23567 thru 0x4a29ca9 (0x6743 bytes) 2024-09-18T04:56:41.615 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:41.615 INFO:teuthology.orchestra.run.smithi002.stdout:1731 read 0x40c5ba0 thru 0x40cc992 (0x6df3 bytes) 2024-09-18T04:56:41.615 INFO:teuthology.orchestra.run.smithi002.stdout:1732 punch from 0x38b7033 to 0x38c2440, (0xb40d bytes) 2024-09-18T04:56:41.615 INFO:teuthology.orchestra.run.smithi002.stdout:1733 read 0x295473a thru 0x295b75c (0x7023 bytes) 2024-09-18T04:56:41.616 INFO:teuthology.orchestra.run.smithi002.stdout:1735 trunc from 0x826fc07 to 0x683dd4d 2024-09-18T04:56:41.625 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:41.626 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:41.626 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:56:41.627 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:56:41.627 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:41.628 INFO:teuthology.orchestra.run.smithi002.stdout:1738 write 0x5c90391 thru 0x5c92fe1 (0x2c51 bytes) 2024-09-18T04:56:41.634 INFO:teuthology.orchestra.run.smithi002.stdout:1739 read 0xd9b2bf thru 0xda3fa6 (0x8ce8 bytes) 2024-09-18T04:56:41.635 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:56:41.636 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:41.636 INFO:teuthology.orchestra.run.smithi002.stdout:1742 read 0x286f049 thru 0x287dd80 (0xed38 bytes) 2024-09-18T04:56:41.636 INFO:teuthology.orchestra.run.smithi002.stdout:1743 clone 14 order 23 su 2097152 sc 5 2024-09-18T04:56:41.637 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:41.634+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:41.738 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:41.738 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:56:41.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:56:41.738 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:56:41.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.739 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:56:41.739 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:56:41.739 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 30.342226107999998, 2024-09-18T04:56:41.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037718511000000003, 2024-09-18T04:56:41.739 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.739 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:41.739 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.739 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:56:41.739 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:56:41.739 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-18T04:56:41.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.739 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:56:41.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:56:41.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393125+0000", 2024-09-18T04:56:41.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:41.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393133+0000", 2024-09-18T04:56:41.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.225e-06 2024-09-18T04:56:41.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393144+0000", 2024-09-18T04:56:41.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1279e-05 2024-09-18T04:56:41.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393153+0000", 2024-09-18T04:56:41.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4540000000000007e-06 2024-09-18T04:56:41.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393203+0000", 2024-09-18T04:56:41.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9963000000000003e-05 2024-09-18T04:56:41.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:56:41.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393210+0000", 2024-09-18T04:56:41.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5689999999999999e-06 2024-09-18T04:56:41.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427399+0000", 2024-09-18T04:56:41.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034188784999999999 2024-09-18T04:56:41.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:41.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427532+0000", 2024-09-18T04:56:41.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013264700000000001 2024-09-18T04:56:41.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:41.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427685+0000", 2024-09-18T04:56:41.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015300599999999999 2024-09-18T04:56:41.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:41.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.428520+0000", 2024-09-18T04:56:41.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00083502499999999998 2024-09-18T04:56:41.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:41.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430772+0000", 2024-09-18T04:56:41.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022518880000000001 2024-09-18T04:56:41.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:41.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430817+0000", 2024-09-18T04:56:41.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5077000000000002e-05 2024-09-18T04:56:41.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430845+0000", 2024-09-18T04:56:41.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8036999999999999e-05 2024-09-18T04:56:41.745 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.745 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.746 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.746 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:805 3.15 3:ab221474:::rbd_data.11ac33624de9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:41.746 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.333817+0000", 2024-09-18T04:56:41.746 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.401535106000001, 2024-09-18T04:56:41.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017602593999999999, 2024-09-18T04:56:41.746 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.746 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:41.746 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.746 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.746 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.746 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 805 2024-09-18T04:56:41.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.747 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.333817+0000", 2024-09-18T04:56:41.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.333817+0000", 2024-09-18T04:56:41.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.333816+0000", 2024-09-18T04:56:41.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:41.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.333821+0000", 2024-09-18T04:56:41.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3079999999999998e-06 2024-09-18T04:56:41.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.333832+0000", 2024-09-18T04:56:41.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0139000000000001e-05 2024-09-18T04:56:41.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.333855+0000", 2024-09-18T04:56:41.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3210999999999999e-05 2024-09-18T04:56:41.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.333903+0000", 2024-09-18T04:56:41.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8613999999999999e-05 2024-09-18T04:56:41.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:41.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.334043+0000", 2024-09-18T04:56:41.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013937199999999999 2024-09-18T04:56:41.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-18T04:56:41.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.337213+0000", 2024-09-18T04:56:41.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003170595 2024-09-18T04:56:41.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:41.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.351345+0000", 2024-09-18T04:56:41.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014131539 2024-09-18T04:56:41.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:41.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.351356+0000", 2024-09-18T04:56:41.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0703999999999999e-05 2024-09-18T04:56:41.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:41.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.351392+0000", 2024-09-18T04:56:41.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6758999999999998e-05 2024-09-18T04:56:41.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.351420+0000", 2024-09-18T04:56:41.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7582000000000001e-05 2024-09-18T04:56:41.752 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.752 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.752 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.753 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:5168 3.11 3:8c2284b2:::rbd_data.11ac14fc1d07.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 10863232~40683] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:41.753 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.504653+0000", 2024-09-18T04:56:41.753 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.2306989599999998, 2024-09-18T04:56:41.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.027990498999999999, 2024-09-18T04:56:41.753 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.753 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:41.753 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.753 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.753 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.753 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 5168 2024-09-18T04:56:41.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.753 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.504653+0000", 2024-09-18T04:56:41.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.504653+0000", 2024-09-18T04:56:41.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.504653+0000", 2024-09-18T04:56:41.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:41.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.516093+0000", 2024-09-18T04:56:41.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011440120999999999 2024-09-18T04:56:41.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.516116+0000", 2024-09-18T04:56:41.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3153999999999999e-05 2024-09-18T04:56:41.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.516127+0000", 2024-09-18T04:56:41.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1056999999999999e-05 2024-09-18T04:56:41.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.516210+0000", 2024-09-18T04:56:41.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3128999999999996e-05 2024-09-18T04:56:41.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:41.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.516339+0000", 2024-09-18T04:56:41.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012917000000000001 2024-09-18T04:56:41.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:41.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.516794+0000", 2024-09-18T04:56:41.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00045514500000000001 2024-09-18T04:56:41.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:41.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.528090+0000", 2024-09-18T04:56:41.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011295423000000001 2024-09-18T04:56:41.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:41.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.532575+0000", 2024-09-18T04:56:41.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0044854220000000002 2024-09-18T04:56:41.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:41.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.532613+0000", 2024-09-18T04:56:41.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7333000000000003e-05 2024-09-18T04:56:41.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.532644+0000", 2024-09-18T04:56:41.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1282999999999998e-05 2024-09-18T04:56:41.758 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.758 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.758 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.759 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:22467 3.11 3:8dcd132e:::rbd_data.11ac82b416ac.00000000000000c1:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-18T04:56:41.759 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:37.484119+0000", 2024-09-18T04:56:41.759 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.2512329869999999, 2024-09-18T04:56:41.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023250593, 2024-09-18T04:56:41.759 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.759 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:41.759 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.759 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.759 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.759 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 22467 2024-09-18T04:56:41.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.759 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.484119+0000", 2024-09-18T04:56:41.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.484119+0000", 2024-09-18T04:56:41.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.484118+0000", 2024-09-18T04:56:41.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:41.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.484125+0000", 2024-09-18T04:56:41.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5860000000000001e-06 2024-09-18T04:56:41.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.484136+0000", 2024-09-18T04:56:41.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1164e-05 2024-09-18T04:56:41.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.484141+0000", 2024-09-18T04:56:41.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6659999999999996e-06 2024-09-18T04:56:41.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.484177+0000", 2024-09-18T04:56:41.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5139999999999999e-05 2024-09-18T04:56:41.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:41.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.484263+0000", 2024-09-18T04:56:41.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6811999999999995e-05 2024-09-18T04:56:41.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:41.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.487896+0000", 2024-09-18T04:56:41.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036321769999999999 2024-09-18T04:56:41.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:41.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.507245+0000", 2024-09-18T04:56:41.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019349094000000001 2024-09-18T04:56:41.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:41.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.507289+0000", 2024-09-18T04:56:41.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4168999999999997e-05 2024-09-18T04:56:41.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:41.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.507327+0000", 2024-09-18T04:56:41.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7811000000000001e-05 2024-09-18T04:56:41.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.507370+0000", 2024-09-18T04:56:41.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3331999999999998e-05 2024-09-18T04:56:41.764 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.764 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.764 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.765 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28690 3.11 3.41a894f1 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.765 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840079+0000", 2024-09-18T04:56:41.765 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.89527390500000004, 2024-09-18T04:56:41.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015023241999999999, 2024-09-18T04:56:41.765 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.765 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:41.765 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.765 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.765 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.765 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28690 2024-09-18T04:56:41.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.765 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840079+0000", 2024-09-18T04:56:41.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840079+0000", 2024-09-18T04:56:41.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840078+0000", 2024-09-18T04:56:41.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:41.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840080+0000", 2024-09-18T04:56:41.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2249999999999999e-06 2024-09-18T04:56:41.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840083+0000", 2024-09-18T04:56:41.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8039999999999999e-06 2024-09-18T04:56:41.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840086+0000", 2024-09-18T04:56:41.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4420000000000001e-06 2024-09-18T04:56:41.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855091+0000", 2024-09-18T04:56:41.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015005136000000001 2024-09-18T04:56:41.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855102+0000", 2024-09-18T04:56:41.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0839000000000001e-05 2024-09-18T04:56:41.768 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.768 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.768 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.769 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28713 3.11 3.41a894f1 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.769 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840088+0000", 2024-09-18T04:56:41.769 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.89526437400000003, 2024-09-18T04:56:41.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015057488000000001, 2024-09-18T04:56:41.769 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.769 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:41.769 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.769 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.769 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.769 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28713 2024-09-18T04:56:41.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.769 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840088+0000", 2024-09-18T04:56:41.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840088+0000", 2024-09-18T04:56:41.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840088+0000", 2024-09-18T04:56:41.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840089+0000", 2024-09-18T04:56:41.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2100000000000001e-06 2024-09-18T04:56:41.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840092+0000", 2024-09-18T04:56:41.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9289999999999998e-06 2024-09-18T04:56:41.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840094+0000", 2024-09-18T04:56:41.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5110000000000002e-06 2024-09-18T04:56:41.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855135+0000", 2024-09-18T04:56:41.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015040820999999999 2024-09-18T04:56:41.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855146+0000", 2024-09-18T04:56:41.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0312999999999999e-05 2024-09-18T04:56:41.773 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.773 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.773 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.773 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28714 3.11 3.5e1e1ab1 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.773 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840096+0000", 2024-09-18T04:56:41.773 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.895255935, 2024-09-18T04:56:41.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015095044, 2024-09-18T04:56:41.773 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.774 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:41.774 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.774 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.774 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.774 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28714 2024-09-18T04:56:41.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.774 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840096+0000", 2024-09-18T04:56:41.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840096+0000", 2024-09-18T04:56:41.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840096+0000", 2024-09-18T04:56:41.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:41.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840098+0000", 2024-09-18T04:56:41.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.299e-06 2024-09-18T04:56:41.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840101+0000", 2024-09-18T04:56:41.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6449999999999998e-06 2024-09-18T04:56:41.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840104+0000", 2024-09-18T04:56:41.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-18T04:56:41.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855180+0000", 2024-09-18T04:56:41.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015076463999999999 2024-09-18T04:56:41.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855192+0000", 2024-09-18T04:56:41.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1314e-05 2024-09-18T04:56:41.777 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.777 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.777 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.777 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28750 3.11 3.41a894f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.778 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840281+0000", 2024-09-18T04:56:41.778 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.89507138399999997, 2024-09-18T04:56:41.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015023635, 2024-09-18T04:56:41.778 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.778 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:41.778 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.778 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.778 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.778 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28750 2024-09-18T04:56:41.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.778 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840281+0000", 2024-09-18T04:56:41.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840281+0000", 2024-09-18T04:56:41.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840281+0000", 2024-09-18T04:56:41.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840287+0000", 2024-09-18T04:56:41.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2040000000000002e-06 2024-09-18T04:56:41.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840293+0000", 2024-09-18T04:56:41.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.19e-06 2024-09-18T04:56:41.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840298+0000", 2024-09-18T04:56:41.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8219999999999997e-06 2024-09-18T04:56:41.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855295+0000", 2024-09-18T04:56:41.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014997323999999999 2024-09-18T04:56:41.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855305+0000", 2024-09-18T04:56:41.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4800000000000007e-06 2024-09-18T04:56:41.782 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.782 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.782 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.782 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28751 3.11 3.41a894f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.782 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840303+0000", 2024-09-18T04:56:41.782 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.89504975099999995, 2024-09-18T04:56:41.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015059965999999999, 2024-09-18T04:56:41.783 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.783 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:41.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.783 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28751 2024-09-18T04:56:41.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.783 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840303+0000", 2024-09-18T04:56:41.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840303+0000", 2024-09-18T04:56:41.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840302+0000", 2024-09-18T04:56:41.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840304+0000", 2024-09-18T04:56:41.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1330000000000002e-06 2024-09-18T04:56:41.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840310+0000", 2024-09-18T04:56:41.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1850000000000001e-06 2024-09-18T04:56:41.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840316+0000", 2024-09-18T04:56:41.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2100000000000001e-06 2024-09-18T04:56:41.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855342+0000", 2024-09-18T04:56:41.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015026214 2024-09-18T04:56:41.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855363+0000", 2024-09-18T04:56:41.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0704e-05 2024-09-18T04:56:41.787 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.787 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.787 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.787 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28752 3.11 3.41a894f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.787 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840320+0000", 2024-09-18T04:56:41.787 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.89503216900000004, 2024-09-18T04:56:41.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01507758, 2024-09-18T04:56:41.788 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.788 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:41.788 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.788 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.788 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.788 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28752 2024-09-18T04:56:41.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.788 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840320+0000", 2024-09-18T04:56:41.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840320+0000", 2024-09-18T04:56:41.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840320+0000", 2024-09-18T04:56:41.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840322+0000", 2024-09-18T04:56:41.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0219999999999999e-06 2024-09-18T04:56:41.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840328+0000", 2024-09-18T04:56:41.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8490000000000002e-06 2024-09-18T04:56:41.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840333+0000", 2024-09-18T04:56:41.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.814e-06 2024-09-18T04:56:41.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855388+0000", 2024-09-18T04:56:41.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015055681 2024-09-18T04:56:41.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855398+0000", 2024-09-18T04:56:41.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5710000000000004e-06 2024-09-18T04:56:41.791 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.791 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.791 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.791 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28756 3.11 3.41a894f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.792 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840506+0000", 2024-09-18T04:56:41.792 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.89484635499999998, 2024-09-18T04:56:41.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015052385, 2024-09-18T04:56:41.792 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.792 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:41.792 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.792 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.792 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.792 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28756 2024-09-18T04:56:41.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.792 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840506+0000", 2024-09-18T04:56:41.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840506+0000", 2024-09-18T04:56:41.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840506+0000", 2024-09-18T04:56:41.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840509+0000", 2024-09-18T04:56:41.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.304e-06 2024-09-18T04:56:41.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840515+0000", 2024-09-18T04:56:41.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8660000000000002e-06 2024-09-18T04:56:41.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840519+0000", 2024-09-18T04:56:41.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2390000000000004e-06 2024-09-18T04:56:41.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855549+0000", 2024-09-18T04:56:41.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015029847000000001 2024-09-18T04:56:41.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855558+0000", 2024-09-18T04:56:41.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5719999999999992e-06 2024-09-18T04:56:41.796 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.796 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.796 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.796 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28812 3.11 3.41a894f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.796 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840523+0000", 2024-09-18T04:56:41.796 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.89482963900000001, 2024-09-18T04:56:41.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015078464999999999, 2024-09-18T04:56:41.796 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.796 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:41.796 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.796 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.797 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.797 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28812 2024-09-18T04:56:41.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.797 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840523+0000", 2024-09-18T04:56:41.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840523+0000", 2024-09-18T04:56:41.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840522+0000", 2024-09-18T04:56:41.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840524+0000", 2024-09-18T04:56:41.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.829e-06 2024-09-18T04:56:41.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840530+0000", 2024-09-18T04:56:41.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7350000000000001e-06 2024-09-18T04:56:41.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840535+0000", 2024-09-18T04:56:41.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0069999999999997e-06 2024-09-18T04:56:41.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855592+0000", 2024-09-18T04:56:41.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015056704000000001 2024-09-18T04:56:41.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855601+0000", 2024-09-18T04:56:41.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5289999999999997e-06 2024-09-18T04:56:41.801 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.801 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.801 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.801 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28813 3.11 3.41a894f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.801 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840539+0000", 2024-09-18T04:56:41.801 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.89481350400000004, 2024-09-18T04:56:41.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015088677, 2024-09-18T04:56:41.801 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.801 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:41.801 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.802 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28813 2024-09-18T04:56:41.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.802 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840539+0000", 2024-09-18T04:56:41.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840539+0000", 2024-09-18T04:56:41.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840539+0000", 2024-09-18T04:56:41.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840541+0000", 2024-09-18T04:56:41.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9489999999999999e-06 2024-09-18T04:56:41.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840546+0000", 2024-09-18T04:56:41.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5040000000000002e-06 2024-09-18T04:56:41.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840550+0000", 2024-09-18T04:56:41.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1819999999999999e-06 2024-09-18T04:56:41.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855619+0000", 2024-09-18T04:56:41.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015068712 2024-09-18T04:56:41.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855628+0000", 2024-09-18T04:56:41.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6449999999999994e-06 2024-09-18T04:56:41.805 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.805 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.805 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.805 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28814 3.11 3.41a894f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.805 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840555+0000", 2024-09-18T04:56:41.805 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.89479741599999996, 2024-09-18T04:56:41.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015115778, 2024-09-18T04:56:41.805 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.805 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:41.805 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.806 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.806 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.806 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28814 2024-09-18T04:56:41.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.806 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840555+0000", 2024-09-18T04:56:41.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840555+0000", 2024-09-18T04:56:41.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840555+0000", 2024-09-18T04:56:41.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840557+0000", 2024-09-18T04:56:41.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9010000000000001e-06 2024-09-18T04:56:41.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840562+0000", 2024-09-18T04:56:41.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.451e-06 2024-09-18T04:56:41.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840566+0000", 2024-09-18T04:56:41.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1189999999999999e-06 2024-09-18T04:56:41.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855662+0000", 2024-09-18T04:56:41.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015095411 2024-09-18T04:56:41.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855671+0000", 2024-09-18T04:56:41.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2380000000000003e-06 2024-09-18T04:56:41.809 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.809 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.809 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.809 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28815 3.11 3.41a894f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.809 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840570+0000", 2024-09-18T04:56:41.809 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.89478245099999998, 2024-09-18T04:56:41.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015143973, 2024-09-18T04:56:41.809 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.809 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:41.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.810 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28815 2024-09-18T04:56:41.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.810 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840570+0000", 2024-09-18T04:56:41.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840570+0000", 2024-09-18T04:56:41.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840570+0000", 2024-09-18T04:56:41.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840572+0000", 2024-09-18T04:56:41.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.012e-06 2024-09-18T04:56:41.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840577+0000", 2024-09-18T04:56:41.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8699999999999997e-06 2024-09-18T04:56:41.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840582+0000", 2024-09-18T04:56:41.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3970000000000004e-06 2024-09-18T04:56:41.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855705+0000", 2024-09-18T04:56:41.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015122834999999999 2024-09-18T04:56:41.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855714+0000", 2024-09-18T04:56:41.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2250000000000006e-06 2024-09-18T04:56:41.813 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.813 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.813 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.813 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28816 3.11 3.41a894f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.813 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840586+0000", 2024-09-18T04:56:41.813 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.894766332, 2024-09-18T04:56:41.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015144272, 2024-09-18T04:56:41.813 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.813 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:41.814 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.814 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.814 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.814 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28816 2024-09-18T04:56:41.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.814 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840586+0000", 2024-09-18T04:56:41.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840586+0000", 2024-09-18T04:56:41.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840586+0000", 2024-09-18T04:56:41.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840588+0000", 2024-09-18T04:56:41.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.153e-06 2024-09-18T04:56:41.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840593+0000", 2024-09-18T04:56:41.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6579999999999999e-06 2024-09-18T04:56:41.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840598+0000", 2024-09-18T04:56:41.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.25e-06 2024-09-18T04:56:41.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855726+0000", 2024-09-18T04:56:41.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015128206 2024-09-18T04:56:41.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855730+0000", 2024-09-18T04:56:41.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4150000000000001e-06 2024-09-18T04:56:41.819 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.819 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.820 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.820 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28817 3.11 3.41a894f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.820 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840602+0000", 2024-09-18T04:56:41.820 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.89475083600000005, 2024-09-18T04:56:41.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015152196, 2024-09-18T04:56:41.820 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.820 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:41.820 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.820 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.820 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.820 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28817 2024-09-18T04:56:41.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.821 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840602+0000", 2024-09-18T04:56:41.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840602+0000", 2024-09-18T04:56:41.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840601+0000", 2024-09-18T04:56:41.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840608+0000", 2024-09-18T04:56:41.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4070000000000004e-06 2024-09-18T04:56:41.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840614+0000", 2024-09-18T04:56:41.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7769999999999999e-06 2024-09-18T04:56:41.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840619+0000", 2024-09-18T04:56:41.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3989999999999997e-06 2024-09-18T04:56:41.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855745+0000", 2024-09-18T04:56:41.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015126539 2024-09-18T04:56:41.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855754+0000", 2024-09-18T04:56:41.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5699999999999993e-06 2024-09-18T04:56:41.824 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.824 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.824 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.824 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28818 3.11 3.41a894f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.824 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840637+0000", 2024-09-18T04:56:41.824 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.89471576200000003, 2024-09-18T04:56:41.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015132655, 2024-09-18T04:56:41.824 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.824 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:41.824 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.824 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.824 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.825 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28818 2024-09-18T04:56:41.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.825 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840637+0000", 2024-09-18T04:56:41.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840637+0000", 2024-09-18T04:56:41.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840636+0000", 2024-09-18T04:56:41.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:41.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840640+0000", 2024-09-18T04:56:41.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6090000000000003e-06 2024-09-18T04:56:41.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840649+0000", 2024-09-18T04:56:41.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6719999999999993e-06 2024-09-18T04:56:41.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840656+0000", 2024-09-18T04:56:41.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3509999999999996e-06 2024-09-18T04:56:41.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855765+0000", 2024-09-18T04:56:41.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015108053999999999 2024-09-18T04:56:41.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855769+0000", 2024-09-18T04:56:41.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7759999999999997e-06 2024-09-18T04:56:41.828 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.828 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.828 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.828 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28819 3.11 3.41a894f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.828 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840662+0000", 2024-09-18T04:56:41.828 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.8946904, 2024-09-18T04:56:41.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015142542, 2024-09-18T04:56:41.828 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.828 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:41.828 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.828 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.828 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.829 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28819 2024-09-18T04:56:41.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.829 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840662+0000", 2024-09-18T04:56:41.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840662+0000", 2024-09-18T04:56:41.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840661+0000", 2024-09-18T04:56:41.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840664+0000", 2024-09-18T04:56:41.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5809999999999999e-06 2024-09-18T04:56:41.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840671+0000", 2024-09-18T04:56:41.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5479999999999998e-06 2024-09-18T04:56:41.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840676+0000", 2024-09-18T04:56:41.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1939999999999997e-06 2024-09-18T04:56:41.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855796+0000", 2024-09-18T04:56:41.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015119850000000001 2024-09-18T04:56:41.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855805+0000", 2024-09-18T04:56:41.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8959999999999992e-06 2024-09-18T04:56:41.832 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.832 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.832 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.832 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28820 3.11 3.41a894f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:41.832 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840682+0000", 2024-09-18T04:56:41.832 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.89467088400000006, 2024-09-18T04:56:41.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015144426000000001, 2024-09-18T04:56:41.832 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:41.832 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:41.832 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:41.833 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:41.833 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:41.833 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28820 2024-09-18T04:56:41.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.833 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:41.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:41.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840682+0000", 2024-09-18T04:56:41.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:41.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840682+0000", 2024-09-18T04:56:41.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:41.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:41.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840681+0000", 2024-09-18T04:56:41.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:41.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:41.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840683+0000", 2024-09-18T04:56:41.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2249999999999999e-06 2024-09-18T04:56:41.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:41.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840690+0000", 2024-09-18T04:56:41.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2820000000000002e-06 2024-09-18T04:56:41.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:41.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840698+0000", 2024-09-18T04:56:41.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2479999999999996e-06 2024-09-18T04:56:41.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:41.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855822+0000", 2024-09-18T04:56:41.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015123763 2024-09-18T04:56:41.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:41.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:41.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:41.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855826+0000", 2024-09-18T04:56:41.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2880000000000003e-06 2024-09-18T04:56:41.836 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.836 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.836 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.836 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:41.836 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:41.836 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:41.836 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:41.735+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:41.837 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:56:41.880 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:41.876+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:41.939 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:41.937+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:41.946 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:56:41.947 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:41.955 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:56:42.040 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:42.038+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:42.062 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:42.062 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:42.063 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:56:42.063 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:56:42.063 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:42.071 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:56:42.141 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:42.139+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:42.179 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:42.179 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:56:42.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:56:42.179 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:56:42.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.180 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9314 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1581056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.180 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767102+0000", 2024-09-18T04:56:42.180 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4092996539999998, 2024-09-18T04:56:42.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043648235000000001, 2024-09-18T04:56:42.180 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.180 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.180 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.180 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.180 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.180 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9314 2024-09-18T04:56:42.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.180 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767102+0000", 2024-09-18T04:56:42.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767102+0000", 2024-09-18T04:56:42.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767101+0000", 2024-09-18T04:56:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767104+0000", 2024-09-18T04:56:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3920000000000001e-06 2024-09-18T04:56:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767110+0000", 2024-09-18T04:56:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0480000000000004e-06 2024-09-18T04:56:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767114+0000", 2024-09-18T04:56:42.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6639999999999997e-06 2024-09-18T04:56:42.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810579+0000", 2024-09-18T04:56:42.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043464931999999998 2024-09-18T04:56:42.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810667+0000", 2024-09-18T04:56:42.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7356999999999997e-05 2024-09-18T04:56:42.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810750+0000", 2024-09-18T04:56:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3374999999999994e-05 2024-09-18T04:56:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9315 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1630208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767118+0000", 2024-09-18T04:56:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4092829770000002, 2024-09-18T04:56:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043838369000000002, 2024-09-18T04:56:42.184 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9315 2024-09-18T04:56:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767118+0000", 2024-09-18T04:56:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767118+0000", 2024-09-18T04:56:42.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767118+0000", 2024-09-18T04:56:42.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767126+0000", 2024-09-18T04:56:42.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6559999999999992e-06 2024-09-18T04:56:42.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767132+0000", 2024-09-18T04:56:42.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3350000000000003e-06 2024-09-18T04:56:42.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767138+0000", 2024-09-18T04:56:42.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5849999999999999e-06 2024-09-18T04:56:42.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810792+0000", 2024-09-18T04:56:42.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043654485999999999 2024-09-18T04:56:42.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810879+0000", 2024-09-18T04:56:42.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6691000000000001e-05 2024-09-18T04:56:42.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810957+0000", 2024-09-18T04:56:42.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7996000000000003e-05 2024-09-18T04:56:42.188 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.188 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.188 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.189 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9316 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.189 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:56:42.189 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4092592320000001, 2024-09-18T04:56:42.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044027208999999998, 2024-09-18T04:56:42.189 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.189 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.189 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.189 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.189 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.189 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9316 2024-09-18T04:56:42.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.189 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:56:42.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:56:42.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:56:42.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767144+0000", 2024-09-18T04:56:42.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3070000000000001e-06 2024-09-18T04:56:42.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767150+0000", 2024-09-18T04:56:42.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1700000000000002e-06 2024-09-18T04:56:42.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767163+0000", 2024-09-18T04:56:42.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2670999999999999e-05 2024-09-18T04:56:42.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810995+0000", 2024-09-18T04:56:42.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043832310999999999 2024-09-18T04:56:42.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811087+0000", 2024-09-18T04:56:42.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1345e-05 2024-09-18T04:56:42.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811169+0000", 2024-09-18T04:56:42.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2848000000000004e-05 2024-09-18T04:56:42.193 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.193 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.193 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.193 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9317 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1728512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.193 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:56:42.193 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4092335970000001, 2024-09-18T04:56:42.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044190170000000001, 2024-09-18T04:56:42.193 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.193 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9317 2024-09-18T04:56:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:56:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:56:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767167+0000", 2024-09-18T04:56:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767170+0000", 2024-09-18T04:56:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7259999999999998e-06 2024-09-18T04:56:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767176+0000", 2024-09-18T04:56:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1460000000000001e-06 2024-09-18T04:56:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767182+0000", 2024-09-18T04:56:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1020000000000002e-06 2024-09-18T04:56:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811204+0000", 2024-09-18T04:56:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044022039999999998 2024-09-18T04:56:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811286+0000", 2024-09-18T04:56:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1623000000000001e-05 2024-09-18T04:56:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811358+0000", 2024-09-18T04:56:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2125000000000005e-05 2024-09-18T04:56:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9318 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1777664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:56:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4092150520000004, 2024-09-18T04:56:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044380181999999997, 2024-09-18T04:56:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9318 2024-09-18T04:56:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:56:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:56:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:56:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767189+0000", 2024-09-18T04:56:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5909999999999998e-06 2024-09-18T04:56:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767194+0000", 2024-09-18T04:56:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7690000000000002e-06 2024-09-18T04:56:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767199+0000", 2024-09-18T04:56:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189999999999999e-06 2024-09-18T04:56:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811394+0000", 2024-09-18T04:56:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044195350000000001 2024-09-18T04:56:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811483+0000", 2024-09-18T04:56:42.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8282999999999995e-05 2024-09-18T04:56:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811567+0000", 2024-09-18T04:56:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3855000000000003e-05 2024-09-18T04:56:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9322 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.768849+0000", 2024-09-18T04:56:42.202 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4075526600000003, 2024-09-18T04:56:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043526553000000003, 2024-09-18T04:56:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9322 2024-09-18T04:56:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768849+0000", 2024-09-18T04:56:42.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768849+0000", 2024-09-18T04:56:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768848+0000", 2024-09-18T04:56:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768851+0000", 2024-09-18T04:56:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2079999999999999e-06 2024-09-18T04:56:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768857+0000", 2024-09-18T04:56:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.942e-06 2024-09-18T04:56:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768867+0000", 2024-09-18T04:56:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0711e-05 2024-09-18T04:56:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812206+0000", 2024-09-18T04:56:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043338347999999999 2024-09-18T04:56:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812293+0000", 2024-09-18T04:56:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7435000000000006e-05 2024-09-18T04:56:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812375+0000", 2024-09-18T04:56:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2279000000000003e-05 2024-09-18T04:56:42.206 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9323 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1990656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.768873+0000", 2024-09-18T04:56:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4075288739999996, 2024-09-18T04:56:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043695031000000002, 2024-09-18T04:56:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.207 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9323 2024-09-18T04:56:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768873+0000", 2024-09-18T04:56:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768873+0000", 2024-09-18T04:56:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768872+0000", 2024-09-18T04:56:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768880+0000", 2024-09-18T04:56:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4179999999999997e-06 2024-09-18T04:56:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768890+0000", 2024-09-18T04:56:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5310000000000007e-06 2024-09-18T04:56:42.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768895+0000", 2024-09-18T04:56:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.186e-06 2024-09-18T04:56:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812412+0000", 2024-09-18T04:56:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043516525 2024-09-18T04:56:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812496+0000", 2024-09-18T04:56:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4039e-05 2024-09-18T04:56:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812568+0000", 2024-09-18T04:56:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1988000000000003e-05 2024-09-18T04:56:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9324 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 2039808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.211 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.768900+0000", 2024-09-18T04:56:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4075015430000004, 2024-09-18T04:56:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043885234000000002, 2024-09-18T04:56:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9324 2024-09-18T04:56:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768900+0000", 2024-09-18T04:56:42.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768900+0000", 2024-09-18T04:56:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768900+0000", 2024-09-18T04:56:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768902+0000", 2024-09-18T04:56:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3070000000000001e-06 2024-09-18T04:56:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768908+0000", 2024-09-18T04:56:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2670000000000003e-06 2024-09-18T04:56:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768913+0000", 2024-09-18T04:56:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8770000000000001e-06 2024-09-18T04:56:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812606+0000", 2024-09-18T04:56:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043692867000000003 2024-09-18T04:56:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812695+0000", 2024-09-18T04:56:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9060000000000005e-05 2024-09-18T04:56:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812785+0000", 2024-09-18T04:56:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0242999999999996e-05 2024-09-18T04:56:42.215 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9325 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:56:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4074845140000001, 2024-09-18T04:56:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044063989999999997, 2024-09-18T04:56:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.216 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9325 2024-09-18T04:56:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:56:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:56:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:56:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768919+0000", 2024-09-18T04:56:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:56:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768925+0000", 2024-09-18T04:56:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2310000000000001e-06 2024-09-18T04:56:42.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768929+0000", 2024-09-18T04:56:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4469999999999999e-06 2024-09-18T04:56:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812823+0000", 2024-09-18T04:56:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043893975000000002 2024-09-18T04:56:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812908+0000", 2024-09-18T04:56:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4862000000000002e-05 2024-09-18T04:56:42.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812981+0000", 2024-09-18T04:56:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2892000000000006e-05 2024-09-18T04:56:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9332 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 274432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.770912+0000", 2024-09-18T04:56:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4054898549999999, 2024-09-18T04:56:42.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043671336999999998, 2024-09-18T04:56:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9332 2024-09-18T04:56:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770912+0000", 2024-09-18T04:56:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770912+0000", 2024-09-18T04:56:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770911+0000", 2024-09-18T04:56:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770916+0000", 2024-09-18T04:56:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6269999999999999e-06 2024-09-18T04:56:42.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770919+0000", 2024-09-18T04:56:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3100000000000001e-06 2024-09-18T04:56:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770922+0000", 2024-09-18T04:56:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3439999999999999e-06 2024-09-18T04:56:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814399+0000", 2024-09-18T04:56:42.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043477403999999997 2024-09-18T04:56:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814484+0000", 2024-09-18T04:56:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4625999999999999e-05 2024-09-18T04:56:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814583+0000", 2024-09-18T04:56:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9403999999999999e-05 2024-09-18T04:56:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.224 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9333 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.770924+0000", 2024-09-18T04:56:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4054775719999997, 2024-09-18T04:56:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043870484000000001, 2024-09-18T04:56:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9333 2024-09-18T04:56:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.225 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770924+0000", 2024-09-18T04:56:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770924+0000", 2024-09-18T04:56:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770924+0000", 2024-09-18T04:56:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770925+0000", 2024-09-18T04:56:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.19e-06 2024-09-18T04:56:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770931+0000", 2024-09-18T04:56:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0789999999999997e-06 2024-09-18T04:56:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770934+0000", 2024-09-18T04:56:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6110000000000001e-06 2024-09-18T04:56:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814609+0000", 2024-09-18T04:56:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043675640000000002 2024-09-18T04:56:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814685+0000", 2024-09-18T04:56:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5506000000000005e-05 2024-09-18T04:56:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814794+0000", 2024-09-18T04:56:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010972300000000001 2024-09-18T04:56:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.231 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9334 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:56:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4054657280000002, 2024-09-18T04:56:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044042853999999999, 2024-09-18T04:56:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.232 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9334 2024-09-18T04:56:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:56:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:56:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770935+0000", 2024-09-18T04:56:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770937+0000", 2024-09-18T04:56:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1489999999999999e-06 2024-09-18T04:56:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770940+0000", 2024-09-18T04:56:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9569999999999998e-06 2024-09-18T04:56:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770945+0000", 2024-09-18T04:56:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:56:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814838+0000", 2024-09-18T04:56:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043892635999999999 2024-09-18T04:56:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814920+0000", 2024-09-18T04:56:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2175000000000005e-05 2024-09-18T04:56:42.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814979+0000", 2024-09-18T04:56:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8372000000000003e-05 2024-09-18T04:56:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9347 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773038+0000", 2024-09-18T04:56:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4033633879999998, 2024-09-18T04:56:42.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043570541999999997, 2024-09-18T04:56:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9347 2024-09-18T04:56:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773038+0000", 2024-09-18T04:56:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773038+0000", 2024-09-18T04:56:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773038+0000", 2024-09-18T04:56:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773040+0000", 2024-09-18T04:56:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.261e-06 2024-09-18T04:56:42.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773046+0000", 2024-09-18T04:56:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9290000000000003e-06 2024-09-18T04:56:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773050+0000", 2024-09-18T04:56:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5340000000000001e-06 2024-09-18T04:56:42.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.816516+0000", 2024-09-18T04:56:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043465472999999998 2024-09-18T04:56:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.816556+0000", 2024-09-18T04:56:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0129000000000002e-05 2024-09-18T04:56:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.816609+0000", 2024-09-18T04:56:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2703e-05 2024-09-18T04:56:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9348 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773057+0000", 2024-09-18T04:56:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4033443630000004, 2024-09-18T04:56:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043651141999999997, 2024-09-18T04:56:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.242 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9348 2024-09-18T04:56:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773057+0000", 2024-09-18T04:56:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773057+0000", 2024-09-18T04:56:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773057+0000", 2024-09-18T04:56:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773059+0000", 2024-09-18T04:56:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.221e-06 2024-09-18T04:56:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773065+0000", 2024-09-18T04:56:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1680000000000001e-06 2024-09-18T04:56:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773070+0000", 2024-09-18T04:56:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6179999999999997e-06 2024-09-18T04:56:42.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.816623+0000", 2024-09-18T04:56:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043553174 2024-09-18T04:56:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.816661+0000", 2024-09-18T04:56:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8389000000000001e-05 2024-09-18T04:56:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.816708+0000", 2024-09-18T04:56:42.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6919999999999998e-05 2024-09-18T04:56:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9349 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773074+0000", 2024-09-18T04:56:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4033276419999998, 2024-09-18T04:56:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043758942000000002, 2024-09-18T04:56:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.247 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9349 2024-09-18T04:56:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773074+0000", 2024-09-18T04:56:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773074+0000", 2024-09-18T04:56:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773073+0000", 2024-09-18T04:56:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773076+0000", 2024-09-18T04:56:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2340000000000001e-06 2024-09-18T04:56:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773081+0000", 2024-09-18T04:56:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8150000000000002e-06 2024-09-18T04:56:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773094+0000", 2024-09-18T04:56:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2191000000000001e-05 2024-09-18T04:56:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.816735+0000", 2024-09-18T04:56:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043641539999999999 2024-09-18T04:56:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.816782+0000", 2024-09-18T04:56:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6866000000000002e-05 2024-09-18T04:56:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.816833+0000", 2024-09-18T04:56:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0844000000000003e-05 2024-09-18T04:56:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.251 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9350 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1060864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773115+0000", 2024-09-18T04:56:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4032865819999998, 2024-09-18T04:56:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043835878000000002, 2024-09-18T04:56:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.252 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9350 2024-09-18T04:56:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773115+0000", 2024-09-18T04:56:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773115+0000", 2024-09-18T04:56:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773114+0000", 2024-09-18T04:56:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773117+0000", 2024-09-18T04:56:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8229999999999999e-06 2024-09-18T04:56:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773124+0000", 2024-09-18T04:56:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3309999999999998e-06 2024-09-18T04:56:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773130+0000", 2024-09-18T04:56:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6570000000000002e-06 2024-09-18T04:56:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.816861+0000", 2024-09-18T04:56:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043730416000000001 2024-09-18T04:56:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.816902+0000", 2024-09-18T04:56:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1016000000000002e-05 2024-09-18T04:56:42.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.816951+0000", 2024-09-18T04:56:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9177e-05 2024-09-18T04:56:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9351 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1110016~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773135+0000", 2024-09-18T04:56:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4032665020000001, 2024-09-18T04:56:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043934572999999998, 2024-09-18T04:56:42.256 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9351 2024-09-18T04:56:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773135+0000", 2024-09-18T04:56:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773135+0000", 2024-09-18T04:56:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773134+0000", 2024-09-18T04:56:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773140+0000", 2024-09-18T04:56:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9780000000000002e-06 2024-09-18T04:56:42.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773147+0000", 2024-09-18T04:56:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5289999999999997e-06 2024-09-18T04:56:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773153+0000", 2024-09-18T04:56:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.648e-06 2024-09-18T04:56:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.816978+0000", 2024-09-18T04:56:42.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043825201000000001 2024-09-18T04:56:42.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817018+0000", 2024-09-18T04:56:42.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0296000000000002e-05 2024-09-18T04:56:42.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817070+0000", 2024-09-18T04:56:42.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1411e-05 2024-09-18T04:56:42.260 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.260 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.260 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.261 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9352 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1159168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.261 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:56:42.261 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4032445510000002, 2024-09-18T04:56:42.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044011269999999998, 2024-09-18T04:56:42.261 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.261 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.261 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.261 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.261 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.261 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9352 2024-09-18T04:56:42.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.261 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:56:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:56:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773156+0000", 2024-09-18T04:56:42.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773159+0000", 2024-09-18T04:56:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2919999999999998e-06 2024-09-18T04:56:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773165+0000", 2024-09-18T04:56:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0580000000000003e-06 2024-09-18T04:56:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773177+0000", 2024-09-18T04:56:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.204e-05 2024-09-18T04:56:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817082+0000", 2024-09-18T04:56:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043905528999999999 2024-09-18T04:56:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817122+0000", 2024-09-18T04:56:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9258999999999998e-05 2024-09-18T04:56:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817168+0000", 2024-09-18T04:56:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6545000000000002e-05 2024-09-18T04:56:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9353 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1208320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:56:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4032202529999998, 2024-09-18T04:56:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044105537, 2024-09-18T04:56:42.265 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9353 2024-09-18T04:56:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:56:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:56:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:56:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773183+0000", 2024-09-18T04:56:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2929999999999999e-06 2024-09-18T04:56:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773190+0000", 2024-09-18T04:56:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4330000000000001e-06 2024-09-18T04:56:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773194+0000", 2024-09-18T04:56:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.865e-06 2024-09-18T04:56:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817196+0000", 2024-09-18T04:56:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044001708 2024-09-18T04:56:42.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817236+0000", 2024-09-18T04:56:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0043999999999998e-05 2024-09-18T04:56:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817287+0000", 2024-09-18T04:56:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0605000000000001e-05 2024-09-18T04:56:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9354 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:56:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4032027420000004, 2024-09-18T04:56:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044199796, 2024-09-18T04:56:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9354 2024-09-18T04:56:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:56:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:56:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773198+0000", 2024-09-18T04:56:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773200+0000", 2024-09-18T04:56:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4159999999999998e-06 2024-09-18T04:56:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773206+0000", 2024-09-18T04:56:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.789e-06 2024-09-18T04:56:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773210+0000", 2024-09-18T04:56:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2679999999999996e-06 2024-09-18T04:56:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817314+0000", 2024-09-18T04:56:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044103187000000002 2024-09-18T04:56:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817353+0000", 2024-09-18T04:56:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8973000000000001e-05 2024-09-18T04:56:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817399+0000", 2024-09-18T04:56:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5877999999999997e-05 2024-09-18T04:56:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.274 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.274 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:42.275 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:56:42.276 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:42.274+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:42.339 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-09-18T04:56:42.377 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:42.374+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:42.412 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:42.413 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:56:42.413 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:42.424 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:56:42.478 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:42.476+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:42.513 INFO:teuthology.orchestra.run.smithi002.stdout:1744 punch from 0x394f659 to 0x3954931, (0x52d8 bytes) 2024-09-18T04:56:42.518 INFO:teuthology.orchestra.run.smithi002.stdout:1746 trunc from 0x683dd4d to 0x54e2408 2024-09-18T04:56:42.533 INFO:teuthology.orchestra.run.smithi002.stdout:1747 read 0x178bd06 thru 0x17935e6 (0x78e1 bytes) 2024-09-18T04:56:42.538 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:42.538 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:42.538 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:56:42.539 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:56:42.539 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:42.541 INFO:teuthology.orchestra.run.smithi002.stdout:1748 trunc from 0x54e2408 to 0xc29a27a 2024-09-18T04:56:42.546 INFO:teuthology.orchestra.run.smithi002.stdout:1750 write 0x66fc59b thru 0x6701b88 (0x55ee bytes) 2024-09-18T04:56:42.548 INFO:teuthology.orchestra.run.smithi002.stdout:1753 read 0xa85d7c1 thru 0xa8683f0 (0xac30 bytes) 2024-09-18T04:56:42.548 INFO:teuthology.orchestra.run.smithi002.stdout:1756 writesame 0x26b883c thru 0x26bb7f7 data_size 0x41(0x2fbc bytes) 2024-09-18T04:56:42.549 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:56:42.560 INFO:teuthology.orchestra.run.smithi002.stdout:1757 write 0xe6c458a thru 0xe6d3363 (0xedda bytes) 2024-09-18T04:56:42.564 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:42.564 INFO:teuthology.orchestra.run.smithi002.stdout:1760 read 0x95b12d9 thru 0x95bf201 (0xdf29 bytes) 2024-09-18T04:56:42.564 INFO:teuthology.orchestra.run.smithi002.stdout:1761 write 0xc02197d thru 0xc030130 (0xe7b4 bytes) 2024-09-18T04:56:42.567 INFO:teuthology.orchestra.run.smithi002.stdout:1764 write 0xbf2ef77 thru 0xbf3627d (0x7307 bytes) 2024-09-18T04:56:42.570 INFO:teuthology.orchestra.run.smithi002.stdout:1765 write 0x962f0b9 thru 0x9631e2a (0x2d72 bytes) 2024-09-18T04:56:42.573 INFO:teuthology.orchestra.run.smithi002.stdout:1766 read 0x92e19bf thru 0x92ef1e4 (0xd826 bytes) 2024-09-18T04:56:42.573 INFO:teuthology.orchestra.run.smithi002.stdout:1767 writesame 0x9d634a thru 0x9d9d1c data_size 0x1(0x39d3 bytes) 2024-09-18T04:56:42.579 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:42.576+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:42.580 INFO:teuthology.orchestra.run.smithi002.stdout:1769 read 0xb7af685 thru 0xb7ba01e (0xa99a bytes) 2024-09-18T04:56:42.580 INFO:teuthology.orchestra.run.smithi002.stdout:1770 read 0xbde929e thru 0xbdf10e4 (0x7e47 bytes) 2024-09-18T04:56:42.580 INFO:teuthology.orchestra.run.smithi002.stdout:1771 write 0x89f17b thru 0x8abb85 (0xca0b bytes) 2024-09-18T04:56:42.582 INFO:teuthology.orchestra.run.smithi002.stdout:1772 read 0xdeae1 thru 0xe1cda (0x31fa bytes) 2024-09-18T04:56:42.582 INFO:teuthology.orchestra.run.smithi002.stdout:1773 write 0x99ae2cd thru 0x99b1a57 (0x378b bytes) 2024-09-18T04:56:42.585 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:42.585 INFO:teuthology.orchestra.run.smithi002.stdout:1775 writesame 0x5964092 thru 0x596fb2c data_size 0x1f(0xba9b bytes) 2024-09-18T04:56:42.603 INFO:teuthology.orchestra.run.smithi002.stdout:1776 trunc from 0xe6d3364 to 0x13b894b 2024-09-18T04:56:42.615 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:56:42.650 INFO:teuthology.orchestra.run.smithi002.stdout:1777 writesame 0xd3222c6 thru 0xd332292 data_size 0x7(0xffcd bytes) 2024-09-18T04:56:42.650 INFO:teuthology.orchestra.run.smithi002.stdout:1778 writesame 0x9affab thru 0x9b3e97 data_size 0x1(0x3eed bytes) 2024-09-18T04:56:42.656 INFO:teuthology.orchestra.run.smithi002.stdout:1779 read 0x9ba7641 thru 0x9badf25 (0x68e5 bytes) 2024-09-18T04:56:42.656 INFO:teuthology.orchestra.run.smithi002.stdout:1782 writesame 0x19eaaa1 thru 0x19f2b93 data_size 0x1(0x80f3 bytes) 2024-09-18T04:56:42.673 INFO:teuthology.orchestra.run.smithi002.stdout:1785 write 0x68c7668 thru 0x68c9b79 (0x2512 bytes) 2024-09-18T04:56:42.676 INFO:teuthology.orchestra.run.smithi002.stdout:1786 write 0xb4a6903 thru 0xb4aff77 (0x9675 bytes) 2024-09-18T04:56:42.679 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:42.679 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:56:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:56:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:56:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14125 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:56:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.5497556450000003, 2024-09-18T04:56:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052956470999999998, 2024-09-18T04:56:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.680 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14125 2024-09-18T04:56:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:56:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:56:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126849+0000", 2024-09-18T04:56:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126852+0000", 2024-09-18T04:56:42.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3659999999999999e-06 2024-09-18T04:56:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126858+0000", 2024-09-18T04:56:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4540000000000004e-06 2024-09-18T04:56:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126863+0000", 2024-09-18T04:56:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1780000000000002e-06 2024-09-18T04:56:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179608+0000", 2024-09-18T04:56:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052745159999999999 2024-09-18T04:56:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179703+0000", 2024-09-18T04:56:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4773999999999998e-05 2024-09-18T04:56:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179806+0000", 2024-09-18T04:56:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010296799999999999 2024-09-18T04:56:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.684 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14136 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1732608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:56:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.5475950940000001, 2024-09-18T04:56:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052973412999999997, 2024-09-18T04:56:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14136 2024-09-18T04:56:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.685 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:56:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:56:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:56:42.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129013+0000", 2024-09-18T04:56:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.96e-06 2024-09-18T04:56:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129020+0000", 2024-09-18T04:56:42.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6809999999999998e-06 2024-09-18T04:56:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129026+0000", 2024-09-18T04:56:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:56:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181873+0000", 2024-09-18T04:56:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052847238999999997 2024-09-18T04:56:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181932+0000", 2024-09-18T04:56:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9339000000000002e-05 2024-09-18T04:56:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181984+0000", 2024-09-18T04:56:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1348999999999999e-05 2024-09-18T04:56:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14137 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1781760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.689 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:56:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.5475746890000002, 2024-09-18T04:56:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053128914999999999, 2024-09-18T04:56:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14137 2024-09-18T04:56:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.690 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:56:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:56:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129030+0000", 2024-09-18T04:56:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129033+0000", 2024-09-18T04:56:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7439999999999999e-06 2024-09-18T04:56:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129039+0000", 2024-09-18T04:56:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.314e-06 2024-09-18T04:56:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129044+0000", 2024-09-18T04:56:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9919999999999998e-06 2024-09-18T04:56:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182015+0000", 2024-09-18T04:56:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052970875000000001 2024-09-18T04:56:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182058+0000", 2024-09-18T04:56:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2997000000000002e-05 2024-09-18T04:56:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182160+0000", 2024-09-18T04:56:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101437 2024-09-18T04:56:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.694 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14138 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1830912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:56:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.5475566550000002, 2024-09-18T04:56:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053256435999999997, 2024-09-18T04:56:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.695 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14138 2024-09-18T04:56:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:56:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:56:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129048+0000", 2024-09-18T04:56:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129056+0000", 2024-09-18T04:56:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.413e-06 2024-09-18T04:56:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129062+0000", 2024-09-18T04:56:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6810000000000001e-06 2024-09-18T04:56:42.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129068+0000", 2024-09-18T04:56:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0459999999999996e-06 2024-09-18T04:56:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182191+0000", 2024-09-18T04:56:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053123210999999997 2024-09-18T04:56:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182238+0000", 2024-09-18T04:56:42.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7404999999999998e-05 2024-09-18T04:56:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182305+0000", 2024-09-18T04:56:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7028999999999998e-05 2024-09-18T04:56:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15910 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2236416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:56:42.699 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4484691229999997, 2024-09-18T04:56:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053241054000000003, 2024-09-18T04:56:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15910 2024-09-18T04:56:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:56:42.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:56:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:56:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228138+0000", 2024-09-18T04:56:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:56:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228144+0000", 2024-09-18T04:56:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5369999999999997e-06 2024-09-18T04:56:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228149+0000", 2024-09-18T04:56:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1209999999999998e-06 2024-09-18T04:56:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280822+0000", 2024-09-18T04:56:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052673194999999999 2024-09-18T04:56:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280903+0000", 2024-09-18T04:56:42.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1179000000000003e-05 2024-09-18T04:56:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281377+0000", 2024-09-18T04:56:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047417899999999997 2024-09-18T04:56:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15911 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2285568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:56:42.704 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.448452391, 2024-09-18T04:56:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053415151000000001, 2024-09-18T04:56:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15911 2024-09-18T04:56:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:56:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:56:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:56:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228155+0000", 2024-09-18T04:56:42.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:56:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228161+0000", 2024-09-18T04:56:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:56:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228166+0000", 2024-09-18T04:56:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2320000000000001e-06 2024-09-18T04:56:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281405+0000", 2024-09-18T04:56:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053239336999999998 2024-09-18T04:56:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281482+0000", 2024-09-18T04:56:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6569999999999994e-05 2024-09-18T04:56:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281568+0000", 2024-09-18T04:56:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6328000000000004e-05 2024-09-18T04:56:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15918 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2662400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:56:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4466834100000003, 2024-09-18T04:56:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053091065, 2024-09-18T04:56:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.709 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15918 2024-09-18T04:56:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:56:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:56:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:56:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229924+0000", 2024-09-18T04:56:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:56:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229930+0000", 2024-09-18T04:56:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.066e-06 2024-09-18T04:56:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229935+0000", 2024-09-18T04:56:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9119999999999997e-06 2024-09-18T04:56:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282840+0000", 2024-09-18T04:56:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052905119 2024-09-18T04:56:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282922+0000", 2024-09-18T04:56:42.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2081000000000002e-05 2024-09-18T04:56:42.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283013+0000", 2024-09-18T04:56:42.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0828000000000005e-05 2024-09-18T04:56:42.714 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.714 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.714 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.714 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15919 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2711552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.714 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:56:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4466661319999998, 2024-09-18T04:56:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053293021000000003, 2024-09-18T04:56:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15919 2024-09-18T04:56:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:56:42.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:56:42.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:56:42.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229941+0000", 2024-09-18T04:56:42.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2639999999999998e-06 2024-09-18T04:56:42.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229947+0000", 2024-09-18T04:56:42.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9750000000000004e-06 2024-09-18T04:56:42.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229952+0000", 2024-09-18T04:56:42.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7929999999999997e-06 2024-09-18T04:56:42.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283062+0000", 2024-09-18T04:56:42.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05310952 2024-09-18T04:56:42.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283162+0000", 2024-09-18T04:56:42.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100098 2024-09-18T04:56:42.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283232+0000", 2024-09-18T04:56:42.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0693999999999998e-05 2024-09-18T04:56:42.719 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.719 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.719 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.719 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15920 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2760704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.719 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:56:42.719 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4466493710000004, 2024-09-18T04:56:42.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053502367000000002, 2024-09-18T04:56:42.719 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.719 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.719 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.719 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.720 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.720 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15920 2024-09-18T04:56:42.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.720 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:56:42.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:56:42.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:56:42.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229958+0000", 2024-09-18T04:56:42.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1770000000000001e-06 2024-09-18T04:56:42.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229964+0000", 2024-09-18T04:56:42.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8470000000000001e-06 2024-09-18T04:56:42.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229968+0000", 2024-09-18T04:56:42.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5680000000000001e-06 2024-09-18T04:56:42.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283269+0000", 2024-09-18T04:56:42.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053300620999999999 2024-09-18T04:56:42.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283359+0000", 2024-09-18T04:56:42.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9971999999999998e-05 2024-09-18T04:56:42.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283458+0000", 2024-09-18T04:56:42.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9483999999999998e-05 2024-09-18T04:56:42.723 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.723 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.723 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.724 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15921 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2809856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.724 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:56:42.724 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4466328209999997, 2024-09-18T04:56:42.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053800977, 2024-09-18T04:56:42.724 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.724 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.724 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.724 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.724 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.724 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15921 2024-09-18T04:56:42.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.724 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:56:42.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:56:42.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229972+0000", 2024-09-18T04:56:42.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229974+0000", 2024-09-18T04:56:42.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2910000000000002e-06 2024-09-18T04:56:42.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229982+0000", 2024-09-18T04:56:42.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7249999999999997e-06 2024-09-18T04:56:42.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229989+0000", 2024-09-18T04:56:42.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6429999999999998e-06 2024-09-18T04:56:42.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283601+0000", 2024-09-18T04:56:42.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053612211 2024-09-18T04:56:42.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283681+0000", 2024-09-18T04:56:42.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0125999999999998e-05 2024-09-18T04:56:42.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283774+0000", 2024-09-18T04:56:42.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2362999999999996e-05 2024-09-18T04:56:42.728 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.728 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.728 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.728 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15922 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2859008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.728 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:56:42.728 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4465878820000002, 2024-09-18T04:56:42.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053882270000000003, 2024-09-18T04:56:42.729 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.729 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.729 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15922 2024-09-18T04:56:42.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.729 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:56:42.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:56:42.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230017+0000", 2024-09-18T04:56:42.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230019+0000", 2024-09-18T04:56:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4210000000000002e-06 2024-09-18T04:56:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230027+0000", 2024-09-18T04:56:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000002e-06 2024-09-18T04:56:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230032+0000", 2024-09-18T04:56:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0089999999999998e-06 2024-09-18T04:56:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283800+0000", 2024-09-18T04:56:42.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053768418999999998 2024-09-18T04:56:42.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283849+0000", 2024-09-18T04:56:42.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9339000000000003e-05 2024-09-18T04:56:42.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283900+0000", 2024-09-18T04:56:42.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0421999999999999e-05 2024-09-18T04:56:42.732 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.733 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.733 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.733 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15923 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2924544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.733 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:56:42.733 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4465690130000004, 2024-09-18T04:56:42.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054073016000000002, 2024-09-18T04:56:42.733 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.733 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:42.733 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.733 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.733 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.733 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15923 2024-09-18T04:56:42.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.734 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:56:42.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:56:42.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:56:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230038+0000", 2024-09-18T04:56:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.463e-06 2024-09-18T04:56:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230045+0000", 2024-09-18T04:56:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3570000000000003e-06 2024-09-18T04:56:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230050+0000", 2024-09-18T04:56:42.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7299999999999996e-06 2024-09-18T04:56:42.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283928+0000", 2024-09-18T04:56:42.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053878218999999998 2024-09-18T04:56:42.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:42.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284010+0000", 2024-09-18T04:56:42.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2577000000000006e-05 2024-09-18T04:56:42.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284109+0000", 2024-09-18T04:56:42.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9102000000000001e-05 2024-09-18T04:56:42.737 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.737 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.737 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.737 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15924 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.737 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:56:42.737 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4465505429999999, 2024-09-18T04:56:42.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054108470999999998, 2024-09-18T04:56:42.738 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.738 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:42.738 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.738 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.738 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.738 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15924 2024-09-18T04:56:42.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.738 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:56:42.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:56:42.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:56:42.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230061+0000", 2024-09-18T04:56:42.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8240000000000002e-06 2024-09-18T04:56:42.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230068+0000", 2024-09-18T04:56:42.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4320000000000004e-06 2024-09-18T04:56:42.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230073+0000", 2024-09-18T04:56:42.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.921e-06 2024-09-18T04:56:42.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284153+0000", 2024-09-18T04:56:42.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054080793000000002 2024-09-18T04:56:42.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284163+0000", 2024-09-18T04:56:42.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.859e-06 2024-09-18T04:56:42.741 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.741 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.741 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.742 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15925 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.742 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:56:42.743 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4465264280000003, 2024-09-18T04:56:42.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054134632000000002, 2024-09-18T04:56:42.743 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.743 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:42.743 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.743 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.743 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.743 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15925 2024-09-18T04:56:42.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.744 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:56:42.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:56:42.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:56:42.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230081+0000", 2024-09-18T04:56:42.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4109999999999999e-06 2024-09-18T04:56:42.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230087+0000", 2024-09-18T04:56:42.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.161e-06 2024-09-18T04:56:42.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230092+0000", 2024-09-18T04:56:42.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9259999999999999e-06 2024-09-18T04:56:42.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284204+0000", 2024-09-18T04:56:42.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054111546000000003 2024-09-18T04:56:42.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284214+0000", 2024-09-18T04:56:42.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9639999999999998e-06 2024-09-18T04:56:42.748 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.748 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.748 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.748 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15926 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.748 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:56:42.748 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4465091040000004, 2024-09-18T04:56:42.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054143193999999999, 2024-09-18T04:56:42.748 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.748 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:42.748 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.748 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.749 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.749 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15926 2024-09-18T04:56:42.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.749 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:56:42.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:56:42.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:56:42.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230098+0000", 2024-09-18T04:56:42.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1399999999999998e-06 2024-09-18T04:56:42.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230104+0000", 2024-09-18T04:56:42.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-18T04:56:42.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230109+0000", 2024-09-18T04:56:42.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9250000000000003e-06 2024-09-18T04:56:42.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284231+0000", 2024-09-18T04:56:42.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054122297999999999 2024-09-18T04:56:42.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284239+0000", 2024-09-18T04:56:42.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1689999999999992e-06 2024-09-18T04:56:42.752 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.752 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.753 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.753 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15927 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.753 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:56:42.753 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4464920570000004, 2024-09-18T04:56:42.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054168435000000001, 2024-09-18T04:56:42.753 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.754 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:42.754 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.754 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.754 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.754 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15927 2024-09-18T04:56:42.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.755 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:56:42.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:56:42.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:56:42.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230116+0000", 2024-09-18T04:56:42.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5150000000000001e-06 2024-09-18T04:56:42.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230122+0000", 2024-09-18T04:56:42.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1190000000000002e-06 2024-09-18T04:56:42.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230126+0000", 2024-09-18T04:56:42.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.792e-06 2024-09-18T04:56:42.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284273+0000", 2024-09-18T04:56:42.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054146086000000003 2024-09-18T04:56:42.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284282+0000", 2024-09-18T04:56:42.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2590000000000006e-06 2024-09-18T04:56:42.760 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.761 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.761 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.761 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15928 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.761 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:56:42.761 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4464746740000001, 2024-09-18T04:56:42.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054189739000000001, 2024-09-18T04:56:42.762 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.762 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:42.762 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.762 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.762 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.762 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15928 2024-09-18T04:56:42.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.763 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:56:42.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:56:42.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230130+0000", 2024-09-18T04:56:42.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230133+0000", 2024-09-18T04:56:42.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.244e-06 2024-09-18T04:56:42.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230139+0000", 2024-09-18T04:56:42.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9830000000000001e-06 2024-09-18T04:56:42.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230143+0000", 2024-09-18T04:56:42.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7530000000000001e-06 2024-09-18T04:56:42.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284313+0000", 2024-09-18T04:56:42.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054169332000000001 2024-09-18T04:56:42.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284320+0000", 2024-09-18T04:56:42.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8699999999999992e-06 2024-09-18T04:56:42.767 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.767 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.767 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.767 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15929 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.767 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:56:42.767 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4464582339999996, 2024-09-18T04:56:42.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054218771999999998, 2024-09-18T04:56:42.768 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.768 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:42.768 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.768 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.768 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.768 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15929 2024-09-18T04:56:42.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.768 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:56:42.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:56:42.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:56:42.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230149+0000", 2024-09-18T04:56:42.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-18T04:56:42.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230155+0000", 2024-09-18T04:56:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2469999999999997e-06 2024-09-18T04:56:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230160+0000", 2024-09-18T04:56:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7129999999999996e-06 2024-09-18T04:56:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284355+0000", 2024-09-18T04:56:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054194900999999997 2024-09-18T04:56:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284366+0000", 2024-09-18T04:56:42.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0993e-05 2024-09-18T04:56:42.772 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.772 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.772 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.772 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15930 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.773 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:56:42.773 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4464412339999999, 2024-09-18T04:56:42.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054244713999999999, 2024-09-18T04:56:42.773 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.773 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:42.773 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.773 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.773 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.773 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15930 2024-09-18T04:56:42.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.773 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:56:42.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:56:42.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:56:42.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230166+0000", 2024-09-18T04:56:42.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1009999999999999e-06 2024-09-18T04:56:42.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230172+0000", 2024-09-18T04:56:42.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.981e-06 2024-09-18T04:56:42.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230184+0000", 2024-09-18T04:56:42.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2449000000000001e-05 2024-09-18T04:56:42.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284400+0000", 2024-09-18T04:56:42.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054215452999999997 2024-09-18T04:56:42.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284409+0000", 2024-09-18T04:56:42.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0839999999999998e-06 2024-09-18T04:56:42.777 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.777 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.777 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.777 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15931 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:42.777 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:56:42.777 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 6.4464154450000004, 2024-09-18T04:56:42.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054261928000000001, 2024-09-18T04:56:42.777 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:42.777 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:42.778 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:42.778 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:42.778 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:42.778 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15931 2024-09-18T04:56:42.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.778 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:42.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:42.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:56:42.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:42.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:56:42.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:42.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:42.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230189+0000", 2024-09-18T04:56:42.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:42.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:42.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230192+0000", 2024-09-18T04:56:42.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6599999999999999e-06 2024-09-18T04:56:42.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:42.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230199+0000", 2024-09-18T04:56:42.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5570000000000001e-06 2024-09-18T04:56:42.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:42.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230204+0000", 2024-09-18T04:56:42.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2270000000000001e-06 2024-09-18T04:56:42.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:42.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284442+0000", 2024-09-18T04:56:42.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054238383000000001 2024-09-18T04:56:42.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:42.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:42.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:42.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284452+0000", 2024-09-18T04:56:42.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6069999999999997e-06 2024-09-18T04:56:42.783 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.783 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.783 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.783 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:42.783 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:42.783 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:42.784 INFO:teuthology.orchestra.run.smithi002.stdout:1787 read 0x1eea73c thru 0x1eeb1ed (0xab2 bytes) 2024-09-18T04:56:42.784 INFO:teuthology.orchestra.run.smithi002.stdout:1789 punch from 0x642b78e to 0x642c4b7, (0xd29 bytes) 2024-09-18T04:56:42.784 INFO:teuthology.orchestra.run.smithi002.stdout:1790 write 0x85096b3 thru 0x851688f (0xd1dd bytes) 2024-09-18T04:56:42.785 INFO:teuthology.orchestra.run.smithi002.stdout:1791 write 0x4cf5386 thru 0x4cfe5ee (0x9269 bytes) 2024-09-18T04:56:42.785 INFO:teuthology.orchestra.run.smithi002.stdout:1795 read 0x5c5d9d2 thru 0x5c69371 (0xb9a0 bytes) 2024-09-18T04:56:42.785 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:42.785 INFO:teuthology.orchestra.run.smithi002.stdout:1797 write 0x567c5b0 thru 0x568a744 (0xe195 bytes) 2024-09-18T04:56:42.785 INFO:teuthology.orchestra.run.smithi002.stdout:1798 read 0x7fb4b8b thru 0x7fb60e7 (0x155d bytes) 2024-09-18T04:56:42.785 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:42.786 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:56:42.786 INFO:teuthology.orchestra.run.smithi002.stdout:1801 write 0x4e37792 thru 0x4e46010 (0xe87f bytes) 2024-09-18T04:56:42.787 INFO:teuthology.orchestra.run.smithi002.stdout:1802 clone 15 order 22 su 16384 sc 12 2024-09-18T04:56:42.787 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:56:42.787 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:56:42.788 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:42.677+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:42.788 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:56:42.788 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:56:42.789 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:56:42.789 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:56:42.793 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:56:42.794 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:56:42.794 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:42.792+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:42.810 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:56:42.811 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:56:42.827 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:56:42.827 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:56:42.843 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:56:42.843 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:56:42.860 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:56:42.861 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:56:42.890 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:42.887+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:42.901 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:42.901 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:42.910 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:56:42.995 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:42.992+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:43.057 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:43.057 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:43.066 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:56:43.108 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:43.106+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:43.192 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:43.189+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:43.216 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:43.216 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:43.218 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T04:56:43.226 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:56:43.267 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone14 from 0xd332293 (overlap 0x13b894b) to 0x0 2024-09-18T04:56:43.292 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:43.289+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:43.313 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T04:56:43.313 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T04:56:43.314 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-18T04:56:43.314 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T04:56:43.317 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-09-18T04:56:43.391 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:43.392 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:43.393 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:43.391+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:43.402 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:56:43.446 INFO:teuthology.orchestra.run.smithi002.stdout:1803 read 0x22c9ee1 thru 0x22d2aba (0x8bda bytes) 2024-09-18T04:56:43.447 INFO:teuthology.orchestra.run.smithi002.stdout:1804 read 0xa76a329 thru 0xa76c008 (0x1ce0 bytes) 2024-09-18T04:56:43.448 INFO:teuthology.orchestra.run.smithi002.stdout:1805 write 0x1464bf7 thru 0x1469e2c (0x5236 bytes) 2024-09-18T04:56:43.494 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:43.491+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:43.495 INFO:teuthology.orchestra.run.smithi002.stdout:1806 read 0x3d91bab thru 0x3d9d734 (0xbb8a bytes) 2024-09-18T04:56:43.496 INFO:teuthology.orchestra.run.smithi002.stdout:1807 writesame 0x9b3efbc thru 0x9b40517 data_size 0x2(0x155c bytes) 2024-09-18T04:56:43.522 INFO:teuthology.orchestra.run.smithi002.stdout:1808 punch from 0x281b959 to 0x2823cef, (0x8396 bytes) 2024-09-18T04:56:43.522 INFO:teuthology.orchestra.run.smithi002.stdout:1811 write 0xaaf71dc thru 0xaafc31c (0x5141 bytes) 2024-09-18T04:56:43.542 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:43.557 INFO:teuthology.orchestra.run.smithi002.stdout:1813 trunc from 0xd332293 to 0x1e73206 2024-09-18T04:56:43.592 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:43.592 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:43.594 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:43.592+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:43.603 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:56:43.695 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:56:43.695 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:43.695 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 39, 2024-09-18T04:56:43.695 INFO:teuthology.orchestra.run.smithi002.stdout: "fsid": "c0761d0c-69d5-4b64-b8b9-c7d098b2d06d", 2024-09-18T04:56:43.695 INFO:teuthology.orchestra.run.smithi002.stdout: "created": "2024-09-18T04:55:58.216538+0000", 2024-09-18T04:56:43.696 INFO:teuthology.orchestra.run.smithi002.stdout: "modified": "2024-09-18T04:56:43.200427+0000", 2024-09-18T04:56:43.696 INFO:teuthology.orchestra.run.smithi002.stdout: "last_up_change": "2024-09-18T04:56:08.369924+0000", 2024-09-18T04:56:43.696 INFO:teuthology.orchestra.run.smithi002.stdout: "last_in_change": "2024-09-18T04:56:00.333800+0000", 2024-09-18T04:56:43.696 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T04:56:43.696 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_num": 5799936, 2024-09-18T04:56:43.696 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_set": [ 2024-09-18T04:56:43.696 INFO:teuthology.orchestra.run.smithi002.stdout: "pglog_hardlimit", 2024-09-18T04:56:43.696 INFO:teuthology.orchestra.run.smithi002.stdout: "purged_snapdirs", 2024-09-18T04:56:43.697 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_deletes", 2024-09-18T04:56:43.697 INFO:teuthology.orchestra.run.smithi002.stdout: "sortbitwise" 2024-09-18T04:56:43.697 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:56:43.697 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_version": 4, 2024-09-18T04:56:43.697 INFO:teuthology.orchestra.run.smithi002.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T04:56:43.697 INFO:teuthology.orchestra.run.smithi002.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T04:56:43.697 INFO:teuthology.orchestra.run.smithi002.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T04:56:43.697 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_snapshot": "", 2024-09-18T04:56:43.697 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_max": 3, 2024-09-18T04:56:43.698 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd": 8, 2024-09-18T04:56:43.698 INFO:teuthology.orchestra.run.smithi002.stdout: "require_min_compat_client": "luminous", 2024-09-18T04:56:43.698 INFO:teuthology.orchestra.run.smithi002.stdout: "min_compat_client": "luminous", 2024-09-18T04:56:43.698 INFO:teuthology.orchestra.run.smithi002.stdout: "require_osd_release": "reef", 2024-09-18T04:56:43.698 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_crimson": false, 2024-09-18T04:56:43.698 INFO:teuthology.orchestra.run.smithi002.stdout: "pools": [ 2024-09-18T04:56:43.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 1, 2024-09-18T04:56:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "rbd", 2024-09-18T04:56:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-18T04:56:08.983344+0000", 2024-09-18T04:56:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-18T04:56:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T04:56:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-18T04:56:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-18T04:56:43.699 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-18T04:56:43.700 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-18T04:56:43.700 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-18T04:56:43.700 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-18T04:56:43.700 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T04:56:43.700 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T04:56:43.701 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-18T04:56:43.701 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-18T04:56:43.701 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 8, 2024-09-18T04:56:43.701 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 8, 2024-09-18T04:56:43.701 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 8, 2024-09-18T04:56:43.701 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 8, 2024-09-18T04:56:43.701 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 8, 2024-09-18T04:56:43.701 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-18T04:56:43.701 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-18T04:56:43.702 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-18T04:56:43.702 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-18T04:56:43.702 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-18T04:56:43.702 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-18T04:56:43.702 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-18T04:56:43.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.703 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "17", 2024-09-18T04:56:43.703 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-18T04:56:43.703 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T04:56:43.703 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T04:56:43.703 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-18T04:56:43.704 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-18T04:56:43.704 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 3, 2024-09-18T04:56:43.704 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 17, 2024-09-18T04:56:43.704 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-18T04:56:43.704 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-18T04:56:43.704 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-18T04:56:43.704 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-18T04:56:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-18T04:56:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-18T04:56:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-18T04:56:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-18T04:56:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-18T04:56:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-18T04:56:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-18T04:56:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T04:56:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T04:56:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T04:56:43.705 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-18T04:56:43.706 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-18T04:56:43.706 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-18T04:56:43.706 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-18T04:56:43.706 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-18T04:56:43.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.706 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-18T04:56:43.707 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-18T04:56:43.707 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-18T04:56:43.707 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-18T04:56:43.707 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-18T04:56:43.707 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T04:56:43.707 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-18T04:56:43.707 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-18T04:56:43.707 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-18T04:56:43.707 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-18T04:56:43.707 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-18T04:56:43.707 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-18T04:56:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-18T04:56:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-18T04:56:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-18T04:56:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 2, 2024-09-18T04:56:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 2, 2024-09-18T04:56:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 1, 2024-09-18T04:56:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 2, 2024-09-18T04:56:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 2, 2024-09-18T04:56:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-18T04:56:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-18T04:56:43.708 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-18T04:56:43.709 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.709 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 2, 2024-09-18T04:56:43.709 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": ".mgr", 2024-09-18T04:56:43.709 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-18T04:56:10.313988+0000", 2024-09-18T04:56:43.709 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-18T04:56:43.709 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-18T04:56:43.709 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-18T04:56:43.709 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-18T04:56:43.709 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-18T04:56:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-18T04:56:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-18T04:56:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-18T04:56:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T04:56:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T04:56:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-18T04:56:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-18T04:56:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 1, 2024-09-18T04:56:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 1, 2024-09-18T04:56:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 1, 2024-09-18T04:56:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 1, 2024-09-18T04:56:43.710 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 1, 2024-09-18T04:56:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-18T04:56:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-18T04:56:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-18T04:56:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-18T04:56:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-18T04:56:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-18T04:56:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-18T04:56:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "17", 2024-09-18T04:56:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-18T04:56:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T04:56:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T04:56:43.711 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-18T04:56:43.712 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-18T04:56:43.712 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-18T04:56:43.712 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-18T04:56:43.712 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-18T04:56:43.712 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-18T04:56:43.712 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-18T04:56:43.712 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-18T04:56:43.712 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-18T04:56:43.712 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-18T04:56:43.712 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-18T04:56:43.712 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-18T04:56:43.712 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-18T04:56:43.713 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-18T04:56:43.713 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-18T04:56:43.713 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T04:56:43.713 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T04:56:43.713 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T04:56:43.713 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-18T04:56:43.713 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-18T04:56:43.713 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-18T04:56:43.713 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-18T04:56:43.713 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-18T04:56:43.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.714 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-18T04:56:43.714 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-18T04:56:43.714 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-18T04:56:43.714 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-18T04:56:43.714 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-18T04:56:43.714 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T04:56:43.714 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-18T04:56:43.714 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-18T04:56:43.714 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-18T04:56:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-18T04:56:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-18T04:56:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-18T04:56:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_max": 32, 2024-09-18T04:56:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 1 2024-09-18T04:56:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-18T04:56:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": {} 2024-09-18T04:56:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-18T04:56:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 8, 2024-09-18T04:56:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 8, 2024-09-18T04:56:43.715 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.25, 2024-09-18T04:56:43.716 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 2, 2024-09-18T04:56:43.716 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 2, 2024-09-18T04:56:43.716 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-18T04:56:43.716 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-18T04:56:43.716 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-18T04:56:43.716 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.716 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-18T04:56:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "pool_client.0", 2024-09-18T04:56:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-18T04:56:19.562579+0000", 2024-09-18T04:56:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-18T04:56:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T04:56:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-18T04:56:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-18T04:56:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-18T04:56:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-18T04:56:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-18T04:56:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-18T04:56:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T04:56:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T04:56:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-18T04:56:43.717 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-18T04:56:43.718 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 32, 2024-09-18T04:56:43.718 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 32, 2024-09-18T04:56:43.718 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 32, 2024-09-18T04:56:43.718 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 32, 2024-09-18T04:56:43.718 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 32, 2024-09-18T04:56:43.718 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-18T04:56:43.718 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-18T04:56:43.718 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-18T04:56:43.718 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-18T04:56:43.718 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-18T04:56:43.718 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-18T04:56:43.719 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-18T04:56:43.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.719 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "39", 2024-09-18T04:56:43.719 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-18T04:56:43.719 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T04:56:43.719 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T04:56:43.719 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-18T04:56:43.719 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-18T04:56:43.719 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 18, 2024-09-18T04:56:43.719 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 39, 2024-09-18T04:56:43.719 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-18T04:56:43.720 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-18T04:56:43.720 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-18T04:56:43.720 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-18T04:56:43.720 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-18T04:56:43.720 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-18T04:56:43.720 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-18T04:56:43.720 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-18T04:56:43.720 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-18T04:56:43.720 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-18T04:56:43.720 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-18T04:56:43.720 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T04:56:43.720 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T04:56:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T04:56:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-18T04:56:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-18T04:56:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-18T04:56:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-18T04:56:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-18T04:56:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-18T04:56:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-18T04:56:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-18T04:56:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-18T04:56:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-18T04:56:43.721 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T04:56:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-18T04:56:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-18T04:56:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-18T04:56:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-18T04:56:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-18T04:56:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-18T04:56:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-18T04:56:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-18T04:56:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-18T04:56:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 1.75, 2024-09-18T04:56:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 1.75, 2024-09-18T04:56:43.722 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 1, 2024-09-18T04:56:43.723 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 1.75, 2024-09-18T04:56:43.723 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 1.75, 2024-09-18T04:56:43.723 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-18T04:56:43.723 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-18T04:56:43.723 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-18T04:56:43.723 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.723 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.723 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:56:43.723 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-18T04:56:43.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.723 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-18T04:56:43.724 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "c43f8024-3a3e-424e-aa54-398310ceb70d", 2024-09-18T04:56:43.724 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:43.724 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:43.724 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:43.724 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:43.724 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:43.724 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:43.724 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:43.724 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 19, 2024-09-18T04:56:43.724 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:43.724 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:43.724 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:43.725 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.725 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.725 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6824", 2024-09-18T04:56:43.725 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:43.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.725 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.725 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6825", 2024-09-18T04:56:43.725 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:43.725 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.725 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6826", 2024-09-18T04:56:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6827", 2024-09-18T04:56:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.726 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6830", 2024-09-18T04:56:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6831", 2024-09-18T04:56:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.727 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6828", 2024-09-18T04:56:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6829", 2024-09-18T04:56:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:43.728 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.729 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.729 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6825/4106704527", 2024-09-18T04:56:43.729 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6827/4106704527", 2024-09-18T04:56:43.729 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6831/4106704527", 2024-09-18T04:56:43.729 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6829/4106704527", 2024-09-18T04:56:43.729 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:43.729 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:43.729 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:43.729 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.729 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-18T04:56:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "c664b970-145c-4b6f-8852-2f6dcf1fa30d", 2024-09-18T04:56:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 19, 2024-09-18T04:56:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.731 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.731 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6808", 2024-09-18T04:56:43.731 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:43.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.731 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.731 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6809", 2024-09-18T04:56:43.731 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:43.731 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.731 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.731 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:43.732 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.732 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.732 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6810", 2024-09-18T04:56:43.732 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:43.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.732 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.732 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6811", 2024-09-18T04:56:43.732 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:43.732 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6814", 2024-09-18T04:56:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6815", 2024-09-18T04:56:43.733 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:43.734 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.734 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.734 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:43.734 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.734 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.734 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6812", 2024-09-18T04:56:43.734 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:43.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6813", 2024-09-18T04:56:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6809/1781959259", 2024-09-18T04:56:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6811/1781959259", 2024-09-18T04:56:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6815/1781959259", 2024-09-18T04:56:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6813/1781959259", 2024-09-18T04:56:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:43.735 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-18T04:56:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "0609ea8c-d975-490e-8844-04a3e4512efd", 2024-09-18T04:56:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 0, 2024-09-18T04:56:43.736 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:43.737 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:43.737 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:43.737 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.737 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.737 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6800", 2024-09-18T04:56:43.737 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:43.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.738 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.738 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6801", 2024-09-18T04:56:43.738 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:43.738 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.739 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.739 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:43.739 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.739 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.739 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6802", 2024-09-18T04:56:43.739 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:43.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.740 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.740 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6803", 2024-09-18T04:56:43.740 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:43.740 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.740 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.740 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:43.740 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.740 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.740 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6806", 2024-09-18T04:56:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6807", 2024-09-18T04:56:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.741 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6804", 2024-09-18T04:56:43.742 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:43.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.742 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.742 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6805", 2024-09-18T04:56:43.742 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:43.742 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.742 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.742 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6801/1153570802", 2024-09-18T04:56:43.742 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6803/1153570802", 2024-09-18T04:56:43.742 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6807/1153570802", 2024-09-18T04:56:43.742 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6805/1153570802", 2024-09-18T04:56:43.742 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:43.743 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:43.743 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:43.743 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.743 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-18T04:56:43.743 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "c6bd5ac8-ab89-4fa2-b843-3b166d1b238b", 2024-09-18T04:56:43.743 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:43.743 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:43.743 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:43.743 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:43.743 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:43.743 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:43.743 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:43.744 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 30, 2024-09-18T04:56:43.744 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:43.744 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:43.744 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:43.744 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.744 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.744 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6816", 2024-09-18T04:56:43.744 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:43.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.744 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.744 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6817", 2024-09-18T04:56:43.744 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:43.745 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.745 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.745 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:43.745 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.745 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.745 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6818", 2024-09-18T04:56:43.745 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:43.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.745 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.745 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6819", 2024-09-18T04:56:43.746 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:43.746 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.746 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.746 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:43.746 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.746 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.746 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6822", 2024-09-18T04:56:43.746 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:43.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.746 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.746 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6823", 2024-09-18T04:56:43.747 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:43.747 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.747 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.747 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:43.747 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.747 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.747 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6820", 2024-09-18T04:56:43.747 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:43.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.747 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.747 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6821", 2024-09-18T04:56:43.748 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:43.748 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.748 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.748 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6817/2008321375", 2024-09-18T04:56:43.748 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6819/2008321375", 2024-09-18T04:56:43.748 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6823/2008321375", 2024-09-18T04:56:43.748 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6821/2008321375", 2024-09-18T04:56:43.748 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:43.748 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:43.748 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:43.748 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.749 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-18T04:56:43.749 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "89d6a371-4f4e-465e-a023-d5083cce6e6c", 2024-09-18T04:56:43.749 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:43.749 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:43.749 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:43.749 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:43.749 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:43.749 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:43.749 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:43.749 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 19, 2024-09-18T04:56:43.749 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:43.749 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:43.749 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:43.750 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.750 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.750 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6824", 2024-09-18T04:56:43.750 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:43.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.750 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.750 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6825", 2024-09-18T04:56:43.750 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:43.750 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.750 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.751 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:43.751 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.751 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.751 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6826", 2024-09-18T04:56:43.751 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:43.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.751 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.751 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6827", 2024-09-18T04:56:43.751 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:43.751 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.751 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.752 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:43.752 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.752 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.752 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6830", 2024-09-18T04:56:43.752 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:43.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.752 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.752 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6831", 2024-09-18T04:56:43.752 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:43.752 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.752 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.753 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:43.753 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.753 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.753 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6828", 2024-09-18T04:56:43.753 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:43.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.753 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.753 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6829", 2024-09-18T04:56:43.753 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:43.754 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.754 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.754 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6825/2939896598", 2024-09-18T04:56:43.754 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6827/2939896598", 2024-09-18T04:56:43.754 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6831/2939896598", 2024-09-18T04:56:43.754 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6829/2939896598", 2024-09-18T04:56:43.754 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:43.754 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:43.754 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:43.754 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.755 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-18T04:56:43.755 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "da99686b-d979-4753-b4aa-a3b26e9a0497", 2024-09-18T04:56:43.755 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:43.755 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:43.755 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:43.755 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:43.755 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:43.755 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:43.755 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:43.755 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 19, 2024-09-18T04:56:43.755 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:43.755 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:43.755 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:43.756 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.756 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.756 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6816", 2024-09-18T04:56:43.756 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:43.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.756 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.756 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6817", 2024-09-18T04:56:43.756 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:43.756 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.756 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.757 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:43.757 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.757 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.757 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6818", 2024-09-18T04:56:43.757 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:43.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.757 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.757 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6819", 2024-09-18T04:56:43.757 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:43.757 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.758 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.758 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:43.758 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.758 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.758 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6822", 2024-09-18T04:56:43.758 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:43.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.758 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.758 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6823", 2024-09-18T04:56:43.758 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:43.759 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.759 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.759 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:43.759 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.759 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.759 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6820", 2024-09-18T04:56:43.759 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:43.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.759 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.759 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6821", 2024-09-18T04:56:43.759 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:43.760 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.760 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.760 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6817/3440879618", 2024-09-18T04:56:43.760 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6819/3440879618", 2024-09-18T04:56:43.760 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6823/3440879618", 2024-09-18T04:56:43.760 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6821/3440879618", 2024-09-18T04:56:43.760 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:43.760 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:43.760 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:43.760 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.760 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-18T04:56:43.761 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "ceec6aa6-603d-4706-bfea-3c07dbdaed3a", 2024-09-18T04:56:43.761 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:43.761 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:43.761 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:43.761 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:43.761 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:43.761 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:43.761 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:43.761 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 19, 2024-09-18T04:56:43.761 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:43.761 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:43.761 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:43.761 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.762 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.762 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6800", 2024-09-18T04:56:43.762 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:43.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.762 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.762 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6801", 2024-09-18T04:56:43.762 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:43.762 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.762 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.762 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:43.762 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.763 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.763 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6802", 2024-09-18T04:56:43.763 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:43.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.763 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.763 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6803", 2024-09-18T04:56:43.763 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:43.763 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.763 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.763 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:43.763 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.764 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.764 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6806", 2024-09-18T04:56:43.764 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:43.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.764 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.764 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6807", 2024-09-18T04:56:43.764 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:43.764 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.764 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.764 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:43.765 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.765 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.765 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6804", 2024-09-18T04:56:43.765 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:43.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.765 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.765 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6805", 2024-09-18T04:56:43.765 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:43.765 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.766 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.766 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6801/2493830504", 2024-09-18T04:56:43.766 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6803/2493830504", 2024-09-18T04:56:43.766 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6807/2493830504", 2024-09-18T04:56:43.766 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6805/2493830504", 2024-09-18T04:56:43.766 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:43.766 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:43.766 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:43.766 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.766 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-18T04:56:43.767 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "6779dfad-f60b-41f4-b666-a99db6dc1814", 2024-09-18T04:56:43.767 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:43.767 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:43.767 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:43.767 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:43.767 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:43.767 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:43.767 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:43.767 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 19, 2024-09-18T04:56:43.767 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:43.767 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:43.767 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:43.767 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.767 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.768 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6808", 2024-09-18T04:56:43.768 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:43.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.768 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.768 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6809", 2024-09-18T04:56:43.768 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:43.768 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.768 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.768 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:43.768 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.769 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.769 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6810", 2024-09-18T04:56:43.769 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:43.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.769 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.769 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6811", 2024-09-18T04:56:43.769 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:43.769 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.769 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.770 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:43.770 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.770 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.770 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6814", 2024-09-18T04:56:43.770 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:43.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.770 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.770 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6815", 2024-09-18T04:56:43.770 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:43.770 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.770 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.771 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:43.771 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:43.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.771 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:43.771 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6812", 2024-09-18T04:56:43.771 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:43.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.771 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:43.771 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6813", 2024-09-18T04:56:43.771 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:43.771 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.771 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.772 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6809/91038858", 2024-09-18T04:56:43.772 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6811/91038858", 2024-09-18T04:56:43.772 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6815/91038858", 2024-09-18T04:56:43.772 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6813/91038858", 2024-09-18T04:56:43.772 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:43.772 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:43.772 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:43.772 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.772 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.772 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:56:43.772 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_xinfo": [ 2024-09-18T04:56:43.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.772 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-18T04:56:43.773 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:43.773 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:43.773 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:43.773 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:43.773 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:43.773 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.468213+0000", 2024-09-18T04:56:43.773 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:43.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.773 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-18T04:56:43.773 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:43.773 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:43.773 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:43.774 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:43.774 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:43.774 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.538610+0000", 2024-09-18T04:56:43.774 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:43.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.774 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-18T04:56:43.774 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:43.774 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:43.774 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:43.774 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:43.774 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:43.774 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.500881+0000", 2024-09-18T04:56:43.774 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:43.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.775 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-18T04:56:43.775 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:43.775 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:43.775 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:43.775 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:43.775 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:43.775 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.474179+0000", 2024-09-18T04:56:43.775 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:43.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.775 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-18T04:56:43.776 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:43.776 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:43.776 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:43.776 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:43.776 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:43.776 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.483737+0000", 2024-09-18T04:56:43.776 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:43.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.776 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-18T04:56:43.776 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:43.776 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:43.776 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:43.776 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:43.777 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:43.777 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.460693+0000", 2024-09-18T04:56:43.777 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:43.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.777 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-18T04:56:43.777 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:43.777 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:43.777 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:43.777 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:43.777 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:43.777 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.475970+0000", 2024-09-18T04:56:43.777 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:43.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.778 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-18T04:56:43.778 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:43.778 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:43.778 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:43.778 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:43.778 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:43.778 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.519287+0000", 2024-09-18T04:56:43.778 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:43.778 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.778 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:56:43.778 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap": [ 2024-09-18T04:56:43.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:43.779 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "3.13", 2024-09-18T04:56:43.779 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-18T04:56:43.779 INFO:teuthology.orchestra.run.smithi002.stdout: 3, 2024-09-18T04:56:43.779 INFO:teuthology.orchestra.run.smithi002.stdout: 4 2024-09-18T04:56:43.779 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:43.779 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.779 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:56:43.779 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_items": [], 2024-09-18T04:56:43.779 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_primaries": [], 2024-09-18T04:56:43.779 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_temp": [], 2024-09-18T04:56:43.779 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_temp": [], 2024-09-18T04:56:43.779 INFO:teuthology.orchestra.run.smithi002.stdout: "blocklist": {}, 2024-09-18T04:56:43.779 INFO:teuthology.orchestra.run.smithi002.stdout: "range_blocklist": {}, 2024-09-18T04:56:43.779 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profiles": { 2024-09-18T04:56:43.780 INFO:teuthology.orchestra.run.smithi002.stdout: "default": { 2024-09-18T04:56:43.780 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-failure-domain": "osd", 2024-09-18T04:56:43.780 INFO:teuthology.orchestra.run.smithi002.stdout: "k": "2", 2024-09-18T04:56:43.780 INFO:teuthology.orchestra.run.smithi002.stdout: "m": "1", 2024-09-18T04:56:43.780 INFO:teuthology.orchestra.run.smithi002.stdout: "plugin": "jerasure", 2024-09-18T04:56:43.780 INFO:teuthology.orchestra.run.smithi002.stdout: "technique": "reed_sol_van" 2024-09-18T04:56:43.780 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:43.780 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps_queue": [], 2024-09-18T04:56:43.780 INFO:teuthology.orchestra.run.smithi002.stdout: "new_removed_snaps": [], 2024-09-18T04:56:43.780 INFO:teuthology.orchestra.run.smithi002.stdout: "new_purged_snaps": [], 2024-09-18T04:56:43.780 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_node_flags": {}, 2024-09-18T04:56:43.780 INFO:teuthology.orchestra.run.smithi002.stdout: "device_class_flags": {}, 2024-09-18T04:56:43.780 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode": { 2024-09-18T04:56:43.781 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_enabled": false, 2024-09-18T04:56:43.781 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_bucket_count": 0, 2024-09-18T04:56:43.781 INFO:teuthology.orchestra.run.smithi002.stdout: "degraded_stretch_mode": 0, 2024-09-18T04:56:43.781 INFO:teuthology.orchestra.run.smithi002.stdout: "recovering_stretch_mode": 0, 2024-09-18T04:56:43.781 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_bucket": 0 2024-09-18T04:56:43.781 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:43.781 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:43.783 INFO:tasks.thrashosds.thrasher:j is {'epoch': 39, 'fsid': 'c0761d0c-69d5-4b64-b8b9-c7d098b2d06d', 'created': '2024-09-18T04:55:58.216538+0000', 'modified': '2024-09-18T04:56:43.200427+0000', 'last_up_change': '2024-09-18T04:56:08.369924+0000', 'last_in_change': '2024-09-18T04:56:00.333800+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-18T04:56:08.983344+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-18T04:56:10.313988+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T04:56:19.562579+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 18, 'snap_epoch': 39, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'c43f8024-3a3e-424e-aa54-398310ceb70d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6824', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6825', 'nonce': 4106704527}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6826', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6827', 'nonce': 4106704527}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6830', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6831', 'nonce': 4106704527}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6828', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6829', 'nonce': 4106704527}]}, 'public_addr': '172.21.15.2:6825/4106704527', 'cluster_addr': '172.21.15.2:6827/4106704527', 'heartbeat_back_addr': '172.21.15.2:6831/4106704527', 'heartbeat_front_addr': '172.21.15.2:6829/4106704527', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c664b970-145c-4b6f-8852-2f6dcf1fa30d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6808', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6809', 'nonce': 1781959259}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6810', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6811', 'nonce': 1781959259}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6814', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6815', 'nonce': 1781959259}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6812', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6813', 'nonce': 1781959259}]}, 'public_addr': '172.21.15.2:6809/1781959259', 'cluster_addr': '172.21.15.2:6811/1781959259', 'heartbeat_back_addr': '172.21.15.2:6815/1781959259', 'heartbeat_front_addr': '172.21.15.2:6813/1781959259', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0609ea8c-d975-490e-8844-04a3e4512efd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6800', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6801', 'nonce': 1153570802}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6802', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6803', 'nonce': 1153570802}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6806', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6807', 'nonce': 1153570802}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6804', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6805', 'nonce': 1153570802}]}, 'public_addr': '172.21.15.2:6801/1153570802', 'cluster_addr': '172.21.15.2:6803/1153570802', 'heartbeat_back_addr': '172.21.15.2:6807/1153570802', 'heartbeat_front_addr': '172.21.15.2:6805/1153570802', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'c6bd5ac8-ab89-4fa2-b843-3b166d1b238b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6816', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6817', 'nonce': 2008321375}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6818', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6819', 'nonce': 2008321375}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6822', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6823', 'nonce': 2008321375}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6820', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6821', 'nonce': 2008321375}]}, 'public_addr': '172.21.15.2:6817/2008321375', 'cluster_addr': '172.21.15.2:6819/2008321375', 'heartbeat_back_addr': '172.21.15.2:6823/2008321375', 'heartbeat_front_addr': '172.21.15.2:6821/2008321375', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '89d6a371-4f4e-465e-a023-d5083cce6e6c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6824', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6825', 'nonce': 2939896598}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6826', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6827', 'nonce': 2939896598}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6830', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6831', 'nonce': 2939896598}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6828', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6829', 'nonce': 2939896598}]}, 'public_addr': '172.21.15.99:6825/2939896598', 'cluster_addr': '172.21.15.99:6827/2939896598', 'heartbeat_back_addr': '172.21.15.99:6831/2939896598', 'heartbeat_front_addr': '172.21.15.99:6829/2939896598', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'da99686b-d979-4753-b4aa-a3b26e9a0497', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6816', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6817', 'nonce': 3440879618}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6818', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6819', 'nonce': 3440879618}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6822', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6823', 'nonce': 3440879618}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6820', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6821', 'nonce': 3440879618}]}, 'public_addr': '172.21.15.99:6817/3440879618', 'cluster_addr': '172.21.15.99:6819/3440879618', 'heartbeat_back_addr': '172.21.15.99:6823/3440879618', 'heartbeat_front_addr': '172.21.15.99:6821/3440879618', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ceec6aa6-603d-4706-bfea-3c07dbdaed3a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6800', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6801', 'nonce': 2493830504}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6802', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6803', 'nonce': 2493830504}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6806', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6807', 'nonce': 2493830504}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6804', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6805', 'nonce': 2493830504}]}, 'public_addr': '172.21.15.99:6801/2493830504', 'cluster_addr': '172.21.15.99:6803/2493830504', 'heartbeat_back_addr': '172.21.15.99:6807/2493830504', 'heartbeat_front_addr': '172.21.15.99:6805/2493830504', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '6779dfad-f60b-41f4-b666-a99db6dc1814', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6808', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6809', 'nonce': 91038858}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6810', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6811', 'nonce': 91038858}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6814', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6815', 'nonce': 91038858}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6812', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6813', 'nonce': 91038858}]}, 'public_addr': '172.21.15.99:6809/91038858', 'cluster_addr': '172.21.15.99:6811/91038858', 'heartbeat_back_addr': '172.21.15.99:6815/91038858', 'heartbeat_front_addr': '172.21.15.99:6813/91038858', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.468213+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.538610+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.500881+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.474179+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.483737+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.460693+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.475970+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.519287+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.13', 'osds': [3, 4]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T04:56:43.783 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T04:56:43.784 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:43.784 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:43.784 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:56:43.785 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:43.780+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:43.785 INFO:teuthology.orchestra.run.smithi002.stdout:1814 writesame 0x136f0c2 thru 0x137a635 data_size 0x31(0xb574 bytes) 2024-09-18T04:56:43.786 INFO:teuthology.orchestra.run.smithi002.stdout:1817 read 0x16cd012 thru 0x16d585c (0x884b bytes) 2024-09-18T04:56:43.786 INFO:teuthology.orchestra.run.smithi002.stdout:1818 trunc from 0x1e73206 to 0x1cdf0ea 2024-09-18T04:56:43.796 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:43.812 INFO:teuthology.orchestra.run.smithi002.stdout:1820 writesame 0xa6684a9 thru 0xa66b7e5 data_size 0x1(0x333d bytes) 2024-09-18T04:56:43.812 INFO:teuthology.orchestra.run.smithi002.stdout:1821 read 0x23d6a87 thru 0x23dc2ba (0x5834 bytes) 2024-09-18T04:56:43.812 INFO:teuthology.orchestra.run.smithi002.stdout:1822 read 0x3ebedd3 thru 0x3eca491 (0xb6bf bytes) 2024-09-18T04:56:43.812 INFO:teuthology.orchestra.run.smithi002.stdout:1823 writesame 0x9d8e238 thru 0x9d977b4 data_size 0xb(0x957d bytes) 2024-09-18T04:56:43.819 INFO:teuthology.orchestra.run.smithi002.stdout:1824 punch from 0xa5d8cca to 0xa5e4298, (0xb5ce bytes) 2024-09-18T04:56:43.819 INFO:teuthology.orchestra.run.smithi002.stdout:1825 read 0x6b03f09 thru 0x6b040b7 (0x1af bytes) 2024-09-18T04:56:43.819 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:43.827 INFO:teuthology.orchestra.run.smithi002.stdout:1827 write 0xc920cd3 thru 0xc92c322 (0xb650 bytes) 2024-09-18T04:56:43.842 INFO:teuthology.orchestra.run.smithi002.stdout:1828 write 0xe692f1e thru 0xe69c711 (0x97f4 bytes) 2024-09-18T04:56:43.844 INFO:teuthology.orchestra.run.smithi002.stdout:1829 read 0xa6b1c43 thru 0xa6bd6a4 (0xba62 bytes) 2024-09-18T04:56:43.844 INFO:teuthology.orchestra.run.smithi002.stdout:1830 write 0x5926096 thru 0x593186c (0xb7d7 bytes) 2024-09-18T04:56:43.849 INFO:teuthology.orchestra.run.smithi002.stdout:1831 read 0x4acbb21 thru 0x4ad3664 (0x7b44 bytes) 2024-09-18T04:56:43.849 INFO:teuthology.orchestra.run.smithi002.stdout:1833 write 0x79afba7 thru 0x79bb98d (0xbde7 bytes) 2024-09-18T04:56:43.855 INFO:teuthology.orchestra.run.smithi002.stdout:1836 write 0x3a7f34 thru 0x3ab33d (0x340a bytes) 2024-09-18T04:56:43.866 INFO:teuthology.orchestra.run.smithi002.stdout:1837 trunc from 0xe69c712 to 0x5cbd5ff 2024-09-18T04:56:43.878 INFO:teuthology.orchestra.run.smithi002.stdout:1838 write 0x3120cec thru 0x3121d55 (0x106a bytes) 2024-09-18T04:56:43.883 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:43.881+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:43.889 INFO:teuthology.orchestra.run.smithi002.stdout:1840 write 0x8b3c94d thru 0x8b4877e (0xbe32 bytes) 2024-09-18T04:56:43.893 INFO:teuthology.orchestra.run.smithi002.stdout:1843 writesame 0xc9dff2 thru 0xcacf33 data_size 0x7d(0xef42 bytes) 2024-09-18T04:56:43.899 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:43.899 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:43.903 INFO:teuthology.orchestra.run.smithi002.stdout:1844 trunc from 0x8b4877f to 0x2816ede 2024-09-18T04:56:43.909 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:56:43.916 INFO:teuthology.orchestra.run.smithi002.stdout:1845 read 0x134a19d thru 0x13503ee (0x6252 bytes) 2024-09-18T04:56:43.930 INFO:teuthology.orchestra.run.smithi002.stdout:1846 write 0x97f8800 thru 0x97fdb6d (0x536e bytes) 2024-09-18T04:56:43.933 INFO:teuthology.orchestra.run.smithi002.stdout:1849 read 0x5306c1c thru 0x5310ae2 (0x9ec7 bytes) 2024-09-18T04:56:43.936 INFO:teuthology.orchestra.run.smithi002.stdout:1852 trunc from 0x97fdb6e to 0x6ced795 2024-09-18T04:56:43.950 INFO:teuthology.orchestra.run.smithi002.stdout:1854 write 0x4552d7e thru 0x455f2ec (0xc56f bytes) 2024-09-18T04:56:43.966 INFO:teuthology.orchestra.run.smithi002.stdout:1857 write 0xb2d483b thru 0xb2dc0fe (0x78c4 bytes) 2024-09-18T04:56:43.970 INFO:teuthology.orchestra.run.smithi002.stdout:1858 writesame 0x199495d thru 0x19a03d8 data_size 0x8c(0xba7c bytes) 2024-09-18T04:56:43.972 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:43.972 INFO:teuthology.orchestra.run.smithi002.stdout:1860 punch from 0x4da53be to 0x4da641d, (0x105f bytes) 2024-09-18T04:56:43.973 INFO:teuthology.orchestra.run.smithi002.stdout:1861 punch from 0x715f908 to 0x716d323, (0xda1b bytes) 2024-09-18T04:56:43.973 INFO:teuthology.orchestra.run.smithi002.stdout:1862 read 0x4d97ff3 thru 0x4d9a66a (0x2678 bytes) 2024-09-18T04:56:43.977 INFO:teuthology.orchestra.run.smithi002.stdout:1863 write 0xb440dc0 thru 0xb44e5ea (0xd82b bytes) 2024-09-18T04:56:43.983 INFO:teuthology.orchestra.run.smithi002.stdout:1864 read 0x4e19c06 thru 0x4e22a50 (0x8e4b bytes) 2024-09-18T04:56:43.983 INFO:teuthology.orchestra.run.smithi002.stdout:1865 read 0xab2f714 thru 0xab3ec53 (0xf540 bytes) 2024-09-18T04:56:43.983 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:43.981+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:43.984 INFO:teuthology.orchestra.run.smithi002.stdout:1866 read 0x8eb1fd7 thru 0x8ec0cbf (0xece9 bytes) 2024-09-18T04:56:43.984 INFO:teuthology.orchestra.run.smithi002.stdout:1867 punch from 0xafd0864 to 0xafdda96, (0xd232 bytes) 2024-09-18T04:56:43.984 INFO:teuthology.orchestra.run.smithi002.stdout:1869 write 0x122cf1 thru 0x12fc4e (0xcf5e bytes) 2024-09-18T04:56:43.988 INFO:teuthology.orchestra.run.smithi002.stdout:1870 trunc from 0xb44e5eb to 0xd749c9e 2024-09-18T04:56:43.993 INFO:teuthology.orchestra.run.smithi002.stdout:1871 read 0x2586b2a thru 0x2591011 (0xa4e8 bytes) 2024-09-18T04:56:43.994 INFO:teuthology.orchestra.run.smithi002.stdout:1872 write 0x845179a thru 0x845a596 (0x8dfd bytes) 2024-09-18T04:56:44.013 INFO:teuthology.orchestra.run.smithi002.stdout:1873 trunc from 0xd749c9e to 0x5c1bee7 2024-09-18T04:56:44.027 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:44.027 INFO:teuthology.orchestra.run.smithi002.stdout:1875 punch from 0x1b45999 to 0x1b50015, (0xa67c bytes) 2024-09-18T04:56:44.029 INFO:teuthology.orchestra.run.smithi002.stdout:1876 trunc from 0x5c1bee7 to 0x8334c7a 2024-09-18T04:56:44.040 INFO:teuthology.orchestra.run.smithi002.stdout:1878 write 0x8a2063e thru 0x8a2e282 (0xdc45 bytes) 2024-09-18T04:56:44.044 INFO:teuthology.orchestra.run.smithi002.stdout:1879 write 0x1640c12 thru 0x1640e7c (0x26b bytes) 2024-09-18T04:56:44.045 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:44.045 INFO:teuthology.orchestra.run.smithi002.stdout:1882 read 0x1bc9aa thru 0x1c7ea0 (0xb4f7 bytes) 2024-09-18T04:56:44.046 INFO:teuthology.orchestra.run.smithi002.stdout:1883 write 0x645f667 thru 0x6467cb8 (0x8652 bytes) 2024-09-18T04:56:44.061 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:44.062 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:44.062 INFO:teuthology.orchestra.run.smithi002.stdout:1884 writesame 0xeac8ab7 thru 0xead6a8a data_size 0x96(0xdfd4 bytes) 2024-09-18T04:56:44.063 INFO:teuthology.orchestra.run.smithi002.stdout:1885 read 0x1628b32 thru 0x1628f72 (0x441 bytes) 2024-09-18T04:56:44.063 INFO:teuthology.orchestra.run.smithi002.stdout:1886 punch from 0x3c29e30 to 0x3c3664d, (0xc81d bytes) 2024-09-18T04:56:44.063 INFO:teuthology.orchestra.run.smithi002.stdout:1887 write 0xb845a7 thru 0xb91f4f (0xd9a9 bytes) 2024-09-18T04:56:44.064 INFO:teuthology.orchestra.run.smithi002.stdout:1888 read 0x645a091 thru 0x6466c8d (0xcbfd bytes) 2024-09-18T04:56:44.065 INFO:teuthology.orchestra.run.smithi002.stdout:1889 write 0xa58a87 thru 0xa622fc (0x9876 bytes) 2024-09-18T04:56:44.067 INFO:teuthology.orchestra.run.smithi002.stdout:1891 write 0xe5cd13b thru 0xe5d53c2 (0x8288 bytes) 2024-09-18T04:56:44.071 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:56:44.074 INFO:teuthology.orchestra.run.smithi002.stdout:1893 punch from 0x6a1a760 to 0x6a1cc63, (0x2503 bytes) 2024-09-18T04:56:44.074 INFO:teuthology.orchestra.run.smithi002.stdout:1894 write 0x418cbb5 thru 0x4197d88 (0xb1d4 bytes) 2024-09-18T04:56:44.078 INFO:teuthology.orchestra.run.smithi002.stdout:1895 read 0x953a338 thru 0x954a232 (0xfefb bytes) 2024-09-18T04:56:44.078 INFO:teuthology.orchestra.run.smithi002.stdout:1896 punch from 0x26bd3be to 0x26ca329, (0xcf6b bytes) 2024-09-18T04:56:44.078 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:44.078 INFO:teuthology.orchestra.run.smithi002.stdout:1898 punch from 0x87bcc56 to 0x87c930f, (0xc6b9 bytes) 2024-09-18T04:56:44.078 INFO:teuthology.orchestra.run.smithi002.stdout:1901 read 0xbf63fe4 thru 0xbf7327b (0xf298 bytes) 2024-09-18T04:56:44.080 INFO:teuthology.orchestra.run.smithi002.stdout:1902 trunc from 0xead6a8b to 0xc21f247 2024-09-18T04:56:44.088 INFO:teuthology.orchestra.run.smithi002.stdout:1905 writesame 0x55c7572 thru 0x55ca315 data_size 0x2e(0x2da4 bytes) 2024-09-18T04:56:44.095 INFO:teuthology.orchestra.run.smithi002.stdout:1906 trunc from 0xc21f247 to 0x8d16c6d 2024-09-18T04:56:44.107 INFO:teuthology.orchestra.run.smithi002.stdout:1907 read 0x6bf7073 thru 0x6bf7fc6 (0xf54 bytes) 2024-09-18T04:56:44.114 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:44.112+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:44.117 INFO:teuthology.orchestra.run.smithi002.stdout:1909 writesame 0xd33e18a thru 0xd3405cd data_size 0x16(0x2444 bytes) 2024-09-18T04:56:44.117 INFO:teuthology.orchestra.run.smithi002.stdout:1910 read 0x5f52917 thru 0x5f54b7f (0x2269 bytes) 2024-09-18T04:56:44.118 INFO:teuthology.orchestra.run.smithi002.stdout:1912 read 0xb12c56 thru 0xb1a58e (0x7939 bytes) 2024-09-18T04:56:44.119 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:56:44.119 INFO:teuthology.orchestra.run.smithi002.stdout:1913 punch from 0x5abdcd9 to 0x5acad34, (0xd05b bytes) 2024-09-18T04:56:44.119 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:44.119 INFO:teuthology.orchestra.run.smithi002.stdout:1915 read 0x215fbb4 thru 0x21629ae (0x2dfb bytes) 2024-09-18T04:56:44.120 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-18T04:56:44.121 INFO:teuthology.orchestra.run.smithi002.stdout:1916 write 0x319240f thru 0x3196215 (0x3e07 bytes) 2024-09-18T04:56:44.122 INFO:teuthology.orchestra.run.smithi002.stdout:1917 clone 16 order 25 su 16777216 sc 13 2024-09-18T04:56:44.131 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":49,"stamp":"2024-09-18T04:56:43.207826+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":485454756,"num_objects":395,"num_object_clones":67,"num_object_copies":790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":395,"num_whiteouts":64,"num_read":33940,"num_read_kb":88797,"num_write":4345,"num_write_kb":88703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":3255539,"num_keys_recovered":9,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3287,"ondisk_log_size":3287,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":272676,"kb_used_data":15520,"kb_used_omap":12,"kb_used_meta":257011,"kb_avail":754702044,"statfs":{"total":773094113280,"available":772814893056,"internally_reserved":0,"allocated":15892480,"data_stored":93684017,"data_compressed":1226127,"data_compressed_allocated":6926336,"data_compressed_original":87086442,"omap_allocated":12742,"internal_metadata":263179834},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":132570057,"num_objects":178,"num_object_clones":19,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":19,"num_read":21929,"num_read_kb":57606,"num_write":1987,"num_write_kb":36010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.020874"},"pg_stats":[{"pgid":"3.1f","version":"34'91","reported_seq":465,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.902475+0000","last_change":"2024-09-18T04:56:23.130295+0000","last_active":"2024-09-18T04:56:38.902475+0000","last_peered":"2024-09-18T04:56:38.902475+0000","last_clean":"2024-09-18T04:56:38.902475+0000","last_became_active":"2024-09-18T04:56:21.132342+0000","last_became_peered":"2024-09-18T04:56:21.132342+0000","last_unstale":"2024-09-18T04:56:38.902475+0000","last_undegraded":"2024-09-18T04:56:38.902475+0000","last_fullsized":"2024-09-18T04:56:38.902475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:43:47.128460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011070279999999999,"stat_sum":{"num_bytes":4927632,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":990,"num_read_kb":1056,"num_write":87,"num_write_kb":308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"34'121","reported_seq":199,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.467747+0000","last_change":"2024-09-18T04:56:23.582447+0000","last_active":"2024-09-18T04:56:38.467747+0000","last_peered":"2024-09-18T04:56:38.467747+0000","last_clean":"2024-09-18T04:56:38.467747+0000","last_became_active":"2024-09-18T04:56:21.126431+0000","last_became_peered":"2024-09-18T04:56:21.126431+0000","last_unstale":"2024-09-18T04:56:38.467747+0000","last_undegraded":"2024-09-18T04:56:38.467747+0000","last_fullsized":"2024-09-18T04:56:38.467747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:59.671751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062644600000000001,"stat_sum":{"num_bytes":11777370,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":460,"num_read_kb":621,"num_write":187,"num_write_kb":2083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"35'66","reported_seq":226,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-18T04:56:39.195434+0000","last_change":"2024-09-18T04:56:23.137064+0000","last_active":"2024-09-18T04:56:39.195434+0000","last_peered":"2024-09-18T04:56:39.195434+0000","last_clean":"2024-09-18T04:56:39.195434+0000","last_became_active":"2024-09-18T04:56:21.131613+0000","last_became_peered":"2024-09-18T04:56:21.131613+0000","last_unstale":"2024-09-18T04:56:39.195434+0000","last_undegraded":"2024-09-18T04:56:39.195434+0000","last_fullsized":"2024-09-18T04:56:39.195434+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:15:27.419409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000455248,"stat_sum":{"num_bytes":19660837,"num_objects":12,"num_object_clones":3,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":177,"num_read_kb":3528,"num_write":78,"num_write_kb":13026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"35'319","reported_seq":998,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-18T04:56:39.277493+0000","last_change":"2024-09-18T04:56:23.137422+0000","last_active":"2024-09-18T04:56:39.277493+0000","last_peered":"2024-09-18T04:56:39.277493+0000","last_clean":"2024-09-18T04:56:39.277493+0000","last_became_active":"2024-09-18T04:56:21.129551+0000","last_became_peered":"2024-09-18T04:56:21.129551+0000","last_unstale":"2024-09-18T04:56:39.277493+0000","last_undegraded":"2024-09-18T04:56:39.277493+0000","last_fullsized":"2024-09-18T04:56:39.277493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":319,"log_dups_size":0,"ondisk_log_size":319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:38:38.231233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081678499999999999,"stat_sum":{"num_bytes":2714132,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":2051,"num_read_kb":1335,"num_write":496,"num_write_kb":828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"34'39","reported_seq":105,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.676340+0000","last_change":"2024-09-18T04:56:23.130841+0000","last_active":"2024-09-18T04:56:38.676340+0000","last_peered":"2024-09-18T04:56:38.676340+0000","last_clean":"2024-09-18T04:56:38.676340+0000","last_became_active":"2024-09-18T04:56:21.129660+0000","last_became_peered":"2024-09-18T04:56:21.129660+0000","last_unstale":"2024-09-18T04:56:38.676340+0000","last_undegraded":"2024-09-18T04:56:38.676340+0000","last_fullsized":"2024-09-18T04:56:38.676340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:43.007677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014550170000000001,"stat_sum":{"num_bytes":1729539,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":68,"num_read_kb":112,"num_write":48,"num_write_kb":839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"34'71","reported_seq":643,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-18T04:56:39.285901+0000","last_change":"2024-09-18T04:56:23.133311+0000","last_active":"2024-09-18T04:56:39.285901+0000","last_peered":"2024-09-18T04:56:39.285901+0000","last_clean":"2024-09-18T04:56:39.285901+0000","last_became_active":"2024-09-18T04:56:21.127888+0000","last_became_peered":"2024-09-18T04:56:21.127888+0000","last_unstale":"2024-09-18T04:56:39.285901+0000","last_undegraded":"2024-09-18T04:56:39.285901+0000","last_fullsized":"2024-09-18T04:56:39.285901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":71,"log_dups_size":0,"ondisk_log_size":71,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:35:20.652713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087073300000000005,"stat_sum":{"num_bytes":9866596,"num_objects":13,"num_object_clones":4,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":3,"num_read":583,"num_read_kb":2863,"num_write":83,"num_write_kb":13918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"33'50","reported_seq":1361,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.809738+0000","last_change":"2024-09-18T04:56:23.130039+0000","last_active":"2024-09-18T04:56:38.809738+0000","last_peered":"2024-09-18T04:56:38.809738+0000","last_clean":"2024-09-18T04:56:38.809738+0000","last_became_active":"2024-09-18T04:56:21.125153+0000","last_became_peered":"2024-09-18T04:56:21.125153+0000","last_unstale":"2024-09-18T04:56:38.809738+0000","last_undegraded":"2024-09-18T04:56:38.809738+0000","last_fullsized":"2024-09-18T04:56:38.809738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:54:02.903679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012370669999999999,"stat_sum":{"num_bytes":8561253,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":1,"num_read":1605,"num_read_kb":4460,"num_write":78,"num_write_kb":4424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"34'101","reported_seq":221,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.590212+0000","last_change":"2024-09-18T04:56:23.582074+0000","last_active":"2024-09-18T04:56:38.590212+0000","last_peered":"2024-09-18T04:56:38.590212+0000","last_clean":"2024-09-18T04:56:38.590212+0000","last_became_active":"2024-09-18T04:56:21.127959+0000","last_became_peered":"2024-09-18T04:56:21.127959+0000","last_unstale":"2024-09-18T04:56:38.590212+0000","last_undegraded":"2024-09-18T04:56:38.590212+0000","last_fullsized":"2024-09-18T04:56:38.590212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:17:32.019845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053040699999999999,"stat_sum":{"num_bytes":16431937,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":2,"num_read":385,"num_read_kb":679,"num_write":166,"num_write_kb":2638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"35'54","reported_seq":329,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-18T04:56:39.286258+0000","last_change":"2024-09-18T04:56:23.130832+0000","last_active":"2024-09-18T04:56:39.286258+0000","last_peered":"2024-09-18T04:56:39.286258+0000","last_clean":"2024-09-18T04:56:39.286258+0000","last_became_active":"2024-09-18T04:56:21.130316+0000","last_became_peered":"2024-09-18T04:56:21.130316+0000","last_unstale":"2024-09-18T04:56:39.286258+0000","last_undegraded":"2024-09-18T04:56:39.286258+0000","last_fullsized":"2024-09-18T04:56:39.286258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:12:05.411040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001612023,"stat_sum":{"num_bytes":7736164,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":681,"num_read_kb":818,"num_write":65,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"34'50","reported_seq":2281,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:39.069289+0000","last_change":"2024-09-18T04:56:23.136846+0000","last_active":"2024-09-18T04:56:39.069289+0000","last_peered":"2024-09-18T04:56:39.069289+0000","last_clean":"2024-09-18T04:56:39.069289+0000","last_became_active":"2024-09-18T04:56:21.131133+0000","last_became_peered":"2024-09-18T04:56:21.131133+0000","last_unstale":"2024-09-18T04:56:39.069289+0000","last_undegraded":"2024-09-18T04:56:39.069289+0000","last_fullsized":"2024-09-18T04:56:39.069289+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:01:57.036904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025847199999999999,"stat_sum":{"num_bytes":35684568,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":5,"num_read":2218,"num_read_kb":5346,"num_write":60,"num_write_kb":2251,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"34'142","reported_seq":1546,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.999747+0000","last_change":"2024-09-18T04:56:23.142657+0000","last_active":"2024-09-18T04:56:38.999747+0000","last_peered":"2024-09-18T04:56:38.999747+0000","last_clean":"2024-09-18T04:56:38.999747+0000","last_became_active":"2024-09-18T04:56:21.131979+0000","last_became_peered":"2024-09-18T04:56:21.131979+0000","last_unstale":"2024-09-18T04:56:38.999747+0000","last_undegraded":"2024-09-18T04:56:38.999747+0000","last_fullsized":"2024-09-18T04:56:38.999747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":142,"log_dups_size":0,"ondisk_log_size":142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:16:43.303511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029355000000000001,"stat_sum":{"num_bytes":57018492,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":2526,"num_read_kb":10840,"num_write":137,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"34'88","reported_seq":201,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.992842+0000","last_change":"2024-09-18T04:56:23.132876+0000","last_active":"2024-09-18T04:56:38.992842+0000","last_peered":"2024-09-18T04:56:38.992842+0000","last_clean":"2024-09-18T04:56:38.992842+0000","last_became_active":"2024-09-18T04:56:21.134979+0000","last_became_peered":"2024-09-18T04:56:21.134979+0000","last_unstale":"2024-09-18T04:56:38.992842+0000","last_undegraded":"2024-09-18T04:56:38.992842+0000","last_fullsized":"2024-09-18T04:56:38.992842+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:36:24.881782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038004100000000002,"stat_sum":{"num_bytes":3343721,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":365,"num_read_kb":1689,"num_write":142,"num_write_kb":2221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":34,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.211149+0000","last_change":"2024-09-18T04:56:12.390222+0000","last_active":"2024-09-18T04:56:38.211149+0000","last_peered":"2024-09-18T04:56:38.211149+0000","last_clean":"2024-09-18T04:56:38.211149+0000","last_became_active":"2024-09-18T04:56:10.401272+0000","last_became_peered":"2024-09-18T04:56:10.401272+0000","last_unstale":"2024-09-18T04:56:38.211149+0000","last_undegraded":"2024-09-18T04:56:38.211149+0000","last_fullsized":"2024-09-18T04:56:38.211149+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:33:50.692096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063436799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"33'116","reported_seq":1314,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.180944+0000","last_change":"2024-09-18T04:56:23.137022+0000","last_active":"2024-09-18T04:56:38.180944+0000","last_peered":"2024-09-18T04:56:38.180944+0000","last_clean":"2024-09-18T04:56:38.180944+0000","last_became_active":"2024-09-18T04:56:21.129964+0000","last_became_peered":"2024-09-18T04:56:21.129964+0000","last_unstale":"2024-09-18T04:56:38.180944+0000","last_undegraded":"2024-09-18T04:56:38.180944+0000","last_fullsized":"2024-09-18T04:56:38.180944+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":116,"log_dups_size":0,"ondisk_log_size":116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:29:32.843883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043204099999999998,"stat_sum":{"num_bytes":11053291,"num_objects":16,"num_object_clones":1,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":1,"num_read":1581,"num_read_kb":4986,"num_write":208,"num_write_kb":5238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.211165+0000","last_change":"2024-09-18T04:56:12.389971+0000","last_active":"2024-09-18T04:56:38.211165+0000","last_peered":"2024-09-18T04:56:38.211165+0000","last_clean":"2024-09-18T04:56:38.211165+0000","last_became_active":"2024-09-18T04:56:10.412595+0000","last_became_peered":"2024-09-18T04:56:10.412595+0000","last_unstale":"2024-09-18T04:56:38.211165+0000","last_undegraded":"2024-09-18T04:56:38.211165+0000","last_fullsized":"2024-09-18T04:56:38.211165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:45.638420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038576799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"34'252","reported_seq":2492,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.676302+0000","last_change":"2024-09-18T04:56:23.129727+0000","last_active":"2024-09-18T04:56:38.676302+0000","last_peered":"2024-09-18T04:56:38.676302+0000","last_clean":"2024-09-18T04:56:38.676302+0000","last_became_active":"2024-09-18T04:56:21.122488+0000","last_became_peered":"2024-09-18T04:56:21.122488+0000","last_unstale":"2024-09-18T04:56:38.676302+0000","last_undegraded":"2024-09-18T04:56:38.676302+0000","last_fullsized":"2024-09-18T04:56:38.676302+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:06:03.886036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072197,"stat_sum":{"num_bytes":26889032,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":3373,"num_read_kb":8970,"num_write":456,"num_write_kb":2811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":34,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.188297+0000","last_change":"2024-09-18T04:56:12.412298+0000","last_active":"2024-09-18T04:56:38.188297+0000","last_peered":"2024-09-18T04:56:38.188297+0000","last_clean":"2024-09-18T04:56:38.188297+0000","last_became_active":"2024-09-18T04:56:10.399718+0000","last_became_peered":"2024-09-18T04:56:10.399718+0000","last_unstale":"2024-09-18T04:56:38.188297+0000","last_undegraded":"2024-09-18T04:56:38.188297+0000","last_fullsized":"2024-09-18T04:56:38.188297+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:41:31.956685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024059099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"35'227","reported_seq":291,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-18T04:56:39.214272+0000","last_change":"2024-09-18T04:56:23.582182+0000","last_active":"2024-09-18T04:56:39.214272+0000","last_peered":"2024-09-18T04:56:39.214272+0000","last_clean":"2024-09-18T04:56:39.214272+0000","last_became_active":"2024-09-18T04:56:21.125785+0000","last_became_peered":"2024-09-18T04:56:21.125785+0000","last_unstale":"2024-09-18T04:56:39.214272+0000","last_undegraded":"2024-09-18T04:56:39.214272+0000","last_fullsized":"2024-09-18T04:56:39.214272+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":227,"log_dups_size":0,"ondisk_log_size":227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:35:40.684580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048263400000000001,"stat_sum":{"num_bytes":5732349,"num_objects":17,"num_object_clones":1,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":3,"num_read":996,"num_read_kb":1077,"num_write":361,"num_write_kb":718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.220074+0000","last_change":"2024-09-18T04:56:12.413007+0000","last_active":"2024-09-18T04:56:38.220074+0000","last_peered":"2024-09-18T04:56:38.220074+0000","last_clean":"2024-09-18T04:56:38.220074+0000","last_became_active":"2024-09-18T04:56:10.399076+0000","last_became_peered":"2024-09-18T04:56:10.399076+0000","last_unstale":"2024-09-18T04:56:38.220074+0000","last_undegraded":"2024-09-18T04:56:38.220074+0000","last_fullsized":"2024-09-18T04:56:38.220074+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:54:29.826665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045144100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"34'60","reported_seq":212,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.753324+0000","last_change":"2024-09-18T04:56:23.129577+0000","last_active":"2024-09-18T04:56:38.753324+0000","last_peered":"2024-09-18T04:56:38.753324+0000","last_clean":"2024-09-18T04:56:38.753324+0000","last_became_active":"2024-09-18T04:56:21.127750+0000","last_became_peered":"2024-09-18T04:56:21.127750+0000","last_unstale":"2024-09-18T04:56:38.753324+0000","last_undegraded":"2024-09-18T04:56:38.753324+0000","last_fullsized":"2024-09-18T04:56:38.753324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:58:18.075339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039168900000000002,"stat_sum":{"num_bytes":30945362,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":169,"num_read_kb":1423,"num_write":76,"num_write_kb":1923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"34'42","reported_seq":147,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.856341+0000","last_change":"2024-09-18T04:56:23.130445+0000","last_active":"2024-09-18T04:56:38.856341+0000","last_peered":"2024-09-18T04:56:38.856341+0000","last_clean":"2024-09-18T04:56:38.856341+0000","last_became_active":"2024-09-18T04:56:21.130514+0000","last_became_peered":"2024-09-18T04:56:21.130514+0000","last_unstale":"2024-09-18T04:56:38.856341+0000","last_undegraded":"2024-09-18T04:56:38.856341+0000","last_fullsized":"2024-09-18T04:56:38.856341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:06:37.491009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001360497,"stat_sum":{"num_bytes":10833268,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":93,"num_read_kb":1166,"num_write":48,"num_write_kb":1563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":178,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.200415+0000","last_change":"2024-09-18T04:56:11.428051+0000","last_active":"2024-09-18T04:56:38.200415+0000","last_peered":"2024-09-18T04:56:38.200415+0000","last_clean":"2024-09-18T04:56:38.200415+0000","last_became_active":"2024-09-18T04:56:11.427170+0000","last_became_peered":"2024-09-18T04:56:11.427170+0000","last_unstale":"2024-09-18T04:56:38.200415+0000","last_undegraded":"2024-09-18T04:56:38.200415+0000","last_fullsized":"2024-09-18T04:56:38.200415+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_clean_scrub_stamp":"2024-09-18T04:56:10.373785+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:16:08.102627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":35,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.259500+0000","last_change":"2024-09-18T04:56:12.413125+0000","last_active":"2024-09-18T04:56:38.259500+0000","last_peered":"2024-09-18T04:56:38.259500+0000","last_clean":"2024-09-18T04:56:38.259500+0000","last_became_active":"2024-09-18T04:56:10.401641+0000","last_became_peered":"2024-09-18T04:56:10.401641+0000","last_unstale":"2024-09-18T04:56:38.259500+0000","last_undegraded":"2024-09-18T04:56:38.259500+0000","last_fullsized":"2024-09-18T04:56:38.259500+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:19:46.375505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000622431,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"33'101","reported_seq":2656,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.259479+0000","last_change":"2024-09-18T04:56:23.129808+0000","last_active":"2024-09-18T04:56:38.259479+0000","last_peered":"2024-09-18T04:56:38.259479+0000","last_clean":"2024-09-18T04:56:38.259479+0000","last_became_active":"2024-09-18T04:56:21.122785+0000","last_became_peered":"2024-09-18T04:56:21.122785+0000","last_unstale":"2024-09-18T04:56:38.259479+0000","last_undegraded":"2024-09-18T04:56:38.259479+0000","last_fullsized":"2024-09-18T04:56:38.259479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":101,"log_dups_size":0,"ondisk_log_size":101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:05:30.391004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010536390000000001,"stat_sum":{"num_bytes":25702790,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":4,"num_read":3362,"num_read_kb":7020,"num_write":91,"num_write_kb":423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":36,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.200513+0000","last_change":"2024-09-18T04:56:12.398631+0000","last_active":"2024-09-18T04:56:38.200513+0000","last_peered":"2024-09-18T04:56:38.200513+0000","last_clean":"2024-09-18T04:56:38.200513+0000","last_became_active":"2024-09-18T04:56:10.412614+0000","last_became_peered":"2024-09-18T04:56:10.412614+0000","last_unstale":"2024-09-18T04:56:38.200513+0000","last_undegraded":"2024-09-18T04:56:38.200513+0000","last_fullsized":"2024-09-18T04:56:38.200513+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:44:13.401320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075126299999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"34'36","reported_seq":77,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.590189+0000","last_change":"2024-09-18T04:56:23.133345+0000","last_active":"2024-09-18T04:56:38.590189+0000","last_peered":"2024-09-18T04:56:38.590189+0000","last_clean":"2024-09-18T04:56:38.590189+0000","last_became_active":"2024-09-18T04:56:21.128793+0000","last_became_peered":"2024-09-18T04:56:21.128793+0000","last_unstale":"2024-09-18T04:56:38.590189+0000","last_undegraded":"2024-09-18T04:56:38.590189+0000","last_fullsized":"2024-09-18T04:56:38.590189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:10:36.658665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067161900000000004,"stat_sum":{"num_bytes":3347318,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":21,"num_read_kb":88,"num_write":37,"num_write_kb":743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":34,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.180813+0000","last_change":"2024-09-18T04:56:12.390044+0000","last_active":"2024-09-18T04:56:38.180813+0000","last_peered":"2024-09-18T04:56:38.180813+0000","last_clean":"2024-09-18T04:56:38.180813+0000","last_became_active":"2024-09-18T04:56:10.413042+0000","last_became_peered":"2024-09-18T04:56:10.413042+0000","last_unstale":"2024-09-18T04:56:38.180813+0000","last_undegraded":"2024-09-18T04:56:38.180813+0000","last_fullsized":"2024-09-18T04:56:38.180813+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:46:11.074243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"34'54","reported_seq":1273,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:39.014997+0000","last_change":"2024-09-18T04:56:23.581937+0000","last_active":"2024-09-18T04:56:39.014997+0000","last_peered":"2024-09-18T04:56:39.014997+0000","last_clean":"2024-09-18T04:56:39.014997+0000","last_became_active":"2024-09-18T04:56:21.130465+0000","last_became_peered":"2024-09-18T04:56:21.130465+0000","last_unstale":"2024-09-18T04:56:39.014997+0000","last_undegraded":"2024-09-18T04:56:39.014997+0000","last_fullsized":"2024-09-18T04:56:39.014997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:38:38.231233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037357100000000001,"stat_sum":{"num_bytes":21785164,"num_objects":12,"num_object_clones":2,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":2,"num_read":1231,"num_read_kb":3535,"num_write":50,"num_write_kb":1337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":34,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.200434+0000","last_change":"2024-09-18T04:56:12.398299+0000","last_active":"2024-09-18T04:56:38.200434+0000","last_peered":"2024-09-18T04:56:38.200434+0000","last_clean":"2024-09-18T04:56:38.200434+0000","last_became_active":"2024-09-18T04:56:10.413140+0000","last_became_peered":"2024-09-18T04:56:10.413140+0000","last_unstale":"2024-09-18T04:56:38.200434+0000","last_undegraded":"2024-09-18T04:56:38.200434+0000","last_fullsized":"2024-09-18T04:56:38.200434+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:37:11.125495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041592799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"34'49","reported_seq":1198,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.846183+0000","last_change":"2024-09-18T04:56:23.582318+0000","last_active":"2024-09-18T04:56:38.846183+0000","last_peered":"2024-09-18T04:56:38.846183+0000","last_clean":"2024-09-18T04:56:38.846183+0000","last_became_active":"2024-09-18T04:56:21.125295+0000","last_became_peered":"2024-09-18T04:56:21.125295+0000","last_unstale":"2024-09-18T04:56:38.846183+0000","last_undegraded":"2024-09-18T04:56:38.846183+0000","last_fullsized":"2024-09-18T04:56:38.846183+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:29:59.427218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053349799999999998,"stat_sum":{"num_bytes":14883571,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":4,"num_read":1138,"num_read_kb":3892,"num_write":47,"num_write_kb":682,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"35'131","reported_seq":1795,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-18T04:56:39.291916+0000","last_change":"2024-09-18T04:56:23.133113+0000","last_active":"2024-09-18T04:56:39.291916+0000","last_peered":"2024-09-18T04:56:39.291916+0000","last_clean":"2024-09-18T04:56:39.291916+0000","last_became_active":"2024-09-18T04:56:21.128135+0000","last_became_peered":"2024-09-18T04:56:21.128135+0000","last_unstale":"2024-09-18T04:56:39.291916+0000","last_undegraded":"2024-09-18T04:56:39.291916+0000","last_fullsized":"2024-09-18T04:56:39.291916+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:46:04.362749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000513702,"stat_sum":{"num_bytes":7330485,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":2,"num_read":2742,"num_read_kb":4168,"num_write":129,"num_write_kb":431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"35'80","reported_seq":165,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-18T04:56:39.230923+0000","last_change":"2024-09-18T04:56:23.641225+0000","last_active":"2024-09-18T04:56:39.230923+0000","last_peered":"2024-09-18T04:56:39.230923+0000","last_clean":"2024-09-18T04:56:39.230923+0000","last_became_active":"2024-09-18T04:56:21.127832+0000","last_became_peered":"2024-09-18T04:56:21.127832+0000","last_unstale":"2024-09-18T04:56:39.230923+0000","last_undegraded":"2024-09-18T04:56:39.230923+0000","last_fullsized":"2024-09-18T04:56:39.230923+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:23:15.821734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029947799999999999,"stat_sum":{"num_bytes":4738251,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":2,"num_read":168,"num_read_kb":412,"num_write":92,"num_write_kb":771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"35'113","reported_seq":1244,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-18T04:56:39.282594+0000","last_change":"2024-09-18T04:56:23.582528+0000","last_active":"2024-09-18T04:56:39.282594+0000","last_peered":"2024-09-18T04:56:39.282594+0000","last_clean":"2024-09-18T04:56:39.282594+0000","last_became_active":"2024-09-18T04:56:21.127444+0000","last_became_peered":"2024-09-18T04:56:21.127444+0000","last_unstale":"2024-09-18T04:56:39.282594+0000","last_undegraded":"2024-09-18T04:56:39.282594+0000","last_fullsized":"2024-09-18T04:56:39.282594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":113,"log_dups_size":0,"ondisk_log_size":113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:15:41.251274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060817200000000003,"stat_sum":{"num_bytes":8104580,"num_objects":14,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":2,"num_read":1419,"num_read_kb":3626,"num_write":160,"num_write_kb":1489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"34'63","reported_seq":111,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.589999+0000","last_change":"2024-09-18T04:56:23.136791+0000","last_active":"2024-09-18T04:56:38.589999+0000","last_peered":"2024-09-18T04:56:38.589999+0000","last_clean":"2024-09-18T04:56:38.589999+0000","last_became_active":"2024-09-18T04:56:21.131676+0000","last_became_peered":"2024-09-18T04:56:21.131676+0000","last_unstale":"2024-09-18T04:56:38.589999+0000","last_undegraded":"2024-09-18T04:56:38.589999+0000","last_fullsized":"2024-09-18T04:56:38.589999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:34:36.563239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000302343,"stat_sum":{"num_bytes":6140478,"num_objects":7,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":33,"num_read_kb":295,"num_write":59,"num_write_kb":748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"34'85","reported_seq":661,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.993002+0000","last_change":"2024-09-18T04:56:23.143279+0000","last_active":"2024-09-18T04:56:38.993002+0000","last_peered":"2024-09-18T04:56:38.993002+0000","last_clean":"2024-09-18T04:56:38.993002+0000","last_became_active":"2024-09-18T04:56:21.133686+0000","last_became_peered":"2024-09-18T04:56:21.133686+0000","last_unstale":"2024-09-18T04:56:38.993002+0000","last_undegraded":"2024-09-18T04:56:38.993002+0000","last_fullsized":"2024-09-18T04:56:38.993002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:50:51.011660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087309599999999996,"stat_sum":{"num_bytes":14766062,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":2,"num_read":1518,"num_read_kb":1381,"num_write":86,"num_write_kb":866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"34'32","reported_seq":81,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.470571+0000","last_change":"2024-09-18T04:56:23.143532+0000","last_active":"2024-09-18T04:56:38.470571+0000","last_peered":"2024-09-18T04:56:38.470571+0000","last_clean":"2024-09-18T04:56:38.470571+0000","last_became_active":"2024-09-18T04:56:21.132364+0000","last_became_peered":"2024-09-18T04:56:21.132364+0000","last_unstale":"2024-09-18T04:56:38.470571+0000","last_undegraded":"2024-09-18T04:56:38.470571+0000","last_fullsized":"2024-09-18T04:56:38.470571+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:25:44.430438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010190329999999999,"stat_sum":{"num_bytes":2609105,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":31,"num_read_kb":103,"num_write":42,"num_write_kb":679,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"34'85","reported_seq":385,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.258131+0000","last_change":"2024-09-18T04:56:23.143500+0000","last_active":"2024-09-18T04:56:38.258131+0000","last_peered":"2024-09-18T04:56:38.258131+0000","last_clean":"2024-09-18T04:56:38.258131+0000","last_became_active":"2024-09-18T04:56:21.130105+0000","last_became_peered":"2024-09-18T04:56:21.130105+0000","last_unstale":"2024-09-18T04:56:38.258131+0000","last_undegraded":"2024-09-18T04:56:38.258131+0000","last_fullsized":"2024-09-18T04:56:38.258131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:31:55.392378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010487090000000001,"stat_sum":{"num_bytes":11583512,"num_objects":7,"num_object_clones":2,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":786,"num_read_kb":935,"num_write":80,"num_write_kb":264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"34'155","reported_seq":314,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:39.004852+0000","last_change":"2024-09-18T04:56:23.641201+0000","last_active":"2024-09-18T04:56:39.004852+0000","last_peered":"2024-09-18T04:56:39.004852+0000","last_clean":"2024-09-18T04:56:39.004852+0000","last_became_active":"2024-09-18T04:56:21.131202+0000","last_became_peered":"2024-09-18T04:56:21.131202+0000","last_unstale":"2024-09-18T04:56:39.004852+0000","last_undegraded":"2024-09-18T04:56:39.004852+0000","last_fullsized":"2024-09-18T04:56:39.004852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:17:35.438723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000304567,"stat_sum":{"num_bytes":45526000,"num_objects":18,"num_object_clones":5,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":2,"num_read":506,"num_read_kb":3357,"num_write":185,"num_write_kb":2722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"34'184","reported_seq":659,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:39.060315+0000","last_change":"2024-09-18T04:56:23.142612+0000","last_active":"2024-09-18T04:56:39.060315+0000","last_peered":"2024-09-18T04:56:39.060315+0000","last_clean":"2024-09-18T04:56:39.060315+0000","last_became_active":"2024-09-18T04:56:21.130594+0000","last_became_peered":"2024-09-18T04:56:21.130594+0000","last_unstale":"2024-09-18T04:56:39.060315+0000","last_undegraded":"2024-09-18T04:56:39.060315+0000","last_fullsized":"2024-09-18T04:56:39.060315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:46:34.324509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030763399999999999,"stat_sum":{"num_bytes":38770065,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":1,"num_read":1417,"num_read_kb":7011,"num_write":236,"num_write_kb":15341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"34'41","reported_seq":581,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-18T04:56:38.795467+0000","last_change":"2024-09-18T04:56:23.129633+0000","last_active":"2024-09-18T04:56:38.795467+0000","last_peered":"2024-09-18T04:56:38.795467+0000","last_clean":"2024-09-18T04:56:38.795467+0000","last_became_active":"2024-09-18T04:56:21.129597+0000","last_became_peered":"2024-09-18T04:56:21.129597+0000","last_unstale":"2024-09-18T04:56:38.795467+0000","last_undegraded":"2024-09-18T04:56:38.795467+0000","last_fullsized":"2024-09-18T04:56:38.795467+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:50.665769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059848499999999995,"stat_sum":{"num_bytes":3521081,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":3,"num_read":520,"num_read_kb":1351,"num_write":40,"num_write_kb":1327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"34'69","reported_seq":53,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-18T04:56:39.295904+0000","last_change":"2024-09-18T04:56:34.195134+0000","last_active":"2024-09-18T04:56:39.295904+0000","last_peered":"2024-09-18T04:56:39.295904+0000","last_clean":"2024-09-18T04:56:39.295904+0000","last_became_active":"2024-09-18T04:56:34.171449+0000","last_became_peered":"2024-09-18T04:56:34.171449+0000","last_unstale":"2024-09-18T04:56:39.295904+0000","last_undegraded":"2024-09-18T04:56:39.295904+0000","last_fullsized":"2024-09-18T04:56:39.295904+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:38:41.302426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085320600000000002,"stat_sum":{"num_bytes":11150364,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":611,"num_read_kb":554,"num_write":74,"num_write_kb":1725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":3255539,"num_keys_recovered":9,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":484864369,"num_objects":391,"num_object_clones":67,"num_object_copies":782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":391,"num_whiteouts":64,"num_read":33824,"num_read_kb":88697,"num_write":4184,"num_write_kb":86285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":3255539,"num_keys_recovered":9,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12865536,"data_stored":91213564,"data_compressed":1208082,"data_compressed_allocated":6844416,"data_compressed_original":85812586,"omap_allocated":0,"internal_metadata":0},"log_size":3167,"ondisk_log_size":3167,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17398,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33248,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338592,"statfs":{"total":96636764160,"available":96602718208,"internally_reserved":0,"allocated":2183168,"data_stored":12799864,"data_compressed":170854,"data_compressed_allocated":880640,"data_compressed_original":11832730,"omap_allocated":1589,"internal_metadata":31848907},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":1044480,"data_stored":6470169,"data_compressed":47203,"data_compressed_allocated":315392,"data_compressed_original":5969052,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36499999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33256,"kb_used_data":2140,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338584,"statfs":{"total":96636764160,"available":96602710016,"internally_reserved":0,"allocated":2191360,"data_stored":10306410,"data_compressed":165319,"data_compressed_allocated":872448,"data_compressed_original":9255635,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33452,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338388,"statfs":{"total":96636764160,"available":96602509312,"internally_reserved":0,"allocated":2396160,"data_stored":18178179,"data_compressed":185343,"data_compressed_allocated":1130496,"data_compressed_original":17123399,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38108,"kb_used_data":2896,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333732,"statfs":{"total":96636764160,"available":96597741568,"internally_reserved":0,"allocated":2965504,"data_stored":19034637,"data_compressed":245582,"data_compressed_allocated":1536000,"data_compressed_original":18004675,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":913408,"data_stored":2756802,"data_compressed":48343,"data_compressed_allocated":249856,"data_compressed_original":2366082,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574858,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33156,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2064384,"data_stored":11363449,"data_compressed":184767,"data_compressed_allocated":954368,"data_compressed_original":10558082,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.443}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37296,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334544,"statfs":{"total":96636764160,"available":96598573056,"internally_reserved":0,"allocated":2134016,"data_stored":12774507,"data_compressed":178716,"data_compressed_allocated":987136,"data_compressed_original":11976787,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1744896,"data_stored":12612636,"data_compressed":178716,"data_compressed_allocated":987136,"data_compressed_original":11976787,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1630208,"data_stored":10616409,"data_compressed":176068,"data_compressed_allocated":917504,"data_compressed_original":9984642,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":524288,"data_stored":2547870,"data_compressed":48116,"data_compressed_allocated":245760,"data_compressed_original":2316930,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2605056,"data_stored":18882856,"data_compressed":245582,"data_compressed_allocated":1536000,"data_compressed_original":18004675,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2080768,"data_stored":17993014,"data_compressed":184806,"data_compressed_allocated":1122304,"data_compressed_original":17033287,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1806336,"data_stored":10143635,"data_compressed":165319,"data_compressed_allocated":872448,"data_compressed_original":9255635,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":712704,"data_stored":6351511,"data_compressed":47203,"data_compressed_allocated":315392,"data_compressed_original":5969052,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1761280,"data_stored":12065633,"data_compressed":162272,"data_compressed_allocated":847872,"data_compressed_original":11271578,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T04:56:44.134 INFO:tasks.thrashosds.thrasher:Setting 1.4 to [6, 7] 2024-09-18T04:56:44.134 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '1.4', '6', '7'] 2024-09-18T04:56:44.134 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 1.4 6 7 2024-09-18T04:56:44.177 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:44.177 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:44.186 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:56:44.213 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-18T04:56:44.213 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:44.211+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:44.274 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone15 from 0xd3405ce (overlap 0x1cdf0ea) to 0x2e31b1 2024-09-18T04:56:44.286 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:44.284+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:44.339 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:44.339 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:44.348 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:56:44.354 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-09-18T04:56:44.356 INFO:teuthology.orchestra.run.smithi002.stdout:1918 write 0xbe8355 thru 0xbed323 (0x4fcf bytes) 2024-09-18T04:56:44.387 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:44.385+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:44.409 INFO:teuthology.orchestra.run.smithi002.stdout:1919 write 0x8224c6d thru 0x82333a2 (0xe736 bytes) 2024-09-18T04:56:44.413 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:44.413 INFO:teuthology.orchestra.run.smithi002.stdout:1921 punch from 0x28e9669 to 0x28ec0d7, (0x2a6e bytes) 2024-09-18T04:56:44.413 INFO:teuthology.orchestra.run.smithi002.stdout:1922 punch from 0x204557c to 0x2045abb, (0x53f bytes) 2024-09-18T04:56:44.418 INFO:teuthology.orchestra.run.smithi002.stdout:1924 trunc from 0xd3405ce to 0x8fe9cb1 2024-09-18T04:56:44.425 INFO:teuthology.orchestra.run.smithi002.stdout:1925 write 0x7c49dcc thru 0x7c5742f (0xd664 bytes) 2024-09-18T04:56:44.439 INFO:teuthology.orchestra.run.smithi002.stdout:1926 write 0xd35b273 thru 0xd35b853 (0x5e1 bytes) 2024-09-18T04:56:44.440 INFO:teuthology.orchestra.run.smithi002.stdout:1927 read 0xd13e835 thru 0xd147d99 (0x9565 bytes) 2024-09-18T04:56:44.441 INFO:teuthology.orchestra.run.smithi002.stdout:1929 punch from 0x8d0e27d to 0x8d119ae, (0x3731 bytes) 2024-09-18T04:56:44.441 INFO:teuthology.orchestra.run.smithi002.stdout:1931 write 0x9fe3527 thru 0x9ff0d33 (0xd80d bytes) 2024-09-18T04:56:44.444 INFO:teuthology.orchestra.run.smithi002.stdout:1932 writesame 0x42f5246 thru 0x42fbae8 data_size 0x1(0x68a3 bytes) 2024-09-18T04:56:44.454 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:44.454 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:44.459 INFO:teuthology.orchestra.run.smithi002.stdout:1933 write 0xc66b7d6 thru 0xc673bd3 (0x83fe bytes) 2024-09-18T04:56:44.462 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:44.462 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:44.462 INFO:teuthology.orchestra.run.smithi002.stdout:1937 punch from 0xceb3360 to 0xceb3796, (0x436 bytes) 2024-09-18T04:56:44.462 INFO:teuthology.orchestra.run.smithi002.stdout:1939 writesame 0x8361426 thru 0x836ec11 data_size 0x1a(0xd7ec bytes) 2024-09-18T04:56:44.464 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:56:44.464 INFO:teuthology.orchestra.run.smithi002.stdout:1940 trunc from 0xd35b854 to 0xcf0b938 2024-09-18T04:56:44.473 INFO:teuthology.orchestra.run.smithi002.stdout:1941 write 0x95cc078 thru 0x95d2546 (0x64cf bytes) 2024-09-18T04:56:44.479 INFO:teuthology.orchestra.run.smithi002.stdout:1942 trunc from 0xcf0b938 to 0x69f27e7 2024-09-18T04:56:44.498 INFO:teuthology.orchestra.run.smithi002.stdout:1944 trunc from 0x69f27e7 to 0x2c40fcd 2024-09-18T04:56:44.507 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:44.505+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:44.509 INFO:teuthology.orchestra.run.smithi002.stdout:1946 punch from 0x28f7b14 to 0x290060a, (0x8af6 bytes) 2024-09-18T04:56:44.510 INFO:teuthology.orchestra.run.smithi002.stdout:1947 flatten 2024-09-18T04:56:44.543 INFO:teuthology.orchestra.run.smithi002.stdout:1951 trunc from 0x2c40fcd to 0xb7b4343 2024-09-18T04:56:44.548 INFO:teuthology.orchestra.run.smithi002.stdout:1952 read 0x6481772 thru 0x6481bbb (0x44a bytes) 2024-09-18T04:56:44.557 INFO:teuthology.orchestra.run.smithi002.stdout:1953 writesame 0xe07a7b5 thru 0xe0894b0 data_size 0x3a(0xecfc bytes) 2024-09-18T04:56:44.558 INFO:teuthology.orchestra.run.smithi002.stdout:1954 read 0x5f5e40f thru 0x5f68670 (0xa262 bytes) 2024-09-18T04:56:44.558 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:44.558 INFO:teuthology.orchestra.run.smithi002.stdout:1956 trunc from 0xe0894b1 to 0xd5712b0 2024-09-18T04:56:44.567 INFO:teuthology.orchestra.run.smithi002.stdout:1959 trunc from 0xd5712b0 to 0xb8f1ad4 2024-09-18T04:56:44.569 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:44.569 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:44.573 INFO:teuthology.orchestra.run.smithi002.stdout:1960 punch from 0xb307c9f to 0xb316878, (0xebd9 bytes) 2024-09-18T04:56:44.573 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:44.573 INFO:teuthology.orchestra.run.smithi002.stdout:1962 punch from 0xb222ea7 to 0xb22ce4d, (0x9fa6 bytes) 2024-09-18T04:56:44.573 INFO:teuthology.orchestra.run.smithi002.stdout:1963 punch from 0x2908809 to 0x29142f6, (0xbaed bytes) 2024-09-18T04:56:44.575 INFO:teuthology.orchestra.run.smithi002.stdout:1966 trunc from 0xb8f1ad4 to 0xdadf32d 2024-09-18T04:56:44.578 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:56:44.580 INFO:teuthology.orchestra.run.smithi002.stdout:1967 write 0x5d8093e thru 0x5d88a49 (0x810c bytes) 2024-09-18T04:56:44.588 INFO:teuthology.orchestra.run.smithi002.stdout:1969 write 0xe22f5a0 thru 0xe230f3c (0x199d bytes) 2024-09-18T04:56:44.590 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:44.590 INFO:teuthology.orchestra.run.smithi002.stdout:1971 write 0x6a61705 thru 0x6a64f45 (0x3841 bytes) 2024-09-18T04:56:44.592 INFO:teuthology.orchestra.run.smithi002.stdout:1972 punch from 0x895d5db to 0x8960cc3, (0x36e8 bytes) 2024-09-18T04:56:44.592 INFO:teuthology.orchestra.run.smithi002.stdout:1974 write 0x73bd8af thru 0x73c3ba6 (0x62f8 bytes) 2024-09-18T04:56:44.595 INFO:teuthology.orchestra.run.smithi002.stdout:1976 read 0x338772f thru 0x338c2a4 (0x4b76 bytes) 2024-09-18T04:56:44.595 INFO:teuthology.orchestra.run.smithi002.stdout:1977 write 0x1081941 thru 0x108ab08 (0x91c8 bytes) 2024-09-18T04:56:44.596 INFO:teuthology.orchestra.run.smithi002.stdout:1978 read 0xd1d1884 thru 0xd1d5455 (0x3bd2 bytes) 2024-09-18T04:56:44.597 INFO:teuthology.orchestra.run.smithi002.stdout:1979 punch from 0xa273d51 to 0xa27dbf6, (0x9ea5 bytes) 2024-09-18T04:56:44.603 INFO:teuthology.orchestra.run.smithi002.stdout:1980 trunc from 0xe230f3d to 0x8368168 2024-09-18T04:56:44.610 INFO:teuthology.orchestra.run.smithi002.stdout:1981 write 0x52edbcb thru 0x52f1e57 (0x428d bytes) 2024-09-18T04:56:44.611 INFO:teuthology.orchestra.run.smithi002.stdout:1982 punch from 0x1ef949f to 0x1f08926, (0xf487 bytes) 2024-09-18T04:56:44.618 INFO:teuthology.orchestra.run.smithi002.stdout:1983 trunc from 0x8368168 to 0xd891fec 2024-09-18T04:56:44.619 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:44.617+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:44.622 INFO:teuthology.orchestra.run.smithi002.stdout:1984 read 0x5d56593 thru 0x5d607b5 (0xa223 bytes) 2024-09-18T04:56:44.623 INFO:teuthology.orchestra.run.smithi002.stdout:1985 read 0x80bba1f thru 0x80c48d5 (0x8eb7 bytes) 2024-09-18T04:56:44.623 INFO:teuthology.orchestra.run.smithi002.stdout:1986 writesame 0xcce126a thru 0xcce7ddb data_size 0x11(0x6b72 bytes) 2024-09-18T04:56:44.641 INFO:teuthology.orchestra.run.smithi002.stdout:1987 trunc from 0xd891fec to 0x6f37bd 2024-09-18T04:56:44.662 INFO:teuthology.orchestra.run.smithi002.stdout:1988 trunc from 0x6f37bd to 0x56465a0 2024-09-18T04:56:44.668 INFO:teuthology.orchestra.run.smithi002.stdout:1989 write 0x1b6804b thru 0x1b7768d (0xf643 bytes) 2024-09-18T04:56:44.678 INFO:teuthology.orchestra.run.smithi002.stdout:1991 writesame 0x6767cf7 thru 0x676849c data_size 0x16(0x7a6 bytes) 2024-09-18T04:56:44.678 INFO:teuthology.orchestra.run.smithi002.stdout:1992 writesame 0x3e39399 thru 0x3e3cd34 data_size 0x6(0x399c bytes) 2024-09-18T04:56:44.687 INFO:teuthology.orchestra.run.smithi002.stdout:1993 write 0x7b500a7 thru 0x7b59f16 (0x9e70 bytes) 2024-09-18T04:56:44.689 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:44.687+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:44.696 INFO:teuthology.orchestra.run.smithi002.stdout:1994 write 0x9a4df5d thru 0x9a501e5 (0x2289 bytes) 2024-09-18T04:56:44.699 INFO:teuthology.orchestra.run.smithi002.stdout:1995 read 0x75fe27 thru 0x76af3a (0xb114 bytes) 2024-09-18T04:56:44.707 INFO:teuthology.orchestra.run.smithi002.stdout:1997 writesame 0xc23f033 thru 0xc248376 data_size 0x34(0x9344 bytes) 2024-09-18T04:56:44.707 INFO:teuthology.orchestra.run.smithi002.stdout:1998 punch from 0x31d2c82 to 0x31d663e, (0x39bc bytes) 2024-09-18T04:56:44.707 INFO:teuthology.orchestra.run.smithi002.stdout:1999 writesame 0xb02d0b thru 0xb0f557 data_size 0x1(0xc84d bytes) 2024-09-18T04:56:44.723 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:44.723 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:44.731 INFO:teuthology.orchestra.run.smithi002.stdout:2000 punch from 0x5f4a16f to 0x5f52611, (0x84a2 bytes) 2024-09-18T04:56:44.732 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:56:44.740 INFO:teuthology.orchestra.run.smithi002.stdout:2002 trunc from 0xc248377 to 0x46201d3 2024-09-18T04:56:44.749 INFO:teuthology.orchestra.run.smithi002.stdout:2003 punch from 0x2ace0f3 to 0x2ad014c, (0x2059 bytes) 2024-09-18T04:56:44.754 INFO:teuthology.orchestra.run.smithi002.stdout:2004 trunc from 0x46201d3 to 0x343e43 2024-09-18T04:56:44.764 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:44.780 INFO:teuthology.orchestra.run.smithi002.stdout:2007 writesame 0x8e493b0 thru 0x8e4d8d1 data_size 0x1(0x4522 bytes) 2024-09-18T04:56:44.780 INFO:teuthology.orchestra.run.smithi002.stdout:2008 clone 17 order 22 su 16384 sc 3 2024-09-18T04:56:44.790 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:44.788+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:44.833 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:44.833 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:44.842 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:56:44.872 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:56:44.917 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:44.915+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:44.945 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:44.945 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:44.953 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:56:44.992 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:44.990+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:44.996 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:56:44.996 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:56:45.013 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:56:45.013 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:56:45.028 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:56:45.028 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:56:45.044 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:56:45.044 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:56:45.055 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:45.056 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:45.061 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:56:45.061 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:56:45.065 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:56:45.078 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:56:45.078 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:56:45.093 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:45.091+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:45.094 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:56:45.094 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:56:45.111 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:56:45.111 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:56:45.169 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:45.169 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:45.169 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:56:45.169 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:56:45.170 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:45.177 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:56:45.214 INFO:teuthology.orchestra.run.smithi002.stderr:set 1.4 pg_upmap mapping to [6,7] 2024-09-18T04:56:45.221 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:45.219+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:45.285 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:45.286 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:56:45.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:56:45.286 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:56:45.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.286 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:56:45.286 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:56:45.286 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.889075837999997, 2024-09-18T04:56:45.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037718511000000003, 2024-09-18T04:56:45.286 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:45.286 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:45.287 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:45.287 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:56:45.287 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:56:45.287 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-18T04:56:45.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.287 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:45.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:45.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:56:45.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:45.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:56:45.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:45.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393125+0000", 2024-09-18T04:56:45.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:45.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:45.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393133+0000", 2024-09-18T04:56:45.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.225e-06 2024-09-18T04:56:45.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:45.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393144+0000", 2024-09-18T04:56:45.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1279e-05 2024-09-18T04:56:45.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393153+0000", 2024-09-18T04:56:45.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4540000000000007e-06 2024-09-18T04:56:45.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:45.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393203+0000", 2024-09-18T04:56:45.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9963000000000003e-05 2024-09-18T04:56:45.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:56:45.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393210+0000", 2024-09-18T04:56:45.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5689999999999999e-06 2024-09-18T04:56:45.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:45.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427399+0000", 2024-09-18T04:56:45.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034188784999999999 2024-09-18T04:56:45.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:45.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427532+0000", 2024-09-18T04:56:45.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013264700000000001 2024-09-18T04:56:45.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:45.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427685+0000", 2024-09-18T04:56:45.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015300599999999999 2024-09-18T04:56:45.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:45.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.428520+0000", 2024-09-18T04:56:45.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00083502499999999998 2024-09-18T04:56:45.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:45.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430772+0000", 2024-09-18T04:56:45.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022518880000000001 2024-09-18T04:56:45.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:45.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430817+0000", 2024-09-18T04:56:45.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5077000000000002e-05 2024-09-18T04:56:45.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:45.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430845+0000", 2024-09-18T04:56:45.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8036999999999999e-05 2024-09-18T04:56:45.295 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.295 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:45.295 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.295 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:805 3.15 3:ab221474:::rbd_data.11ac33624de9.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-09-18T04:56:45.295 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:27.333817+0000", 2024-09-18T04:56:45.295 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.948384835999999, 2024-09-18T04:56:45.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.017602593999999999, 2024-09-18T04:56:45.295 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:45.295 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:45.296 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:45.296 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:45.296 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.296 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 805 2024-09-18T04:56:45.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.296 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:45.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:45.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.333817+0000", 2024-09-18T04:56:45.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:45.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.333817+0000", 2024-09-18T04:56:45.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:45.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.333816+0000", 2024-09-18T04:56:45.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:45.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:45.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.333821+0000", 2024-09-18T04:56:45.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3079999999999998e-06 2024-09-18T04:56:45.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:45.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.333832+0000", 2024-09-18T04:56:45.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0139000000000001e-05 2024-09-18T04:56:45.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.333855+0000", 2024-09-18T04:56:45.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3210999999999999e-05 2024-09-18T04:56:45.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:45.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.333903+0000", 2024-09-18T04:56:45.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8613999999999999e-05 2024-09-18T04:56:45.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:45.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.334043+0000", 2024-09-18T04:56:45.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013937199999999999 2024-09-18T04:56:45.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 3", 2024-09-18T04:56:45.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.337213+0000", 2024-09-18T04:56:45.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003170595 2024-09-18T04:56:45.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:45.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.351345+0000", 2024-09-18T04:56:45.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014131539 2024-09-18T04:56:45.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:45.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.351356+0000", 2024-09-18T04:56:45.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0703999999999999e-05 2024-09-18T04:56:45.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:45.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.351392+0000", 2024-09-18T04:56:45.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6758999999999998e-05 2024-09-18T04:56:45.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:45.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:27.351420+0000", 2024-09-18T04:56:45.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7582000000000001e-05 2024-09-18T04:56:45.302 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.302 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:45.302 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.302 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:5168 3.11 3:8c2284b2:::rbd_data.11ac14fc1d07.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 10863232~40683] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:45.302 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.504653+0000", 2024-09-18T04:56:45.302 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 12.77754869, 2024-09-18T04:56:45.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.027990498999999999, 2024-09-18T04:56:45.302 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:45.302 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:45.302 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:45.303 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:45.303 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.303 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 5168 2024-09-18T04:56:45.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.303 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:45.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:45.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.504653+0000", 2024-09-18T04:56:45.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:45.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.504653+0000", 2024-09-18T04:56:45.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:45.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.504653+0000", 2024-09-18T04:56:45.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:45.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:45.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.516093+0000", 2024-09-18T04:56:45.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011440120999999999 2024-09-18T04:56:45.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:45.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.516116+0000", 2024-09-18T04:56:45.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3153999999999999e-05 2024-09-18T04:56:45.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.516127+0000", 2024-09-18T04:56:45.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1056999999999999e-05 2024-09-18T04:56:45.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:45.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.516210+0000", 2024-09-18T04:56:45.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3128999999999996e-05 2024-09-18T04:56:45.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:45.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.516339+0000", 2024-09-18T04:56:45.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012917000000000001 2024-09-18T04:56:45.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:45.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.516794+0000", 2024-09-18T04:56:45.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00045514500000000001 2024-09-18T04:56:45.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:45.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.528090+0000", 2024-09-18T04:56:45.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011295423000000001 2024-09-18T04:56:45.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:45.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.532575+0000", 2024-09-18T04:56:45.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0044854220000000002 2024-09-18T04:56:45.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:45.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.532613+0000", 2024-09-18T04:56:45.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7333000000000003e-05 2024-09-18T04:56:45.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:45.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.532644+0000", 2024-09-18T04:56:45.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.1282999999999998e-05 2024-09-18T04:56:45.309 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.309 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:45.309 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.309 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:22467 3.11 3:8dcd132e:::rbd_data.11ac82b416ac.00000000000000c1:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-09-18T04:56:45.309 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:37.484119+0000", 2024-09-18T04:56:45.309 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.7980827169999998, 2024-09-18T04:56:45.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023250593, 2024-09-18T04:56:45.309 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:45.309 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:45.310 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:45.310 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:45.310 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.310 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 22467 2024-09-18T04:56:45.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.310 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:45.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:45.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.484119+0000", 2024-09-18T04:56:45.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:45.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.484119+0000", 2024-09-18T04:56:45.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:45.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.484118+0000", 2024-09-18T04:56:45.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:45.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:45.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.484125+0000", 2024-09-18T04:56:45.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5860000000000001e-06 2024-09-18T04:56:45.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:45.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.484136+0000", 2024-09-18T04:56:45.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1164e-05 2024-09-18T04:56:45.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.484141+0000", 2024-09-18T04:56:45.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6659999999999996e-06 2024-09-18T04:56:45.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:45.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.484177+0000", 2024-09-18T04:56:45.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5139999999999999e-05 2024-09-18T04:56:45.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:45.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.484263+0000", 2024-09-18T04:56:45.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6811999999999995e-05 2024-09-18T04:56:45.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:56:45.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.487896+0000", 2024-09-18T04:56:45.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0036321769999999999 2024-09-18T04:56:45.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:45.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.507245+0000", 2024-09-18T04:56:45.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.019349094000000001 2024-09-18T04:56:45.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:45.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.507289+0000", 2024-09-18T04:56:45.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4168999999999997e-05 2024-09-18T04:56:45.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:45.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.507327+0000", 2024-09-18T04:56:45.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7811000000000001e-05 2024-09-18T04:56:45.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:45.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:37.507370+0000", 2024-09-18T04:56:45.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3331999999999998e-05 2024-09-18T04:56:45.317 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.317 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:45.317 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.317 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28690 3.11 3.41a894f1 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:45.318 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840079+0000", 2024-09-18T04:56:45.318 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4421236349999997, 2024-09-18T04:56:45.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015023241999999999, 2024-09-18T04:56:45.318 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:45.318 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:45.318 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:45.318 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:45.318 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.318 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28690 2024-09-18T04:56:45.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.318 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:45.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:45.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840079+0000", 2024-09-18T04:56:45.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:45.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840079+0000", 2024-09-18T04:56:45.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:45.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840078+0000", 2024-09-18T04:56:45.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:45.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:45.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840080+0000", 2024-09-18T04:56:45.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2249999999999999e-06 2024-09-18T04:56:45.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:45.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840083+0000", 2024-09-18T04:56:45.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8039999999999999e-06 2024-09-18T04:56:45.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840086+0000", 2024-09-18T04:56:45.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4420000000000001e-06 2024-09-18T04:56:45.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:45.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855091+0000", 2024-09-18T04:56:45.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015005136000000001 2024-09-18T04:56:45.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:45.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855102+0000", 2024-09-18T04:56:45.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0839000000000001e-05 2024-09-18T04:56:45.323 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.323 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:45.323 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.323 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28713 3.11 3.41a894f1 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:45.323 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840088+0000", 2024-09-18T04:56:45.323 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4421141039999998, 2024-09-18T04:56:45.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015057488000000001, 2024-09-18T04:56:45.324 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:45.324 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:45.324 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:45.324 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:45.324 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.324 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28713 2024-09-18T04:56:45.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.324 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:45.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:45.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840088+0000", 2024-09-18T04:56:45.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:45.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840088+0000", 2024-09-18T04:56:45.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:45.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840088+0000", 2024-09-18T04:56:45.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:45.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840089+0000", 2024-09-18T04:56:45.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2100000000000001e-06 2024-09-18T04:56:45.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:45.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840092+0000", 2024-09-18T04:56:45.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9289999999999998e-06 2024-09-18T04:56:45.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840094+0000", 2024-09-18T04:56:45.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5110000000000002e-06 2024-09-18T04:56:45.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:45.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855135+0000", 2024-09-18T04:56:45.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015040820999999999 2024-09-18T04:56:45.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:45.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855146+0000", 2024-09-18T04:56:45.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0312999999999999e-05 2024-09-18T04:56:45.327 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.328 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:45.328 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.328 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28714 3.11 3.5e1e1ab1 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:45.328 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840096+0000", 2024-09-18T04:56:45.329 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4421056649999997, 2024-09-18T04:56:45.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015095044, 2024-09-18T04:56:45.329 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:45.329 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:45.329 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:45.329 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:45.329 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.329 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28714 2024-09-18T04:56:45.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.329 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:45.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:45.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840096+0000", 2024-09-18T04:56:45.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:45.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840096+0000", 2024-09-18T04:56:45.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:45.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840096+0000", 2024-09-18T04:56:45.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:56:45.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:45.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840098+0000", 2024-09-18T04:56:45.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.299e-06 2024-09-18T04:56:45.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:45.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840101+0000", 2024-09-18T04:56:45.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6449999999999998e-06 2024-09-18T04:56:45.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840104+0000", 2024-09-18T04:56:45.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.542e-06 2024-09-18T04:56:45.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:45.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855180+0000", 2024-09-18T04:56:45.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015076463999999999 2024-09-18T04:56:45.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:45.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855192+0000", 2024-09-18T04:56:45.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1314e-05 2024-09-18T04:56:45.334 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.334 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:45.334 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.334 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28750 3.11 3.41a894f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:45.334 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840281+0000", 2024-09-18T04:56:45.334 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4419211140000003, 2024-09-18T04:56:45.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015023635, 2024-09-18T04:56:45.334 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:45.334 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:45.334 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:45.334 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:45.334 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.335 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28750 2024-09-18T04:56:45.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.335 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:45.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:45.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840281+0000", 2024-09-18T04:56:45.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:45.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840281+0000", 2024-09-18T04:56:45.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:45.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840281+0000", 2024-09-18T04:56:45.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:45.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840287+0000", 2024-09-18T04:56:45.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2040000000000002e-06 2024-09-18T04:56:45.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:45.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840293+0000", 2024-09-18T04:56:45.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.19e-06 2024-09-18T04:56:45.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840298+0000", 2024-09-18T04:56:45.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8219999999999997e-06 2024-09-18T04:56:45.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:45.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855295+0000", 2024-09-18T04:56:45.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.014997323999999999 2024-09-18T04:56:45.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:45.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855305+0000", 2024-09-18T04:56:45.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4800000000000007e-06 2024-09-18T04:56:45.338 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.338 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:45.338 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.338 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28751 3.11 3.41a894f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:45.338 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840303+0000", 2024-09-18T04:56:45.338 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4418994810000001, 2024-09-18T04:56:45.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015059965999999999, 2024-09-18T04:56:45.338 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:45.338 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:45.339 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:45.339 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:45.339 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.339 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28751 2024-09-18T04:56:45.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.339 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:45.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:45.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840303+0000", 2024-09-18T04:56:45.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:45.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840303+0000", 2024-09-18T04:56:45.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:45.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840302+0000", 2024-09-18T04:56:45.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:45.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840304+0000", 2024-09-18T04:56:45.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1330000000000002e-06 2024-09-18T04:56:45.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:45.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840310+0000", 2024-09-18T04:56:45.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1850000000000001e-06 2024-09-18T04:56:45.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840316+0000", 2024-09-18T04:56:45.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2100000000000001e-06 2024-09-18T04:56:45.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:45.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855342+0000", 2024-09-18T04:56:45.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015026214 2024-09-18T04:56:45.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:45.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855363+0000", 2024-09-18T04:56:45.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0704e-05 2024-09-18T04:56:45.342 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.342 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:45.342 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.343 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28752 3.11 3.41a894f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:45.343 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840320+0000", 2024-09-18T04:56:45.343 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4418818990000002, 2024-09-18T04:56:45.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.01507758, 2024-09-18T04:56:45.343 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:45.343 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:45.343 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:45.343 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:45.343 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.343 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28752 2024-09-18T04:56:45.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.343 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:45.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:45.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840320+0000", 2024-09-18T04:56:45.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:45.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840320+0000", 2024-09-18T04:56:45.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:45.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840320+0000", 2024-09-18T04:56:45.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:45.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840322+0000", 2024-09-18T04:56:45.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0219999999999999e-06 2024-09-18T04:56:45.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:45.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840328+0000", 2024-09-18T04:56:45.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8490000000000002e-06 2024-09-18T04:56:45.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840333+0000", 2024-09-18T04:56:45.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.814e-06 2024-09-18T04:56:45.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:45.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855388+0000", 2024-09-18T04:56:45.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015055681 2024-09-18T04:56:45.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:45.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855398+0000", 2024-09-18T04:56:45.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5710000000000004e-06 2024-09-18T04:56:45.346 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.346 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:45.347 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.347 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28756 3.11 3.41a894f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:45.347 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840506+0000", 2024-09-18T04:56:45.347 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4416960850000002, 2024-09-18T04:56:45.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015052385, 2024-09-18T04:56:45.347 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:45.347 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:45.347 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:45.347 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:45.347 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.347 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28756 2024-09-18T04:56:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840506+0000", 2024-09-18T04:56:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840506+0000", 2024-09-18T04:56:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840506+0000", 2024-09-18T04:56:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840509+0000", 2024-09-18T04:56:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.304e-06 2024-09-18T04:56:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840515+0000", 2024-09-18T04:56:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8660000000000002e-06 2024-09-18T04:56:45.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840519+0000", 2024-09-18T04:56:45.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2390000000000004e-06 2024-09-18T04:56:45.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:45.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855549+0000", 2024-09-18T04:56:45.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015029847000000001 2024-09-18T04:56:45.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:45.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855558+0000", 2024-09-18T04:56:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5719999999999992e-06 2024-09-18T04:56:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28812 3.11 3.41a894f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840523+0000", 2024-09-18T04:56:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.441679369, 2024-09-18T04:56:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015078464999999999, 2024-09-18T04:56:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:45.351 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:45.352 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:45.352 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.352 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28812 2024-09-18T04:56:45.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.352 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:45.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:45.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840523+0000", 2024-09-18T04:56:45.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:45.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840523+0000", 2024-09-18T04:56:45.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:45.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840522+0000", 2024-09-18T04:56:45.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:45.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840524+0000", 2024-09-18T04:56:45.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.829e-06 2024-09-18T04:56:45.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:45.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840530+0000", 2024-09-18T04:56:45.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7350000000000001e-06 2024-09-18T04:56:45.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840535+0000", 2024-09-18T04:56:45.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0069999999999997e-06 2024-09-18T04:56:45.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:45.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855592+0000", 2024-09-18T04:56:45.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015056704000000001 2024-09-18T04:56:45.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:45.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855601+0000", 2024-09-18T04:56:45.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5289999999999997e-06 2024-09-18T04:56:45.357 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.357 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:45.357 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.358 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28813 3.11 3.41a894f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:45.358 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840539+0000", 2024-09-18T04:56:45.358 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.441663234, 2024-09-18T04:56:45.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015088677, 2024-09-18T04:56:45.358 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:45.358 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:45.358 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:45.358 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:45.358 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.359 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28813 2024-09-18T04:56:45.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.359 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:45.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:45.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840539+0000", 2024-09-18T04:56:45.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:45.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840539+0000", 2024-09-18T04:56:45.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:45.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840539+0000", 2024-09-18T04:56:45.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:45.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840541+0000", 2024-09-18T04:56:45.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9489999999999999e-06 2024-09-18T04:56:45.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:45.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840546+0000", 2024-09-18T04:56:45.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5040000000000002e-06 2024-09-18T04:56:45.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840550+0000", 2024-09-18T04:56:45.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1819999999999999e-06 2024-09-18T04:56:45.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:45.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855619+0000", 2024-09-18T04:56:45.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015068712 2024-09-18T04:56:45.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:45.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855628+0000", 2024-09-18T04:56:45.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6449999999999994e-06 2024-09-18T04:56:45.362 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.362 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:45.362 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.362 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28814 3.11 3.41a894f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:45.362 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840555+0000", 2024-09-18T04:56:45.362 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4416471460000002, 2024-09-18T04:56:45.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015115778, 2024-09-18T04:56:45.363 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:45.363 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:45.363 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:45.363 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:45.363 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.363 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28814 2024-09-18T04:56:45.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.363 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:45.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:45.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840555+0000", 2024-09-18T04:56:45.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:45.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840555+0000", 2024-09-18T04:56:45.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:45.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840555+0000", 2024-09-18T04:56:45.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:45.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840557+0000", 2024-09-18T04:56:45.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9010000000000001e-06 2024-09-18T04:56:45.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:45.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840562+0000", 2024-09-18T04:56:45.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.451e-06 2024-09-18T04:56:45.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840566+0000", 2024-09-18T04:56:45.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1189999999999999e-06 2024-09-18T04:56:45.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:45.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855662+0000", 2024-09-18T04:56:45.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015095411 2024-09-18T04:56:45.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:45.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855671+0000", 2024-09-18T04:56:45.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2380000000000003e-06 2024-09-18T04:56:45.368 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.368 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:45.368 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.369 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28815 3.11 3.41a894f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:45.369 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840570+0000", 2024-09-18T04:56:45.369 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4416321810000001, 2024-09-18T04:56:45.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015143973, 2024-09-18T04:56:45.369 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:45.369 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:45.369 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:45.369 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:45.369 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.369 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28815 2024-09-18T04:56:45.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.370 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:45.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:45.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840570+0000", 2024-09-18T04:56:45.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:45.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840570+0000", 2024-09-18T04:56:45.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:45.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840570+0000", 2024-09-18T04:56:45.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:45.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840572+0000", 2024-09-18T04:56:45.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.012e-06 2024-09-18T04:56:45.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:45.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840577+0000", 2024-09-18T04:56:45.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8699999999999997e-06 2024-09-18T04:56:45.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840582+0000", 2024-09-18T04:56:45.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3970000000000004e-06 2024-09-18T04:56:45.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:45.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855705+0000", 2024-09-18T04:56:45.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015122834999999999 2024-09-18T04:56:45.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:45.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855714+0000", 2024-09-18T04:56:45.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2250000000000006e-06 2024-09-18T04:56:45.373 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.373 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:45.373 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.373 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28816 3.11 3.41a894f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:45.373 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840586+0000", 2024-09-18T04:56:45.373 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4416160619999996, 2024-09-18T04:56:45.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015144272, 2024-09-18T04:56:45.374 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:45.374 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:45.374 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:45.374 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:45.374 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.374 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28816 2024-09-18T04:56:45.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.374 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:45.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:45.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840586+0000", 2024-09-18T04:56:45.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:45.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840586+0000", 2024-09-18T04:56:45.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:45.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840586+0000", 2024-09-18T04:56:45.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:45.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840588+0000", 2024-09-18T04:56:45.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.153e-06 2024-09-18T04:56:45.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:45.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840593+0000", 2024-09-18T04:56:45.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6579999999999999e-06 2024-09-18T04:56:45.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840598+0000", 2024-09-18T04:56:45.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.25e-06 2024-09-18T04:56:45.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:45.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855726+0000", 2024-09-18T04:56:45.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015128206 2024-09-18T04:56:45.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:45.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855730+0000", 2024-09-18T04:56:45.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4150000000000001e-06 2024-09-18T04:56:45.377 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.377 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:45.377 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.378 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28817 3.11 3.41a894f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:45.378 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840602+0000", 2024-09-18T04:56:45.378 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.441600566, 2024-09-18T04:56:45.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015152196, 2024-09-18T04:56:45.378 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:45.378 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:45.378 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:45.378 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:45.378 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.378 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28817 2024-09-18T04:56:45.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.378 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:45.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:45.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840602+0000", 2024-09-18T04:56:45.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:45.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840602+0000", 2024-09-18T04:56:45.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:45.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840601+0000", 2024-09-18T04:56:45.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:45.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840608+0000", 2024-09-18T04:56:45.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4070000000000004e-06 2024-09-18T04:56:45.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:45.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840614+0000", 2024-09-18T04:56:45.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7769999999999999e-06 2024-09-18T04:56:45.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840619+0000", 2024-09-18T04:56:45.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3989999999999997e-06 2024-09-18T04:56:45.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:45.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855745+0000", 2024-09-18T04:56:45.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015126539 2024-09-18T04:56:45.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:45.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855754+0000", 2024-09-18T04:56:45.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5699999999999993e-06 2024-09-18T04:56:45.381 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.382 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:45.382 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.382 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28818 3.11 3.41a894f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:45.382 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840637+0000", 2024-09-18T04:56:45.382 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4415654919999996, 2024-09-18T04:56:45.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015132655, 2024-09-18T04:56:45.382 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:45.382 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:45.382 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:45.382 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:45.382 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.382 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28818 2024-09-18T04:56:45.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.383 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:45.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:45.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840637+0000", 2024-09-18T04:56:45.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:45.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840637+0000", 2024-09-18T04:56:45.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:45.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840636+0000", 2024-09-18T04:56:45.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:45.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:45.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840640+0000", 2024-09-18T04:56:45.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6090000000000003e-06 2024-09-18T04:56:45.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:45.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840649+0000", 2024-09-18T04:56:45.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6719999999999993e-06 2024-09-18T04:56:45.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840656+0000", 2024-09-18T04:56:45.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3509999999999996e-06 2024-09-18T04:56:45.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:45.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855765+0000", 2024-09-18T04:56:45.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015108053999999999 2024-09-18T04:56:45.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:45.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855769+0000", 2024-09-18T04:56:45.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7759999999999997e-06 2024-09-18T04:56:45.386 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.386 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:45.386 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.386 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28819 3.11 3.41a894f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:45.386 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840662+0000", 2024-09-18T04:56:45.386 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4415401299999999, 2024-09-18T04:56:45.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015142542, 2024-09-18T04:56:45.386 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:45.386 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:45.386 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:45.386 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:45.386 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.387 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28819 2024-09-18T04:56:45.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.387 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:45.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:45.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840662+0000", 2024-09-18T04:56:45.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:45.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840662+0000", 2024-09-18T04:56:45.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:45.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840661+0000", 2024-09-18T04:56:45.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:45.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840664+0000", 2024-09-18T04:56:45.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5809999999999999e-06 2024-09-18T04:56:45.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:45.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840671+0000", 2024-09-18T04:56:45.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5479999999999998e-06 2024-09-18T04:56:45.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840676+0000", 2024-09-18T04:56:45.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1939999999999997e-06 2024-09-18T04:56:45.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:45.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855796+0000", 2024-09-18T04:56:45.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015119850000000001 2024-09-18T04:56:45.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:45.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855805+0000", 2024-09-18T04:56:45.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8959999999999992e-06 2024-09-18T04:56:45.390 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.390 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:45.390 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.390 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:28820 3.11 3.41a894f1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:45.390 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.840682+0000", 2024-09-18T04:56:45.390 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.4415206139999999, 2024-09-18T04:56:45.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015144426000000001, 2024-09-18T04:56:45.390 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:45.390 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:45.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:45.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:45.391 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.391 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 28820 2024-09-18T04:56:45.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.391 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:45.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:45.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840682+0000", 2024-09-18T04:56:45.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:45.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840682+0000", 2024-09-18T04:56:45.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:45.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:45.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840681+0000", 2024-09-18T04:56:45.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:45.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840683+0000", 2024-09-18T04:56:45.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2249999999999999e-06 2024-09-18T04:56:45.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:45.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840690+0000", 2024-09-18T04:56:45.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2820000000000002e-06 2024-09-18T04:56:45.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.840698+0000", 2024-09-18T04:56:45.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2479999999999996e-06 2024-09-18T04:56:45.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:45.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855822+0000", 2024-09-18T04:56:45.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.015123763 2024-09-18T04:56:45.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:45.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:45.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:45.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.855826+0000", 2024-09-18T04:56:45.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2880000000000003e-06 2024-09-18T04:56:45.395 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.395 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:45.396 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.396 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:45.396 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:45.396 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:45.400 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-09-18T04:56:45.400 INFO:teuthology.orchestra.run.smithi002.stdout:2009 punch from 0x8bd490e to 0x8be3595, (0xec87 bytes) 2024-09-18T04:56:45.400 INFO:teuthology.orchestra.run.smithi002.stdout:2010 read 0x1cdc3a6 thru 0x1ce8dcd (0xca28 bytes) 2024-09-18T04:56:45.400 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:45.400 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:45.401 INFO:teuthology.orchestra.run.smithi002.stdout:2013 write 0x2bd2465 thru 0x2bd5a56 (0x35f2 bytes) 2024-09-18T04:56:45.401 INFO:teuthology.orchestra.run.smithi002.stdout:2014 punch from 0x1372517 to 0x137f472, (0xcf5b bytes) 2024-09-18T04:56:45.401 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:45.401 INFO:teuthology.orchestra.run.smithi002.stdout:2017 writesame 0x8d3eb0c thru 0x8d40c03 data_size 0x14(0x20f8 bytes) 2024-09-18T04:56:45.402 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:56:45.402 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:45.395+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:45.403 INFO:teuthology.orchestra.run.smithi002.stdout:2018 writesame 0x314a08e thru 0x314a9fd data_size 0x8(0x970 bytes) 2024-09-18T04:56:45.414 INFO:teuthology.orchestra.run.smithi002.stdout:2019 write 0xb08feb3 thru 0xb096af6 (0x6c44 bytes) 2024-09-18T04:56:45.419 INFO:teuthology.orchestra.run.smithi002.stdout:2020 clone 18 order 21 su 8192 sc 7 2024-09-18T04:56:45.499 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:45.497+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:45.512 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:45.512 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:45.512 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:56:45.513 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:45.519 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:56:45.600 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:45.597+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:45.666 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:56:45.667 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:56:45.667 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:45.675 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:56:45.704 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:45.701+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:45.784 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:45.784 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:56:45.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:56:45.784 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:56:45.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.784 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9314 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1581056~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:45.784 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767102+0000", 2024-09-18T04:56:45.784 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.013744792000001, 2024-09-18T04:56:45.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043648235000000001, 2024-09-18T04:56:45.785 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:45.785 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:45.785 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:45.785 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:45.785 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.785 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9314 2024-09-18T04:56:45.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.785 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:45.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:45.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767102+0000", 2024-09-18T04:56:45.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767102+0000", 2024-09-18T04:56:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767101+0000", 2024-09-18T04:56:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767104+0000", 2024-09-18T04:56:45.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3920000000000001e-06 2024-09-18T04:56:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767110+0000", 2024-09-18T04:56:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0480000000000004e-06 2024-09-18T04:56:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767114+0000", 2024-09-18T04:56:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6639999999999997e-06 2024-09-18T04:56:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810579+0000", 2024-09-18T04:56:45.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043464931999999998 2024-09-18T04:56:45.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:45.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810667+0000", 2024-09-18T04:56:45.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7356999999999997e-05 2024-09-18T04:56:45.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:45.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810750+0000", 2024-09-18T04:56:45.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3374999999999994e-05 2024-09-18T04:56:45.788 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.788 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9315 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1630208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767118+0000", 2024-09-18T04:56:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.013728114999999, 2024-09-18T04:56:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043838369000000002, 2024-09-18T04:56:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9315 2024-09-18T04:56:45.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767118+0000", 2024-09-18T04:56:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767118+0000", 2024-09-18T04:56:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767118+0000", 2024-09-18T04:56:45.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767126+0000", 2024-09-18T04:56:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6559999999999992e-06 2024-09-18T04:56:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767132+0000", 2024-09-18T04:56:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3350000000000003e-06 2024-09-18T04:56:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767138+0000", 2024-09-18T04:56:45.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5849999999999999e-06 2024-09-18T04:56:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810792+0000", 2024-09-18T04:56:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043654485999999999 2024-09-18T04:56:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810879+0000", 2024-09-18T04:56:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6691000000000001e-05 2024-09-18T04:56:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:45.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810957+0000", 2024-09-18T04:56:45.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7996000000000003e-05 2024-09-18T04:56:45.793 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.793 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:45.793 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.793 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9316 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:45.793 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:56:45.793 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.013704369999999, 2024-09-18T04:56:45.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044027208999999998, 2024-09-18T04:56:45.793 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:45.793 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:45.794 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:45.794 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:45.794 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.794 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9316 2024-09-18T04:56:45.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.794 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:45.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:56:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:56:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:56:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:45.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767144+0000", 2024-09-18T04:56:45.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3070000000000001e-06 2024-09-18T04:56:45.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:45.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767150+0000", 2024-09-18T04:56:45.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1700000000000002e-06 2024-09-18T04:56:45.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767163+0000", 2024-09-18T04:56:45.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2670999999999999e-05 2024-09-18T04:56:45.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:45.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810995+0000", 2024-09-18T04:56:45.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043832310999999999 2024-09-18T04:56:45.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:45.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811087+0000", 2024-09-18T04:56:45.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1345e-05 2024-09-18T04:56:45.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811169+0000", 2024-09-18T04:56:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2848000000000004e-05 2024-09-18T04:56:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9317 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1728512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:56:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.013678734999999, 2024-09-18T04:56:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044190170000000001, 2024-09-18T04:56:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:45.798 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:45.799 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:45.799 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:45.799 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.799 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9317 2024-09-18T04:56:45.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.799 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:45.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:45.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:56:45.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:45.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:56:45.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:45.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767167+0000", 2024-09-18T04:56:45.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:45.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767170+0000", 2024-09-18T04:56:45.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7259999999999998e-06 2024-09-18T04:56:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767176+0000", 2024-09-18T04:56:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1460000000000001e-06 2024-09-18T04:56:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767182+0000", 2024-09-18T04:56:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1020000000000002e-06 2024-09-18T04:56:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811204+0000", 2024-09-18T04:56:45.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044022039999999998 2024-09-18T04:56:45.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:45.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811286+0000", 2024-09-18T04:56:45.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1623000000000001e-05 2024-09-18T04:56:45.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:45.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811358+0000", 2024-09-18T04:56:45.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2125000000000005e-05 2024-09-18T04:56:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9318 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1777664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:56:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.01366019, 2024-09-18T04:56:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044380181999999997, 2024-09-18T04:56:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:45.803 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9318 2024-09-18T04:56:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:56:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:56:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:56:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767189+0000", 2024-09-18T04:56:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5909999999999998e-06 2024-09-18T04:56:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:45.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767194+0000", 2024-09-18T04:56:45.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7690000000000002e-06 2024-09-18T04:56:45.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767199+0000", 2024-09-18T04:56:45.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189999999999999e-06 2024-09-18T04:56:45.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:45.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811394+0000", 2024-09-18T04:56:45.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044195350000000001 2024-09-18T04:56:45.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:45.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811483+0000", 2024-09-18T04:56:45.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8282999999999995e-05 2024-09-18T04:56:45.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811567+0000", 2024-09-18T04:56:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3855000000000003e-05 2024-09-18T04:56:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9322 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1941504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.768849+0000", 2024-09-18T04:56:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.011997797999999, 2024-09-18T04:56:45.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043526553000000003, 2024-09-18T04:56:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9322 2024-09-18T04:56:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768849+0000", 2024-09-18T04:56:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768849+0000", 2024-09-18T04:56:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768848+0000", 2024-09-18T04:56:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768851+0000", 2024-09-18T04:56:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2079999999999999e-06 2024-09-18T04:56:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768857+0000", 2024-09-18T04:56:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.942e-06 2024-09-18T04:56:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768867+0000", 2024-09-18T04:56:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0711e-05 2024-09-18T04:56:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812206+0000", 2024-09-18T04:56:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043338347999999999 2024-09-18T04:56:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812293+0000", 2024-09-18T04:56:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7435000000000006e-05 2024-09-18T04:56:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812375+0000", 2024-09-18T04:56:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2279000000000003e-05 2024-09-18T04:56:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.813 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9323 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1990656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.768873+0000", 2024-09-18T04:56:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.011974012, 2024-09-18T04:56:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043695031000000002, 2024-09-18T04:56:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.814 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9323 2024-09-18T04:56:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768873+0000", 2024-09-18T04:56:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768873+0000", 2024-09-18T04:56:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768872+0000", 2024-09-18T04:56:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768880+0000", 2024-09-18T04:56:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4179999999999997e-06 2024-09-18T04:56:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:45.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768890+0000", 2024-09-18T04:56:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5310000000000007e-06 2024-09-18T04:56:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768895+0000", 2024-09-18T04:56:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.186e-06 2024-09-18T04:56:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812412+0000", 2024-09-18T04:56:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043516525 2024-09-18T04:56:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812496+0000", 2024-09-18T04:56:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4039e-05 2024-09-18T04:56:45.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812568+0000", 2024-09-18T04:56:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1988000000000003e-05 2024-09-18T04:56:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.823 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9324 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 2039808~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:45.824 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.768900+0000", 2024-09-18T04:56:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.011946681, 2024-09-18T04:56:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043885234000000002, 2024-09-18T04:56:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9324 2024-09-18T04:56:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768900+0000", 2024-09-18T04:56:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:45.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768900+0000", 2024-09-18T04:56:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768900+0000", 2024-09-18T04:56:45.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768902+0000", 2024-09-18T04:56:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3070000000000001e-06 2024-09-18T04:56:45.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768908+0000", 2024-09-18T04:56:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2670000000000003e-06 2024-09-18T04:56:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768913+0000", 2024-09-18T04:56:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8770000000000001e-06 2024-09-18T04:56:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:45.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812606+0000", 2024-09-18T04:56:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043692867000000003 2024-09-18T04:56:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:45.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812695+0000", 2024-09-18T04:56:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9060000000000005e-05 2024-09-18T04:56:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812785+0000", 2024-09-18T04:56:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0242999999999996e-05 2024-09-18T04:56:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.833 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9325 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:56:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.011929651999999, 2024-09-18T04:56:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044063989999999997, 2024-09-18T04:56:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9325 2024-09-18T04:56:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.834 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:56:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:56:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:45.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:56:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768919+0000", 2024-09-18T04:56:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:56:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768925+0000", 2024-09-18T04:56:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2310000000000001e-06 2024-09-18T04:56:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768929+0000", 2024-09-18T04:56:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4469999999999999e-06 2024-09-18T04:56:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812823+0000", 2024-09-18T04:56:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043893975000000002 2024-09-18T04:56:45.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812908+0000", 2024-09-18T04:56:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4862000000000002e-05 2024-09-18T04:56:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812981+0000", 2024-09-18T04:56:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2892000000000006e-05 2024-09-18T04:56:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9332 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 274432~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.770912+0000", 2024-09-18T04:56:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.009934993, 2024-09-18T04:56:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043671336999999998, 2024-09-18T04:56:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9332 2024-09-18T04:56:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:45.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770912+0000", 2024-09-18T04:56:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770912+0000", 2024-09-18T04:56:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:45.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770911+0000", 2024-09-18T04:56:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770916+0000", 2024-09-18T04:56:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6269999999999999e-06 2024-09-18T04:56:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770919+0000", 2024-09-18T04:56:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3100000000000001e-06 2024-09-18T04:56:45.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770922+0000", 2024-09-18T04:56:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3439999999999999e-06 2024-09-18T04:56:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814399+0000", 2024-09-18T04:56:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043477403999999997 2024-09-18T04:56:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:45.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814484+0000", 2024-09-18T04:56:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4625999999999999e-05 2024-09-18T04:56:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814583+0000", 2024-09-18T04:56:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9403999999999999e-05 2024-09-18T04:56:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9333 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 323584~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:45.843 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.770924+0000", 2024-09-18T04:56:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.00992271, 2024-09-18T04:56:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043870484000000001, 2024-09-18T04:56:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9333 2024-09-18T04:56:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:45.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770924+0000", 2024-09-18T04:56:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770924+0000", 2024-09-18T04:56:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770924+0000", 2024-09-18T04:56:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770925+0000", 2024-09-18T04:56:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.19e-06 2024-09-18T04:56:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770931+0000", 2024-09-18T04:56:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0789999999999997e-06 2024-09-18T04:56:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770934+0000", 2024-09-18T04:56:45.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6110000000000001e-06 2024-09-18T04:56:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814609+0000", 2024-09-18T04:56:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043675640000000002 2024-09-18T04:56:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814685+0000", 2024-09-18T04:56:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5506000000000005e-05 2024-09-18T04:56:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814794+0000", 2024-09-18T04:56:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010972300000000001 2024-09-18T04:56:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9334 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:56:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.009910866, 2024-09-18T04:56:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044042853999999999, 2024-09-18T04:56:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:45.850 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9334 2024-09-18T04:56:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:56:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:45.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:56:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770935+0000", 2024-09-18T04:56:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770937+0000", 2024-09-18T04:56:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1489999999999999e-06 2024-09-18T04:56:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770940+0000", 2024-09-18T04:56:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9569999999999998e-06 2024-09-18T04:56:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770945+0000", 2024-09-18T04:56:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:56:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814838+0000", 2024-09-18T04:56:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043892635999999999 2024-09-18T04:56:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814920+0000", 2024-09-18T04:56:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2175000000000005e-05 2024-09-18T04:56:45.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814979+0000", 2024-09-18T04:56:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8372000000000003e-05 2024-09-18T04:56:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9347 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 946176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773038+0000", 2024-09-18T04:56:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.007808526, 2024-09-18T04:56:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043570541999999997, 2024-09-18T04:56:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:45.856 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9347 2024-09-18T04:56:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773038+0000", 2024-09-18T04:56:45.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773038+0000", 2024-09-18T04:56:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:45.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773038+0000", 2024-09-18T04:56:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773040+0000", 2024-09-18T04:56:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.261e-06 2024-09-18T04:56:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773046+0000", 2024-09-18T04:56:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9290000000000003e-06 2024-09-18T04:56:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773050+0000", 2024-09-18T04:56:45.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5340000000000001e-06 2024-09-18T04:56:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.816516+0000", 2024-09-18T04:56:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043465472999999998 2024-09-18T04:56:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.816556+0000", 2024-09-18T04:56:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0129000000000002e-05 2024-09-18T04:56:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.816609+0000", 2024-09-18T04:56:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2703e-05 2024-09-18T04:56:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.862 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9348 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 995328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773057+0000", 2024-09-18T04:56:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.007789501, 2024-09-18T04:56:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043651141999999997, 2024-09-18T04:56:45.863 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9348 2024-09-18T04:56:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773057+0000", 2024-09-18T04:56:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773057+0000", 2024-09-18T04:56:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773057+0000", 2024-09-18T04:56:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:45.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773059+0000", 2024-09-18T04:56:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.221e-06 2024-09-18T04:56:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773065+0000", 2024-09-18T04:56:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1680000000000001e-06 2024-09-18T04:56:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773070+0000", 2024-09-18T04:56:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6179999999999997e-06 2024-09-18T04:56:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.816623+0000", 2024-09-18T04:56:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043553174 2024-09-18T04:56:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:45.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.816661+0000", 2024-09-18T04:56:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8389000000000001e-05 2024-09-18T04:56:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.816708+0000", 2024-09-18T04:56:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6919999999999998e-05 2024-09-18T04:56:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9349 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1044480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:45.869 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773074+0000", 2024-09-18T04:56:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.00777278, 2024-09-18T04:56:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043758942000000002, 2024-09-18T04:56:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9349 2024-09-18T04:56:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:45.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773074+0000", 2024-09-18T04:56:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773074+0000", 2024-09-18T04:56:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773073+0000", 2024-09-18T04:56:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773076+0000", 2024-09-18T04:56:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2340000000000001e-06 2024-09-18T04:56:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773081+0000", 2024-09-18T04:56:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8150000000000002e-06 2024-09-18T04:56:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773094+0000", 2024-09-18T04:56:45.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2191000000000001e-05 2024-09-18T04:56:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.816735+0000", 2024-09-18T04:56:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043641539999999999 2024-09-18T04:56:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.816782+0000", 2024-09-18T04:56:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6866000000000002e-05 2024-09-18T04:56:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:45.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.816833+0000", 2024-09-18T04:56:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0844000000000003e-05 2024-09-18T04:56:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9350 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1060864~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773115+0000", 2024-09-18T04:56:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.007731720000001, 2024-09-18T04:56:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043835878000000002, 2024-09-18T04:56:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:45.874 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9350 2024-09-18T04:56:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773115+0000", 2024-09-18T04:56:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773115+0000", 2024-09-18T04:56:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773114+0000", 2024-09-18T04:56:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773117+0000", 2024-09-18T04:56:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8229999999999999e-06 2024-09-18T04:56:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:45.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773124+0000", 2024-09-18T04:56:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3309999999999998e-06 2024-09-18T04:56:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773130+0000", 2024-09-18T04:56:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6570000000000002e-06 2024-09-18T04:56:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.816861+0000", 2024-09-18T04:56:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043730416000000001 2024-09-18T04:56:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.816902+0000", 2024-09-18T04:56:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1016000000000002e-05 2024-09-18T04:56:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.816951+0000", 2024-09-18T04:56:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9177e-05 2024-09-18T04:56:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9351 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1110016~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:45.878 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773135+0000", 2024-09-18T04:56:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.00771164, 2024-09-18T04:56:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043934572999999998, 2024-09-18T04:56:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9351 2024-09-18T04:56:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773135+0000", 2024-09-18T04:56:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773135+0000", 2024-09-18T04:56:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773134+0000", 2024-09-18T04:56:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773140+0000", 2024-09-18T04:56:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9780000000000002e-06 2024-09-18T04:56:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773147+0000", 2024-09-18T04:56:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5289999999999997e-06 2024-09-18T04:56:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773153+0000", 2024-09-18T04:56:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.648e-06 2024-09-18T04:56:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.816978+0000", 2024-09-18T04:56:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043825201000000001 2024-09-18T04:56:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817018+0000", 2024-09-18T04:56:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0296000000000002e-05 2024-09-18T04:56:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817070+0000", 2024-09-18T04:56:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1411e-05 2024-09-18T04:56:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9352 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1159168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:56:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.007689688999999, 2024-09-18T04:56:45.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044011269999999998, 2024-09-18T04:56:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9352 2024-09-18T04:56:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:56:45.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:56:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773156+0000", 2024-09-18T04:56:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773159+0000", 2024-09-18T04:56:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2919999999999998e-06 2024-09-18T04:56:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773165+0000", 2024-09-18T04:56:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0580000000000003e-06 2024-09-18T04:56:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773177+0000", 2024-09-18T04:56:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.204e-05 2024-09-18T04:56:45.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817082+0000", 2024-09-18T04:56:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043905528999999999 2024-09-18T04:56:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817122+0000", 2024-09-18T04:56:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9258999999999998e-05 2024-09-18T04:56:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817168+0000", 2024-09-18T04:56:45.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6545000000000002e-05 2024-09-18T04:56:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9353 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1208320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:56:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.007665391, 2024-09-18T04:56:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044105537, 2024-09-18T04:56:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:45.888 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9353 2024-09-18T04:56:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:56:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:56:45.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:56:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773183+0000", 2024-09-18T04:56:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2929999999999999e-06 2024-09-18T04:56:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:45.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773190+0000", 2024-09-18T04:56:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4330000000000001e-06 2024-09-18T04:56:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773194+0000", 2024-09-18T04:56:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.865e-06 2024-09-18T04:56:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817196+0000", 2024-09-18T04:56:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044001708 2024-09-18T04:56:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817236+0000", 2024-09-18T04:56:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0043999999999998e-05 2024-09-18T04:56:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817287+0000", 2024-09-18T04:56:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0605000000000001e-05 2024-09-18T04:56:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.892 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9354 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:56:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.00764788, 2024-09-18T04:56:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044199796, 2024-09-18T04:56:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9354 2024-09-18T04:56:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:45.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:56:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:56:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773198+0000", 2024-09-18T04:56:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773200+0000", 2024-09-18T04:56:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4159999999999998e-06 2024-09-18T04:56:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773206+0000", 2024-09-18T04:56:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.789e-06 2024-09-18T04:56:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773210+0000", 2024-09-18T04:56:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2679999999999996e-06 2024-09-18T04:56:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817314+0000", 2024-09-18T04:56:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044103187000000002 2024-09-18T04:56:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817353+0000", 2024-09-18T04:56:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8973000000000001e-05 2024-09-18T04:56:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817399+0000", 2024-09-18T04:56:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5877999999999997e-05 2024-09-18T04:56:45.896 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:45.897 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:45.897 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:45.899 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:56:45.899 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:45.896+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:45.999 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:45.997+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:46.047 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:46.047 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:56:46.047 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:46.055 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:56:46.100 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:46.097+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:46.201 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:56:46.201 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:56:46.201 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:46.202 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:46.199+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:46.209 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:56:46.228 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T04:56:46.275 INFO:teuthology.orchestra.run.smithi002.stdout:flattening image image_client.0-clone17 2024-09-18T04:56:46.302 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:46.300+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:46.337 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:56:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:56:46.337 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:56:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14125 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:56:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.207157558, 2024-09-18T04:56:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052956470999999998, 2024-09-18T04:56:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:46.338 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14125 2024-09-18T04:56:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:56:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:56:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126849+0000", 2024-09-18T04:56:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126852+0000", 2024-09-18T04:56:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3659999999999999e-06 2024-09-18T04:56:46.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126858+0000", 2024-09-18T04:56:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4540000000000004e-06 2024-09-18T04:56:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126863+0000", 2024-09-18T04:56:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1780000000000002e-06 2024-09-18T04:56:46.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179608+0000", 2024-09-18T04:56:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052745159999999999 2024-09-18T04:56:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179703+0000", 2024-09-18T04:56:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4773999999999998e-05 2024-09-18T04:56:46.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179806+0000", 2024-09-18T04:56:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010296799999999999 2024-09-18T04:56:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14136 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1732608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:56:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.204997006999999, 2024-09-18T04:56:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052973412999999997, 2024-09-18T04:56:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:46.344 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14136 2024-09-18T04:56:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:56:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:56:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:56:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129013+0000", 2024-09-18T04:56:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.96e-06 2024-09-18T04:56:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129020+0000", 2024-09-18T04:56:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6809999999999998e-06 2024-09-18T04:56:46.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129026+0000", 2024-09-18T04:56:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:56:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181873+0000", 2024-09-18T04:56:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052847238999999997 2024-09-18T04:56:46.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181932+0000", 2024-09-18T04:56:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9339000000000002e-05 2024-09-18T04:56:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181984+0000", 2024-09-18T04:56:46.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1348999999999999e-05 2024-09-18T04:56:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14137 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1781760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:56:46.350 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.204976602, 2024-09-18T04:56:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053128914999999999, 2024-09-18T04:56:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14137 2024-09-18T04:56:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.351 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:56:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:56:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129030+0000", 2024-09-18T04:56:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129033+0000", 2024-09-18T04:56:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7439999999999999e-06 2024-09-18T04:56:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129039+0000", 2024-09-18T04:56:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.314e-06 2024-09-18T04:56:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129044+0000", 2024-09-18T04:56:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9919999999999998e-06 2024-09-18T04:56:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182015+0000", 2024-09-18T04:56:46.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052970875000000001 2024-09-18T04:56:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182058+0000", 2024-09-18T04:56:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2997000000000002e-05 2024-09-18T04:56:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182160+0000", 2024-09-18T04:56:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101437 2024-09-18T04:56:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.355 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14138 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1830912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:56:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.204958568, 2024-09-18T04:56:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053256435999999997, 2024-09-18T04:56:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:46.356 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14138 2024-09-18T04:56:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:56:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:56:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129048+0000", 2024-09-18T04:56:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129056+0000", 2024-09-18T04:56:46.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.413e-06 2024-09-18T04:56:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129062+0000", 2024-09-18T04:56:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6810000000000001e-06 2024-09-18T04:56:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129068+0000", 2024-09-18T04:56:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0459999999999996e-06 2024-09-18T04:56:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182191+0000", 2024-09-18T04:56:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053123210999999997 2024-09-18T04:56:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182238+0000", 2024-09-18T04:56:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7404999999999998e-05 2024-09-18T04:56:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182305+0000", 2024-09-18T04:56:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7028999999999998e-05 2024-09-18T04:56:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.361 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15910 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2236416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:56:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.105871036, 2024-09-18T04:56:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053241054000000003, 2024-09-18T04:56:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15910 2024-09-18T04:56:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:56:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:56:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:56:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228138+0000", 2024-09-18T04:56:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:56:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228144+0000", 2024-09-18T04:56:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5369999999999997e-06 2024-09-18T04:56:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228149+0000", 2024-09-18T04:56:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1209999999999998e-06 2024-09-18T04:56:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280822+0000", 2024-09-18T04:56:46.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052673194999999999 2024-09-18T04:56:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280903+0000", 2024-09-18T04:56:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1179000000000003e-05 2024-09-18T04:56:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281377+0000", 2024-09-18T04:56:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047417899999999997 2024-09-18T04:56:46.366 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15911 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2285568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:56:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.105854303999999, 2024-09-18T04:56:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053415151000000001, 2024-09-18T04:56:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:46.367 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15911 2024-09-18T04:56:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:56:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:56:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:56:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228155+0000", 2024-09-18T04:56:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:56:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228161+0000", 2024-09-18T04:56:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:56:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228166+0000", 2024-09-18T04:56:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2320000000000001e-06 2024-09-18T04:56:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281405+0000", 2024-09-18T04:56:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053239336999999998 2024-09-18T04:56:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281482+0000", 2024-09-18T04:56:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6569999999999994e-05 2024-09-18T04:56:46.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281568+0000", 2024-09-18T04:56:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6328000000000004e-05 2024-09-18T04:56:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15918 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2662400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:56:46.372 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.104085323, 2024-09-18T04:56:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053091065, 2024-09-18T04:56:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15918 2024-09-18T04:56:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:56:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:56:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:56:46.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229924+0000", 2024-09-18T04:56:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:56:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229930+0000", 2024-09-18T04:56:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.066e-06 2024-09-18T04:56:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229935+0000", 2024-09-18T04:56:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9119999999999997e-06 2024-09-18T04:56:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282840+0000", 2024-09-18T04:56:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052905119 2024-09-18T04:56:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282922+0000", 2024-09-18T04:56:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2081000000000002e-05 2024-09-18T04:56:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283013+0000", 2024-09-18T04:56:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0828000000000005e-05 2024-09-18T04:56:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.377 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15919 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2711552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:56:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.104068045, 2024-09-18T04:56:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053293021000000003, 2024-09-18T04:56:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15919 2024-09-18T04:56:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:56:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:56:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:56:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229941+0000", 2024-09-18T04:56:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2639999999999998e-06 2024-09-18T04:56:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229947+0000", 2024-09-18T04:56:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9750000000000004e-06 2024-09-18T04:56:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229952+0000", 2024-09-18T04:56:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7929999999999997e-06 2024-09-18T04:56:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283062+0000", 2024-09-18T04:56:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05310952 2024-09-18T04:56:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283162+0000", 2024-09-18T04:56:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100098 2024-09-18T04:56:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283232+0000", 2024-09-18T04:56:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0693999999999998e-05 2024-09-18T04:56:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15920 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2760704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:56:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.104051284000001, 2024-09-18T04:56:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053502367000000002, 2024-09-18T04:56:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.383 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15920 2024-09-18T04:56:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:56:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:56:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:56:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229958+0000", 2024-09-18T04:56:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1770000000000001e-06 2024-09-18T04:56:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229964+0000", 2024-09-18T04:56:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8470000000000001e-06 2024-09-18T04:56:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229968+0000", 2024-09-18T04:56:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5680000000000001e-06 2024-09-18T04:56:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283269+0000", 2024-09-18T04:56:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053300620999999999 2024-09-18T04:56:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283359+0000", 2024-09-18T04:56:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9971999999999998e-05 2024-09-18T04:56:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283458+0000", 2024-09-18T04:56:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9483999999999998e-05 2024-09-18T04:56:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15921 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2809856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:56:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.104034734000001, 2024-09-18T04:56:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053800977, 2024-09-18T04:56:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:46.389 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15921 2024-09-18T04:56:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:56:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:56:46.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229972+0000", 2024-09-18T04:56:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229974+0000", 2024-09-18T04:56:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2910000000000002e-06 2024-09-18T04:56:46.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229982+0000", 2024-09-18T04:56:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7249999999999997e-06 2024-09-18T04:56:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229989+0000", 2024-09-18T04:56:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6429999999999998e-06 2024-09-18T04:56:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283601+0000", 2024-09-18T04:56:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053612211 2024-09-18T04:56:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283681+0000", 2024-09-18T04:56:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0125999999999998e-05 2024-09-18T04:56:46.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283774+0000", 2024-09-18T04:56:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2362999999999996e-05 2024-09-18T04:56:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15922 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2859008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:46.394 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:56:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.103989795, 2024-09-18T04:56:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053882270000000003, 2024-09-18T04:56:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15922 2024-09-18T04:56:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:56:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:56:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230017+0000", 2024-09-18T04:56:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230019+0000", 2024-09-18T04:56:46.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4210000000000002e-06 2024-09-18T04:56:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230027+0000", 2024-09-18T04:56:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000002e-06 2024-09-18T04:56:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230032+0000", 2024-09-18T04:56:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0089999999999998e-06 2024-09-18T04:56:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283800+0000", 2024-09-18T04:56:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053768418999999998 2024-09-18T04:56:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283849+0000", 2024-09-18T04:56:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9339000000000003e-05 2024-09-18T04:56:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283900+0000", 2024-09-18T04:56:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0421999999999999e-05 2024-09-18T04:56:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15923 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2924544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:56:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.103970926000001, 2024-09-18T04:56:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054073016000000002, 2024-09-18T04:56:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.400 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15923 2024-09-18T04:56:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:56:46.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:56:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:56:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230038+0000", 2024-09-18T04:56:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.463e-06 2024-09-18T04:56:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230045+0000", 2024-09-18T04:56:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3570000000000003e-06 2024-09-18T04:56:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230050+0000", 2024-09-18T04:56:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7299999999999996e-06 2024-09-18T04:56:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283928+0000", 2024-09-18T04:56:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053878218999999998 2024-09-18T04:56:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284010+0000", 2024-09-18T04:56:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2577000000000006e-05 2024-09-18T04:56:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284109+0000", 2024-09-18T04:56:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9102000000000001e-05 2024-09-18T04:56:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.405 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15924 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:56:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.103952456, 2024-09-18T04:56:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054108470999999998, 2024-09-18T04:56:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15924 2024-09-18T04:56:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:56:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:56:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:56:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230061+0000", 2024-09-18T04:56:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8240000000000002e-06 2024-09-18T04:56:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230068+0000", 2024-09-18T04:56:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4320000000000004e-06 2024-09-18T04:56:46.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230073+0000", 2024-09-18T04:56:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.921e-06 2024-09-18T04:56:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284153+0000", 2024-09-18T04:56:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054080793000000002 2024-09-18T04:56:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284163+0000", 2024-09-18T04:56:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.859e-06 2024-09-18T04:56:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15925 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:56:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.103928341, 2024-09-18T04:56:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054134632000000002, 2024-09-18T04:56:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15925 2024-09-18T04:56:46.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:56:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:56:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:56:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230081+0000", 2024-09-18T04:56:46.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4109999999999999e-06 2024-09-18T04:56:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230087+0000", 2024-09-18T04:56:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.161e-06 2024-09-18T04:56:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230092+0000", 2024-09-18T04:56:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9259999999999999e-06 2024-09-18T04:56:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284204+0000", 2024-09-18T04:56:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054111546000000003 2024-09-18T04:56:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284214+0000", 2024-09-18T04:56:46.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9639999999999998e-06 2024-09-18T04:56:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15926 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:56:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.103911017, 2024-09-18T04:56:46.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054143193999999999, 2024-09-18T04:56:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15926 2024-09-18T04:56:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:56:46.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:56:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:56:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230098+0000", 2024-09-18T04:56:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1399999999999998e-06 2024-09-18T04:56:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230104+0000", 2024-09-18T04:56:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-18T04:56:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230109+0000", 2024-09-18T04:56:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9250000000000003e-06 2024-09-18T04:56:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284231+0000", 2024-09-18T04:56:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054122297999999999 2024-09-18T04:56:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284239+0000", 2024-09-18T04:56:46.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1689999999999992e-06 2024-09-18T04:56:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15927 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:56:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.10389397, 2024-09-18T04:56:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054168435000000001, 2024-09-18T04:56:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.422 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15927 2024-09-18T04:56:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:56:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:56:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:56:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230116+0000", 2024-09-18T04:56:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5150000000000001e-06 2024-09-18T04:56:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230122+0000", 2024-09-18T04:56:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1190000000000002e-06 2024-09-18T04:56:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230126+0000", 2024-09-18T04:56:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.792e-06 2024-09-18T04:56:46.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284273+0000", 2024-09-18T04:56:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054146086000000003 2024-09-18T04:56:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284282+0000", 2024-09-18T04:56:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2590000000000006e-06 2024-09-18T04:56:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.426 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15928 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:56:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.103876587, 2024-09-18T04:56:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054189739000000001, 2024-09-18T04:56:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:46.427 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15928 2024-09-18T04:56:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:56:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:56:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230130+0000", 2024-09-18T04:56:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230133+0000", 2024-09-18T04:56:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.244e-06 2024-09-18T04:56:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230139+0000", 2024-09-18T04:56:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9830000000000001e-06 2024-09-18T04:56:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230143+0000", 2024-09-18T04:56:46.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7530000000000001e-06 2024-09-18T04:56:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284313+0000", 2024-09-18T04:56:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054169332000000001 2024-09-18T04:56:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284320+0000", 2024-09-18T04:56:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8699999999999992e-06 2024-09-18T04:56:46.432 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15929 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:56:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.103860147000001, 2024-09-18T04:56:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054218771999999998, 2024-09-18T04:56:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:46.433 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15929 2024-09-18T04:56:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:56:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:56:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:56:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230149+0000", 2024-09-18T04:56:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-18T04:56:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230155+0000", 2024-09-18T04:56:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2469999999999997e-06 2024-09-18T04:56:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230160+0000", 2024-09-18T04:56:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7129999999999996e-06 2024-09-18T04:56:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284355+0000", 2024-09-18T04:56:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054194900999999997 2024-09-18T04:56:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284366+0000", 2024-09-18T04:56:46.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0993e-05 2024-09-18T04:56:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15930 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:56:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.103843146999999, 2024-09-18T04:56:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054244713999999999, 2024-09-18T04:56:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:46.438 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15930 2024-09-18T04:56:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:56:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:56:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:56:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230166+0000", 2024-09-18T04:56:46.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1009999999999999e-06 2024-09-18T04:56:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230172+0000", 2024-09-18T04:56:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.981e-06 2024-09-18T04:56:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230184+0000", 2024-09-18T04:56:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2449000000000001e-05 2024-09-18T04:56:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284400+0000", 2024-09-18T04:56:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054215452999999997 2024-09-18T04:56:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284409+0000", 2024-09-18T04:56:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0839999999999998e-06 2024-09-18T04:56:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15931 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:56:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.103817358000001, 2024-09-18T04:56:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054261928000000001, 2024-09-18T04:56:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15931 2024-09-18T04:56:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.443 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:56:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:56:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230189+0000", 2024-09-18T04:56:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230192+0000", 2024-09-18T04:56:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6599999999999999e-06 2024-09-18T04:56:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230199+0000", 2024-09-18T04:56:46.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5570000000000001e-06 2024-09-18T04:56:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230204+0000", 2024-09-18T04:56:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2270000000000001e-06 2024-09-18T04:56:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284442+0000", 2024-09-18T04:56:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054238383000000001 2024-09-18T04:56:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284452+0000", 2024-09-18T04:56:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6069999999999997e-06 2024-09-18T04:56:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.447 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.447 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:46.449 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:56:46.449 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:46.446+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:46.549 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:46.547+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:46.579 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-09-18T04:56:46.603 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:46.603 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:56:46.603 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:46.616 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:56:46.650 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:46.647+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:46.736 INFO:teuthology.orchestra.run.smithi002.stdout:2022 read 0x1043176 thru 0x104fe33 (0xccbe bytes) 2024-09-18T04:56:46.736 INFO:teuthology.orchestra.run.smithi002.stdout:2024 write 0x5495d84 thru 0x549d4a1 (0x771e bytes) 2024-09-18T04:56:46.750 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:46.748+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:46.752 INFO:teuthology.orchestra.run.smithi002.stdout:2025 punch from 0x8c0f6b4 to 0x8c18fe8, (0x9934 bytes) 2024-09-18T04:56:46.752 INFO:teuthology.orchestra.run.smithi002.stdout:2026 read 0x5b08336 thru 0x5b17749 (0xf414 bytes) 2024-09-18T04:56:46.753 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:46.753 INFO:teuthology.orchestra.run.smithi002.stdout:2029 punch from 0x1223377 to 0x1229fd9, (0x6c62 bytes) 2024-09-18T04:56:46.753 INFO:teuthology.orchestra.run.smithi002.stdout:2030 punch from 0x183ea51 to 0x183f50e, (0xabd bytes) 2024-09-18T04:56:46.753 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:46.753 INFO:teuthology.orchestra.run.smithi002.stdout:2032 clone 19 order 23 su 524288 sc 11 2024-09-18T04:56:46.767 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:56:46.767 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:56:46.767 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:46.776 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:56:46.852 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:46.850+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:46.883 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:46.883 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:56:46.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:56:46.883 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:56:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:689 3.4 e24/19 3:21ae0c4f:::rbd_data.11acdc0eab99.0000000000000029:head v 24'7, mlcod=23'4)", 2024-09-18T04:56:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.214992+0000", 2024-09-18T04:56:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.665298984, 2024-09-18T04:56:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001477602, 2024-09-18T04:56:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214992+0000", 2024-09-18T04:56:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214992+0000", 2024-09-18T04:56:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214991+0000", 2024-09-18T04:56:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214997+0000", 2024-09-18T04:56:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1709999999999999e-06 2024-09-18T04:56:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.215006+0000", 2024-09-18T04:56:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8799999999999997e-06 2024-09-18T04:56:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.215012+0000", 2024-09-18T04:56:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0870000000000003e-06 2024-09-18T04:56:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.215068+0000", 2024-09-18T04:56:46.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5350999999999999e-05 2024-09-18T04:56:46.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.215087+0000", 2024-09-18T04:56:46.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9565e-05 2024-09-18T04:56:46.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:46.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.216432+0000", 2024-09-18T04:56:46.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013447439999999999 2024-09-18T04:56:46.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.216469+0000", 2024-09-18T04:56:46.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7370999999999998e-05 2024-09-18T04:56:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:4372 3.9 e28/19 3:91d61960:::rbd_data.11ac7d9658bc.0000000000000027:head v 28'25, mlcod=25'22)", 2024-09-18T04:56:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:31.233447+0000", 2024-09-18T04:56:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.646843865999999, 2024-09-18T04:56:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012563520000000001, 2024-09-18T04:56:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:31.233447+0000", 2024-09-18T04:56:46.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:31.233447+0000", 2024-09-18T04:56:46.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:31.233446+0000", 2024-09-18T04:56:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:31.233454+0000", 2024-09-18T04:56:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.627e-06 2024-09-18T04:56:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:31.233738+0000", 2024-09-18T04:56:46.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028350199999999998 2024-09-18T04:56:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:31.233749+0000", 2024-09-18T04:56:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1289e-05 2024-09-18T04:56:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:31.233809+0000", 2024-09-18T04:56:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0563999999999998e-05 2024-09-18T04:56:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:31.233833+0000", 2024-09-18T04:56:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4043999999999999e-05 2024-09-18T04:56:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:31.234664+0000", 2024-09-18T04:56:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00083019500000000004 2024-09-18T04:56:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:31.234703+0000", 2024-09-18T04:56:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9461e-05 2024-09-18T04:56:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.893 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.894 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:11220 3.4 e32/19 3:267b21b8:::rbd_object_map.11ac7a27a554:head v 32'111, mlcod=32'109)", 2024-09-18T04:56:46.894 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.931314+0000", 2024-09-18T04:56:46.894 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.948976498, 2024-09-18T04:56:46.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012730090000000001, 2024-09-18T04:56:46.894 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.894 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:46.894 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.931314+0000", 2024-09-18T04:56:46.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.931314+0000", 2024-09-18T04:56:46.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.931313+0000", 2024-09-18T04:56:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.931324+0000", 2024-09-18T04:56:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.084e-05 2024-09-18T04:56:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.931345+0000", 2024-09-18T04:56:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1311999999999999e-05 2024-09-18T04:56:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.931354+0000", 2024-09-18T04:56:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2750000000000001e-06 2024-09-18T04:56:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.931427+0000", 2024-09-18T04:56:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2978999999999994e-05 2024-09-18T04:56:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.931465+0000", 2024-09-18T04:56:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7815999999999998e-05 2024-09-18T04:56:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.932548+0000", 2024-09-18T04:56:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010825800000000001 2024-09-18T04:56:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.932587+0000", 2024-09-18T04:56:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9526000000000001e-05 2024-09-18T04:56:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.898 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:13694 3.4 e32/19 3:267b21b8:::rbd_object_map.11ac7a27a554:head v 32'141, mlcod=32'139)", 2024-09-18T04:56:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.103977+0000", 2024-09-18T04:56:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.776313501000001, 2024-09-18T04:56:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012994040000000001, 2024-09-18T04:56:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.103977+0000", 2024-09-18T04:56:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.103977+0000", 2024-09-18T04:56:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.103975+0000", 2024-09-18T04:56:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999981 2024-09-18T04:56:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.103988+0000", 2024-09-18T04:56:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2367e-05 2024-09-18T04:56:46.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.104009+0000", 2024-09-18T04:56:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1256000000000001e-05 2024-09-18T04:56:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.104018+0000", 2024-09-18T04:56:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0259999999999996e-06 2024-09-18T04:56:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.104064+0000", 2024-09-18T04:56:46.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6326000000000003e-05 2024-09-18T04:56:46.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.104088+0000", 2024-09-18T04:56:46.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3754e-05 2024-09-18T04:56:46.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:46.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.105203+0000", 2024-09-18T04:56:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011151379999999999 2024-09-18T04:56:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.105277+0000", 2024-09-18T04:56:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3357999999999998e-05 2024-09-18T04:56:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:15989 3.4 e32/19 3:238b8fe4:::rbd_data.11ac7a27a554.0000000000000005:head v 32'150, mlcod=32'147)", 2024-09-18T04:56:46.903 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.245449+0000", 2024-09-18T04:56:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.634841714, 2024-09-18T04:56:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028002180000000002, 2024-09-18T04:56:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.245449+0000", 2024-09-18T04:56:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.245449+0000", 2024-09-18T04:56:46.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.245448+0000", 2024-09-18T04:56:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.246149+0000", 2024-09-18T04:56:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00070047499999999999 2024-09-18T04:56:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.246164+0000", 2024-09-18T04:56:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5532999999999998e-05 2024-09-18T04:56:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.246171+0000", 2024-09-18T04:56:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8990000000000004e-06 2024-09-18T04:56:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.246218+0000", 2024-09-18T04:56:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6789000000000002e-05 2024-09-18T04:56:46.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.246242+0000", 2024-09-18T04:56:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3706999999999999e-05 2024-09-18T04:56:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.248217+0000", 2024-09-18T04:56:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019752060000000002 2024-09-18T04:56:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.248249+0000", 2024-09-18T04:56:46.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2252000000000002e-05 2024-09-18T04:56:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:15991 3.4 e32/19 3:21a074f1:::rbd_data.11ac7a27a554.0000000000000006:head v 32'152, mlcod=32'148)", 2024-09-18T04:56:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.249009+0000", 2024-09-18T04:56:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.631281161, 2024-09-18T04:56:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021236060000000001, 2024-09-18T04:56:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249009+0000", 2024-09-18T04:56:46.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249009+0000", 2024-09-18T04:56:46.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249009+0000", 2024-09-18T04:56:46.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:46.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249228+0000", 2024-09-18T04:56:46.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021908999999999999 2024-09-18T04:56:46.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249243+0000", 2024-09-18T04:56:46.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5031999999999999e-05 2024-09-18T04:56:46.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249250+0000", 2024-09-18T04:56:46.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2919999999999998e-06 2024-09-18T04:56:46.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249294+0000", 2024-09-18T04:56:46.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3479000000000002e-05 2024-09-18T04:56:46.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249314+0000", 2024-09-18T04:56:46.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0596000000000001e-05 2024-09-18T04:56:46.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:46.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.251093+0000", 2024-09-18T04:56:46.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001778342 2024-09-18T04:56:46.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.251133+0000", 2024-09-18T04:56:46.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0503999999999997e-05 2024-09-18T04:56:46.913 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.913 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.913 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.913 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:18783 3.4 e32/19 3:267b21b8:::rbd_object_map.11ac7a27a554:head v 32'161, mlcod=32'159)", 2024-09-18T04:56:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.663855+0000", 2024-09-18T04:56:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.216435858000001, 2024-09-18T04:56:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012940969999999999, 2024-09-18T04:56:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.663855+0000", 2024-09-18T04:56:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.663855+0000", 2024-09-18T04:56:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.663853+0000", 2024-09-18T04:56:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.663865+0000", 2024-09-18T04:56:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1554e-05 2024-09-18T04:56:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.663886+0000", 2024-09-18T04:56:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0993999999999999e-05 2024-09-18T04:56:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.663901+0000", 2024-09-18T04:56:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5396999999999998e-05 2024-09-18T04:56:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.663978+0000", 2024-09-18T04:56:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6310999999999995e-05 2024-09-18T04:56:46.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.664008+0000", 2024-09-18T04:56:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0057e-05 2024-09-18T04:56:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.665112+0000", 2024-09-18T04:56:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001103885 2024-09-18T04:56:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.665149+0000", 2024-09-18T04:56:46.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7283999999999997e-05 2024-09-18T04:56:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:18826 3.4 e32/19 3:23a67d65:::rbd_data.11ac7a27a554.0000000000000010:head v 32'162, mlcod=32'159)", 2024-09-18T04:56:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.666171+0000", 2024-09-18T04:56:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.214119488, 2024-09-18T04:56:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022865569999999998, 2024-09-18T04:56:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666171+0000", 2024-09-18T04:56:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666171+0000", 2024-09-18T04:56:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666170+0000", 2024-09-18T04:56:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:46.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666654+0000", 2024-09-18T04:56:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00048368400000000001 2024-09-18T04:56:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666670+0000", 2024-09-18T04:56:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5818e-05 2024-09-18T04:56:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666677+0000", 2024-09-18T04:56:46.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5750000000000003e-06 2024-09-18T04:56:46.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666734+0000", 2024-09-18T04:56:46.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6663999999999998e-05 2024-09-18T04:56:46.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666757+0000", 2024-09-18T04:56:46.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2691e-05 2024-09-18T04:56:46.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:46.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.668417+0000", 2024-09-18T04:56:46.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001660501 2024-09-18T04:56:46.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.668458+0000", 2024-09-18T04:56:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0364999999999997e-05 2024-09-18T04:56:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:19048 3.4 e32/19 3:267b21b8:::rbd_object_map.11ac7a27a554:head v 32'164, mlcod=32'162)", 2024-09-18T04:56:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.680521+0000", 2024-09-18T04:56:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.1997698, 2024-09-18T04:56:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00134738, 2024-09-18T04:56:46.926 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.680521+0000", 2024-09-18T04:56:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.680521+0000", 2024-09-18T04:56:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.680519+0000", 2024-09-18T04:56:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.680530+0000", 2024-09-18T04:56:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0200000000000001e-05 2024-09-18T04:56:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.680551+0000", 2024-09-18T04:56:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1106000000000001e-05 2024-09-18T04:56:46.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.680560+0000", 2024-09-18T04:56:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0459999999999994e-06 2024-09-18T04:56:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.680651+0000", 2024-09-18T04:56:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0781999999999999e-05 2024-09-18T04:56:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.680676+0000", 2024-09-18T04:56:46.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5357000000000001e-05 2024-09-18T04:56:46.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:46.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.681825+0000", 2024-09-18T04:56:46.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011489569999999999 2024-09-18T04:56:46.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.681868+0000", 2024-09-18T04:56:46.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3259e-05 2024-09-18T04:56:46.930 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.931 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.931 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.931 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:20114 3.4 e32/19 3:267b21b8:::rbd_object_map.11ac7a27a554:head v 32'167, mlcod=32'165)", 2024-09-18T04:56:46.931 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.722905+0000", 2024-09-18T04:56:46.931 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.157385223, 2024-09-18T04:56:46.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014794739999999999, 2024-09-18T04:56:46.931 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.931 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:46.931 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.722905+0000", 2024-09-18T04:56:46.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.722905+0000", 2024-09-18T04:56:46.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.722904+0000", 2024-09-18T04:56:46.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:46.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.722915+0000", 2024-09-18T04:56:46.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0162e-05 2024-09-18T04:56:46.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.722931+0000", 2024-09-18T04:56:46.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6174000000000001e-05 2024-09-18T04:56:46.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.722940+0000", 2024-09-18T04:56:46.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0129999999999999e-06 2024-09-18T04:56:46.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.723013+0000", 2024-09-18T04:56:46.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3798000000000001e-05 2024-09-18T04:56:46.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.723044+0000", 2024-09-18T04:56:46.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0726999999999999e-05 2024-09-18T04:56:46.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:46.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.724350+0000", 2024-09-18T04:56:46.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001305891 2024-09-18T04:56:46.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.724385+0000", 2024-09-18T04:56:46.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4770000000000001e-05 2024-09-18T04:56:46.937 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.937 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.937 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.937 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:21999 3.4 e32/19 3:267b21b8:::rbd_object_map.11ac7a27a554:head v 32'190, mlcod=32'188)", 2024-09-18T04:56:46.937 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:37.004308+0000", 2024-09-18T04:56:46.937 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.8759827750000007, 2024-09-18T04:56:46.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001259568, 2024-09-18T04:56:46.937 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.937 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:46.938 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.004308+0000", 2024-09-18T04:56:46.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.004308+0000", 2024-09-18T04:56:46.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.004307+0000", 2024-09-18T04:56:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.004314+0000", 2024-09-18T04:56:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7050000000000002e-06 2024-09-18T04:56:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.004329+0000", 2024-09-18T04:56:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5194000000000001e-05 2024-09-18T04:56:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.004336+0000", 2024-09-18T04:56:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7610000000000002e-06 2024-09-18T04:56:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.004395+0000", 2024-09-18T04:56:46.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9577999999999998e-05 2024-09-18T04:56:46.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.004413+0000", 2024-09-18T04:56:46.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8206000000000002e-05 2024-09-18T04:56:46.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:46.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.005535+0000", 2024-09-18T04:56:46.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001121677 2024-09-18T04:56:46.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.005567+0000", 2024-09-18T04:56:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2359e-05 2024-09-18T04:56:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:22065 3.4 e32/19 3:267b21b8:::rbd_object_map.11ac7a27a554:head v 32'217, mlcod=32'213)", 2024-09-18T04:56:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:37.044092+0000", 2024-09-18T04:56:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.8361986160000008, 2024-09-18T04:56:46.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001661636, 2024-09-18T04:56:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044092+0000", 2024-09-18T04:56:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044092+0000", 2024-09-18T04:56:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044091+0000", 2024-09-18T04:56:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044096+0000", 2024-09-18T04:56:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9230000000000001e-06 2024-09-18T04:56:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044106+0000", 2024-09-18T04:56:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7569999999999999e-06 2024-09-18T04:56:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044111+0000", 2024-09-18T04:56:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4839999999999998e-06 2024-09-18T04:56:46.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044147+0000", 2024-09-18T04:56:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6736999999999997e-05 2024-09-18T04:56:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044158+0000", 2024-09-18T04:56:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1178999999999999e-05 2024-09-18T04:56:46.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.045717+0000", 2024-09-18T04:56:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015584850000000001 2024-09-18T04:56:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.045754+0000", 2024-09-18T04:56:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6671000000000002e-05 2024-09-18T04:56:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.947 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:22066 3.4 e32/19 3:267b21b8:::rbd_object_map.11ac7a27a554:head v 32'218, mlcod=32'214)", 2024-09-18T04:56:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:37.044877+0000", 2024-09-18T04:56:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.8354137290000008, 2024-09-18T04:56:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014606160000000001, 2024-09-18T04:56:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044877+0000", 2024-09-18T04:56:46.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044877+0000", 2024-09-18T04:56:46.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044876+0000", 2024-09-18T04:56:46.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044884+0000", 2024-09-18T04:56:46.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4220000000000003e-06 2024-09-18T04:56:46.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044896+0000", 2024-09-18T04:56:46.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2418999999999999e-05 2024-09-18T04:56:46.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044902+0000", 2024-09-18T04:56:46.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3380000000000001e-06 2024-09-18T04:56:46.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044944+0000", 2024-09-18T04:56:46.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1865e-05 2024-09-18T04:56:46.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044963+0000", 2024-09-18T04:56:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8454999999999999e-05 2024-09-18T04:56:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.046302+0000", 2024-09-18T04:56:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013391259999999999 2024-09-18T04:56:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.046338+0000", 2024-09-18T04:56:46.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5695000000000003e-05 2024-09-18T04:56:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:22098 3.4 e32/19 3:267b21b8:::rbd_object_map.11ac7a27a554:head v 32'225, mlcod=32'221)", 2024-09-18T04:56:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:37.062135+0000", 2024-09-18T04:56:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.8181557949999991, 2024-09-18T04:56:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001486558, 2024-09-18T04:56:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.062135+0000", 2024-09-18T04:56:46.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.062135+0000", 2024-09-18T04:56:46.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.062134+0000", 2024-09-18T04:56:46.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.062140+0000", 2024-09-18T04:56:46.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2550000000000003e-06 2024-09-18T04:56:46.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.062153+0000", 2024-09-18T04:56:46.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2629000000000001e-05 2024-09-18T04:56:46.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.062160+0000", 2024-09-18T04:56:46.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9249999999999998e-06 2024-09-18T04:56:46.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.062201+0000", 2024-09-18T04:56:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0921999999999999e-05 2024-09-18T04:56:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.062222+0000", 2024-09-18T04:56:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1628999999999999e-05 2024-09-18T04:56:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:46.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.063558+0000", 2024-09-18T04:56:46.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001335934 2024-09-18T04:56:46.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.063621+0000", 2024-09-18T04:56:46.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2964000000000002e-05 2024-09-18T04:56:46.959 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.959 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.959 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:22125 3.4 e32/19 3:267b21b8:::rbd_object_map.11ac7a27a554:head v 32'237, mlcod=32'232)", 2024-09-18T04:56:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:37.076006+0000", 2024-09-18T04:56:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.8042847690000006, 2024-09-18T04:56:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014754970000000001, 2024-09-18T04:56:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.076006+0000", 2024-09-18T04:56:46.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.076006+0000", 2024-09-18T04:56:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.076005+0000", 2024-09-18T04:56:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.076012+0000", 2024-09-18T04:56:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4760000000000003e-06 2024-09-18T04:56:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.076024+0000", 2024-09-18T04:56:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2228e-05 2024-09-18T04:56:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.076031+0000", 2024-09-18T04:56:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7469999999999999e-06 2024-09-18T04:56:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.076066+0000", 2024-09-18T04:56:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5182999999999998e-05 2024-09-18T04:56:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.076079+0000", 2024-09-18T04:56:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3155e-05 2024-09-18T04:56:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.077443+0000", 2024-09-18T04:56:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001363946 2024-09-18T04:56:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.077481+0000", 2024-09-18T04:56:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8556999999999997e-05 2024-09-18T04:56:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:22126 3.4 e32/19 3:267b21b8:::rbd_object_map.11ac7a27a554:head v 32'238, mlcod=32'234)", 2024-09-18T04:56:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:37.076984+0000", 2024-09-18T04:56:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.8033062490000002, 2024-09-18T04:56:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013169290000000001, 2024-09-18T04:56:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.964 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.076984+0000", 2024-09-18T04:56:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.076984+0000", 2024-09-18T04:56:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.076984+0000", 2024-09-18T04:56:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.076990+0000", 2024-09-18T04:56:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6889999999999998e-06 2024-09-18T04:56:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.077004+0000", 2024-09-18T04:56:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3635000000000001e-05 2024-09-18T04:56:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.077010+0000", 2024-09-18T04:56:46.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1840000000000004e-06 2024-09-18T04:56:46.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.077074+0000", 2024-09-18T04:56:46.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3996000000000002e-05 2024-09-18T04:56:46.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.077095+0000", 2024-09-18T04:56:46.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0632000000000002e-05 2024-09-18T04:56:46.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:46.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.078254+0000", 2024-09-18T04:56:46.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001158858 2024-09-18T04:56:46.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.078301+0000", 2024-09-18T04:56:46.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7775999999999998e-05 2024-09-18T04:56:46.969 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.969 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.969 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.970 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:22140 3.0 e32/19 3:04e0ba1b:::rbd_data.11ac7a27a554.00000000000000d3:head v 32'98, mlcod=32'96)", 2024-09-18T04:56:46.970 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:37.083209+0000", 2024-09-18T04:56:46.970 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.7970814829999995, 2024-09-18T04:56:46.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001278176, 2024-09-18T04:56:46.970 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.970 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:46.970 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.083209+0000", 2024-09-18T04:56:46.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.083209+0000", 2024-09-18T04:56:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.083208+0000", 2024-09-18T04:56:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.083219+0000", 2024-09-18T04:56:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.028e-05 2024-09-18T04:56:46.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.083292+0000", 2024-09-18T04:56:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3627999999999994e-05 2024-09-18T04:56:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.083301+0000", 2024-09-18T04:56:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4489999999999999e-06 2024-09-18T04:56:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.083349+0000", 2024-09-18T04:56:46.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8712999999999999e-05 2024-09-18T04:56:46.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.083369+0000", 2024-09-18T04:56:46.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9364999999999998e-05 2024-09-18T04:56:46.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:46.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.084449+0000", 2024-09-18T04:56:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010797510000000001 2024-09-18T04:56:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.084487+0000", 2024-09-18T04:56:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8748999999999998e-05 2024-09-18T04:56:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:29070 3.4 e36/19 3:26169c24:::rbd_data.11ac138032e7.0000000000000004:head v 36'263, mlcod=36'261)", 2024-09-18T04:56:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:40.883605+0000", 2024-09-18T04:56:46.974 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9966860689999999, 2024-09-18T04:56:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014912059999999999, 2024-09-18T04:56:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:40.883605+0000", 2024-09-18T04:56:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:40.883605+0000", 2024-09-18T04:56:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:40.883603+0000", 2024-09-18T04:56:46.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999981 2024-09-18T04:56:46.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:40.883776+0000", 2024-09-18T04:56:46.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000173518 2024-09-18T04:56:46.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:40.883795+0000", 2024-09-18T04:56:46.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8190999999999999e-05 2024-09-18T04:56:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:40.883801+0000", 2024-09-18T04:56:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7440000000000001e-06 2024-09-18T04:56:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:40.883851+0000", 2024-09-18T04:56:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9360000000000002e-05 2024-09-18T04:56:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:40.883871+0000", 2024-09-18T04:56:46.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0182000000000001e-05 2024-09-18T04:56:46.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:46.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:40.885065+0000", 2024-09-18T04:56:46.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001194325 2024-09-18T04:56:46.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:40.885096+0000", 2024-09-18T04:56:46.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0568999999999998e-05 2024-09-18T04:56:46.978 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:36051 3.4 e42/19 3:25d849c5:::rbd_data.11ac728e6e9a.0000000000000002:head v 42'294, mlcod=40'292)", 2024-09-18T04:56:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:46.443141+0000", 2024-09-18T04:56:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.43714973400000001, 2024-09-18T04:56:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037584760000000002, 2024-09-18T04:56:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443141+0000", 2024-09-18T04:56:46.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443141+0000", 2024-09-18T04:56:46.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443141+0000", 2024-09-18T04:56:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443812+0000", 2024-09-18T04:56:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067104899999999995 2024-09-18T04:56:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443831+0000", 2024-09-18T04:56:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9074999999999999e-05 2024-09-18T04:56:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443841+0000", 2024-09-18T04:56:46.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8919999999999995e-06 2024-09-18T04:56:46.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443933+0000", 2024-09-18T04:56:46.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2348000000000004e-05 2024-09-18T04:56:46.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443957+0000", 2024-09-18T04:56:46.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3567e-05 2024-09-18T04:56:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.446862+0000", 2024-09-18T04:56:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029052919999999999 2024-09-18T04:56:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.446899+0000", 2024-09-18T04:56:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7515999999999998e-05 2024-09-18T04:56:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:36052 3.4 e42/19 3:25d849c5:::rbd_data.11ac728e6e9a.0000000000000002:head v 42'296, mlcod=40'292)", 2024-09-18T04:56:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:46.445721+0000", 2024-09-18T04:56:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.434569396, 2024-09-18T04:56:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001516749, 2024-09-18T04:56:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.445721+0000", 2024-09-18T04:56:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.445721+0000", 2024-09-18T04:56:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.445720+0000", 2024-09-18T04:56:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.445729+0000", 2024-09-18T04:56:46.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2099999999999999e-06 2024-09-18T04:56:46.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:46.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.445745+0000", 2024-09-18T04:56:46.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5852999999999999e-05 2024-09-18T04:56:46.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:46.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.445754+0000", 2024-09-18T04:56:46.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3219999999999993e-06 2024-09-18T04:56:46.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:46.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.445834+0000", 2024-09-18T04:56:46.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0307000000000002e-05 2024-09-18T04:56:46.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:46.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.445857+0000", 2024-09-18T04:56:46.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3294000000000001e-05 2024-09-18T04:56:46.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.447206+0000", 2024-09-18T04:56:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001349233 2024-09-18T04:56:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.447238+0000", 2024-09-18T04:56:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1489000000000003e-05 2024-09-18T04:56:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:46.988 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:46.989 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:46.990 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:56:46.990 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:46.988+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:47.091 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:47.088+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:47.121 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:56:47.138 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:47.138 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:47.139 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:56:47.139 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:47.147 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:56:47.192 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:47.190+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:47.224 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-18T04:56:47.251 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:56:47.251 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:56:47.271 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:56:47.272 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:56:47.290 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:56:47.290 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:56:47.292 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone18 from 0xb096af7 (overlap 0xb096af7) to 0x8d45592 2024-09-18T04:56:47.292 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:47.290+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:47.321 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:56:47.322 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:56:47.323 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:47.323 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:47.336 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:56:47.351 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:56:47.351 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:56:47.377 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:56:47.378 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:56:47.393 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:47.391+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:47.400 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:56:47.401 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:56:47.420 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:56:47.420 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:56:47.490 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:47.490 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:47.495 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:47.493+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:47.500 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:56:47.543 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-09-18T04:56:47.596 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:47.593+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:47.605 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:47.605 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:47.614 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:56:47.698 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:47.696+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:47.746 INFO:teuthology.orchestra.run.smithi002.stdout:2033 writesame 0xb5fdd7a thru 0xb60b178 data_size 0x1(0xd3ff bytes) 2024-09-18T04:56:47.755 INFO:teuthology.orchestra.run.smithi002.stdout:2034 write 0x897d685 thru 0x897f51b (0x1e97 bytes) 2024-09-18T04:56:47.759 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:47.759 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:47.769 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:56:47.771 INFO:teuthology.orchestra.run.smithi002.stdout:2035 write 0x78055fb thru 0x780dfe3 (0x89e9 bytes) 2024-09-18T04:56:47.790 INFO:teuthology.orchestra.run.smithi002.stdout:2036 trunc from 0xb60b179 to 0x908effe 2024-09-18T04:56:47.810 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:47.808+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:47.899 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:47.897+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:47.908 INFO:teuthology.orchestra.run.smithi002.stdout:2041 write 0xdf45563 thru 0xdf4e289 (0x8d27 bytes) 2024-09-18T04:56:47.910 INFO:teuthology.orchestra.run.smithi002.stdout:2043 read 0xbe00f0 thru 0xbe65d1 (0x64e2 bytes) 2024-09-18T04:56:47.911 INFO:teuthology.orchestra.run.smithi002.stdout:2046 punch from 0x2d8521c to 0x2d86d63, (0x1b47 bytes) 2024-09-18T04:56:47.911 INFO:teuthology.orchestra.run.smithi002.stdout:2047 read 0x7be7f6 thru 0x7ccb37 (0xe342 bytes) 2024-09-18T04:56:47.912 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:47.912 INFO:teuthology.orchestra.run.smithi002.stdout:2049 punch from 0x63e0e14 to 0x63f03af, (0xf59b bytes) 2024-09-18T04:56:47.912 INFO:teuthology.orchestra.run.smithi002.stdout:2050 read 0xbc9c611 thru 0xbca8fb0 (0xc9a0 bytes) 2024-09-18T04:56:47.913 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:47.913 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:47.913 INFO:teuthology.orchestra.run.smithi002.stdout:2056 read 0xd42623b thru 0xd431dfb (0xbbc1 bytes) 2024-09-18T04:56:47.913 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:47.913 INFO:teuthology.orchestra.run.smithi002.stdout:2058 read 0x7ad48e4 thru 0x7ae0a20 (0xc13d bytes) 2024-09-18T04:56:47.913 INFO:teuthology.orchestra.run.smithi002.stdout:2059 punch from 0x6429df0 to 0x6438ade, (0xecee bytes) 2024-09-18T04:56:47.914 INFO:teuthology.orchestra.run.smithi002.stdout:2060 read 0x912d365 thru 0x9131a8c (0x4728 bytes) 2024-09-18T04:56:47.915 INFO:teuthology.orchestra.run.smithi002.stdout:2061 trunc from 0xdf4e28a to 0xc094d1b 2024-09-18T04:56:47.921 INFO:teuthology.orchestra.run.smithi002.stdout:2063 writesame 0x6ff6749 thru 0x6ff7a8e data_size 0x1(0x1346 bytes) 2024-09-18T04:56:47.923 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:47.924 INFO:teuthology.orchestra.run.smithi002.stdout:2065 read 0x261c640 thru 0x2627220 (0xabe1 bytes) 2024-09-18T04:56:47.924 INFO:teuthology.orchestra.run.smithi002.stdout:2066 writesame 0x8243c59 thru 0x8252b18 data_size 0x50(0xeec0 bytes) 2024-09-18T04:56:47.926 INFO:teuthology.orchestra.run.smithi002.stdout:2068 read 0x8e43936 thru 0x8e50832 (0xcefd bytes) 2024-09-18T04:56:47.926 INFO:teuthology.orchestra.run.smithi002.stdout:2069 read 0x2ab04de thru 0x2ab9243 (0x8d66 bytes) 2024-09-18T04:56:47.927 INFO:teuthology.orchestra.run.smithi002.stdout:2071 write 0x3f87a53 thru 0x3f8bca8 (0x4256 bytes) 2024-09-18T04:56:47.945 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:47.945 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:47.953 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:56:47.953 INFO:teuthology.orchestra.run.smithi002.stdout:2072 read 0x2e90631 thru 0x2e93199 (0x2b69 bytes) 2024-09-18T04:56:47.953 INFO:teuthology.orchestra.run.smithi002.stdout:2074 write 0x4d34d44 thru 0x4d40063 (0xb320 bytes) 2024-09-18T04:56:48.000 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:47.998+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:48.001 INFO:teuthology.orchestra.run.smithi002.stdout:2075 trunc from 0xc094d1b to 0xabe69af 2024-09-18T04:56:48.012 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:48.021 INFO:teuthology.orchestra.run.smithi002.stdout:2077 writesame 0xd2e195f thru 0xd2e858e data_size 0xc(0x6c30 bytes) 2024-09-18T04:56:48.021 INFO:teuthology.orchestra.run.smithi002.stdout:2078 writesame 0x448a47f thru 0x448c093 data_size 0xd(0x1c15 bytes) 2024-09-18T04:56:48.043 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:48.056 INFO:teuthology.orchestra.run.smithi002.stdout:2082 trunc from 0xd2e858f to 0x1b3af78 2024-09-18T04:56:48.061 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:48.061 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:48.073 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:56:48.101 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:48.098+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:48.177 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:48.177 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:48.186 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:56:48.201 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:48.199+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:48.237 INFO:teuthology.orchestra.run.smithi002.stdout:2083 trunc from 0x1b3af78 to 0xeb596c8 2024-09-18T04:56:48.242 INFO:teuthology.orchestra.run.smithi002.stdout:2084 read 0x87c4dc1 thru 0x87cff4c (0xb18c bytes) 2024-09-18T04:56:48.243 INFO:teuthology.orchestra.run.smithi002.stdout:2085 writesame 0x615495 thru 0x622811 data_size 0x1(0xd37d bytes) 2024-09-18T04:56:48.268 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:48.268 INFO:teuthology.orchestra.run.smithi002.stdout:2088 read 0x4abe969 thru 0x4ac07de (0x1e76 bytes) 2024-09-18T04:56:48.269 INFO:teuthology.orchestra.run.smithi002.stdout:2089 punch from 0x1e7923c to 0x1e7a736, (0x14fa bytes) 2024-09-18T04:56:48.269 INFO:teuthology.orchestra.run.smithi002.stdout:2091 read 0x805d15d thru 0x806b631 (0xe4d5 bytes) 2024-09-18T04:56:48.269 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:48.269 INFO:teuthology.orchestra.run.smithi002.stdout:2093 writesame 0xad2b8d thru 0xad5051 data_size 0x1(0x24c5 bytes) 2024-09-18T04:56:48.274 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:48.274 INFO:teuthology.orchestra.run.smithi002.stdout:2097 read 0xd6d692e thru 0xd6e09f9 (0xa0cc bytes) 2024-09-18T04:56:48.274 INFO:teuthology.orchestra.run.smithi002.stdout:2098 writesame 0x258b81e thru 0x259b51b data_size 0x5e(0xfcfe bytes) 2024-09-18T04:56:48.275 INFO:teuthology.orchestra.run.smithi002.stdout:2100 write 0x9673f93 thru 0x96817a2 (0xd810 bytes) 2024-09-18T04:56:48.279 INFO:teuthology.orchestra.run.smithi002.stdout:2101 write 0xfb1d35 thru 0xfba486 (0x8752 bytes) 2024-09-18T04:56:48.281 INFO:teuthology.orchestra.run.smithi002.stdout:2102 punch from 0x3381863 to 0x338bd8d, (0xa52a bytes) 2024-09-18T04:56:48.281 INFO:teuthology.orchestra.run.smithi002.stdout:2103 read 0x9b13ba3 thru 0x9b1e52c (0xa98a bytes) 2024-09-18T04:56:48.281 INFO:teuthology.orchestra.run.smithi002.stdout:2104 writesame 0x33c976f thru 0x33d2234 data_size 0x5d(0x8ac6 bytes) 2024-09-18T04:56:48.282 INFO:teuthology.orchestra.run.smithi002.stdout:2105 write 0xd2fa533 thru 0xd2fb639 (0x1107 bytes) 2024-09-18T04:56:48.285 INFO:teuthology.orchestra.run.smithi002.stdout:2107 writesame 0x7da4ad2 thru 0x7db388e data_size 0x1(0xedbd bytes) 2024-09-18T04:56:48.292 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:48.292 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:48.301 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:56:48.302 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:48.300+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:48.315 INFO:teuthology.orchestra.run.smithi002.stdout:2108 writesame 0xead9f53 thru 0xeae4585 data_size 0x1(0xa633 bytes) 2024-09-18T04:56:48.337 INFO:teuthology.orchestra.run.smithi002.stdout:2109 write 0x813f133 thru 0x8142ade (0x39ac bytes) 2024-09-18T04:56:48.340 INFO:teuthology.orchestra.run.smithi002.stdout:2110 read 0x47a202d thru 0x47ad533 (0xb507 bytes) 2024-09-18T04:56:48.341 INFO:teuthology.orchestra.run.smithi002.stdout:2111 read 0x785cb6 thru 0x793531 (0xd87c bytes) 2024-09-18T04:56:48.342 INFO:teuthology.orchestra.run.smithi002.stdout:2112 read 0x8d42b49 thru 0x8d49a41 (0x6ef9 bytes) 2024-09-18T04:56:48.358 INFO:teuthology.orchestra.run.smithi002.stdout:2113 trunc from 0xeb596c8 to 0x6d58dd 2024-09-18T04:56:48.385 INFO:teuthology.orchestra.run.smithi002.stdout:2114 write 0x8356dbd thru 0x8364708 (0xd94c bytes) 2024-09-18T04:56:48.387 INFO:teuthology.orchestra.run.smithi002.stdout:2119 write 0x2f9836e thru 0x2f9af03 (0x2b96 bytes) 2024-09-18T04:56:48.389 INFO:teuthology.orchestra.run.smithi002.stdout:2120 read 0x613ae88 thru 0x6144e92 (0xa00b bytes) 2024-09-18T04:56:48.389 INFO:teuthology.orchestra.run.smithi002.stdout:2121 write 0x71a8eb1 thru 0x71abb3d (0x2c8d bytes) 2024-09-18T04:56:48.392 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:48.392 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:48.392 INFO:teuthology.orchestra.run.smithi002.stdout:2125 punch from 0x5420252 to 0x542b2a1, (0xb04f bytes) 2024-09-18T04:56:48.392 INFO:teuthology.orchestra.run.smithi002.stdout:2128 write 0x441147b thru 0x441af7e (0x9b04 bytes) 2024-09-18T04:56:48.393 INFO:teuthology.orchestra.run.smithi002.stdout:2134 writesame 0xc15872 thru 0xc16849 data_size 0x1a(0xfd8 bytes) 2024-09-18T04:56:48.395 INFO:teuthology.orchestra.run.smithi002.stdout:2136 write 0x6a415c1 thru 0x6a41d5b (0x79b bytes) 2024-09-18T04:56:48.397 INFO:teuthology.orchestra.run.smithi002.stdout:2137 writesame 0x1eb1df8 thru 0x1eb5540 data_size 0x1(0x3749 bytes) 2024-09-18T04:56:48.403 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:48.401+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:48.405 INFO:teuthology.orchestra.run.smithi002.stdout:2138 write 0x7c03037 thru 0x7c09361 (0x632b bytes) 2024-09-18T04:56:48.407 INFO:teuthology.orchestra.run.smithi002.stdout:2140 read 0x6620db0 thru 0x662ea64 (0xdcb5 bytes) 2024-09-18T04:56:48.408 INFO:teuthology.orchestra.run.smithi002.stdout:2141 write 0x28152b0 thru 0x281d256 (0x7fa7 bytes) 2024-09-18T04:56:48.409 INFO:teuthology.orchestra.run.smithi002.stdout:2143 writesame 0x7fb69cf thru 0x7fbe608 data_size 0x1(0x7c3a bytes) 2024-09-18T04:56:48.412 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:48.412 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:48.420 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:56:48.426 INFO:teuthology.orchestra.run.smithi002.stdout:2144 write 0x227f43e thru 0x228204c (0x2c0f bytes) 2024-09-18T04:56:48.428 INFO:teuthology.orchestra.run.smithi002.stdout:2146 punch from 0x6a1ced7 to 0x6a1e12e, (0x1257 bytes) 2024-09-18T04:56:48.438 INFO:teuthology.orchestra.run.smithi002.stdout:2147 write 0xc432f60 thru 0xc43e0c6 (0xb167 bytes) 2024-09-18T04:56:48.441 INFO:teuthology.orchestra.run.smithi002.stdout:2148 write 0x4466113 thru 0x4475eff (0xfded bytes) 2024-09-18T04:56:48.443 INFO:teuthology.orchestra.run.smithi002.stdout:2149 write 0xdbc3c7 thru 0xdc1b84 (0x57be bytes) 2024-09-18T04:56:48.444 INFO:teuthology.orchestra.run.smithi002.stdout:2150 write 0x797083c thru 0x7973a5b (0x3220 bytes) 2024-09-18T04:56:48.447 INFO:teuthology.orchestra.run.smithi002.stdout:2151 writesame 0x20169db thru 0x20265df data_size 0x1(0xfc05 bytes) 2024-09-18T04:56:48.476 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:48.477 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:48.477 INFO:teuthology.orchestra.run.smithi002.stdout:2155 punch from 0x3a4b83b to 0x3a545a5, (0x8d6a bytes) 2024-09-18T04:56:48.477 INFO:teuthology.orchestra.run.smithi002.stdout:2156 punch from 0x1644990 to 0x1645aef, (0x115f bytes) 2024-09-18T04:56:48.477 INFO:teuthology.orchestra.run.smithi002.stdout:2157 write 0xb289f34 thru 0xb293e75 (0x9f42 bytes) 2024-09-18T04:56:48.479 INFO:teuthology.orchestra.run.smithi002.stdout:2158 write 0x7cf2e0f thru 0x7cf55a2 (0x2794 bytes) 2024-09-18T04:56:48.481 INFO:teuthology.orchestra.run.smithi002.stdout:2159 write 0x1868b99 thru 0x1874cd2 (0xc13a bytes) 2024-09-18T04:56:48.483 INFO:teuthology.orchestra.run.smithi002.stdout:2160 punch from 0x7cbbf3b to 0x7cca9ae, (0xea73 bytes) 2024-09-18T04:56:48.484 INFO:teuthology.orchestra.run.smithi002.stdout:2162 trunc from 0xc43e0c7 to 0xb54f89f 2024-09-18T04:56:48.499 INFO:teuthology.orchestra.run.smithi002.stdout:2163 writesame 0xe5af956 thru 0xe5b4a89 data_size 0x2(0x5134 bytes) 2024-09-18T04:56:48.499 INFO:teuthology.orchestra.run.smithi002.stdout:2165 read 0x928f663 thru 0x92929ba (0x3358 bytes) 2024-09-18T04:56:48.502 INFO:teuthology.orchestra.run.smithi002.stdout:2166 trunc from 0xe5b4a8a to 0xbf037ba 2024-09-18T04:56:48.504 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:48.502+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:48.509 INFO:teuthology.orchestra.run.smithi002.stdout:2167 write 0x13993c6 thru 0x139c1c4 (0x2dff bytes) 2024-09-18T04:56:48.511 INFO:teuthology.orchestra.run.smithi002.stdout:2169 read 0x3a7330c thru 0x3a76b6f (0x3864 bytes) 2024-09-18T04:56:48.511 INFO:teuthology.orchestra.run.smithi002.stdout:2170 read 0x114963a thru 0x114f438 (0x5dff bytes) 2024-09-18T04:56:48.511 INFO:teuthology.orchestra.run.smithi002.stdout:2171 write 0x9775484 thru 0x977a0b6 (0x4c33 bytes) 2024-09-18T04:56:48.513 INFO:teuthology.orchestra.run.smithi002.stdout:2172 read 0x2600c50 thru 0x260aaae (0x9e5f bytes) 2024-09-18T04:56:48.514 INFO:teuthology.orchestra.run.smithi002.stdout:2173 punch from 0x8fd9cac to 0x8fe53f5, (0xb749 bytes) 2024-09-18T04:56:48.514 INFO:teuthology.orchestra.run.smithi002.stdout:2176 write 0x9e7db76 thru 0x9e8089f (0x2d2a bytes) 2024-09-18T04:56:48.517 INFO:teuthology.orchestra.run.smithi002.stdout:2177 punch from 0x993c374 to 0x993dcda, (0x1966 bytes) 2024-09-18T04:56:48.522 INFO:teuthology.orchestra.run.smithi002.stdout:2178 write 0xc516ab0 thru 0xc52278e (0xbcdf bytes) 2024-09-18T04:56:48.525 INFO:teuthology.orchestra.run.smithi002.stdout:2179 read 0x3078732 thru 0x3081a8a (0x9359 bytes) 2024-09-18T04:56:48.526 INFO:teuthology.orchestra.run.smithi002.stdout:2180 read 0x7f5faef thru 0x7f6b272 (0xb784 bytes) 2024-09-18T04:56:48.526 INFO:teuthology.orchestra.run.smithi002.stdout:2181 read 0xed6863 thru 0xed99fc (0x319a bytes) 2024-09-18T04:56:48.527 INFO:teuthology.orchestra.run.smithi002.stdout:2182 punch from 0x13161a5 to 0x13230e8, (0xcf43 bytes) 2024-09-18T04:56:48.527 INFO:teuthology.orchestra.run.smithi002.stdout:2183 write 0x6e6105a thru 0x6e6f2b1 (0xe258 bytes) 2024-09-18T04:56:48.528 INFO:teuthology.orchestra.run.smithi002.stdout:2184 read 0x988a0ad thru 0x9892508 (0x845c bytes) 2024-09-18T04:56:48.528 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:48.532 INFO:teuthology.orchestra.run.smithi002.stdout:2187 trunc from 0xc52278f to 0x85e0316 2024-09-18T04:56:48.550 INFO:teuthology.orchestra.run.smithi002.stdout:2188 write 0xa0675aa thru 0xa07664e (0xf0a5 bytes) 2024-09-18T04:56:48.553 INFO:teuthology.orchestra.run.smithi002.stdout:2191 punch from 0x4018929 to 0x401dbf3, (0x52ca bytes) 2024-09-18T04:56:48.561 INFO:teuthology.orchestra.run.smithi002.stdout:2192 write 0xbd75e92 thru 0xbd816c6 (0xb835 bytes) 2024-09-18T04:56:48.564 INFO:teuthology.orchestra.run.smithi002.stdout:2194 read 0x13fa8bd thru 0x13fe2fa (0x3a3e bytes) 2024-09-18T04:56:48.564 INFO:teuthology.orchestra.run.smithi002.stdout:2195 read 0x5c58ef4 thru 0x5c68c21 (0xfd2e bytes) 2024-09-18T04:56:48.565 INFO:teuthology.orchestra.run.smithi002.stdout:2196 punch from 0x4fdc5fa to 0x4fe1a7e, (0x5484 bytes) 2024-09-18T04:56:48.565 INFO:teuthology.orchestra.run.smithi002.stdout:2197 writesame 0x20cbd0f thru 0x20d6656 data_size 0x4(0xa948 bytes) 2024-09-18T04:56:48.569 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:48.569 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:48.571 INFO:teuthology.orchestra.run.smithi002.stdout:2199 read 0x46e2cf6 thru 0x46f0a26 (0xdd31 bytes) 2024-09-18T04:56:48.571 INFO:teuthology.orchestra.run.smithi002.stdout:2201 clone 20 order 18 su 1024 sc 15 2024-09-18T04:56:48.577 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:56:48.605 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:48.603+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:48.680 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:48.680 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:48.688 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:56:48.731 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:48.729+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:48.791 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:48.791 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:48.800 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:56:48.842 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:48.839+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:48.907 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:48.905+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:48.943 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:48.944 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:48.952 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:56:49.009 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:49.007+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:49.053 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:49.054 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:49.062 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:56:49.110 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:49.107+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:49.173 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:49.173 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:49.181 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:56:49.211 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:49.208+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:49.241 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T04:56:49.295 INFO:teuthology.orchestra.run.smithi002.stdout:leaving image image_client.0-clone19 intact 2024-09-18T04:56:49.297 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:49.297 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:49.307 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:56:49.311 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:49.308+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:49.411 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:49.409+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:49.422 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:49.422 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:49.431 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:56:49.432 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:56:49.450 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-09-18T04:56:49.512 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:49.510+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:49.601 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:49.601 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:49.615 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:56:49.617 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:49.611+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:49.665 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:56:49.665 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:56:49.704 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:56:49.704 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:56:49.714 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:49.712+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:49.726 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:56:49.726 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:56:49.778 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:49.778 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:49.778 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:56:49.778 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:56:49.778 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:49.790 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:56:49.834 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:49.832+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:49.905 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:49.906 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:56:49.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:56:49.906 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:56:49.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.906 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14125 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:49.906 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:56:49.906 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.77563614, 2024-09-18T04:56:49.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052956470999999998, 2024-09-18T04:56:49.906 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:49.906 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:49.907 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:49.907 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:49.907 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:49.907 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14125 2024-09-18T04:56:49.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.907 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:49.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:49.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:56:49.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:49.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:56:49.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:49.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126849+0000", 2024-09-18T04:56:49.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:49.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:49.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126852+0000", 2024-09-18T04:56:49.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3659999999999999e-06 2024-09-18T04:56:49.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:49.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126858+0000", 2024-09-18T04:56:49.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4540000000000004e-06 2024-09-18T04:56:49.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:49.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126863+0000", 2024-09-18T04:56:49.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1780000000000002e-06 2024-09-18T04:56:49.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:49.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179608+0000", 2024-09-18T04:56:49.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052745159999999999 2024-09-18T04:56:49.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:49.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179703+0000", 2024-09-18T04:56:49.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4773999999999998e-05 2024-09-18T04:56:49.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:49.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179806+0000", 2024-09-18T04:56:49.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010296799999999999 2024-09-18T04:56:49.910 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.910 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:49.910 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.911 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14136 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1732608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:49.911 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:56:49.911 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.773475589, 2024-09-18T04:56:49.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052973412999999997, 2024-09-18T04:56:49.911 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:49.911 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:49.911 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:49.911 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:49.911 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:49.911 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14136 2024-09-18T04:56:49.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.911 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:49.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:49.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:56:49.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:49.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:56:49.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:49.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:56:49.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:49.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:49.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129013+0000", 2024-09-18T04:56:49.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.96e-06 2024-09-18T04:56:49.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:49.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129020+0000", 2024-09-18T04:56:49.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6809999999999998e-06 2024-09-18T04:56:49.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:49.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129026+0000", 2024-09-18T04:56:49.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:56:49.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:49.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181873+0000", 2024-09-18T04:56:49.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052847238999999997 2024-09-18T04:56:49.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:49.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181932+0000", 2024-09-18T04:56:49.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9339000000000002e-05 2024-09-18T04:56:49.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:49.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181984+0000", 2024-09-18T04:56:49.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1348999999999999e-05 2024-09-18T04:56:49.915 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.915 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:49.915 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.915 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14137 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1781760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:49.915 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:56:49.915 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.773455183999999, 2024-09-18T04:56:49.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053128914999999999, 2024-09-18T04:56:49.915 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:49.915 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:49.915 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:49.916 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:49.916 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:49.916 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14137 2024-09-18T04:56:49.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.916 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:49.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:49.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:56:49.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:49.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:56:49.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:49.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129030+0000", 2024-09-18T04:56:49.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:49.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:49.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129033+0000", 2024-09-18T04:56:49.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7439999999999999e-06 2024-09-18T04:56:49.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:49.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129039+0000", 2024-09-18T04:56:49.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.314e-06 2024-09-18T04:56:49.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:49.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129044+0000", 2024-09-18T04:56:49.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9919999999999998e-06 2024-09-18T04:56:49.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:49.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182015+0000", 2024-09-18T04:56:49.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052970875000000001 2024-09-18T04:56:49.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:49.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182058+0000", 2024-09-18T04:56:49.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2997000000000002e-05 2024-09-18T04:56:49.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:49.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182160+0000", 2024-09-18T04:56:49.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101437 2024-09-18T04:56:49.919 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.919 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:49.919 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.919 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14138 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1830912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:49.919 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:56:49.920 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.773437149999999, 2024-09-18T04:56:49.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053256435999999997, 2024-09-18T04:56:49.920 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:49.920 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:49.920 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:49.920 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:49.920 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:49.920 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14138 2024-09-18T04:56:49.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.920 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:49.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:49.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:56:49.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:49.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:56:49.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:49.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129048+0000", 2024-09-18T04:56:49.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:49.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:49.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129056+0000", 2024-09-18T04:56:49.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.413e-06 2024-09-18T04:56:49.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:49.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129062+0000", 2024-09-18T04:56:49.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6810000000000001e-06 2024-09-18T04:56:49.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:49.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129068+0000", 2024-09-18T04:56:49.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0459999999999996e-06 2024-09-18T04:56:49.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:49.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182191+0000", 2024-09-18T04:56:49.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053123210999999997 2024-09-18T04:56:49.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:49.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182238+0000", 2024-09-18T04:56:49.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7404999999999998e-05 2024-09-18T04:56:49.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:49.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182305+0000", 2024-09-18T04:56:49.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7028999999999998e-05 2024-09-18T04:56:49.923 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.924 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:49.924 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.924 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15910 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2236416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:49.924 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:56:49.924 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.674349618000001, 2024-09-18T04:56:49.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053241054000000003, 2024-09-18T04:56:49.924 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:49.924 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:49.924 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:49.924 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:49.924 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:49.925 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15910 2024-09-18T04:56:49.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.925 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:49.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:49.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:56:49.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:49.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:56:49.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:49.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:56:49.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:49.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:49.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228138+0000", 2024-09-18T04:56:49.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:56:49.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:49.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228144+0000", 2024-09-18T04:56:49.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5369999999999997e-06 2024-09-18T04:56:49.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:49.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228149+0000", 2024-09-18T04:56:49.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1209999999999998e-06 2024-09-18T04:56:49.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:49.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280822+0000", 2024-09-18T04:56:49.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052673194999999999 2024-09-18T04:56:49.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:49.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280903+0000", 2024-09-18T04:56:49.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1179000000000003e-05 2024-09-18T04:56:49.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:49.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281377+0000", 2024-09-18T04:56:49.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047417899999999997 2024-09-18T04:56:49.928 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.928 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:49.928 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.928 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15911 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2285568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:49.928 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:56:49.929 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.674332886, 2024-09-18T04:56:49.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053415151000000001, 2024-09-18T04:56:49.929 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:49.929 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:49.929 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:49.929 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:49.929 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:49.929 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15911 2024-09-18T04:56:49.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.929 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:49.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:49.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:56:49.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:49.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:56:49.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:49.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:56:49.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:49.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:49.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228155+0000", 2024-09-18T04:56:49.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:56:49.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:49.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228161+0000", 2024-09-18T04:56:49.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:56:49.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:49.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228166+0000", 2024-09-18T04:56:49.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2320000000000001e-06 2024-09-18T04:56:49.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:49.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281405+0000", 2024-09-18T04:56:49.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053239336999999998 2024-09-18T04:56:49.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:49.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281482+0000", 2024-09-18T04:56:49.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6569999999999994e-05 2024-09-18T04:56:49.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:49.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281568+0000", 2024-09-18T04:56:49.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6328000000000004e-05 2024-09-18T04:56:49.933 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.933 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:49.933 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.933 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15918 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2662400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:49.933 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:56:49.933 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.672563905000001, 2024-09-18T04:56:49.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053091065, 2024-09-18T04:56:49.933 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:49.933 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:49.933 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:49.934 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:49.934 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:49.934 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15918 2024-09-18T04:56:49.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.934 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:49.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:49.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:56:49.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:49.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:56:49.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:49.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:56:49.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:49.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:49.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229924+0000", 2024-09-18T04:56:49.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:56:49.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:49.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229930+0000", 2024-09-18T04:56:49.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.066e-06 2024-09-18T04:56:49.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:49.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229935+0000", 2024-09-18T04:56:49.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9119999999999997e-06 2024-09-18T04:56:49.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:49.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282840+0000", 2024-09-18T04:56:49.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052905119 2024-09-18T04:56:49.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:49.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282922+0000", 2024-09-18T04:56:49.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2081000000000002e-05 2024-09-18T04:56:49.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:49.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283013+0000", 2024-09-18T04:56:49.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0828000000000005e-05 2024-09-18T04:56:49.937 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.937 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:49.937 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.938 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15919 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2711552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:49.938 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:56:49.938 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.672546626999999, 2024-09-18T04:56:49.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053293021000000003, 2024-09-18T04:56:49.938 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:49.938 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:49.938 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:49.938 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:49.938 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:49.938 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15919 2024-09-18T04:56:49.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.939 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:49.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:49.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:56:49.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:49.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:56:49.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:49.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:56:49.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:49.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:49.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229941+0000", 2024-09-18T04:56:49.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2639999999999998e-06 2024-09-18T04:56:49.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:49.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229947+0000", 2024-09-18T04:56:49.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9750000000000004e-06 2024-09-18T04:56:49.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:49.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229952+0000", 2024-09-18T04:56:49.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7929999999999997e-06 2024-09-18T04:56:49.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:49.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283062+0000", 2024-09-18T04:56:49.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05310952 2024-09-18T04:56:49.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:49.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283162+0000", 2024-09-18T04:56:49.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100098 2024-09-18T04:56:49.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:49.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283232+0000", 2024-09-18T04:56:49.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0693999999999998e-05 2024-09-18T04:56:49.942 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.942 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:49.942 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.943 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15920 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2760704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:49.943 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:56:49.943 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.672529866, 2024-09-18T04:56:49.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053502367000000002, 2024-09-18T04:56:49.943 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:49.943 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:49.943 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:49.943 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:49.943 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:49.943 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15920 2024-09-18T04:56:49.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.943 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:49.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:49.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:56:49.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:49.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:56:49.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:49.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:56:49.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:49.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:49.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229958+0000", 2024-09-18T04:56:49.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1770000000000001e-06 2024-09-18T04:56:49.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:49.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229964+0000", 2024-09-18T04:56:49.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8470000000000001e-06 2024-09-18T04:56:49.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:49.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229968+0000", 2024-09-18T04:56:49.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5680000000000001e-06 2024-09-18T04:56:49.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:49.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283269+0000", 2024-09-18T04:56:49.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053300620999999999 2024-09-18T04:56:49.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:49.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283359+0000", 2024-09-18T04:56:49.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9971999999999998e-05 2024-09-18T04:56:49.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:49.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283458+0000", 2024-09-18T04:56:49.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9483999999999998e-05 2024-09-18T04:56:49.947 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.947 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:49.947 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.947 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15921 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2809856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:49.947 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:56:49.948 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.672513316, 2024-09-18T04:56:49.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053800977, 2024-09-18T04:56:49.948 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:49.948 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:49.948 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:49.948 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:49.948 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:49.948 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15921 2024-09-18T04:56:49.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.948 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:49.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:49.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:56:49.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:49.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:56:49.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:49.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229972+0000", 2024-09-18T04:56:49.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:49.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:49.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229974+0000", 2024-09-18T04:56:49.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2910000000000002e-06 2024-09-18T04:56:49.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:49.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229982+0000", 2024-09-18T04:56:49.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7249999999999997e-06 2024-09-18T04:56:49.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:49.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229989+0000", 2024-09-18T04:56:49.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6429999999999998e-06 2024-09-18T04:56:49.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:49.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283601+0000", 2024-09-18T04:56:49.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053612211 2024-09-18T04:56:49.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:49.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283681+0000", 2024-09-18T04:56:49.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0125999999999998e-05 2024-09-18T04:56:49.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:49.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283774+0000", 2024-09-18T04:56:49.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2362999999999996e-05 2024-09-18T04:56:49.952 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.952 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:49.952 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.952 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15922 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2859008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:49.952 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:56:49.952 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.672468377, 2024-09-18T04:56:49.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053882270000000003, 2024-09-18T04:56:49.952 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:49.952 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:49.952 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:49.953 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:49.953 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:49.954 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15922 2024-09-18T04:56:49.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.954 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:49.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:49.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:56:49.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:49.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:56:49.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:49.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230017+0000", 2024-09-18T04:56:49.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:49.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:49.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230019+0000", 2024-09-18T04:56:49.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4210000000000002e-06 2024-09-18T04:56:49.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:49.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230027+0000", 2024-09-18T04:56:49.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000002e-06 2024-09-18T04:56:49.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:49.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230032+0000", 2024-09-18T04:56:49.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0089999999999998e-06 2024-09-18T04:56:49.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:49.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283800+0000", 2024-09-18T04:56:49.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053768418999999998 2024-09-18T04:56:49.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:49.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283849+0000", 2024-09-18T04:56:49.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9339000000000003e-05 2024-09-18T04:56:49.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:49.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283900+0000", 2024-09-18T04:56:49.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0421999999999999e-05 2024-09-18T04:56:49.958 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.958 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:49.958 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.958 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15923 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2924544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:49.958 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:56:49.958 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.672449508, 2024-09-18T04:56:49.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054073016000000002, 2024-09-18T04:56:49.958 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:49.958 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:49.958 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:49.959 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:49.959 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:49.959 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15923 2024-09-18T04:56:49.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.959 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:49.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:49.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:56:49.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:49.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:56:49.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:49.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:56:49.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:49.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:49.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230038+0000", 2024-09-18T04:56:49.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.463e-06 2024-09-18T04:56:49.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:49.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230045+0000", 2024-09-18T04:56:49.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3570000000000003e-06 2024-09-18T04:56:49.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:49.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230050+0000", 2024-09-18T04:56:49.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7299999999999996e-06 2024-09-18T04:56:49.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:49.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283928+0000", 2024-09-18T04:56:49.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053878218999999998 2024-09-18T04:56:49.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:49.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284010+0000", 2024-09-18T04:56:49.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2577000000000006e-05 2024-09-18T04:56:49.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:49.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284109+0000", 2024-09-18T04:56:49.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9102000000000001e-05 2024-09-18T04:56:49.962 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.962 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:49.962 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.962 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15924 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:49.963 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:56:49.963 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.672431037999999, 2024-09-18T04:56:49.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054108470999999998, 2024-09-18T04:56:49.963 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:49.963 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:49.963 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:49.963 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:49.963 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:49.963 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15924 2024-09-18T04:56:49.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.963 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:49.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:49.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:56:49.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:49.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:56:49.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:49.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:56:49.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:49.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:49.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230061+0000", 2024-09-18T04:56:49.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8240000000000002e-06 2024-09-18T04:56:49.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:49.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230068+0000", 2024-09-18T04:56:49.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4320000000000004e-06 2024-09-18T04:56:49.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:49.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230073+0000", 2024-09-18T04:56:49.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.921e-06 2024-09-18T04:56:49.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:49.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284153+0000", 2024-09-18T04:56:49.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054080793000000002 2024-09-18T04:56:49.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:49.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284163+0000", 2024-09-18T04:56:49.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.859e-06 2024-09-18T04:56:49.967 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.967 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:49.967 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.967 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15925 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:49.967 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:56:49.967 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.672406923, 2024-09-18T04:56:49.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054134632000000002, 2024-09-18T04:56:49.967 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:49.967 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:49.967 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:49.967 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:49.968 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:49.968 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15925 2024-09-18T04:56:49.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.968 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:49.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:49.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:56:49.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:49.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:56:49.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:49.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:56:49.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:49.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:49.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230081+0000", 2024-09-18T04:56:49.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4109999999999999e-06 2024-09-18T04:56:49.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:49.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230087+0000", 2024-09-18T04:56:49.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.161e-06 2024-09-18T04:56:49.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:49.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230092+0000", 2024-09-18T04:56:49.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9259999999999999e-06 2024-09-18T04:56:49.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:49.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284204+0000", 2024-09-18T04:56:49.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054111546000000003 2024-09-18T04:56:49.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:49.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284214+0000", 2024-09-18T04:56:49.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9639999999999998e-06 2024-09-18T04:56:49.971 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.971 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:49.971 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.971 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15926 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:49.971 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:56:49.971 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.672389599000001, 2024-09-18T04:56:49.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054143193999999999, 2024-09-18T04:56:49.971 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:49.971 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:49.972 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:49.972 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:49.972 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:49.972 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15926 2024-09-18T04:56:49.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.972 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:49.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:49.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:56:49.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:49.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:56:49.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:49.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:56:49.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:49.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:49.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230098+0000", 2024-09-18T04:56:49.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1399999999999998e-06 2024-09-18T04:56:49.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:49.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230104+0000", 2024-09-18T04:56:49.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-18T04:56:49.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:49.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230109+0000", 2024-09-18T04:56:49.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9250000000000003e-06 2024-09-18T04:56:49.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:49.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284231+0000", 2024-09-18T04:56:49.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054122297999999999 2024-09-18T04:56:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284239+0000", 2024-09-18T04:56:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1689999999999992e-06 2024-09-18T04:56:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15927 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:56:49.975 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.672372552000001, 2024-09-18T04:56:49.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054168435000000001, 2024-09-18T04:56:49.976 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:49.976 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:49.976 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:49.976 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:49.976 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:49.976 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15927 2024-09-18T04:56:49.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.976 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:49.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:49.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:56:49.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:56:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:56:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230116+0000", 2024-09-18T04:56:49.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5150000000000001e-06 2024-09-18T04:56:49.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:49.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230122+0000", 2024-09-18T04:56:49.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1190000000000002e-06 2024-09-18T04:56:49.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:49.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230126+0000", 2024-09-18T04:56:49.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.792e-06 2024-09-18T04:56:49.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284273+0000", 2024-09-18T04:56:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054146086000000003 2024-09-18T04:56:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284282+0000", 2024-09-18T04:56:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2590000000000006e-06 2024-09-18T04:56:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15928 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:49.979 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:56:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.672355168999999, 2024-09-18T04:56:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054189739000000001, 2024-09-18T04:56:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15928 2024-09-18T04:56:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:49.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:56:49.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:49.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:56:49.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:49.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230130+0000", 2024-09-18T04:56:49.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:49.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230133+0000", 2024-09-18T04:56:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.244e-06 2024-09-18T04:56:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230139+0000", 2024-09-18T04:56:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9830000000000001e-06 2024-09-18T04:56:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230143+0000", 2024-09-18T04:56:49.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7530000000000001e-06 2024-09-18T04:56:49.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:49.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284313+0000", 2024-09-18T04:56:49.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054169332000000001 2024-09-18T04:56:49.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:49.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284320+0000", 2024-09-18T04:56:49.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8699999999999992e-06 2024-09-18T04:56:49.983 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.983 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15929 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:56:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.672338729, 2024-09-18T04:56:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054218771999999998, 2024-09-18T04:56:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:49.984 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15929 2024-09-18T04:56:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:56:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:56:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:56:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230149+0000", 2024-09-18T04:56:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-18T04:56:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230155+0000", 2024-09-18T04:56:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2469999999999997e-06 2024-09-18T04:56:49.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230160+0000", 2024-09-18T04:56:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7129999999999996e-06 2024-09-18T04:56:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284355+0000", 2024-09-18T04:56:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054194900999999997 2024-09-18T04:56:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:49.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284366+0000", 2024-09-18T04:56:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0993e-05 2024-09-18T04:56:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15930 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:56:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.672321729, 2024-09-18T04:56:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054244713999999999, 2024-09-18T04:56:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:49.988 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:49.989 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:49.989 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:49.989 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15930 2024-09-18T04:56:49.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.989 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:49.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:49.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:56:49.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:49.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:56:49.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:49.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:56:49.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:49.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:49.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230166+0000", 2024-09-18T04:56:49.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1009999999999999e-06 2024-09-18T04:56:49.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:49.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230172+0000", 2024-09-18T04:56:49.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.981e-06 2024-09-18T04:56:49.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230184+0000", 2024-09-18T04:56:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2449000000000001e-05 2024-09-18T04:56:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284400+0000", 2024-09-18T04:56:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054215452999999997 2024-09-18T04:56:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284409+0000", 2024-09-18T04:56:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0839999999999998e-06 2024-09-18T04:56:49.992 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15931 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:56:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.67229594, 2024-09-18T04:56:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054261928000000001, 2024-09-18T04:56:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:49.993 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15931 2024-09-18T04:56:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:56:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:56:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230189+0000", 2024-09-18T04:56:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230192+0000", 2024-09-18T04:56:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6599999999999999e-06 2024-09-18T04:56:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230199+0000", 2024-09-18T04:56:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5570000000000001e-06 2024-09-18T04:56:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230204+0000", 2024-09-18T04:56:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2270000000000001e-06 2024-09-18T04:56:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284442+0000", 2024-09-18T04:56:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054238383000000001 2024-09-18T04:56:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284452+0000", 2024-09-18T04:56:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6069999999999997e-06 2024-09-18T04:56:49.996 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.997 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:49.997 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.997 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:49.997 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:49.997 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:49.998 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:56:49.999 INFO:teuthology.orchestra.run.smithi002.stdout:2202 write 0x3eece1c thru 0x3ef7518 (0xa6fd bytes) 2024-09-18T04:56:49.999 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:56:49.999 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:56:50.000 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:56:50.000 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:56:50.000 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:49.996+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:50.020 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:56:50.021 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:56:50.047 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:56:50.047 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:56:50.066 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:56:50.067 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:56:50.099 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:50.097+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:50.137 INFO:teuthology.orchestra.run.smithi002.stdout:2203 writesame 0xa2efc26 thru 0xa2f01b1 data_size 0x5(0x58c bytes) 2024-09-18T04:56:50.138 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:50.139 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:50.139 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:56:50.139 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:50.152 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:56:50.157 INFO:teuthology.orchestra.run.smithi002.stdout:2204 read 0x25db079 thru 0x25e8eae (0xde36 bytes) 2024-09-18T04:56:50.165 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:50.176 INFO:teuthology.orchestra.run.smithi002.stdout:2206 trunc from 0xbd816c7 to 0x21f0209 2024-09-18T04:56:50.201 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:50.198+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:50.230 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T04:56:50.230 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T04:56:50.230 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-18T04:56:50.231 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T04:56:50.236 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-18T04:56:50.302 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:50.300+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:50.308 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:50.308 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:50.308 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:56:50.308 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:56:50.308 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:50.317 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:56:50.403 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:50.401+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:50.469 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:50.470 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:56:50.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:56:50.470 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:56:50.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.470 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:689 3.4 e24/19 3:21ae0c4f:::rbd_data.11acdc0eab99.0000000000000029:head v 24'7, mlcod=23'4)", 2024-09-18T04:56:50.470 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:27.214992+0000", 2024-09-18T04:56:50.470 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.250379138, 2024-09-18T04:56:50.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001477602, 2024-09-18T04:56:50.471 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:50.471 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:50.471 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:50.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:50.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214992+0000", 2024-09-18T04:56:50.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:50.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214992+0000", 2024-09-18T04:56:50.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:50.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214991+0000", 2024-09-18T04:56:50.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:50.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:50.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.214997+0000", 2024-09-18T04:56:50.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1709999999999999e-06 2024-09-18T04:56:50.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:50.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.215006+0000", 2024-09-18T04:56:50.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8799999999999997e-06 2024-09-18T04:56:50.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:50.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.215012+0000", 2024-09-18T04:56:50.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0870000000000003e-06 2024-09-18T04:56:50.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:50.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.215068+0000", 2024-09-18T04:56:50.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5350999999999999e-05 2024-09-18T04:56:50.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:50.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.215087+0000", 2024-09-18T04:56:50.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9565e-05 2024-09-18T04:56:50.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:50.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.216432+0000", 2024-09-18T04:56:50.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013447439999999999 2024-09-18T04:56:50.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:50.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:27.216469+0000", 2024-09-18T04:56:50.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7370999999999998e-05 2024-09-18T04:56:50.476 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.477 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.477 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.477 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:4372 3.9 e28/19 3:91d61960:::rbd_data.11ac7d9658bc.0000000000000027:head v 28'25, mlcod=25'22)", 2024-09-18T04:56:50.477 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:31.233447+0000", 2024-09-18T04:56:50.478 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.231924020000001, 2024-09-18T04:56:50.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012563520000000001, 2024-09-18T04:56:50.478 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:50.478 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:50.478 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:50.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:50.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:31.233447+0000", 2024-09-18T04:56:50.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:50.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:31.233447+0000", 2024-09-18T04:56:50.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:50.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:31.233446+0000", 2024-09-18T04:56:50.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:50.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:50.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:31.233454+0000", 2024-09-18T04:56:50.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.627e-06 2024-09-18T04:56:50.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:50.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:31.233738+0000", 2024-09-18T04:56:50.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00028350199999999998 2024-09-18T04:56:50.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:50.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:31.233749+0000", 2024-09-18T04:56:50.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1289e-05 2024-09-18T04:56:50.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:50.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:31.233809+0000", 2024-09-18T04:56:50.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0563999999999998e-05 2024-09-18T04:56:50.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:50.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:31.233833+0000", 2024-09-18T04:56:50.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4043999999999999e-05 2024-09-18T04:56:50.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:50.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:31.234664+0000", 2024-09-18T04:56:50.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00083019500000000004 2024-09-18T04:56:50.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:50.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:31.234703+0000", 2024-09-18T04:56:50.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9461e-05 2024-09-18T04:56:50.484 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.485 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.485 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.485 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:11220 3.4 e32/19 3:267b21b8:::rbd_object_map.11ac7a27a554:head v 32'111, mlcod=32'109)", 2024-09-18T04:56:50.485 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.931314+0000", 2024-09-18T04:56:50.485 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.534056652, 2024-09-18T04:56:50.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012730090000000001, 2024-09-18T04:56:50.486 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:50.486 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:50.486 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:50.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:50.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.931314+0000", 2024-09-18T04:56:50.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:50.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.931314+0000", 2024-09-18T04:56:50.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:50.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.931313+0000", 2024-09-18T04:56:50.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:50.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:50.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.931324+0000", 2024-09-18T04:56:50.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.084e-05 2024-09-18T04:56:50.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:50.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.931345+0000", 2024-09-18T04:56:50.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1311999999999999e-05 2024-09-18T04:56:50.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:50.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.931354+0000", 2024-09-18T04:56:50.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2750000000000001e-06 2024-09-18T04:56:50.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:50.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.931427+0000", 2024-09-18T04:56:50.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2978999999999994e-05 2024-09-18T04:56:50.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:50.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.931465+0000", 2024-09-18T04:56:50.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7815999999999998e-05 2024-09-18T04:56:50.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:50.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.932548+0000", 2024-09-18T04:56:50.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010825800000000001 2024-09-18T04:56:50.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:50.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.932587+0000", 2024-09-18T04:56:50.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9526000000000001e-05 2024-09-18T04:56:50.491 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.492 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.492 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.492 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:13694 3.4 e32/19 3:267b21b8:::rbd_object_map.11ac7a27a554:head v 32'141, mlcod=32'139)", 2024-09-18T04:56:50.492 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.103977+0000", 2024-09-18T04:56:50.492 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.361393655000001, 2024-09-18T04:56:50.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012994040000000001, 2024-09-18T04:56:50.492 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:50.492 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:50.492 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:50.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:50.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.103977+0000", 2024-09-18T04:56:50.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:50.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.103977+0000", 2024-09-18T04:56:50.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:50.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.103975+0000", 2024-09-18T04:56:50.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999981 2024-09-18T04:56:50.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:50.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.103988+0000", 2024-09-18T04:56:50.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2367e-05 2024-09-18T04:56:50.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:50.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.104009+0000", 2024-09-18T04:56:50.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1256000000000001e-05 2024-09-18T04:56:50.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:50.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.104018+0000", 2024-09-18T04:56:50.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0259999999999996e-06 2024-09-18T04:56:50.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:50.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.104064+0000", 2024-09-18T04:56:50.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6326000000000003e-05 2024-09-18T04:56:50.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:50.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.104088+0000", 2024-09-18T04:56:50.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3754e-05 2024-09-18T04:56:50.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:50.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.105203+0000", 2024-09-18T04:56:50.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011151379999999999 2024-09-18T04:56:50.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:50.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.105277+0000", 2024-09-18T04:56:50.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3357999999999998e-05 2024-09-18T04:56:50.496 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.497 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.497 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.497 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:15989 3.4 e32/19 3:238b8fe4:::rbd_data.11ac7a27a554.0000000000000005:head v 32'150, mlcod=32'147)", 2024-09-18T04:56:50.497 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.245449+0000", 2024-09-18T04:56:50.497 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.219921868, 2024-09-18T04:56:50.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028002180000000002, 2024-09-18T04:56:50.497 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:50.497 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:50.497 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:50.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:50.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.245449+0000", 2024-09-18T04:56:50.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:50.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.245449+0000", 2024-09-18T04:56:50.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:50.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.245448+0000", 2024-09-18T04:56:50.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:50.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:50.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.246149+0000", 2024-09-18T04:56:50.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00070047499999999999 2024-09-18T04:56:50.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:50.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.246164+0000", 2024-09-18T04:56:50.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5532999999999998e-05 2024-09-18T04:56:50.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:50.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.246171+0000", 2024-09-18T04:56:50.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8990000000000004e-06 2024-09-18T04:56:50.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:50.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.246218+0000", 2024-09-18T04:56:50.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6789000000000002e-05 2024-09-18T04:56:50.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:50.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.246242+0000", 2024-09-18T04:56:50.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3706999999999999e-05 2024-09-18T04:56:50.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:50.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.248217+0000", 2024-09-18T04:56:50.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019752060000000002 2024-09-18T04:56:50.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:50.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.248249+0000", 2024-09-18T04:56:50.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2252000000000002e-05 2024-09-18T04:56:50.501 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.501 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.501 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.501 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:15991 3.4 e32/19 3:21a074f1:::rbd_data.11ac7a27a554.0000000000000006:head v 32'152, mlcod=32'148)", 2024-09-18T04:56:50.501 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.249009+0000", 2024-09-18T04:56:50.502 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.216361315, 2024-09-18T04:56:50.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021236060000000001, 2024-09-18T04:56:50.502 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:50.502 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:50.502 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:50.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:50.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249009+0000", 2024-09-18T04:56:50.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:50.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249009+0000", 2024-09-18T04:56:50.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:50.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249009+0000", 2024-09-18T04:56:50.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:50.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:50.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249228+0000", 2024-09-18T04:56:50.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021908999999999999 2024-09-18T04:56:50.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:50.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249243+0000", 2024-09-18T04:56:50.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5031999999999999e-05 2024-09-18T04:56:50.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:50.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249250+0000", 2024-09-18T04:56:50.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2919999999999998e-06 2024-09-18T04:56:50.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:50.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249294+0000", 2024-09-18T04:56:50.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3479000000000002e-05 2024-09-18T04:56:50.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:50.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249314+0000", 2024-09-18T04:56:50.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0596000000000001e-05 2024-09-18T04:56:50.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:50.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.251093+0000", 2024-09-18T04:56:50.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001778342 2024-09-18T04:56:50.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:50.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.251133+0000", 2024-09-18T04:56:50.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0503999999999997e-05 2024-09-18T04:56:50.506 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.506 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.506 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.506 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:18783 3.4 e32/19 3:267b21b8:::rbd_object_map.11ac7a27a554:head v 32'161, mlcod=32'159)", 2024-09-18T04:56:50.506 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.663855+0000", 2024-09-18T04:56:50.507 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.801516012, 2024-09-18T04:56:50.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0012940969999999999, 2024-09-18T04:56:50.507 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:50.507 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:50.507 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:50.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:50.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.663855+0000", 2024-09-18T04:56:50.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:50.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.663855+0000", 2024-09-18T04:56:50.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:50.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.663853+0000", 2024-09-18T04:56:50.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:50.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:50.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.663865+0000", 2024-09-18T04:56:50.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1554e-05 2024-09-18T04:56:50.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:50.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.663886+0000", 2024-09-18T04:56:50.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0993999999999999e-05 2024-09-18T04:56:50.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:50.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.663901+0000", 2024-09-18T04:56:50.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5396999999999998e-05 2024-09-18T04:56:50.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:50.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.663978+0000", 2024-09-18T04:56:50.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6310999999999995e-05 2024-09-18T04:56:50.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:50.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.664008+0000", 2024-09-18T04:56:50.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0057e-05 2024-09-18T04:56:50.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:50.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.665112+0000", 2024-09-18T04:56:50.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001103885 2024-09-18T04:56:50.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:50.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.665149+0000", 2024-09-18T04:56:50.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7283999999999997e-05 2024-09-18T04:56:50.512 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.512 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.512 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.513 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:18826 3.4 e32/19 3:23a67d65:::rbd_data.11ac7a27a554.0000000000000010:head v 32'162, mlcod=32'159)", 2024-09-18T04:56:50.513 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.666171+0000", 2024-09-18T04:56:50.513 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.799199642, 2024-09-18T04:56:50.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022865569999999998, 2024-09-18T04:56:50.513 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:50.513 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:50.513 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:50.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:50.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666171+0000", 2024-09-18T04:56:50.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:50.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666171+0000", 2024-09-18T04:56:50.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:50.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666170+0000", 2024-09-18T04:56:50.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:50.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:50.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666654+0000", 2024-09-18T04:56:50.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00048368400000000001 2024-09-18T04:56:50.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:50.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666670+0000", 2024-09-18T04:56:50.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5818e-05 2024-09-18T04:56:50.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:50.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666677+0000", 2024-09-18T04:56:50.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5750000000000003e-06 2024-09-18T04:56:50.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:50.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666734+0000", 2024-09-18T04:56:50.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6663999999999998e-05 2024-09-18T04:56:50.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:50.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666757+0000", 2024-09-18T04:56:50.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2691e-05 2024-09-18T04:56:50.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:50.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.668417+0000", 2024-09-18T04:56:50.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001660501 2024-09-18T04:56:50.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:50.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.668458+0000", 2024-09-18T04:56:50.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0364999999999997e-05 2024-09-18T04:56:50.517 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.517 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.517 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.517 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:19048 3.4 e32/19 3:267b21b8:::rbd_object_map.11ac7a27a554:head v 32'164, mlcod=32'162)", 2024-09-18T04:56:50.517 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.680521+0000", 2024-09-18T04:56:50.517 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.784849954, 2024-09-18T04:56:50.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00134738, 2024-09-18T04:56:50.517 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:50.518 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:50.518 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:50.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:50.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.680521+0000", 2024-09-18T04:56:50.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:50.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.680521+0000", 2024-09-18T04:56:50.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:50.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.680519+0000", 2024-09-18T04:56:50.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:50.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:50.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.680530+0000", 2024-09-18T04:56:50.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0200000000000001e-05 2024-09-18T04:56:50.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:50.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.680551+0000", 2024-09-18T04:56:50.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1106000000000001e-05 2024-09-18T04:56:50.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:50.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.680560+0000", 2024-09-18T04:56:50.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0459999999999994e-06 2024-09-18T04:56:50.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:50.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.680651+0000", 2024-09-18T04:56:50.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0781999999999999e-05 2024-09-18T04:56:50.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:50.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.680676+0000", 2024-09-18T04:56:50.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5357000000000001e-05 2024-09-18T04:56:50.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:50.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.681825+0000", 2024-09-18T04:56:50.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011489569999999999 2024-09-18T04:56:50.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:50.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.681868+0000", 2024-09-18T04:56:50.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3259e-05 2024-09-18T04:56:50.521 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.521 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.521 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.522 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:20114 3.4 e32/19 3:267b21b8:::rbd_object_map.11ac7a27a554:head v 32'167, mlcod=32'165)", 2024-09-18T04:56:50.522 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.722905+0000", 2024-09-18T04:56:50.522 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.742465377, 2024-09-18T04:56:50.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014794739999999999, 2024-09-18T04:56:50.522 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:50.522 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:50.522 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:50.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:50.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.722905+0000", 2024-09-18T04:56:50.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:50.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.722905+0000", 2024-09-18T04:56:50.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:50.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.722904+0000", 2024-09-18T04:56:50.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:50.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:50.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.722915+0000", 2024-09-18T04:56:50.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0162e-05 2024-09-18T04:56:50.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:50.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.722931+0000", 2024-09-18T04:56:50.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6174000000000001e-05 2024-09-18T04:56:50.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:50.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.722940+0000", 2024-09-18T04:56:50.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0129999999999999e-06 2024-09-18T04:56:50.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:50.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.723013+0000", 2024-09-18T04:56:50.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3798000000000001e-05 2024-09-18T04:56:50.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:50.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.723044+0000", 2024-09-18T04:56:50.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0726999999999999e-05 2024-09-18T04:56:50.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:50.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.724350+0000", 2024-09-18T04:56:50.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001305891 2024-09-18T04:56:50.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:50.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.724385+0000", 2024-09-18T04:56:50.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4770000000000001e-05 2024-09-18T04:56:50.526 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.526 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.527 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.527 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:21999 3.4 e32/19 3:267b21b8:::rbd_object_map.11ac7a27a554:head v 32'190, mlcod=32'188)", 2024-09-18T04:56:50.527 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:37.004308+0000", 2024-09-18T04:56:50.527 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.461062929000001, 2024-09-18T04:56:50.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001259568, 2024-09-18T04:56:50.527 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:50.527 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:50.527 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:50.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:50.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.004308+0000", 2024-09-18T04:56:50.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:50.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.004308+0000", 2024-09-18T04:56:50.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:50.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.004307+0000", 2024-09-18T04:56:50.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:50.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:50.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.004314+0000", 2024-09-18T04:56:50.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7050000000000002e-06 2024-09-18T04:56:50.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:50.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.004329+0000", 2024-09-18T04:56:50.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5194000000000001e-05 2024-09-18T04:56:50.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:50.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.004336+0000", 2024-09-18T04:56:50.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7610000000000002e-06 2024-09-18T04:56:50.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:50.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.004395+0000", 2024-09-18T04:56:50.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9577999999999998e-05 2024-09-18T04:56:50.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:50.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.004413+0000", 2024-09-18T04:56:50.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8206000000000002e-05 2024-09-18T04:56:50.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:50.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.005535+0000", 2024-09-18T04:56:50.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001121677 2024-09-18T04:56:50.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:50.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.005567+0000", 2024-09-18T04:56:50.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2359e-05 2024-09-18T04:56:50.531 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.531 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.531 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.532 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:22065 3.4 e32/19 3:267b21b8:::rbd_object_map.11ac7a27a554:head v 32'217, mlcod=32'213)", 2024-09-18T04:56:50.532 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:37.044092+0000", 2024-09-18T04:56:50.532 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.421278770000001, 2024-09-18T04:56:50.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001661636, 2024-09-18T04:56:50.532 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:50.532 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:50.532 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:50.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:50.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044092+0000", 2024-09-18T04:56:50.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:50.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044092+0000", 2024-09-18T04:56:50.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:50.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044091+0000", 2024-09-18T04:56:50.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:50.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:50.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044096+0000", 2024-09-18T04:56:50.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9230000000000001e-06 2024-09-18T04:56:50.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:50.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044106+0000", 2024-09-18T04:56:50.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7569999999999999e-06 2024-09-18T04:56:50.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:50.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044111+0000", 2024-09-18T04:56:50.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4839999999999998e-06 2024-09-18T04:56:50.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:50.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044147+0000", 2024-09-18T04:56:50.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6736999999999997e-05 2024-09-18T04:56:50.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:50.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044158+0000", 2024-09-18T04:56:50.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1178999999999999e-05 2024-09-18T04:56:50.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:50.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.045717+0000", 2024-09-18T04:56:50.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015584850000000001 2024-09-18T04:56:50.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:50.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.045754+0000", 2024-09-18T04:56:50.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6671000000000002e-05 2024-09-18T04:56:50.536 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.536 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.536 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.537 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:22066 3.4 e32/19 3:267b21b8:::rbd_object_map.11ac7a27a554:head v 32'218, mlcod=32'214)", 2024-09-18T04:56:50.537 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:37.044877+0000", 2024-09-18T04:56:50.537 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.420493883000001, 2024-09-18T04:56:50.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014606160000000001, 2024-09-18T04:56:50.537 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:50.537 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:50.537 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:50.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:50.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044877+0000", 2024-09-18T04:56:50.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044877+0000", 2024-09-18T04:56:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044876+0000", 2024-09-18T04:56:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:50.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044884+0000", 2024-09-18T04:56:50.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4220000000000003e-06 2024-09-18T04:56:50.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:50.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044896+0000", 2024-09-18T04:56:50.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2418999999999999e-05 2024-09-18T04:56:50.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:50.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044902+0000", 2024-09-18T04:56:50.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3380000000000001e-06 2024-09-18T04:56:50.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:50.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044944+0000", 2024-09-18T04:56:50.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1865e-05 2024-09-18T04:56:50.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:50.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044963+0000", 2024-09-18T04:56:50.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8454999999999999e-05 2024-09-18T04:56:50.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:50.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.046302+0000", 2024-09-18T04:56:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013391259999999999 2024-09-18T04:56:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.046338+0000", 2024-09-18T04:56:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5695000000000003e-05 2024-09-18T04:56:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:22098 3.4 e32/19 3:267b21b8:::rbd_object_map.11ac7a27a554:head v 32'225, mlcod=32'221)", 2024-09-18T04:56:50.541 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:37.062135+0000", 2024-09-18T04:56:50.542 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.403235949000001, 2024-09-18T04:56:50.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001486558, 2024-09-18T04:56:50.542 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:50.542 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:50.542 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:50.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:50.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.062135+0000", 2024-09-18T04:56:50.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:50.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.062135+0000", 2024-09-18T04:56:50.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:50.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.062134+0000", 2024-09-18T04:56:50.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:50.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:50.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.062140+0000", 2024-09-18T04:56:50.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2550000000000003e-06 2024-09-18T04:56:50.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.062153+0000", 2024-09-18T04:56:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2629000000000001e-05 2024-09-18T04:56:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.062160+0000", 2024-09-18T04:56:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9249999999999998e-06 2024-09-18T04:56:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.062201+0000", 2024-09-18T04:56:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0921999999999999e-05 2024-09-18T04:56:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:50.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.062222+0000", 2024-09-18T04:56:50.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1628999999999999e-05 2024-09-18T04:56:50.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:50.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.063558+0000", 2024-09-18T04:56:50.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001335934 2024-09-18T04:56:50.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:50.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.063621+0000", 2024-09-18T04:56:50.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2964000000000002e-05 2024-09-18T04:56:50.546 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.546 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.546 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.546 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:22125 3.4 e32/19 3:267b21b8:::rbd_object_map.11ac7a27a554:head v 32'237, mlcod=32'232)", 2024-09-18T04:56:50.546 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:37.076006+0000", 2024-09-18T04:56:50.546 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.389364923, 2024-09-18T04:56:50.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014754970000000001, 2024-09-18T04:56:50.546 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:50.546 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:50.546 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.076006+0000", 2024-09-18T04:56:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.076006+0000", 2024-09-18T04:56:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.076005+0000", 2024-09-18T04:56:50.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.076012+0000", 2024-09-18T04:56:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4760000000000003e-06 2024-09-18T04:56:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.076024+0000", 2024-09-18T04:56:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2228e-05 2024-09-18T04:56:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.076031+0000", 2024-09-18T04:56:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7469999999999999e-06 2024-09-18T04:56:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.076066+0000", 2024-09-18T04:56:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5182999999999998e-05 2024-09-18T04:56:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.076079+0000", 2024-09-18T04:56:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3155e-05 2024-09-18T04:56:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.077443+0000", 2024-09-18T04:56:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001363946 2024-09-18T04:56:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.077481+0000", 2024-09-18T04:56:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8556999999999997e-05 2024-09-18T04:56:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.550 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:22126 3.4 e32/19 3:267b21b8:::rbd_object_map.11ac7a27a554:head v 32'238, mlcod=32'234)", 2024-09-18T04:56:50.551 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:37.076984+0000", 2024-09-18T04:56:50.551 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.388386403, 2024-09-18T04:56:50.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013169290000000001, 2024-09-18T04:56:50.551 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:50.551 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:50.551 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:50.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:50.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.076984+0000", 2024-09-18T04:56:50.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:50.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.076984+0000", 2024-09-18T04:56:50.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:50.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.076984+0000", 2024-09-18T04:56:50.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:50.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:50.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.076990+0000", 2024-09-18T04:56:50.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6889999999999998e-06 2024-09-18T04:56:50.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.077004+0000", 2024-09-18T04:56:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3635000000000001e-05 2024-09-18T04:56:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.077010+0000", 2024-09-18T04:56:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1840000000000004e-06 2024-09-18T04:56:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.077074+0000", 2024-09-18T04:56:50.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3996000000000002e-05 2024-09-18T04:56:50.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:50.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.077095+0000", 2024-09-18T04:56:50.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0632000000000002e-05 2024-09-18T04:56:50.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:50.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.078254+0000", 2024-09-18T04:56:50.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001158858 2024-09-18T04:56:50.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:50.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.078301+0000", 2024-09-18T04:56:50.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7775999999999998e-05 2024-09-18T04:56:50.555 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.555 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.555 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.555 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:22140 3.0 e32/19 3:04e0ba1b:::rbd_data.11ac7a27a554.00000000000000d3:head v 32'98, mlcod=32'96)", 2024-09-18T04:56:50.555 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:37.083209+0000", 2024-09-18T04:56:50.555 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.382161636999999, 2024-09-18T04:56:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001278176, 2024-09-18T04:56:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.083209+0000", 2024-09-18T04:56:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.083209+0000", 2024-09-18T04:56:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.083208+0000", 2024-09-18T04:56:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.083219+0000", 2024-09-18T04:56:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.028e-05 2024-09-18T04:56:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.083292+0000", 2024-09-18T04:56:50.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3627999999999994e-05 2024-09-18T04:56:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.083301+0000", 2024-09-18T04:56:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4489999999999999e-06 2024-09-18T04:56:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.083349+0000", 2024-09-18T04:56:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8712999999999999e-05 2024-09-18T04:56:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:50.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.083369+0000", 2024-09-18T04:56:50.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9364999999999998e-05 2024-09-18T04:56:50.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:50.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.084449+0000", 2024-09-18T04:56:50.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010797510000000001 2024-09-18T04:56:50.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:50.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.084487+0000", 2024-09-18T04:56:50.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8748999999999998e-05 2024-09-18T04:56:50.559 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.559 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:29070 3.4 e36/19 3:26169c24:::rbd_data.11ac138032e7.0000000000000004:head v 36'263, mlcod=36'261)", 2024-09-18T04:56:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:40.883605+0000", 2024-09-18T04:56:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.5817662230000007, 2024-09-18T04:56:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0014912059999999999, 2024-09-18T04:56:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:50.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:40.883605+0000", 2024-09-18T04:56:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:40.883605+0000", 2024-09-18T04:56:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:40.883603+0000", 2024-09-18T04:56:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999981 2024-09-18T04:56:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:40.883776+0000", 2024-09-18T04:56:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000173518 2024-09-18T04:56:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:40.883795+0000", 2024-09-18T04:56:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8190999999999999e-05 2024-09-18T04:56:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:40.883801+0000", 2024-09-18T04:56:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7440000000000001e-06 2024-09-18T04:56:50.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:40.883851+0000", 2024-09-18T04:56:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9360000000000002e-05 2024-09-18T04:56:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:40.883871+0000", 2024-09-18T04:56:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0182000000000001e-05 2024-09-18T04:56:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:40.885065+0000", 2024-09-18T04:56:50.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001194325 2024-09-18T04:56:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:40.885096+0000", 2024-09-18T04:56:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0568999999999998e-05 2024-09-18T04:56:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:36051 3.4 e42/19 3:25d849c5:::rbd_data.11ac728e6e9a.0000000000000002:head v 42'294, mlcod=40'292)", 2024-09-18T04:56:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:46.443141+0000", 2024-09-18T04:56:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.022229888, 2024-09-18T04:56:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037584760000000002, 2024-09-18T04:56:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443141+0000", 2024-09-18T04:56:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443141+0000", 2024-09-18T04:56:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:50.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443141+0000", 2024-09-18T04:56:50.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:50.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:50.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443812+0000", 2024-09-18T04:56:50.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067104899999999995 2024-09-18T04:56:50.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:50.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443831+0000", 2024-09-18T04:56:50.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9074999999999999e-05 2024-09-18T04:56:50.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443841+0000", 2024-09-18T04:56:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8919999999999995e-06 2024-09-18T04:56:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443933+0000", 2024-09-18T04:56:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2348000000000004e-05 2024-09-18T04:56:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443957+0000", 2024-09-18T04:56:50.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3567e-05 2024-09-18T04:56:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.446862+0000", 2024-09-18T04:56:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029052919999999999 2024-09-18T04:56:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.446899+0000", 2024-09-18T04:56:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7515999999999998e-05 2024-09-18T04:56:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:36052 3.4 e42/19 3:25d849c5:::rbd_data.11ac728e6e9a.0000000000000002:head v 42'296, mlcod=40'292)", 2024-09-18T04:56:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:46.445721+0000", 2024-09-18T04:56:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.0196495499999996, 2024-09-18T04:56:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001516749, 2024-09-18T04:56:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.445721+0000", 2024-09-18T04:56:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.445721+0000", 2024-09-18T04:56:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.445720+0000", 2024-09-18T04:56:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.445729+0000", 2024-09-18T04:56:50.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2099999999999999e-06 2024-09-18T04:56:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.445745+0000", 2024-09-18T04:56:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5852999999999999e-05 2024-09-18T04:56:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.445754+0000", 2024-09-18T04:56:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3219999999999993e-06 2024-09-18T04:56:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:50.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.445834+0000", 2024-09-18T04:56:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0307000000000002e-05 2024-09-18T04:56:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.445857+0000", 2024-09-18T04:56:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3294000000000001e-05 2024-09-18T04:56:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:56:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.447206+0000", 2024-09-18T04:56:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001349233 2024-09-18T04:56:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:50.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.447238+0000", 2024-09-18T04:56:50.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1489000000000003e-05 2024-09-18T04:56:50.573 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.573 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.573 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.573 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.573 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.573 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:50.575 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:56:50.576 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:56:50.576 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:50.576 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 45, 2024-09-18T04:56:50.576 INFO:teuthology.orchestra.run.smithi002.stdout: "fsid": "c0761d0c-69d5-4b64-b8b9-c7d098b2d06d", 2024-09-18T04:56:50.576 INFO:teuthology.orchestra.run.smithi002.stdout: "created": "2024-09-18T04:55:58.216538+0000", 2024-09-18T04:56:50.576 INFO:teuthology.orchestra.run.smithi002.stdout: "modified": "2024-09-18T04:56:50.224497+0000", 2024-09-18T04:56:50.576 INFO:teuthology.orchestra.run.smithi002.stdout: "last_up_change": "2024-09-18T04:56:08.369924+0000", 2024-09-18T04:56:50.577 INFO:teuthology.orchestra.run.smithi002.stdout: "last_in_change": "2024-09-18T04:56:00.333800+0000", 2024-09-18T04:56:50.577 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T04:56:50.577 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_num": 5804032, 2024-09-18T04:56:50.577 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_set": [ 2024-09-18T04:56:50.577 INFO:teuthology.orchestra.run.smithi002.stdout: "nodeep-scrub", 2024-09-18T04:56:50.577 INFO:teuthology.orchestra.run.smithi002.stdout: "pglog_hardlimit", 2024-09-18T04:56:50.577 INFO:teuthology.orchestra.run.smithi002.stdout: "purged_snapdirs", 2024-09-18T04:56:50.577 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_deletes", 2024-09-18T04:56:50.577 INFO:teuthology.orchestra.run.smithi002.stdout: "sortbitwise" 2024-09-18T04:56:50.577 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:56:50.577 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_version": 4, 2024-09-18T04:56:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T04:56:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T04:56:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T04:56:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_snapshot": "", 2024-09-18T04:56:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_max": 3, 2024-09-18T04:56:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd": 8, 2024-09-18T04:56:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: "require_min_compat_client": "luminous", 2024-09-18T04:56:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: "min_compat_client": "luminous", 2024-09-18T04:56:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: "require_osd_release": "reef", 2024-09-18T04:56:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_crimson": false, 2024-09-18T04:56:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: "pools": [ 2024-09-18T04:56:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.578 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 1, 2024-09-18T04:56:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "rbd", 2024-09-18T04:56:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-18T04:56:08.983344+0000", 2024-09-18T04:56:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-18T04:56:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T04:56:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-18T04:56:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-18T04:56:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-18T04:56:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-18T04:56:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-18T04:56:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-18T04:56:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T04:56:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T04:56:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-18T04:56:50.579 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-18T04:56:50.580 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 8, 2024-09-18T04:56:50.580 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 8, 2024-09-18T04:56:50.580 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 8, 2024-09-18T04:56:50.580 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 8, 2024-09-18T04:56:50.580 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 8, 2024-09-18T04:56:50.580 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-18T04:56:50.580 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-18T04:56:50.580 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-18T04:56:50.580 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-18T04:56:50.580 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-18T04:56:50.580 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-18T04:56:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-18T04:56:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "17", 2024-09-18T04:56:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-18T04:56:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T04:56:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T04:56:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-18T04:56:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-18T04:56:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 3, 2024-09-18T04:56:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 17, 2024-09-18T04:56:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-18T04:56:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-18T04:56:50.581 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-18T04:56:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-18T04:56:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-18T04:56:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-18T04:56:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-18T04:56:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-18T04:56:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-18T04:56:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-18T04:56:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-18T04:56:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T04:56:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T04:56:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T04:56:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-18T04:56:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-18T04:56:50.582 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-18T04:56:50.583 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-18T04:56:50.583 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-18T04:56:50.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.583 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-18T04:56:50.583 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-18T04:56:50.583 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-18T04:56:50.583 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-18T04:56:50.583 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-18T04:56:50.583 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T04:56:50.583 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-18T04:56:50.583 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-18T04:56:50.584 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-18T04:56:50.584 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-18T04:56:50.584 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-18T04:56:50.584 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-18T04:56:50.584 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-18T04:56:50.584 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-18T04:56:50.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.584 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-18T04:56:50.584 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 3, 2024-09-18T04:56:50.584 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 3, 2024-09-18T04:56:50.584 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 1, 2024-09-18T04:56:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 3, 2024-09-18T04:56:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 3, 2024-09-18T04:56:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-18T04:56:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-18T04:56:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-18T04:56:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 2, 2024-09-18T04:56:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": ".mgr", 2024-09-18T04:56:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-18T04:56:10.313988+0000", 2024-09-18T04:56:50.585 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-18T04:56:50.586 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-18T04:56:50.586 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-18T04:56:50.586 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-18T04:56:50.586 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-18T04:56:50.586 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-18T04:56:50.586 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-18T04:56:50.586 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-18T04:56:50.586 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T04:56:50.586 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T04:56:50.586 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-18T04:56:50.586 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-18T04:56:50.586 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 1, 2024-09-18T04:56:50.586 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 1, 2024-09-18T04:56:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 1, 2024-09-18T04:56:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 1, 2024-09-18T04:56:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 1, 2024-09-18T04:56:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-18T04:56:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-18T04:56:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-18T04:56:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-18T04:56:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-18T04:56:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-18T04:56:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-18T04:56:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "17", 2024-09-18T04:56:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-18T04:56:50.587 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T04:56:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T04:56:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-18T04:56:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-18T04:56:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-18T04:56:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-18T04:56:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-18T04:56:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-18T04:56:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-18T04:56:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-18T04:56:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-18T04:56:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-18T04:56:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-18T04:56:50.588 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-18T04:56:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-18T04:56:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-18T04:56:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-18T04:56:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T04:56:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T04:56:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T04:56:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-18T04:56:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-18T04:56:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-18T04:56:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-18T04:56:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-18T04:56:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.589 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-18T04:56:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-18T04:56:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-18T04:56:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-18T04:56:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-18T04:56:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T04:56:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-18T04:56:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-18T04:56:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-18T04:56:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-18T04:56:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-18T04:56:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-18T04:56:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_max": 32, 2024-09-18T04:56:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 1 2024-09-18T04:56:50.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-18T04:56:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": {} 2024-09-18T04:56:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-18T04:56:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 8, 2024-09-18T04:56:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 8, 2024-09-18T04:56:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.25, 2024-09-18T04:56:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 2, 2024-09-18T04:56:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 2, 2024-09-18T04:56:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-18T04:56:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-18T04:56:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-18T04:56:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-18T04:56:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "pool_client.0", 2024-09-18T04:56:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-18T04:56:19.562579+0000", 2024-09-18T04:56:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-18T04:56:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T04:56:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-18T04:56:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-18T04:56:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-18T04:56:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-18T04:56:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-18T04:56:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-18T04:56:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T04:56:50.592 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T04:56:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-18T04:56:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-18T04:56:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 32, 2024-09-18T04:56:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 32, 2024-09-18T04:56:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 32, 2024-09-18T04:56:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 32, 2024-09-18T04:56:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 32, 2024-09-18T04:56:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-18T04:56:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-18T04:56:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-18T04:56:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-18T04:56:50.593 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-18T04:56:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-18T04:56:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-18T04:56:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "44", 2024-09-18T04:56:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-18T04:56:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T04:56:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T04:56:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-18T04:56:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-18T04:56:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 23, 2024-09-18T04:56:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 44, 2024-09-18T04:56:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-18T04:56:50.594 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-18T04:56:50.595 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-18T04:56:50.595 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-18T04:56:50.595 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-18T04:56:50.595 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-18T04:56:50.595 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-18T04:56:50.595 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-18T04:56:50.595 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-18T04:56:50.595 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-18T04:56:50.595 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-18T04:56:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T04:56:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T04:56:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T04:56:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-18T04:56:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-18T04:56:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-18T04:56:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-18T04:56:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-18T04:56:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-18T04:56:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-18T04:56:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-18T04:56:50.596 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-18T04:56:50.597 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-18T04:56:50.597 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T04:56:50.597 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-18T04:56:50.597 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-18T04:56:50.597 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-18T04:56:50.597 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-18T04:56:50.597 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-18T04:56:50.597 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-18T04:56:50.597 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-18T04:56:50.597 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-18T04:56:50.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.597 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-18T04:56:50.597 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 1.75, 2024-09-18T04:56:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 1.75, 2024-09-18T04:56:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 1, 2024-09-18T04:56:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 1.75, 2024-09-18T04:56:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 1.75, 2024-09-18T04:56:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-18T04:56:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-18T04:56:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-18T04:56:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:56:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-18T04:56:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-18T04:56:50.598 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "c43f8024-3a3e-424e-aa54-398310ceb70d", 2024-09-18T04:56:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 19, 2024-09-18T04:56:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:50.599 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6824", 2024-09-18T04:56:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6825", 2024-09-18T04:56:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.601 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:50.601 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.601 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.601 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6826", 2024-09-18T04:56:50.601 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:50.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6827", 2024-09-18T04:56:50.606 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6830", 2024-09-18T04:56:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6831", 2024-09-18T04:56:50.607 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6828", 2024-09-18T04:56:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6829", 2024-09-18T04:56:50.608 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6825/4106704527", 2024-09-18T04:56:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6827/4106704527", 2024-09-18T04:56:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6831/4106704527", 2024-09-18T04:56:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6829/4106704527", 2024-09-18T04:56:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.609 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-18T04:56:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "c664b970-145c-4b6f-8852-2f6dcf1fa30d", 2024-09-18T04:56:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 19, 2024-09-18T04:56:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6808", 2024-09-18T04:56:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6809", 2024-09-18T04:56:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6810", 2024-09-18T04:56:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6811", 2024-09-18T04:56:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6814", 2024-09-18T04:56:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6815", 2024-09-18T04:56:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6812", 2024-09-18T04:56:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6813", 2024-09-18T04:56:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6809/1781959259", 2024-09-18T04:56:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6811/1781959259", 2024-09-18T04:56:50.614 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6815/1781959259", 2024-09-18T04:56:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6813/1781959259", 2024-09-18T04:56:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-18T04:56:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "0609ea8c-d975-490e-8844-04a3e4512efd", 2024-09-18T04:56:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:50.615 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 0, 2024-09-18T04:56:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6800", 2024-09-18T04:56:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.616 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6801", 2024-09-18T04:56:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6802", 2024-09-18T04:56:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.617 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6803", 2024-09-18T04:56:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6806", 2024-09-18T04:56:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:50.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6807", 2024-09-18T04:56:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6804", 2024-09-18T04:56:50.619 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6805", 2024-09-18T04:56:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6801/1153570802", 2024-09-18T04:56:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6803/1153570802", 2024-09-18T04:56:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6807/1153570802", 2024-09-18T04:56:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6805/1153570802", 2024-09-18T04:56:50.620 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-18T04:56:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "c6bd5ac8-ab89-4fa2-b843-3b166d1b238b", 2024-09-18T04:56:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:50.621 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 30, 2024-09-18T04:56:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6816", 2024-09-18T04:56:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6817", 2024-09-18T04:56:50.622 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6818", 2024-09-18T04:56:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6819", 2024-09-18T04:56:50.623 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6822", 2024-09-18T04:56:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.624 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6823", 2024-09-18T04:56:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6820", 2024-09-18T04:56:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.625 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6821", 2024-09-18T04:56:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6817/2008321375", 2024-09-18T04:56:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6819/2008321375", 2024-09-18T04:56:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6823/2008321375", 2024-09-18T04:56:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6821/2008321375", 2024-09-18T04:56:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:50.626 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-18T04:56:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "89d6a371-4f4e-465e-a023-d5083cce6e6c", 2024-09-18T04:56:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 19, 2024-09-18T04:56:50.627 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6824", 2024-09-18T04:56:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6825", 2024-09-18T04:56:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.628 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6826", 2024-09-18T04:56:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6827", 2024-09-18T04:56:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:50.629 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6830", 2024-09-18T04:56:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6831", 2024-09-18T04:56:50.630 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6828", 2024-09-18T04:56:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.631 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6829", 2024-09-18T04:56:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6825/2939896598", 2024-09-18T04:56:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6827/2939896598", 2024-09-18T04:56:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6831/2939896598", 2024-09-18T04:56:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6829/2939896598", 2024-09-18T04:56:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:50.632 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-18T04:56:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "da99686b-d979-4753-b4aa-a3b26e9a0497", 2024-09-18T04:56:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 19, 2024-09-18T04:56:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:50.633 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6816", 2024-09-18T04:56:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6817", 2024-09-18T04:56:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6818", 2024-09-18T04:56:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6819", 2024-09-18T04:56:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6822", 2024-09-18T04:56:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6823", 2024-09-18T04:56:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.636 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6820", 2024-09-18T04:56:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6821", 2024-09-18T04:56:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:50.637 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6817/3440879618", 2024-09-18T04:56:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6819/3440879618", 2024-09-18T04:56:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6823/3440879618", 2024-09-18T04:56:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6821/3440879618", 2024-09-18T04:56:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-18T04:56:50.638 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "ceec6aa6-603d-4706-bfea-3c07dbdaed3a", 2024-09-18T04:56:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 41, 2024-09-18T04:56:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:50.639 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6800", 2024-09-18T04:56:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6801", 2024-09-18T04:56:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.640 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6802", 2024-09-18T04:56:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6803", 2024-09-18T04:56:50.641 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6806", 2024-09-18T04:56:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6807", 2024-09-18T04:56:50.642 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6804", 2024-09-18T04:56:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.643 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6805", 2024-09-18T04:56:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6801/2493830504", 2024-09-18T04:56:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6803/2493830504", 2024-09-18T04:56:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6807/2493830504", 2024-09-18T04:56:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6805/2493830504", 2024-09-18T04:56:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-18T04:56:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "6779dfad-f60b-41f4-b666-a99db6dc1814", 2024-09-18T04:56:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 19, 2024-09-18T04:56:50.645 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6808", 2024-09-18T04:56:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6809", 2024-09-18T04:56:50.646 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6810", 2024-09-18T04:56:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6811", 2024-09-18T04:56:50.647 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6814", 2024-09-18T04:56:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.648 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6815", 2024-09-18T04:56:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6812", 2024-09-18T04:56:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:50.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6813", 2024-09-18T04:56:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6809/91038858", 2024-09-18T04:56:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6811/91038858", 2024-09-18T04:56:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6815/91038858", 2024-09-18T04:56:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6813/91038858", 2024-09-18T04:56:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:50.650 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:56:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_xinfo": [ 2024-09-18T04:56:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-18T04:56:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.468213+0000", 2024-09-18T04:56:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:50.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-18T04:56:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.538610+0000", 2024-09-18T04:56:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-18T04:56:50.652 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.500881+0000", 2024-09-18T04:56:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-18T04:56:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:50.653 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.474179+0000", 2024-09-18T04:56:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-18T04:56:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:50.654 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.483737+0000", 2024-09-18T04:56:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-18T04:56:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.460693+0000", 2024-09-18T04:56:50.655 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-18T04:56:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.475970+0000", 2024-09-18T04:56:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.656 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-18T04:56:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.519287+0000", 2024-09-18T04:56:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:56:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap": [ 2024-09-18T04:56:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "1.4", 2024-09-18T04:56:50.657 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-18T04:56:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: 6, 2024-09-18T04:56:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: 7 2024-09-18T04:56:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "3.13", 2024-09-18T04:56:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-18T04:56:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: 3, 2024-09-18T04:56:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: 4 2024-09-18T04:56:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.658 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:56:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_items": [], 2024-09-18T04:56:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_primaries": [], 2024-09-18T04:56:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_temp": [], 2024-09-18T04:56:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_temp": [], 2024-09-18T04:56:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "blocklist": {}, 2024-09-18T04:56:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "range_blocklist": {}, 2024-09-18T04:56:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profiles": { 2024-09-18T04:56:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "default": { 2024-09-18T04:56:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-failure-domain": "osd", 2024-09-18T04:56:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "k": "2", 2024-09-18T04:56:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "m": "1", 2024-09-18T04:56:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "plugin": "jerasure", 2024-09-18T04:56:50.659 INFO:teuthology.orchestra.run.smithi002.stdout: "technique": "reed_sol_van" 2024-09-18T04:56:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps_queue": [], 2024-09-18T04:56:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "new_removed_snaps": [], 2024-09-18T04:56:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "new_purged_snaps": [], 2024-09-18T04:56:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_node_flags": {}, 2024-09-18T04:56:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "device_class_flags": {}, 2024-09-18T04:56:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode": { 2024-09-18T04:56:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_enabled": false, 2024-09-18T04:56:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_bucket_count": 0, 2024-09-18T04:56:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "degraded_stretch_mode": 0, 2024-09-18T04:56:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "recovering_stretch_mode": 0, 2024-09-18T04:56:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_bucket": 0 2024-09-18T04:56:50.660 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:50.661 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:50.661 INFO:tasks.thrashosds.thrasher:j is {'epoch': 45, 'fsid': 'c0761d0c-69d5-4b64-b8b9-c7d098b2d06d', 'created': '2024-09-18T04:55:58.216538+0000', 'modified': '2024-09-18T04:56:50.224497+0000', 'last_up_change': '2024-09-18T04:56:08.369924+0000', 'last_in_change': '2024-09-18T04:56:00.333800+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-18T04:56:08.983344+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 3, 'score_stable': 3, 'optimal_score': 1, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-18T04:56:10.313988+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T04:56:19.562579+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 23, 'snap_epoch': 44, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'c43f8024-3a3e-424e-aa54-398310ceb70d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6824', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6825', 'nonce': 4106704527}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6826', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6827', 'nonce': 4106704527}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6830', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6831', 'nonce': 4106704527}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6828', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6829', 'nonce': 4106704527}]}, 'public_addr': '172.21.15.2:6825/4106704527', 'cluster_addr': '172.21.15.2:6827/4106704527', 'heartbeat_back_addr': '172.21.15.2:6831/4106704527', 'heartbeat_front_addr': '172.21.15.2:6829/4106704527', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c664b970-145c-4b6f-8852-2f6dcf1fa30d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6808', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6809', 'nonce': 1781959259}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6810', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6811', 'nonce': 1781959259}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6814', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6815', 'nonce': 1781959259}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6812', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6813', 'nonce': 1781959259}]}, 'public_addr': '172.21.15.2:6809/1781959259', 'cluster_addr': '172.21.15.2:6811/1781959259', 'heartbeat_back_addr': '172.21.15.2:6815/1781959259', 'heartbeat_front_addr': '172.21.15.2:6813/1781959259', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0609ea8c-d975-490e-8844-04a3e4512efd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6800', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6801', 'nonce': 1153570802}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6802', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6803', 'nonce': 1153570802}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6806', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6807', 'nonce': 1153570802}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6804', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6805', 'nonce': 1153570802}]}, 'public_addr': '172.21.15.2:6801/1153570802', 'cluster_addr': '172.21.15.2:6803/1153570802', 'heartbeat_back_addr': '172.21.15.2:6807/1153570802', 'heartbeat_front_addr': '172.21.15.2:6805/1153570802', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'c6bd5ac8-ab89-4fa2-b843-3b166d1b238b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 30, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6816', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6817', 'nonce': 2008321375}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6818', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6819', 'nonce': 2008321375}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6822', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6823', 'nonce': 2008321375}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6820', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6821', 'nonce': 2008321375}]}, 'public_addr': '172.21.15.2:6817/2008321375', 'cluster_addr': '172.21.15.2:6819/2008321375', 'heartbeat_back_addr': '172.21.15.2:6823/2008321375', 'heartbeat_front_addr': '172.21.15.2:6821/2008321375', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '89d6a371-4f4e-465e-a023-d5083cce6e6c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6824', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6825', 'nonce': 2939896598}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6826', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6827', 'nonce': 2939896598}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6830', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6831', 'nonce': 2939896598}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6828', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6829', 'nonce': 2939896598}]}, 'public_addr': '172.21.15.99:6825/2939896598', 'cluster_addr': '172.21.15.99:6827/2939896598', 'heartbeat_back_addr': '172.21.15.99:6831/2939896598', 'heartbeat_front_addr': '172.21.15.99:6829/2939896598', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'da99686b-d979-4753-b4aa-a3b26e9a0497', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6816', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6817', 'nonce': 3440879618}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6818', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6819', 'nonce': 3440879618}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6822', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6823', 'nonce': 3440879618}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6820', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6821', 'nonce': 3440879618}]}, 'public_addr': '172.21.15.99:6817/3440879618', 'cluster_addr': '172.21.15.99:6819/3440879618', 'heartbeat_back_addr': '172.21.15.99:6823/3440879618', 'heartbeat_front_addr': '172.21.15.99:6821/3440879618', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ceec6aa6-603d-4706-bfea-3c07dbdaed3a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 41, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6800', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6801', 'nonce': 2493830504}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6802', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6803', 'nonce': 2493830504}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6806', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6807', 'nonce': 2493830504}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6804', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6805', 'nonce': 2493830504}]}, 'public_addr': '172.21.15.99:6801/2493830504', 'cluster_addr': '172.21.15.99:6803/2493830504', 'heartbeat_back_addr': '172.21.15.99:6807/2493830504', 'heartbeat_front_addr': '172.21.15.99:6805/2493830504', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '6779dfad-f60b-41f4-b666-a99db6dc1814', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6808', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6809', 'nonce': 91038858}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6810', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6811', 'nonce': 91038858}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6814', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6815', 'nonce': 91038858}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6812', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6813', 'nonce': 91038858}]}, 'public_addr': '172.21.15.99:6809/91038858', 'cluster_addr': '172.21.15.99:6811/91038858', 'heartbeat_back_addr': '172.21.15.99:6815/91038858', 'heartbeat_front_addr': '172.21.15.99:6813/91038858', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.468213+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.538610+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.500881+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.474179+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.483737+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.460693+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.475970+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.519287+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '1.4', 'osds': [6, 7]}, {'pgid': '3.13', 'osds': [3, 4]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T04:56:50.662 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 3.13 2024-09-18T04:56:50.662 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 3.13 2024-09-18T04:56:50.663 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:50.572+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:50.664 INFO:teuthology.orchestra.run.smithi002.stdout:2207 read 0x55afb7 thru 0x56563e (0xa688 bytes) 2024-09-18T04:56:50.664 INFO:teuthology.orchestra.run.smithi002.stdout:2208 writesame 0xac89244 thru 0xac8d942 data_size 0x5(0x46ff bytes) 2024-09-18T04:56:50.664 INFO:teuthology.orchestra.run.smithi002.stdout:2210 writesame 0x7969d34 thru 0x7975c9c data_size 0x1(0xbf69 bytes) 2024-09-18T04:56:50.664 INFO:teuthology.orchestra.run.smithi002.stdout:2211 punch from 0x9e16f06 to 0x9e26952, (0xfa4c bytes) 2024-09-18T04:56:50.664 INFO:teuthology.orchestra.run.smithi002.stdout:2213 read 0xa392c3d thru 0xa39f4fc (0xc8c0 bytes) 2024-09-18T04:56:50.664 INFO:teuthology.orchestra.run.smithi002.stdout:2214 write 0x1bb2b43 thru 0x1bb4c00 (0x20be bytes) 2024-09-18T04:56:50.707 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:50.705+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:50.777 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:50.774+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:50.795 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:50.796 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:50.796 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:56:50.796 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:50.805 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:56:50.877 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:50.875+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:50.957 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:50.957 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:56:50.957 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:56:50.957 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:56:50.957 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:50.966 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:56:50.979 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:50.976+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:50.986 INFO:teuthology.orchestra.run.smithi002.stdout:2215 read 0x4314be2 thru 0x4324402 (0xf821 bytes) 2024-09-18T04:56:50.986 INFO:teuthology.orchestra.run.smithi002.stdout:2216 read 0x5f2c69d thru 0x5f3b137 (0xea9b bytes) 2024-09-18T04:56:50.986 INFO:teuthology.orchestra.run.smithi002.stdout:2217 read 0x4d198c9 thru 0x4d2735d (0xda95 bytes) 2024-09-18T04:56:50.986 INFO:teuthology.orchestra.run.smithi002.stdout:2218 read 0x3acfce6 thru 0x3ad454b (0x4866 bytes) 2024-09-18T04:56:50.986 INFO:teuthology.orchestra.run.smithi002.stdout:2220 read 0x110fede thru 0x1111892 (0x19b5 bytes) 2024-09-18T04:56:50.990 INFO:teuthology.orchestra.run.smithi002.stdout:2221 trunc from 0xac8d943 to 0x8bd70f9 2024-09-18T04:56:50.996 INFO:teuthology.orchestra.run.smithi002.stdout:2222 write 0x42f7166 thru 0x4306fd7 (0xfe72 bytes) 2024-09-18T04:56:51.014 INFO:teuthology.orchestra.run.smithi002.stdout:2224 punch from 0x6270c40 to 0x6274630, (0x39f0 bytes) 2024-09-18T04:56:51.014 INFO:teuthology.orchestra.run.smithi002.stdout:2225 write 0x5f29adf thru 0x5f37a0b (0xdf2d bytes) 2024-09-18T04:56:51.025 INFO:teuthology.orchestra.run.smithi002.stdout:2226 read 0x585a149 thru 0x5860fa6 (0x6e5e bytes) 2024-09-18T04:56:51.027 INFO:teuthology.orchestra.run.smithi002.stdout:2227 trunc from 0x8bd70f9 to 0x9f705f9 2024-09-18T04:56:51.031 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:51.031 INFO:teuthology.orchestra.run.smithi002.stdout:2233 read 0x8fb1f04 thru 0x8fb9d13 (0x7e10 bytes) 2024-09-18T04:56:51.031 INFO:teuthology.orchestra.run.smithi002.stdout:2234 punch from 0x89691a7 to 0x896b047, (0x1ea0 bytes) 2024-09-18T04:56:51.031 INFO:teuthology.orchestra.run.smithi002.stdout:2236 read 0x8208ce8 thru 0x820ec77 (0x5f90 bytes) 2024-09-18T04:56:51.031 INFO:teuthology.orchestra.run.smithi002.stdout:2240 read 0x4fac268 thru 0x4fad3b4 (0x114d bytes) 2024-09-18T04:56:51.032 INFO:teuthology.orchestra.run.smithi002.stdout:2241 read 0x1158f9e thru 0x115ef10 (0x5f73 bytes) 2024-09-18T04:56:51.035 INFO:teuthology.orchestra.run.smithi002.stdout:2243 write 0x6b57259 thru 0x6b5c3c6 (0x516e bytes) 2024-09-18T04:56:51.056 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:51.056 INFO:teuthology.orchestra.run.smithi002.stdout:2246 write 0xebd4ea thru 0xec2b0a (0x5621 bytes) 2024-09-18T04:56:51.080 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:51.077+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:51.111 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:51.111 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:56:51.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:56:51.112 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:56:51.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.112 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11151 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1802240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:51.112 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889328+0000", 2024-09-18T04:56:51.112 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.217479724, 2024-09-18T04:56:51.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040037528000000003, 2024-09-18T04:56:51.113 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:51.113 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:51.113 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:51.113 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:51.113 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.113 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11151 2024-09-18T04:56:51.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.113 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:51.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:51.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889328+0000", 2024-09-18T04:56:51.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:51.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889328+0000", 2024-09-18T04:56:51.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:51.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889328+0000", 2024-09-18T04:56:51.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:51.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889330+0000", 2024-09-18T04:56:51.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.148e-06 2024-09-18T04:56:51.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:51.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889336+0000", 2024-09-18T04:56:51.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8220000000000003e-06 2024-09-18T04:56:51.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889340+0000", 2024-09-18T04:56:51.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.318e-06 2024-09-18T04:56:51.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:51.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929171+0000", 2024-09-18T04:56:51.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039830466000000002 2024-09-18T04:56:51.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:51.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929268+0000", 2024-09-18T04:56:51.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7569e-05 2024-09-18T04:56:51.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:51.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929366+0000", 2024-09-18T04:56:51.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7573999999999998e-05 2024-09-18T04:56:51.120 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.120 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:51.120 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.120 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11152 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1867776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:51.121 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889344+0000", 2024-09-18T04:56:51.121 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.217464139000001, 2024-09-18T04:56:51.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040227974, 2024-09-18T04:56:51.121 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:51.121 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:51.121 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:51.121 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:51.121 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11152 2024-09-18T04:56:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889344+0000", 2024-09-18T04:56:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889344+0000", 2024-09-18T04:56:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889344+0000", 2024-09-18T04:56:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889346+0000", 2024-09-18T04:56:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.99e-06 2024-09-18T04:56:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:51.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889351+0000", 2024-09-18T04:56:51.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4260000000000002e-06 2024-09-18T04:56:51.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889370+0000", 2024-09-18T04:56:51.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9262999999999998e-05 2024-09-18T04:56:51.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:51.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929397+0000", 2024-09-18T04:56:51.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040026679000000003 2024-09-18T04:56:51.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929479+0000", 2024-09-18T04:56:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2467999999999997e-05 2024-09-18T04:56:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929572+0000", 2024-09-18T04:56:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2519e-05 2024-09-18T04:56:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11153 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1916928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889377+0000", 2024-09-18T04:56:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.217431165000001, 2024-09-18T04:56:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040407884999999998, 2024-09-18T04:56:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11153 2024-09-18T04:56:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.126 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889377+0000", 2024-09-18T04:56:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889377+0000", 2024-09-18T04:56:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:51.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889376+0000", 2024-09-18T04:56:51.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:51.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889385+0000", 2024-09-18T04:56:51.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1030000000000002e-06 2024-09-18T04:56:51.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:51.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889392+0000", 2024-09-18T04:56:51.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1799999999999999e-06 2024-09-18T04:56:51.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889396+0000", 2024-09-18T04:56:51.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6890000000000003e-06 2024-09-18T04:56:51.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:51.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929603+0000", 2024-09-18T04:56:51.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040206652000000002 2024-09-18T04:56:51.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:51.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929688+0000", 2024-09-18T04:56:51.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5327000000000005e-05 2024-09-18T04:56:51.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:51.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929785+0000", 2024-09-18T04:56:51.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6362999999999999e-05 2024-09-18T04:56:51.131 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.131 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:51.131 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.131 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11154 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:51.131 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:56:51.131 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.217407335000001, 2024-09-18T04:56:51.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040591919999999997, 2024-09-18T04:56:51.131 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:51.131 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:51.132 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:51.132 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:51.132 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.132 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11154 2024-09-18T04:56:51.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.133 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:51.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:51.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:56:51.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:51.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:56:51.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:51.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889400+0000", 2024-09-18T04:56:51.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:51.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889403+0000", 2024-09-18T04:56:51.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3999999999999999e-06 2024-09-18T04:56:51.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:51.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889409+0000", 2024-09-18T04:56:51.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3380000000000001e-06 2024-09-18T04:56:51.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889414+0000", 2024-09-18T04:56:51.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5789999999999997e-06 2024-09-18T04:56:51.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:51.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929811+0000", 2024-09-18T04:56:51.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040397486000000003 2024-09-18T04:56:51.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:51.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929898+0000", 2024-09-18T04:56:51.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6370000000000001e-05 2024-09-18T04:56:51.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:51.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929993+0000", 2024-09-18T04:56:51.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5136000000000005e-05 2024-09-18T04:56:51.141 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.141 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:51.141 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.141 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11155 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2015232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:51.142 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:56:51.142 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.217390157000001, 2024-09-18T04:56:51.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040769156000000001, 2024-09-18T04:56:51.142 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:51.142 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:51.142 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:51.143 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:51.143 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.143 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11155 2024-09-18T04:56:51.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.143 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:51.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:51.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:56:51.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:51.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:56:51.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:51.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:56:51.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:51.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889420+0000", 2024-09-18T04:56:51.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.181e-06 2024-09-18T04:56:51.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:51.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889426+0000", 2024-09-18T04:56:51.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8599999999999998e-06 2024-09-18T04:56:51.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889430+0000", 2024-09-18T04:56:51.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4399999999999998e-06 2024-09-18T04:56:51.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:51.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930020+0000", 2024-09-18T04:56:51.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040589698 2024-09-18T04:56:51.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:51.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930105+0000", 2024-09-18T04:56:51.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4812e-05 2024-09-18T04:56:51.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:51.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930187+0000", 2024-09-18T04:56:51.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2540999999999995e-05 2024-09-18T04:56:51.151 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.151 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:51.151 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.152 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11156 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2064384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:51.152 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:56:51.152 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.217349863000001, 2024-09-18T04:56:51.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040934419, 2024-09-18T04:56:51.153 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:51.153 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:51.153 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:51.153 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:51.153 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.154 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11156 2024-09-18T04:56:51.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.154 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:51.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:51.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:56:51.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:51.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:56:51.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:51.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889457+0000", 2024-09-18T04:56:51.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:51.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:51.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889462+0000", 2024-09-18T04:56:51.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8350000000000003e-06 2024-09-18T04:56:51.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:51.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889481+0000", 2024-09-18T04:56:51.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8845999999999999e-05 2024-09-18T04:56:51.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889495+0000", 2024-09-18T04:56:51.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4229e-05 2024-09-18T04:56:51.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:51.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930216+0000", 2024-09-18T04:56:51.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040721081999999999 2024-09-18T04:56:51.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:51.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930299+0000", 2024-09-18T04:56:51.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2995000000000001e-05 2024-09-18T04:56:51.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:51.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930393+0000", 2024-09-18T04:56:51.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3185999999999999e-05 2024-09-18T04:56:51.160 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.160 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:51.160 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.160 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11402 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1495040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:51.161 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896220+0000", 2024-09-18T04:56:51.161 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.210587893, 2024-09-18T04:56:51.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040050388999999999, 2024-09-18T04:56:51.161 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:51.161 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:51.161 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:51.161 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:51.161 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.161 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11402 2024-09-18T04:56:51.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.161 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:51.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:51.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896220+0000", 2024-09-18T04:56:51.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:51.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896220+0000", 2024-09-18T04:56:51.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:51.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896220+0000", 2024-09-18T04:56:51.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:51.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896222+0000", 2024-09-18T04:56:51.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0049999999999999e-06 2024-09-18T04:56:51.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:51.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896227+0000", 2024-09-18T04:56:51.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6130000000000003e-06 2024-09-18T04:56:51.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896232+0000", 2024-09-18T04:56:51.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4100000000000001e-06 2024-09-18T04:56:51.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:51.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936072+0000", 2024-09-18T04:56:51.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039840253999999999 2024-09-18T04:56:51.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:51.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936175+0000", 2024-09-18T04:56:51.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103037 2024-09-18T04:56:51.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:51.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936271+0000", 2024-09-18T04:56:51.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5383999999999993e-05 2024-09-18T04:56:51.166 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.167 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:51.167 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.167 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11403 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1544192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:51.167 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896250+0000", 2024-09-18T04:56:51.167 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.210558514000001, 2024-09-18T04:56:51.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040244017999999999, 2024-09-18T04:56:51.168 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:51.168 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:51.168 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:51.168 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:51.168 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.168 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11403 2024-09-18T04:56:51.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.168 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:51.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:51.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896250+0000", 2024-09-18T04:56:51.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:51.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896250+0000", 2024-09-18T04:56:51.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:51.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896249+0000", 2024-09-18T04:56:51.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:51.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896251+0000", 2024-09-18T04:56:51.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.136e-06 2024-09-18T04:56:51.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:51.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896256+0000", 2024-09-18T04:56:51.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9350000000000002e-06 2024-09-18T04:56:51.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896259+0000", 2024-09-18T04:56:51.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8839999999999998e-06 2024-09-18T04:56:51.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:51.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936320+0000", 2024-09-18T04:56:51.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040061173999999998 2024-09-18T04:56:51.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:51.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936401+0000", 2024-09-18T04:56:51.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0987000000000002e-05 2024-09-18T04:56:51.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:51.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936494+0000", 2024-09-18T04:56:51.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2281999999999995e-05 2024-09-18T04:56:51.174 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.174 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:51.174 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.175 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11404 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1609728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:51.175 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:56:51.175 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.210545700999999, 2024-09-18T04:56:51.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040669012999999997, 2024-09-18T04:56:51.175 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:51.175 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:51.175 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:51.175 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:51.175 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.175 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11404 2024-09-18T04:56:51.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.176 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:51.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:51.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:56:51.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:51.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:56:51.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:51.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:56:51.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:56:51.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:51.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896263+0000", 2024-09-18T04:56:51.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2729999999999999e-06 2024-09-18T04:56:51.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:51.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896266+0000", 2024-09-18T04:56:51.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8200000000000001e-06 2024-09-18T04:56:51.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896269+0000", 2024-09-18T04:56:51.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3199999999999998e-06 2024-09-18T04:56:51.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:51.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936538+0000", 2024-09-18T04:56:51.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040269394999999999 2024-09-18T04:56:51.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:51.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936657+0000", 2024-09-18T04:56:51.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119498 2024-09-18T04:56:51.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:51.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936931+0000", 2024-09-18T04:56:51.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027391900000000002 2024-09-18T04:56:51.179 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.179 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:51.179 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.179 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11405 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1658880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:51.180 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:56:51.180 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.210537443, 2024-09-18T04:56:51.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040891034999999999, 2024-09-18T04:56:51.180 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:51.180 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:51.180 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:51.180 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:51.180 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.180 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11405 2024-09-18T04:56:51.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.180 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:51.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:51.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:56:51.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:51.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:56:51.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:51.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896270+0000", 2024-09-18T04:56:51.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:56:51.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:51.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896272+0000", 2024-09-18T04:56:51.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.124e-06 2024-09-18T04:56:51.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:51.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896274+0000", 2024-09-18T04:56:51.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8660000000000002e-06 2024-09-18T04:56:51.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896285+0000", 2024-09-18T04:56:51.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0101e-05 2024-09-18T04:56:51.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:51.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936986+0000", 2024-09-18T04:56:51.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040701213999999999 2024-09-18T04:56:51.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:51.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937070+0000", 2024-09-18T04:56:51.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4211999999999999e-05 2024-09-18T04:56:51.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:51.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937162+0000", 2024-09-18T04:56:51.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1728e-05 2024-09-18T04:56:51.184 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.184 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:51.184 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.184 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11406 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1708032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:51.184 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:56:51.184 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.210521274, 2024-09-18T04:56:51.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041085251000000003, 2024-09-18T04:56:51.185 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:51.185 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:51.185 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:51.185 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:51.185 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.185 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11406 2024-09-18T04:56:51.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.185 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:51.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:51.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:56:51.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:51.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:56:51.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:51.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:56:51.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:56:51.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:51.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896288+0000", 2024-09-18T04:56:51.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2440000000000001e-06 2024-09-18T04:56:51.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:51.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896291+0000", 2024-09-18T04:56:51.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8289999999999999e-06 2024-09-18T04:56:51.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896293+0000", 2024-09-18T04:56:51.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5399999999999998e-06 2024-09-18T04:56:51.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:51.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937201+0000", 2024-09-18T04:56:51.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040907418000000001 2024-09-18T04:56:51.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:51.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937283+0000", 2024-09-18T04:56:51.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2460999999999995e-05 2024-09-18T04:56:51.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:51.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937372+0000", 2024-09-18T04:56:51.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8987999999999996e-05 2024-09-18T04:56:51.189 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.189 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:51.189 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.189 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11407 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:51.189 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:56:51.189 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.210511454000001, 2024-09-18T04:56:51.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041438580000000003, 2024-09-18T04:56:51.189 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:51.190 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:51.190 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:51.190 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:51.190 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.190 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11407 2024-09-18T04:56:51.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.190 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:51.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:51.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:56:51.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:51.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:56:51.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:51.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896296+0000", 2024-09-18T04:56:51.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:51.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896298+0000", 2024-09-18T04:56:51.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:56:51.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:51.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896304+0000", 2024-09-18T04:56:51.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6570000000000002e-06 2024-09-18T04:56:51.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896308+0000", 2024-09-18T04:56:51.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0940000000000001e-06 2024-09-18T04:56:51.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:51.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937540+0000", 2024-09-18T04:56:51.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041231469 2024-09-18T04:56:51.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:51.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937625+0000", 2024-09-18T04:56:51.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5767999999999996e-05 2024-09-18T04:56:51.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:51.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937735+0000", 2024-09-18T04:56:51.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010974 2024-09-18T04:56:51.194 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.194 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:51.194 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.194 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11408 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:51.194 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:56:51.194 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.210495815, 2024-09-18T04:56:51.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041653055000000001, 2024-09-18T04:56:51.194 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:51.194 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:51.194 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:51.194 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:51.194 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.195 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11408 2024-09-18T04:56:51.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.195 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:51.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:51.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:56:51.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:51.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:56:51.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:51.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:56:51.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:51.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896314+0000", 2024-09-18T04:56:51.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0140000000000001e-06 2024-09-18T04:56:51.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:51.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896320+0000", 2024-09-18T04:56:51.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-18T04:56:51.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896324+0000", 2024-09-18T04:56:51.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4410000000000003e-06 2024-09-18T04:56:51.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:51.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937769+0000", 2024-09-18T04:56:51.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041445243 2024-09-18T04:56:51.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:51.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937861+0000", 2024-09-18T04:56:51.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1471000000000006e-05 2024-09-18T04:56:51.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:51.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937965+0000", 2024-09-18T04:56:51.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104516 2024-09-18T04:56:51.198 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.198 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:51.198 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.198 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11409 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:51.199 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:56:51.199 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.210479759, 2024-09-18T04:56:51.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041840467999999999, 2024-09-18T04:56:51.199 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:51.199 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:51.199 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:51.199 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:51.199 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.199 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11409 2024-09-18T04:56:51.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.199 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:51.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:51.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:56:51.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:51.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:56:51.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:51.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:56:51.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:51.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896331+0000", 2024-09-18T04:56:51.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5950000000000001e-06 2024-09-18T04:56:51.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:51.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896336+0000", 2024-09-18T04:56:51.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7259999999999999e-06 2024-09-18T04:56:51.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896341+0000", 2024-09-18T04:56:51.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5619999999999997e-06 2024-09-18T04:56:51.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:51.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937995+0000", 2024-09-18T04:56:51.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041654060999999999 2024-09-18T04:56:51.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:51.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938078+0000", 2024-09-18T04:56:51.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2869999999999998e-05 2024-09-18T04:56:51.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:51.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938169+0000", 2024-09-18T04:56:51.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1040999999999998e-05 2024-09-18T04:56:51.203 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.203 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:51.203 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.203 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11410 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1921024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:51.203 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:56:51.203 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.210463276, 2024-09-18T04:56:51.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042060342000000001, 2024-09-18T04:56:51.204 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:51.204 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:51.204 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:51.204 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:51.204 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.204 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11410 2024-09-18T04:56:51.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.204 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:51.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:51.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:56:51.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:51.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:56:51.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:51.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896344+0000", 2024-09-18T04:56:51.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:51.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896360+0000", 2024-09-18T04:56:51.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5068e-05 2024-09-18T04:56:51.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:51.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896366+0000", 2024-09-18T04:56:51.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5119999999999997e-06 2024-09-18T04:56:51.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896371+0000", 2024-09-18T04:56:51.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:56:51.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:51.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938198+0000", 2024-09-18T04:56:51.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041826579000000003 2024-09-18T04:56:51.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:51.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938297+0000", 2024-09-18T04:56:51.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9067000000000005e-05 2024-09-18T04:56:51.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:51.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938405+0000", 2024-09-18T04:56:51.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108492 2024-09-18T04:56:51.207 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.207 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:51.208 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.208 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11411 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1970176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:51.208 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:56:51.208 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.210432433999999, 2024-09-18T04:56:51.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042259191000000002, 2024-09-18T04:56:51.208 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:51.208 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:51.208 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:51.208 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:51.208 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.208 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11411 2024-09-18T04:56:51.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.209 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:51.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:51.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:56:51.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:51.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:56:51.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:51.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896375+0000", 2024-09-18T04:56:51.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:51.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896377+0000", 2024-09-18T04:56:51.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2699999999999999e-06 2024-09-18T04:56:51.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:51.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896383+0000", 2024-09-18T04:56:51.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6740000000000002e-06 2024-09-18T04:56:51.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896388+0000", 2024-09-18T04:56:51.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:56:51.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:51.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938435+0000", 2024-09-18T04:56:51.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042047169000000002 2024-09-18T04:56:51.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:51.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938534+0000", 2024-09-18T04:56:51.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8939999999999998e-05 2024-09-18T04:56:51.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:51.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938635+0000", 2024-09-18T04:56:51.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101208 2024-09-18T04:56:51.212 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.212 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:51.212 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.212 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11412 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:51.212 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:56:51.212 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.210416452, 2024-09-18T04:56:51.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042828079999999998, 2024-09-18T04:56:51.213 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:51.213 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:51.213 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:51.213 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:51.213 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.217 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11412 2024-09-18T04:56:51.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.217 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:51.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:51.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:56:51.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:51.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:56:51.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:51.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896391+0000", 2024-09-18T04:56:51.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:51.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896393+0000", 2024-09-18T04:56:51.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0810000000000001e-06 2024-09-18T04:56:51.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:51.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896399+0000", 2024-09-18T04:56:51.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7230000000000001e-06 2024-09-18T04:56:51.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896403+0000", 2024-09-18T04:56:51.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3470000000000001e-06 2024-09-18T04:56:51.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:51.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939043+0000", 2024-09-18T04:56:51.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042639829999999997 2024-09-18T04:56:51.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:51.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939124+0000", 2024-09-18T04:56:51.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0809000000000005e-05 2024-09-18T04:56:51.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:51.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939220+0000", 2024-09-18T04:56:51.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5659e-05 2024-09-18T04:56:51.221 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.221 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:51.221 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.221 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11413 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2068480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:51.221 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:56:51.221 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.210400757, 2024-09-18T04:56:51.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043042108000000003, 2024-09-18T04:56:51.221 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:51.222 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:51.222 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:51.222 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:51.222 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.222 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11413 2024-09-18T04:56:51.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.222 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:51.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:51.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:56:51.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:51.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:56:51.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:51.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:56:51.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:51.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896409+0000", 2024-09-18T04:56:51.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.165e-06 2024-09-18T04:56:51.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:51.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896415+0000", 2024-09-18T04:56:51.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6149999999999996e-06 2024-09-18T04:56:51.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896419+0000", 2024-09-18T04:56:51.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2649999999999998e-06 2024-09-18T04:56:51.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:51.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939262+0000", 2024-09-18T04:56:51.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042843444000000001 2024-09-18T04:56:51.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:51.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939351+0000", 2024-09-18T04:56:51.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8944999999999996e-05 2024-09-18T04:56:51.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:51.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939449+0000", 2024-09-18T04:56:51.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8029000000000006e-05 2024-09-18T04:56:51.225 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.225 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:51.226 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.226 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11669 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2023424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:51.226 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.907990+0000", 2024-09-18T04:56:51.226 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.198817863, 2024-09-18T04:56:51.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040045942000000001, 2024-09-18T04:56:51.226 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:51.226 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:51.226 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:51.226 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:51.226 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.226 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11669 2024-09-18T04:56:51.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.227 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:51.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:51.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.907990+0000", 2024-09-18T04:56:51.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:51.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.907990+0000", 2024-09-18T04:56:51.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:51.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.907989+0000", 2024-09-18T04:56:51.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:51.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:51.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.907993+0000", 2024-09-18T04:56:51.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6109999999999998e-06 2024-09-18T04:56:51.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:51.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908003+0000", 2024-09-18T04:56:51.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7610000000000002e-06 2024-09-18T04:56:51.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908009+0000", 2024-09-18T04:56:51.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7660000000000004e-06 2024-09-18T04:56:51.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:51.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.947929+0000", 2024-09-18T04:56:51.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039920297 2024-09-18T04:56:51.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:51.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.947982+0000", 2024-09-18T04:56:51.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3176e-05 2024-09-18T04:56:51.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:51.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.948036+0000", 2024-09-18T04:56:51.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4052999999999999e-05 2024-09-18T04:56:51.230 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.230 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:51.230 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.230 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11670 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2072576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:51.230 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.908014+0000", 2024-09-18T04:56:51.230 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.198793982, 2024-09-18T04:56:51.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040179074000000002, 2024-09-18T04:56:51.231 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:51.231 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:51.231 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:51.231 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:51.231 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.231 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11670 2024-09-18T04:56:51.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.231 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:51.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:51.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908014+0000", 2024-09-18T04:56:51.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:51.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908014+0000", 2024-09-18T04:56:51.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:51.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:51.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908014+0000", 2024-09-18T04:56:51.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:51.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908016+0000", 2024-09-18T04:56:51.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3889999999999999e-06 2024-09-18T04:56:51.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:51.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908023+0000", 2024-09-18T04:56:51.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5599999999999999e-06 2024-09-18T04:56:51.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908028+0000", 2024-09-18T04:56:51.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.242e-06 2024-09-18T04:56:51.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:51.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.948075+0000", 2024-09-18T04:56:51.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040047117 2024-09-18T04:56:51.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:51.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.948128+0000", 2024-09-18T04:56:51.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3127999999999997e-05 2024-09-18T04:56:51.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:51.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:51.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:51.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.948193+0000", 2024-09-18T04:56:51.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5042000000000005e-05 2024-09-18T04:56:51.234 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.235 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:51.235 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.235 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:51.235 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:51.235 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:51.236 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:56:51.237 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:51.234+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:51.237 INFO:teuthology.orchestra.run.smithi002.stderr:clear 3.13 pg_upmap mapping 2024-09-18T04:56:51.248 INFO:teuthology.orchestra.run.smithi002.stdout:2248 punch from 0x1c1a422 to 0x1c1b79d, (0x137b bytes) 2024-09-18T04:56:51.254 INFO:teuthology.orchestra.run.smithi002.stdout:2250 trunc from 0x9f705f9 to 0xe64c65c 2024-09-18T04:56:51.259 INFO:teuthology.orchestra.run.smithi002.stdout:2252 punch from 0x10720 to 0x1b3f4, (0xacd4 bytes) 2024-09-18T04:56:51.259 INFO:teuthology.orchestra.run.smithi002.stdout:2253 punch from 0xd1bf217 to 0xd1cc287, (0xd070 bytes) 2024-09-18T04:56:51.259 INFO:teuthology.orchestra.run.smithi002.stdout:2254 writesame 0xade3477 thru 0xade365f data_size 0x1(0x1e9 bytes) 2024-09-18T04:56:51.262 INFO:teuthology.orchestra.run.smithi002.stdout:2256 read 0x8bdae8 thru 0x8c9063 (0xb57c bytes) 2024-09-18T04:56:51.276 INFO:teuthology.orchestra.run.smithi002.stdout:2257 trunc from 0xe64c65c to 0x7e7b69a 2024-09-18T04:56:51.286 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:51.286 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:51.286 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:51.286 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:51.298 INFO:teuthology.orchestra.run.smithi002.stdout:2263 writesame 0xd61d1b7 thru 0xd624bcf data_size 0x45(0x7a19 bytes) 2024-09-18T04:56:51.298 INFO:teuthology.orchestra.run.smithi002.stdout:2264 write 0x7864a80 thru 0x786d444 (0x89c5 bytes) 2024-09-18T04:56:51.337 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:51.335+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:51.344 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:51.344 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:51.344 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:56:51.344 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:51.353 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:56:51.438 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:51.436+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:51.454 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:51.455 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:51.455 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:56:51.455 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:56:51.455 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:51.463 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:56:51.539 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:51.537+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:51.567 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:51.567 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:56:51.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:56:51.567 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:56:51.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.567 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:5082 3.1a 3:5d34c0ec:::rbd_data.11ac14fc1d07.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 26797226~22514] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:51.567 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.444662+0000", 2024-09-18T04:56:51.567 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.119279769999999, 2024-09-18T04:56:51.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048360470000000003, 2024-09-18T04:56:51.568 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:51.568 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:51.568 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:51.568 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:51.568 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.568 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 5082 2024-09-18T04:56:51.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.568 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:51.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:51.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.444662+0000", 2024-09-18T04:56:51.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:51.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.444662+0000", 2024-09-18T04:56:51.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:51.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.444661+0000", 2024-09-18T04:56:51.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:51.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:51.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.469347+0000", 2024-09-18T04:56:51.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024685830999999998 2024-09-18T04:56:51.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:51.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.469368+0000", 2024-09-18T04:56:51.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1166999999999999e-05 2024-09-18T04:56:51.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.469377+0000", 2024-09-18T04:56:51.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5790000000000004e-06 2024-09-18T04:56:51.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:51.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.469420+0000", 2024-09-18T04:56:51.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3238999999999997e-05 2024-09-18T04:56:51.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:51.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.469526+0000", 2024-09-18T04:56:51.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010607699999999999 2024-09-18T04:56:51.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:51.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.469895+0000", 2024-09-18T04:56:51.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036904 2024-09-18T04:56:51.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:51.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.481413+0000", 2024-09-18T04:56:51.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011517583 2024-09-18T04:56:51.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:51.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.492856+0000", 2024-09-18T04:56:51.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011443511999999999 2024-09-18T04:56:51.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:51.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.492944+0000", 2024-09-18T04:56:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7835000000000002e-05 2024-09-18T04:56:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.493023+0000", 2024-09-18T04:56:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8617999999999998e-05 2024-09-18T04:56:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61086 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 1538048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.108826+0000", 2024-09-18T04:56:51.573 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.45511583300000003, 2024-09-18T04:56:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043247130000000002, 2024-09-18T04:56:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61086 2024-09-18T04:56:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:51.574 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108826+0000", 2024-09-18T04:56:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108826+0000", 2024-09-18T04:56:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108826+0000", 2024-09-18T04:56:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108828+0000", 2024-09-18T04:56:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.035e-06 2024-09-18T04:56:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108833+0000", 2024-09-18T04:56:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6489999999999996e-06 2024-09-18T04:56:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108838+0000", 2024-09-18T04:56:51.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3220000000000003e-06 2024-09-18T04:56:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.151968+0000", 2024-09-18T04:56:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043130773999999997 2024-09-18T04:56:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.152017+0000", 2024-09-18T04:56:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8439999999999997e-05 2024-09-18T04:56:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.152073+0000", 2024-09-18T04:56:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6270000000000002e-05 2024-09-18T04:56:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61087 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 1553408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.108842+0000", 2024-09-18T04:56:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.45509999400000001, 2024-09-18T04:56:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043416264000000003, 2024-09-18T04:56:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:51.578 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61087 2024-09-18T04:56:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108842+0000", 2024-09-18T04:56:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108842+0000", 2024-09-18T04:56:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108842+0000", 2024-09-18T04:56:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108844+0000", 2024-09-18T04:56:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9929999999999998e-06 2024-09-18T04:56:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108849+0000", 2024-09-18T04:56:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6760000000000004e-06 2024-09-18T04:56:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108854+0000", 2024-09-18T04:56:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8180000000000003e-06 2024-09-18T04:56:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.152098+0000", 2024-09-18T04:56:51.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043243990000000003 2024-09-18T04:56:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.152179+0000", 2024-09-18T04:56:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1043000000000003e-05 2024-09-18T04:56:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.152258+0000", 2024-09-18T04:56:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9060000000000005e-05 2024-09-18T04:56:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:51.582 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61088 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 1568768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.108858+0000", 2024-09-18T04:56:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.45508380900000001, 2024-09-18T04:56:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043569878999999999, 2024-09-18T04:56:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61088 2024-09-18T04:56:51.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108858+0000", 2024-09-18T04:56:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108858+0000", 2024-09-18T04:56:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108858+0000", 2024-09-18T04:56:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108860+0000", 2024-09-18T04:56:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.88e-06 2024-09-18T04:56:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108865+0000", 2024-09-18T04:56:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5310000000000001e-06 2024-09-18T04:56:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108870+0000", 2024-09-18T04:56:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8640000000000004e-06 2024-09-18T04:56:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:51.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.152285+0000", 2024-09-18T04:56:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043415020999999998 2024-09-18T04:56:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.152347+0000", 2024-09-18T04:56:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2459000000000006e-05 2024-09-18T04:56:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.152428+0000", 2024-09-18T04:56:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0433999999999996e-05 2024-09-18T04:56:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:51.587 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.588 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61130 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1413120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.110351+0000", 2024-09-18T04:56:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.45359040099999998, 2024-09-18T04:56:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043155852000000001, 2024-09-18T04:56:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61130 2024-09-18T04:56:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:51.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110351+0000", 2024-09-18T04:56:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110351+0000", 2024-09-18T04:56:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110351+0000", 2024-09-18T04:56:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110353+0000", 2024-09-18T04:56:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2539999999999999e-06 2024-09-18T04:56:51.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110360+0000", 2024-09-18T04:56:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3119999999999999e-06 2024-09-18T04:56:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110365+0000", 2024-09-18T04:56:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2270000000000001e-06 2024-09-18T04:56:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:51.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.153376+0000", 2024-09-18T04:56:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043011484000000003 2024-09-18T04:56:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.153443+0000", 2024-09-18T04:56:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7187000000000006e-05 2024-09-18T04:56:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.153507+0000", 2024-09-18T04:56:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3825000000000006e-05 2024-09-18T04:56:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.593 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61131 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1428480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.110394+0000", 2024-09-18T04:56:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.45354787299999999, 2024-09-18T04:56:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043272860000000003, 2024-09-18T04:56:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.594 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61131 2024-09-18T04:56:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110394+0000", 2024-09-18T04:56:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110394+0000", 2024-09-18T04:56:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110393+0000", 2024-09-18T04:56:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110396+0000", 2024-09-18T04:56:51.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.999e-06 2024-09-18T04:56:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110404+0000", 2024-09-18T04:56:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.661e-06 2024-09-18T04:56:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110410+0000", 2024-09-18T04:56:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5130000000000004e-06 2024-09-18T04:56:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.153533+0000", 2024-09-18T04:56:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043123131000000002 2024-09-18T04:56:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.153600+0000", 2024-09-18T04:56:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7361999999999996e-05 2024-09-18T04:56:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.153667+0000", 2024-09-18T04:56:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6762999999999998e-05 2024-09-18T04:56:51.598 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61132 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1443840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.110414+0000", 2024-09-18T04:56:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.45352747599999998, 2024-09-18T04:56:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043417771000000001, 2024-09-18T04:56:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:51.599 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61132 2024-09-18T04:56:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110414+0000", 2024-09-18T04:56:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:51.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110414+0000", 2024-09-18T04:56:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110414+0000", 2024-09-18T04:56:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110416+0000", 2024-09-18T04:56:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3089999999999998e-06 2024-09-18T04:56:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110422+0000", 2024-09-18T04:56:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9540000000000001e-06 2024-09-18T04:56:51.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110427+0000", 2024-09-18T04:56:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1100000000000002e-06 2024-09-18T04:56:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.153694+0000", 2024-09-18T04:56:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043266616000000001 2024-09-18T04:56:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:51.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.153765+0000", 2024-09-18T04:56:51.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1557999999999995e-05 2024-09-18T04:56:51.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:51.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.153832+0000", 2024-09-18T04:56:51.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6674999999999994e-05 2024-09-18T04:56:51.604 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.604 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:51.604 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.604 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61133 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1459200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:51.605 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.110438+0000", 2024-09-18T04:56:51.605 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.45350430800000002, 2024-09-18T04:56:51.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043566147999999999, 2024-09-18T04:56:51.605 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:51.605 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:51.605 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:51.605 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:51.605 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.605 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61133 2024-09-18T04:56:51.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110438+0000", 2024-09-18T04:56:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110438+0000", 2024-09-18T04:56:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110437+0000", 2024-09-18T04:56:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110440+0000", 2024-09-18T04:56:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5639999999999999e-06 2024-09-18T04:56:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110447+0000", 2024-09-18T04:56:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0639999999999996e-06 2024-09-18T04:56:51.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110452+0000", 2024-09-18T04:56:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0649999999999998e-06 2024-09-18T04:56:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.153861+0000", 2024-09-18T04:56:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043409534 2024-09-18T04:56:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:51.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.153933+0000", 2024-09-18T04:56:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1730999999999995e-05 2024-09-18T04:56:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154004+0000", 2024-09-18T04:56:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0610000000000003e-05 2024-09-18T04:56:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61134 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:51.609 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.110456+0000", 2024-09-18T04:56:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.45348577699999998, 2024-09-18T04:56:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043713114999999997, 2024-09-18T04:56:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61134 2024-09-18T04:56:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:51.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110456+0000", 2024-09-18T04:56:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110456+0000", 2024-09-18T04:56:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110456+0000", 2024-09-18T04:56:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110458+0000", 2024-09-18T04:56:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1380000000000001e-06 2024-09-18T04:56:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110464+0000", 2024-09-18T04:56:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.212e-06 2024-09-18T04:56:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110469+0000", 2024-09-18T04:56:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7779999999999998e-06 2024-09-18T04:56:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154033+0000", 2024-09-18T04:56:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043563892999999999 2024-09-18T04:56:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154104+0000", 2024-09-18T04:56:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1699000000000006e-05 2024-09-18T04:56:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:51.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154169+0000", 2024-09-18T04:56:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4775000000000002e-05 2024-09-18T04:56:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61135 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1489920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.110473+0000", 2024-09-18T04:56:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.45346871399999999, 2024-09-18T04:56:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.04384685, 2024-09-18T04:56:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:51.614 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61135 2024-09-18T04:56:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110473+0000", 2024-09-18T04:56:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110473+0000", 2024-09-18T04:56:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110473+0000", 2024-09-18T04:56:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:51.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110475+0000", 2024-09-18T04:56:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1129999999999999e-06 2024-09-18T04:56:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110487+0000", 2024-09-18T04:56:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2507000000000001e-05 2024-09-18T04:56:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110494+0000", 2024-09-18T04:56:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1310000000000003e-06 2024-09-18T04:56:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154197+0000", 2024-09-18T04:56:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043702986999999999 2024-09-18T04:56:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154262+0000", 2024-09-18T04:56:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5236999999999999e-05 2024-09-18T04:56:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:51.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154320+0000", 2024-09-18T04:56:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8241e-05 2024-09-18T04:56:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61136 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1505280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.110499+0000", 2024-09-18T04:56:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.45344330399999999, 2024-09-18T04:56:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043976113999999997, 2024-09-18T04:56:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:51.619 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61136 2024-09-18T04:56:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110499+0000", 2024-09-18T04:56:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110499+0000", 2024-09-18T04:56:51.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:51.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110498+0000", 2024-09-18T04:56:51.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:51.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110501+0000", 2024-09-18T04:56:51.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6910000000000002e-06 2024-09-18T04:56:51.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:51.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110507+0000", 2024-09-18T04:56:51.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.516e-06 2024-09-18T04:56:51.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110512+0000", 2024-09-18T04:56:51.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9409999999999998e-06 2024-09-18T04:56:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154348+0000", 2024-09-18T04:56:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043836075000000002 2024-09-18T04:56:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154415+0000", 2024-09-18T04:56:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6748999999999994e-05 2024-09-18T04:56:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:51.626 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154475+0000", 2024-09-18T04:56:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9635000000000001e-05 2024-09-18T04:56:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61137 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1520640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.110516+0000", 2024-09-18T04:56:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.45342537100000002, 2024-09-18T04:56:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044112431000000001, 2024-09-18T04:56:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:51.627 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61137 2024-09-18T04:56:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110516+0000", 2024-09-18T04:56:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110516+0000", 2024-09-18T04:56:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:51.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110516+0000", 2024-09-18T04:56:51.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:51.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110518+0000", 2024-09-18T04:56:51.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3e-06 2024-09-18T04:56:51.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:51.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110524+0000", 2024-09-18T04:56:51.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8320000000000002e-06 2024-09-18T04:56:51.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110529+0000", 2024-09-18T04:56:51.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7559999999999999e-06 2024-09-18T04:56:51.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:51.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154507+0000", 2024-09-18T04:56:51.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043978189000000001 2024-09-18T04:56:51.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154571+0000", 2024-09-18T04:56:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3614000000000005e-05 2024-09-18T04:56:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154629+0000", 2024-09-18T04:56:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8118999999999997e-05 2024-09-18T04:56:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61138 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1536000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.110533+0000", 2024-09-18T04:56:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.45340840900000001, 2024-09-18T04:56:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044287528999999999, 2024-09-18T04:56:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61138 2024-09-18T04:56:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110533+0000", 2024-09-18T04:56:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110533+0000", 2024-09-18T04:56:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110533+0000", 2024-09-18T04:56:51.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:51.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110535+0000", 2024-09-18T04:56:51.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1679999999999998e-06 2024-09-18T04:56:51.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:51.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110541+0000", 2024-09-18T04:56:51.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8900000000000004e-06 2024-09-18T04:56:51.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110546+0000", 2024-09-18T04:56:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6759999999999998e-06 2024-09-18T04:56:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154669+0000", 2024-09-18T04:56:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044122863999999998 2024-09-18T04:56:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:51.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154752+0000", 2024-09-18T04:56:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3008000000000002e-05 2024-09-18T04:56:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154821+0000", 2024-09-18T04:56:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9283999999999996e-05 2024-09-18T04:56:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61139 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1551360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:51.637 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.110582+0000", 2024-09-18T04:56:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.453359658, 2024-09-18T04:56:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044399427999999998, 2024-09-18T04:56:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61139 2024-09-18T04:56:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:51.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110582+0000", 2024-09-18T04:56:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110582+0000", 2024-09-18T04:56:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110582+0000", 2024-09-18T04:56:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110585+0000", 2024-09-18T04:56:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.286e-06 2024-09-18T04:56:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110593+0000", 2024-09-18T04:56:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4260000000000003e-06 2024-09-18T04:56:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110600+0000", 2024-09-18T04:56:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1970000000000001e-06 2024-09-18T04:56:51.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154854+0000", 2024-09-18T04:56:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044254427999999998 2024-09-18T04:56:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154921+0000", 2024-09-18T04:56:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7453999999999995e-05 2024-09-18T04:56:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154982+0000", 2024-09-18T04:56:51.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0191e-05 2024-09-18T04:56:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61140 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1566720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.110605+0000", 2024-09-18T04:56:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.45333706800000001, 2024-09-18T04:56:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044541944999999999, 2024-09-18T04:56:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:51.642 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61140 2024-09-18T04:56:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110605+0000", 2024-09-18T04:56:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:51.643 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110605+0000", 2024-09-18T04:56:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110604+0000", 2024-09-18T04:56:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110607+0000", 2024-09-18T04:56:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.21e-06 2024-09-18T04:56:51.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110613+0000", 2024-09-18T04:56:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3670000000000002e-06 2024-09-18T04:56:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110618+0000", 2024-09-18T04:56:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7829999999999998e-06 2024-09-18T04:56:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:51.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.155013+0000", 2024-09-18T04:56:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044395028000000003 2024-09-18T04:56:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.155085+0000", 2024-09-18T04:56:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1768999999999997e-05 2024-09-18T04:56:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.155147+0000", 2024-09-18T04:56:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2197000000000001e-05 2024-09-18T04:56:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:51.646 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61390 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1460224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.119198+0000", 2024-09-18T04:56:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.44474360099999999, 2024-09-18T04:56:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043152569000000002, 2024-09-18T04:56:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61390 2024-09-18T04:56:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.647 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119198+0000", 2024-09-18T04:56:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119198+0000", 2024-09-18T04:56:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:51.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119198+0000", 2024-09-18T04:56:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119200+0000", 2024-09-18T04:56:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3640000000000002e-06 2024-09-18T04:56:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119206+0000", 2024-09-18T04:56:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9610000000000002e-06 2024-09-18T04:56:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119211+0000", 2024-09-18T04:56:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6059999999999996e-06 2024-09-18T04:56:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162225+0000", 2024-09-18T04:56:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043014189000000001 2024-09-18T04:56:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162289+0000", 2024-09-18T04:56:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4218000000000001e-05 2024-09-18T04:56:51.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162351+0000", 2024-09-18T04:56:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1773999999999995e-05 2024-09-18T04:56:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61394 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1521664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.119713+0000", 2024-09-18T04:56:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.44422888999999999, 2024-09-18T04:56:51.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043168156999999999, 2024-09-18T04:56:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61394 2024-09-18T04:56:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119713+0000", 2024-09-18T04:56:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119713+0000", 2024-09-18T04:56:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119712+0000", 2024-09-18T04:56:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119715+0000", 2024-09-18T04:56:51.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7010000000000001e-06 2024-09-18T04:56:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119722+0000", 2024-09-18T04:56:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1330000000000001e-06 2024-09-18T04:56:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119728+0000", 2024-09-18T04:56:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.186e-06 2024-09-18T04:56:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162751+0000", 2024-09-18T04:56:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043023790999999999 2024-09-18T04:56:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162819+0000", 2024-09-18T04:56:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7970000000000001e-05 2024-09-18T04:56:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162881+0000", 2024-09-18T04:56:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1841999999999995e-05 2024-09-18T04:56:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61395 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1537024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.119735+0000", 2024-09-18T04:56:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.44420672500000002, 2024-09-18T04:56:51.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043266315, 2024-09-18T04:56:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61395 2024-09-18T04:56:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119735+0000", 2024-09-18T04:56:51.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:51.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119735+0000", 2024-09-18T04:56:51.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:51.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119732+0000", 2024-09-18T04:56:51.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-18T04:56:51.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:51.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119737+0000", 2024-09-18T04:56:51.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9150000000000004e-06 2024-09-18T04:56:51.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:51.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119744+0000", 2024-09-18T04:56:51.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4189999999999999e-06 2024-09-18T04:56:51.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119748+0000", 2024-09-18T04:56:51.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8659999999999997e-06 2024-09-18T04:56:51.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:51.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162913+0000", 2024-09-18T04:56:51.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043164730999999998 2024-09-18T04:56:51.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:51.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162959+0000", 2024-09-18T04:56:51.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6190999999999999e-05 2024-09-18T04:56:51.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:51.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.163001+0000", 2024-09-18T04:56:51.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2089000000000003e-05 2024-09-18T04:56:51.661 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.661 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:51.661 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.661 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61396 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1552384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:51.661 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.119753+0000", 2024-09-18T04:56:51.661 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.44418895200000003, 2024-09-18T04:56:51.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043432158999999998, 2024-09-18T04:56:51.661 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:51.661 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:51.661 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:51.662 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:51.662 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.662 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61396 2024-09-18T04:56:51.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.662 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:51.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:51.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119753+0000", 2024-09-18T04:56:51.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:51.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119753+0000", 2024-09-18T04:56:51.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:51.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119753+0000", 2024-09-18T04:56:51.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:51.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119755+0000", 2024-09-18T04:56:51.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0980000000000001e-06 2024-09-18T04:56:51.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:51.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119771+0000", 2024-09-18T04:56:51.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6056999999999999e-05 2024-09-18T04:56:51.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119776+0000", 2024-09-18T04:56:51.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0719999999999999e-06 2024-09-18T04:56:51.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:51.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.163037+0000", 2024-09-18T04:56:51.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043261130000000002 2024-09-18T04:56:51.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:51.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.163114+0000", 2024-09-18T04:56:51.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7545000000000004e-05 2024-09-18T04:56:51.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:51.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.163185+0000", 2024-09-18T04:56:51.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0598000000000004e-05 2024-09-18T04:56:51.665 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.665 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:51.665 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.666 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61397 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1567744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:51.666 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.119782+0000", 2024-09-18T04:56:51.666 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.44415985600000002, 2024-09-18T04:56:51.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043533611999999999, 2024-09-18T04:56:51.666 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:51.666 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:51.666 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:51.666 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:51.666 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:51.666 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61397 2024-09-18T04:56:51.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.667 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:51.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:51.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119782+0000", 2024-09-18T04:56:51.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:51.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119782+0000", 2024-09-18T04:56:51.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:51.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:51.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119782+0000", 2024-09-18T04:56:51.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:51.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:51.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119784+0000", 2024-09-18T04:56:51.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7690000000000001e-06 2024-09-18T04:56:51.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:51.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119791+0000", 2024-09-18T04:56:51.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1960000000000002e-06 2024-09-18T04:56:51.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:51.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119798+0000", 2024-09-18T04:56:51.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0310000000000004e-06 2024-09-18T04:56:51.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:51.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.163230+0000", 2024-09-18T04:56:51.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043432456000000001 2024-09-18T04:56:51.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:51.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.163273+0000", 2024-09-18T04:56:51.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3234e-05 2024-09-18T04:56:51.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:51.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:51.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:51.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.163316+0000", 2024-09-18T04:56:51.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2382000000000002e-05 2024-09-18T04:56:51.670 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.670 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:51.670 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.670 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:51.670 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:51.670 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:51.671 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:51.669+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:51.672 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:56:51.772 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:51.770+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:51.774 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:51.774 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:51.774 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:56:51.774 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:51.782 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:56:51.873 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:51.871+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:51.884 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:51.884 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:51.884 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:56:51.884 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:56:51.884 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:51.892 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:56:51.974 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:51.972+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:52.035 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:52.036 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:56:52.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:56:52.036 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:56:52.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.036 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60344 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1456128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:52.036 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:56:52.036 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.92032951900000004, 2024-09-18T04:56:52.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044026324999999998, 2024-09-18T04:56:52.036 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:52.036 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:52.036 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:52.036 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:52.036 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:52.037 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60344 2024-09-18T04:56:52.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.037 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:52.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:52.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:56:52.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:52.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:56:52.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:52.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:56:52.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:52.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:52.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112224+0000", 2024-09-18T04:56:52.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1320000000000001e-06 2024-09-18T04:56:52.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:52.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112230+0000", 2024-09-18T04:56:52.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9619999999999998e-06 2024-09-18T04:56:52.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:52.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112234+0000", 2024-09-18T04:56:52.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7890000000000002e-06 2024-09-18T04:56:52.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:52.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156105+0000", 2024-09-18T04:56:52.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043870531999999997 2024-09-18T04:56:52.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:52.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156170+0000", 2024-09-18T04:56:52.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5036000000000005e-05 2024-09-18T04:56:52.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:52.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156248+0000", 2024-09-18T04:56:52.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8295999999999997e-05 2024-09-18T04:56:52.040 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.040 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:52.040 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.040 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60345 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1471488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:52.041 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:56:52.041 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.92031275400000001, 2024-09-18T04:56:52.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044180090999999998, 2024-09-18T04:56:52.041 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:52.041 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:52.041 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:52.041 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:52.041 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:52.041 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60345 2024-09-18T04:56:52.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.041 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:52.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:52.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:56:52.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:52.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:56:52.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:52.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112238+0000", 2024-09-18T04:56:52.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:52.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:52.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112240+0000", 2024-09-18T04:56:52.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9910000000000001e-06 2024-09-18T04:56:52.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:52.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112246+0000", 2024-09-18T04:56:52.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0859999999999998e-06 2024-09-18T04:56:52.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:52.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112251+0000", 2024-09-18T04:56:52.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5020000000000003e-06 2024-09-18T04:56:52.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:52.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156274+0000", 2024-09-18T04:56:52.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044023063000000001 2024-09-18T04:56:52.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:52.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156337+0000", 2024-09-18T04:56:52.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3077000000000006e-05 2024-09-18T04:56:52.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:52.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156419+0000", 2024-09-18T04:56:52.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1699000000000005e-05 2024-09-18T04:56:52.045 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.045 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:52.045 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.045 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60346 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1486848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:52.045 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:56:52.045 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.92029497299999996, 2024-09-18T04:56:52.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044336331999999999, 2024-09-18T04:56:52.045 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:52.046 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:52.046 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:52.046 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:52.046 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:52.046 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60346 2024-09-18T04:56:52.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.046 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:52.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:52.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:56:52.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:52.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:56:52.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:52.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112256+0000", 2024-09-18T04:56:52.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:52.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:52.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112258+0000", 2024-09-18T04:56:52.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1720000000000001e-06 2024-09-18T04:56:52.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:52.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112265+0000", 2024-09-18T04:56:52.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4280000000000001e-06 2024-09-18T04:56:52.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:52.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112269+0000", 2024-09-18T04:56:52.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.741e-06 2024-09-18T04:56:52.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:52.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156445+0000", 2024-09-18T04:56:52.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044175341 2024-09-18T04:56:52.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:52.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156512+0000", 2024-09-18T04:56:52.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6723000000000005e-05 2024-09-18T04:56:52.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:52.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156593+0000", 2024-09-18T04:56:52.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1284000000000003e-05 2024-09-18T04:56:52.052 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.052 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:52.052 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.052 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60347 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1502208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:52.052 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:56:52.052 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.92026656200000001, 2024-09-18T04:56:52.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044482560999999997, 2024-09-18T04:56:52.053 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:52.053 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:52.053 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:52.053 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:52.053 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:52.053 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60347 2024-09-18T04:56:52.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.053 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:52.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:52.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:56:52.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:52.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:56:52.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:52.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112284+0000", 2024-09-18T04:56:52.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:52.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:52.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112286+0000", 2024-09-18T04:56:52.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9709999999999998e-06 2024-09-18T04:56:52.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:52.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112290+0000", 2024-09-18T04:56:52.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9199999999999997e-06 2024-09-18T04:56:52.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:52.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112293+0000", 2024-09-18T04:56:52.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870000000000001e-06 2024-09-18T04:56:52.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:52.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156617+0000", 2024-09-18T04:56:52.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044323960000000003 2024-09-18T04:56:52.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:52.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156687+0000", 2024-09-18T04:56:52.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0141000000000005e-05 2024-09-18T04:56:52.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:52.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156767+0000", 2024-09-18T04:56:52.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0140000000000002e-05 2024-09-18T04:56:52.057 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.057 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:52.057 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.057 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60348 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1517568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:52.057 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:56:52.057 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.92025555800000003, 2024-09-18T04:56:52.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044645562, 2024-09-18T04:56:52.057 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:52.057 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:52.057 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:52.058 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:52.058 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:52.058 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60348 2024-09-18T04:56:52.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.058 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:52.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:52.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:56:52.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:52.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:56:52.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:52.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:56:52.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:52.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:52.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112297+0000", 2024-09-18T04:56:52.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.384e-06 2024-09-18T04:56:52.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:52.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112301+0000", 2024-09-18T04:56:52.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0570000000000003e-06 2024-09-18T04:56:52.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:52.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112304+0000", 2024-09-18T04:56:52.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6000000000000001e-06 2024-09-18T04:56:52.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:52.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156793+0000", 2024-09-18T04:56:52.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044489494999999997 2024-09-18T04:56:52.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:52.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156859+0000", 2024-09-18T04:56:52.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5483999999999998e-05 2024-09-18T04:56:52.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:52.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156941+0000", 2024-09-18T04:56:52.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2856999999999996e-05 2024-09-18T04:56:52.062 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.062 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:52.062 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.062 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60349 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1532928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:52.062 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:56:52.062 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.92024558199999995, 2024-09-18T04:56:52.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044832167999999999, 2024-09-18T04:56:52.063 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:52.063 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:52.063 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:52.063 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:52.063 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:52.063 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60349 2024-09-18T04:56:52.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.063 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:52.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:52.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:56:52.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:56:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:56:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112307+0000", 2024-09-18T04:56:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.308e-06 2024-09-18T04:56:52.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112310+0000", 2024-09-18T04:56:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-18T04:56:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112312+0000", 2024-09-18T04:56:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4889999999999998e-06 2024-09-18T04:56:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156977+0000", 2024-09-18T04:56:52.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044664598999999999 2024-09-18T04:56:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157053+0000", 2024-09-18T04:56:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6014999999999996e-05 2024-09-18T04:56:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157138+0000", 2024-09-18T04:56:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5162999999999998e-05 2024-09-18T04:56:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60350 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1548288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:56:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.92023666800000004, 2024-09-18T04:56:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045011858000000002, 2024-09-18T04:56:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:52.067 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60350 2024-09-18T04:56:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:56:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:56:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:56:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112316+0000", 2024-09-18T04:56:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7579999999999999e-06 2024-09-18T04:56:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112321+0000", 2024-09-18T04:56:52.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1649999999999997e-06 2024-09-18T04:56:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112325+0000", 2024-09-18T04:56:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.726e-06 2024-09-18T04:56:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157172+0000", 2024-09-18T04:56:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.04484718 2024-09-18T04:56:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157242+0000", 2024-09-18T04:56:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9675e-05 2024-09-18T04:56:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157327+0000", 2024-09-18T04:56:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4614e-05 2024-09-18T04:56:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60351 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1563648~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:52.071 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:56:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.92022325199999999, 2024-09-18T04:56:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045132113000000001, 2024-09-18T04:56:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60351 2024-09-18T04:56:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:52.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:56:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:56:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:56:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:52.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112330+0000", 2024-09-18T04:56:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1660000000000001e-06 2024-09-18T04:56:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112336+0000", 2024-09-18T04:56:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5940000000000001e-06 2024-09-18T04:56:52.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112340+0000", 2024-09-18T04:56:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5589999999999999e-06 2024-09-18T04:56:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157365+0000", 2024-09-18T04:56:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045024675 2024-09-18T04:56:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:52.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157409+0000", 2024-09-18T04:56:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4356e-05 2024-09-18T04:56:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157460+0000", 2024-09-18T04:56:52.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1072000000000002e-05 2024-09-18T04:56:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60501 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1494016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:56:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.91416255599999996, 2024-09-18T04:56:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044007728000000003, 2024-09-18T04:56:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:52.080 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60501 2024-09-18T04:56:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.081 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:56:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:56:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:56:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:52.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118391+0000", 2024-09-18T04:56:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:56:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118396+0000", 2024-09-18T04:56:52.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2680000000000002e-06 2024-09-18T04:56:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118400+0000", 2024-09-18T04:56:52.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2080000000000002e-06 2024-09-18T04:56:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162317+0000", 2024-09-18T04:56:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043916753000000003 2024-09-18T04:56:52.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162351+0000", 2024-09-18T04:56:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4372000000000002e-05 2024-09-18T04:56:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162397+0000", 2024-09-18T04:56:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5207999999999998e-05 2024-09-18T04:56:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60502 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1509376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:56:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.91414707900000003, 2024-09-18T04:56:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044092438999999997, 2024-09-18T04:56:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:52.090 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60502 2024-09-18T04:56:52.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:56:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:56:52.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:56:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118406+0000", 2024-09-18T04:56:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:56:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118412+0000", 2024-09-18T04:56:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.524e-06 2024-09-18T04:56:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118416+0000", 2024-09-18T04:56:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:56:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162418+0000", 2024-09-18T04:56:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001508000000002 2024-09-18T04:56:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162451+0000", 2024-09-18T04:56:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3662999999999999e-05 2024-09-18T04:56:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162497+0000", 2024-09-18T04:56:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5550000000000003e-05 2024-09-18T04:56:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60503 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1524736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:56:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.91413092100000004, 2024-09-18T04:56:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044178780000000001, 2024-09-18T04:56:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60503 2024-09-18T04:56:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:52.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:56:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:56:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118420+0000", 2024-09-18T04:56:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118422+0000", 2024-09-18T04:56:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.08e-06 2024-09-18T04:56:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:52.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118428+0000", 2024-09-18T04:56:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.964e-06 2024-09-18T04:56:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118433+0000", 2024-09-18T04:56:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.425e-06 2024-09-18T04:56:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162519+0000", 2024-09-18T04:56:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044086082999999998 2024-09-18T04:56:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162552+0000", 2024-09-18T04:56:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3719e-05 2024-09-18T04:56:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162599+0000", 2024-09-18T04:56:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6873999999999999e-05 2024-09-18T04:56:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.100 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60504 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1540096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:56:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.914114816, 2024-09-18T04:56:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044261921000000003, 2024-09-18T04:56:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60504 2024-09-18T04:56:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.101 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:56:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:56:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118436+0000", 2024-09-18T04:56:52.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118438+0000", 2024-09-18T04:56:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0250000000000001e-06 2024-09-18T04:56:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118444+0000", 2024-09-18T04:56:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4360000000000001e-06 2024-09-18T04:56:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:52.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118448+0000", 2024-09-18T04:56:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.357e-06 2024-09-18T04:56:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162620+0000", 2024-09-18T04:56:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044171821 2024-09-18T04:56:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162653+0000", 2024-09-18T04:56:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3479999999999998e-05 2024-09-18T04:56:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162699+0000", 2024-09-18T04:56:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5113e-05 2024-09-18T04:56:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60584 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1196032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:56:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.91232179899999999, 2024-09-18T04:56:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044048727000000003, 2024-09-18T04:56:52.105 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60584 2024-09-18T04:56:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:56:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:52.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:56:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120229+0000", 2024-09-18T04:56:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120231+0000", 2024-09-18T04:56:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:56:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120237+0000", 2024-09-18T04:56:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8379999999999998e-06 2024-09-18T04:56:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120242+0000", 2024-09-18T04:56:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3889999999999998e-06 2024-09-18T04:56:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164187+0000", 2024-09-18T04:56:52.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043945045000000002 2024-09-18T04:56:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164227+0000", 2024-09-18T04:56:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0627999999999998e-05 2024-09-18T04:56:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164278+0000", 2024-09-18T04:56:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1146000000000001e-05 2024-09-18T04:56:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.109 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60585 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1211392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:56:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.91230569100000003, 2024-09-18T04:56:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044389814, 2024-09-18T04:56:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60585 2024-09-18T04:56:52.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:56:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:56:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120245+0000", 2024-09-18T04:56:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:52.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120247+0000", 2024-09-18T04:56:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0150000000000002e-06 2024-09-18T04:56:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120255+0000", 2024-09-18T04:56:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.858e-06 2024-09-18T04:56:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120260+0000", 2024-09-18T04:56:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6530000000000002e-06 2024-09-18T04:56:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164489+0000", 2024-09-18T04:56:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044229483999999999 2024-09-18T04:56:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164555+0000", 2024-09-18T04:56:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.533e-05 2024-09-18T04:56:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164636+0000", 2024-09-18T04:56:52.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0819e-05 2024-09-18T04:56:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60586 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1226752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:56:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.91228724000000005, 2024-09-18T04:56:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044578214999999997, 2024-09-18T04:56:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:52.115 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60586 2024-09-18T04:56:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:56:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:52.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:56:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:56:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120266+0000", 2024-09-18T04:56:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0660000000000002e-06 2024-09-18T04:56:52.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120272+0000", 2024-09-18T04:56:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-18T04:56:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120276+0000", 2024-09-18T04:56:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.476e-06 2024-09-18T04:56:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164673+0000", 2024-09-18T04:56:52.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044396672999999998 2024-09-18T04:56:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164749+0000", 2024-09-18T04:56:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5811999999999999e-05 2024-09-18T04:56:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164842+0000", 2024-09-18T04:56:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3560000000000005e-05 2024-09-18T04:56:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:52.119 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60587 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1242112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:56:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.912238141, 2024-09-18T04:56:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044634794999999998, 2024-09-18T04:56:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60587 2024-09-18T04:56:52.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:56:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:56:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:52.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120280+0000", 2024-09-18T04:56:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999671 2024-09-18T04:56:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120316+0000", 2024-09-18T04:56:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5728000000000001e-05 2024-09-18T04:56:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120329+0000", 2024-09-18T04:56:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2914e-05 2024-09-18T04:56:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120335+0000", 2024-09-18T04:56:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8289999999999996e-06 2024-09-18T04:56:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:52.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164858+0000", 2024-09-18T04:56:52.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044523417000000003 2024-09-18T04:56:52.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:52.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164896+0000", 2024-09-18T04:56:52.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7969999999999997e-05 2024-09-18T04:56:52.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:52.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164948+0000", 2024-09-18T04:56:52.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1996000000000002e-05 2024-09-18T04:56:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.126 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60595 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1364992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:52.127 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:56:52.127 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.91039492499999997, 2024-09-18T04:56:52.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001360000000003, 2024-09-18T04:56:52.127 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60595 2024-09-18T04:56:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.128 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:56:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:56:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:52.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122156+0000", 2024-09-18T04:56:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122158+0000", 2024-09-18T04:56:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0040000000000002e-06 2024-09-18T04:56:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122164+0000", 2024-09-18T04:56:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8719999999999999e-06 2024-09-18T04:56:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122169+0000", 2024-09-18T04:56:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5090000000000004e-06 2024-09-18T04:56:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166040+0000", 2024-09-18T04:56:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043871226999999999 2024-09-18T04:56:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166096+0000", 2024-09-18T04:56:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6495e-05 2024-09-18T04:56:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166158+0000", 2024-09-18T04:56:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1626999999999998e-05 2024-09-18T04:56:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60596 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1380352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:56:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.91037860400000004, 2024-09-18T04:56:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044120800000000002, 2024-09-18T04:56:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:52.132 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60596 2024-09-18T04:56:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:56:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:52.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:56:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:56:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122175+0000", 2024-09-18T04:56:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:56:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122182+0000", 2024-09-18T04:56:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3509999999999996e-06 2024-09-18T04:56:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122187+0000", 2024-09-18T04:56:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7940000000000002e-06 2024-09-18T04:56:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166184+0000", 2024-09-18T04:56:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043997236000000002 2024-09-18T04:56:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166227+0000", 2024-09-18T04:56:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2561000000000001e-05 2024-09-18T04:56:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166294+0000", 2024-09-18T04:56:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7068000000000002e-05 2024-09-18T04:56:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60597 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1395712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:56:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.91036057599999998, 2024-09-18T04:56:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044242166999999999, 2024-09-18T04:56:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60597 2024-09-18T04:56:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:56:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:56:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:56:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122193+0000", 2024-09-18T04:56:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:56:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122199+0000", 2024-09-18T04:56:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6449999999999999e-06 2024-09-18T04:56:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122204+0000", 2024-09-18T04:56:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4819999999999996e-06 2024-09-18T04:56:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166329+0000", 2024-09-18T04:56:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044125059000000001 2024-09-18T04:56:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166370+0000", 2024-09-18T04:56:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1012e-05 2024-09-18T04:56:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166433+0000", 2024-09-18T04:56:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3178999999999999e-05 2024-09-18T04:56:52.140 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60608 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1564672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:56:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.90836271400000002, 2024-09-18T04:56:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044082723999999997, 2024-09-18T04:56:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:52.141 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60608 2024-09-18T04:56:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:56:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:56:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124188+0000", 2024-09-18T04:56:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124190+0000", 2024-09-18T04:56:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0329999999999998e-06 2024-09-18T04:56:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124196+0000", 2024-09-18T04:56:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.575e-06 2024-09-18T04:56:52.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124201+0000", 2024-09-18T04:56:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5000000000000001e-06 2024-09-18T04:56:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168166+0000", 2024-09-18T04:56:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043965763999999997 2024-09-18T04:56:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168218+0000", 2024-09-18T04:56:52.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1785e-05 2024-09-18T04:56:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168271+0000", 2024-09-18T04:56:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3429e-05 2024-09-18T04:56:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:52.145 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:52.145 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:52.146 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:56:52.147 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:52.144+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:52.148 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:56:52.248 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:52.245+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:52.252 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:52.252 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:52.252 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:56:52.252 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:52.256 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T04:56:52.261 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:56:52.287 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:56:52.287 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:56:52.304 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:56:52.304 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:56:52.320 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:56:52.321 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:56:52.337 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:56:52.338 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:56:52.349 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:52.346+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:52.354 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:56:52.354 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:56:52.371 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:56:52.371 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:56:52.401 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:56:52.401 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:56:52.406 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:52.406 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:52.415 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:56:52.423 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:56:52.423 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:56:52.475 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:52.473+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:52.551 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:52.549+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:52.557 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:52.557 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:52.566 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:56:52.652 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:52.650+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:52.708 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:52.708 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:52.716 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:56:52.761 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:52.759+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:52.818 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:52.818 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:52.827 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:56:52.854 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:52.852+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:52.956 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:52.953+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:52.969 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:52.969 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:52.978 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:56:53.057 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:53.055+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:53.090 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:53.090 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:53.099 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:56:53.158 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:53.156+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:53.207 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:53.207 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:53.216 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:56:53.250 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-18T04:56:53.263 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:53.261+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:53.357 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:53.357 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:53.361 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:53.359+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:53.366 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:56:53.463 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:53.460+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:53.513 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:53.513 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:53.522 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:56:53.563 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:53.561+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:53.629 INFO:teuthology.orchestra.run.smithi002.stdout:2266 write 0x75705c8 thru 0x757feaa (0xf8e3 bytes) 2024-09-18T04:56:53.647 INFO:teuthology.orchestra.run.smithi002.stdout:2267 read 0xd1bb739 thru 0xd1c4eda (0x97a2 bytes) 2024-09-18T04:56:53.647 INFO:teuthology.orchestra.run.smithi002.stdout:2268 write 0x4d1190d thru 0x4d14bfc (0x32f0 bytes) 2024-09-18T04:56:53.662 INFO:teuthology.orchestra.run.smithi002.stdout:2269 read 0x59fafb7 thru 0x59fca2f (0x1a79 bytes) 2024-09-18T04:56:53.662 INFO:teuthology.orchestra.run.smithi002.stdout:2271 read 0xa3092b6 thru 0xa3125ad (0x92f8 bytes) 2024-09-18T04:56:53.664 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:53.662+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:53.676 INFO:teuthology.orchestra.run.smithi002.stdout:2272 trunc from 0xd624bd0 to 0xe1a2f8 2024-09-18T04:56:53.676 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:53.676 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:53.690 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:56:53.713 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:53.713 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:53.713 INFO:teuthology.orchestra.run.smithi002.stdout:2275 read 0x4881c3 thru 0x490446 (0x8284 bytes) 2024-09-18T04:56:53.718 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:53.726 INFO:teuthology.orchestra.run.smithi002.stdout:2277 write 0x2c3e77e thru 0x2c4b40b (0xcc8e bytes) 2024-09-18T04:56:53.746 INFO:teuthology.orchestra.run.smithi002.stdout:2278 writesame 0x4d9d8e2 thru 0x4d9e394 data_size 0x21(0xab3 bytes) 2024-09-18T04:56:53.746 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:53.758 INFO:teuthology.orchestra.run.smithi002.stdout:2280 writesame 0xab5250d thru 0xab53179 data_size 0x1(0xc6d bytes) 2024-09-18T04:56:53.758 INFO:teuthology.orchestra.run.smithi002.stdout:2283 read 0x1490471 thru 0x1497154 (0x6ce4 bytes) 2024-09-18T04:56:53.758 INFO:teuthology.orchestra.run.smithi002.stdout:2285 punch from 0x129cbf4 to 0x12a41ef, (0x75fb bytes) 2024-09-18T04:56:53.758 INFO:teuthology.orchestra.run.smithi002.stdout:2287 writesame 0x483286a thru 0x4838aa8 data_size 0x1(0x623f bytes) 2024-09-18T04:56:53.765 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:53.763+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:53.774 INFO:teuthology.orchestra.run.smithi002.stdout:2288 punch from 0x778a0a9 to 0x7790bbf, (0x6b16 bytes) 2024-09-18T04:56:53.779 INFO:teuthology.orchestra.run.smithi002.stdout:2289 trunc from 0xab5317a to 0xe9a0470 2024-09-18T04:56:53.784 INFO:teuthology.orchestra.run.smithi002.stdout:2290 punch from 0x5224069 to 0x5226416, (0x23ad bytes) 2024-09-18T04:56:53.784 INFO:teuthology.orchestra.run.smithi002.stdout:2291 write 0x837e46f thru 0x8386d8c (0x891e bytes) 2024-09-18T04:56:53.801 INFO:teuthology.orchestra.run.smithi002.stdout:2293 punch from 0xc790b00 to 0xc79b152, (0xa652 bytes) 2024-09-18T04:56:53.801 INFO:teuthology.orchestra.run.smithi002.stdout:2294 writesame 0xc291731 thru 0xc294596 data_size 0x1(0x2e66 bytes) 2024-09-18T04:56:53.816 INFO:teuthology.orchestra.run.smithi002.stdout:2295 writesame 0xbf27825 thru 0xbf29de8 data_size 0x2(0x25c4 bytes) 2024-09-18T04:56:53.816 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:53.817 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:53.826 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:56:53.829 INFO:teuthology.orchestra.run.smithi002.stdout:2296 write 0xb7e0dd2 thru 0xb7e27c8 (0x19f7 bytes) 2024-09-18T04:56:53.842 INFO:teuthology.orchestra.run.smithi002.stdout:2297 punch from 0xd8016c8 to 0xd807209, (0x5b41 bytes) 2024-09-18T04:56:53.842 INFO:teuthology.orchestra.run.smithi002.stdout:2299 write 0x9a18c68 thru 0x9a1bcb4 (0x304d bytes) 2024-09-18T04:56:53.861 INFO:teuthology.orchestra.run.smithi002.stdout:2300 trunc from 0xe9a0470 to 0xad256d9 2024-09-18T04:56:53.867 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:53.864+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:53.875 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:53.885 INFO:teuthology.orchestra.run.smithi002.stdout:2302 writesame 0xdb6d87b thru 0xdb799dc data_size 0x25(0xc162 bytes) 2024-09-18T04:56:53.885 INFO:teuthology.orchestra.run.smithi002.stdout:2303 writesame 0xbb19286 thru 0xbb21c3e data_size 0x1(0x89b9 bytes) 2024-09-18T04:56:53.910 INFO:teuthology.orchestra.run.smithi002.stdout:2304 trunc from 0xdb799dd to 0x85d748f 2024-09-18T04:56:53.929 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:53.932 INFO:teuthology.orchestra.run.smithi002.stdout:2306 trunc from 0x85d748f to 0x50e18c0 2024-09-18T04:56:53.950 INFO:teuthology.orchestra.run.smithi002.stdout:2307 write 0x64528c5 thru 0x6459bab (0x72e7 bytes) 2024-09-18T04:56:53.952 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:53.952 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:53.962 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:56:53.968 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:53.965+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:53.968 INFO:teuthology.orchestra.run.smithi002.stdout:2309 punch from 0x84d4ce to 0x8573dc, (0x9f0e bytes) 2024-09-18T04:56:53.974 INFO:teuthology.orchestra.run.smithi002.stdout:2310 write 0x67cecfc thru 0x67d5c90 (0x6f95 bytes) 2024-09-18T04:56:53.990 INFO:teuthology.orchestra.run.smithi002.stdout:2311 write 0x675eaca thru 0x676795e (0x8e95 bytes) 2024-09-18T04:56:53.995 INFO:teuthology.orchestra.run.smithi002.stdout:2312 read 0x62dc7c6 thru 0x62e3b6b (0x73a6 bytes) 2024-09-18T04:56:53.996 INFO:teuthology.orchestra.run.smithi002.stdout:2314 read 0x4ceb1f7 thru 0x4cfa77e (0xf588 bytes) 2024-09-18T04:56:53.996 INFO:teuthology.orchestra.run.smithi002.stdout:2315 writesame 0x2dc0242 thru 0x2dca846 data_size 0x1f(0xa605 bytes) 2024-09-18T04:56:54.018 INFO:teuthology.orchestra.run.smithi002.stdout:2316 writesame 0x771a9fa thru 0x7727d69 data_size 0x44(0xd370 bytes) 2024-09-18T04:56:54.020 INFO:teuthology.orchestra.run.smithi002.stdout:2318 trunc from 0x7727d6a to 0x9d4069e 2024-09-18T04:56:54.030 INFO:teuthology.orchestra.run.smithi002.stdout:2319 write 0xa0eaac4 thru 0xa0f00ad (0x55ea bytes) 2024-09-18T04:56:54.051 INFO:teuthology.orchestra.run.smithi002.stdout:2320 write 0xb43305b thru 0xb436680 (0x3626 bytes) 2024-09-18T04:56:54.067 INFO:teuthology.orchestra.run.smithi002.stdout:2321 writesame 0x58fa7a2 thru 0x59085d3 data_size 0x22(0xde32 bytes) 2024-09-18T04:56:54.068 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:54.066+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:54.086 INFO:teuthology.orchestra.run.smithi002.stdout:2322 write 0x8b8a452 thru 0x8b8ec9c (0x484b bytes) 2024-09-18T04:56:54.087 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:54.087 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:54.097 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:56:54.104 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:54.106 INFO:teuthology.orchestra.run.smithi002.stdout:2325 trunc from 0xb436681 to 0xc804564 2024-09-18T04:56:54.111 INFO:teuthology.orchestra.run.smithi002.stdout:2326 read 0xc5dd69b thru 0xc5e4d44 (0x76aa bytes) 2024-09-18T04:56:54.111 INFO:teuthology.orchestra.run.smithi002.stdout:2327 writesame 0xc4997bb thru 0xc49c337 data_size 0x3(0x2b7d bytes) 2024-09-18T04:56:54.133 INFO:teuthology.orchestra.run.smithi002.stdout:2328 write 0xd9f7a3c thru 0xda068e9 (0xeeae bytes) 2024-09-18T04:56:54.151 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:54.152 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:54.152 INFO:teuthology.orchestra.run.smithi002.stdout:2331 write 0x992488 thru 0x995006 (0x2b7f bytes) 2024-09-18T04:56:54.169 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:54.167+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:54.240 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:54.240 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:54.251 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:56:54.293 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:54.290+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:54.370 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:54.368+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:54.401 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:54.401 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:54.410 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:56:54.434 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:56:54.497 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:54.495+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:54.517 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:54.517 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:54.526 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:56:54.573 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:54.570+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:54.577 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:56:54.578 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:56:54.596 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:56:54.596 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:56:54.612 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:56:54.612 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:56:54.629 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:56:54.629 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:56:54.646 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:56:54.646 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:56:54.662 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:56:54.663 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:56:54.674 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:54.671+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:54.677 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:54.678 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:56:54.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:56:54.678 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:56:54.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.678 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11151 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1802240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:54.678 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889328+0000", 2024-09-18T04:56:54.678 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.785179282000001, 2024-09-18T04:56:54.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040037528000000003, 2024-09-18T04:56:54.679 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:54.679 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:54.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:54.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:54.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:54.679 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11151 2024-09-18T04:56:54.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.679 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:54.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:54.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889328+0000", 2024-09-18T04:56:54.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:54.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889328+0000", 2024-09-18T04:56:54.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:54.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889328+0000", 2024-09-18T04:56:54.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:54.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:54.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889330+0000", 2024-09-18T04:56:54.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.148e-06 2024-09-18T04:56:54.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:54.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889336+0000", 2024-09-18T04:56:54.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8220000000000003e-06 2024-09-18T04:56:54.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:54.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889340+0000", 2024-09-18T04:56:54.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.318e-06 2024-09-18T04:56:54.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:54.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929171+0000", 2024-09-18T04:56:54.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039830466000000002 2024-09-18T04:56:54.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:54.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929268+0000", 2024-09-18T04:56:54.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7569e-05 2024-09-18T04:56:54.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:54.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929366+0000", 2024-09-18T04:56:54.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7573999999999998e-05 2024-09-18T04:56:54.683 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.683 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:54.683 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.684 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11152 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1867776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:54.684 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889344+0000", 2024-09-18T04:56:54.684 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.785163697000002, 2024-09-18T04:56:54.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040227974, 2024-09-18T04:56:54.685 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:54.685 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:54.685 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:54.685 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:54.685 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:54.685 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11152 2024-09-18T04:56:54.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.685 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:54.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:54.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889344+0000", 2024-09-18T04:56:54.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:54.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889344+0000", 2024-09-18T04:56:54.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:54.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889344+0000", 2024-09-18T04:56:54.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:54.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:54.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889346+0000", 2024-09-18T04:56:54.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.99e-06 2024-09-18T04:56:54.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:54.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889351+0000", 2024-09-18T04:56:54.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4260000000000002e-06 2024-09-18T04:56:54.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:54.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889370+0000", 2024-09-18T04:56:54.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9262999999999998e-05 2024-09-18T04:56:54.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:54.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929397+0000", 2024-09-18T04:56:54.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040026679000000003 2024-09-18T04:56:54.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:54.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929479+0000", 2024-09-18T04:56:54.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2467999999999997e-05 2024-09-18T04:56:54.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:54.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929572+0000", 2024-09-18T04:56:54.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2519e-05 2024-09-18T04:56:54.688 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.688 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:54.689 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.689 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11153 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1916928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:54.689 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889377+0000", 2024-09-18T04:56:54.689 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.785130723000002, 2024-09-18T04:56:54.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040407884999999998, 2024-09-18T04:56:54.689 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:54.689 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:54.689 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:54.689 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:54.689 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:54.689 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11153 2024-09-18T04:56:54.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.690 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:54.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:54.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889377+0000", 2024-09-18T04:56:54.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:54.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889377+0000", 2024-09-18T04:56:54.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:54.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889376+0000", 2024-09-18T04:56:54.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:54.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:54.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889385+0000", 2024-09-18T04:56:54.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1030000000000002e-06 2024-09-18T04:56:54.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:54.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889392+0000", 2024-09-18T04:56:54.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1799999999999999e-06 2024-09-18T04:56:54.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:54.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889396+0000", 2024-09-18T04:56:54.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6890000000000003e-06 2024-09-18T04:56:54.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:54.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929603+0000", 2024-09-18T04:56:54.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040206652000000002 2024-09-18T04:56:54.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:54.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929688+0000", 2024-09-18T04:56:54.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5327000000000005e-05 2024-09-18T04:56:54.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:54.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929785+0000", 2024-09-18T04:56:54.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6362999999999999e-05 2024-09-18T04:56:54.693 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.693 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:54.693 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.693 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11154 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:54.693 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:56:54.693 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.785106892999998, 2024-09-18T04:56:54.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040591919999999997, 2024-09-18T04:56:54.693 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:54.693 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:54.694 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:54.694 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:54.694 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:54.694 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11154 2024-09-18T04:56:54.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.694 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:54.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:54.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:56:54.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:54.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:56:54.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:54.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889400+0000", 2024-09-18T04:56:54.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:54.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:54.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889403+0000", 2024-09-18T04:56:54.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3999999999999999e-06 2024-09-18T04:56:54.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:54.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889409+0000", 2024-09-18T04:56:54.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3380000000000001e-06 2024-09-18T04:56:54.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:54.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889414+0000", 2024-09-18T04:56:54.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5789999999999997e-06 2024-09-18T04:56:54.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:54.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929811+0000", 2024-09-18T04:56:54.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040397486000000003 2024-09-18T04:56:54.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:54.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929898+0000", 2024-09-18T04:56:54.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6370000000000001e-05 2024-09-18T04:56:54.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:54.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929993+0000", 2024-09-18T04:56:54.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5136000000000005e-05 2024-09-18T04:56:54.697 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.697 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:54.697 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.701 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11155 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2015232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:54.702 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:56:54.702 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.785089715000002, 2024-09-18T04:56:54.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040769156000000001, 2024-09-18T04:56:54.702 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:54.702 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:54.702 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:54.702 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:54.702 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:54.703 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11155 2024-09-18T04:56:54.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.703 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:54.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:54.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:56:54.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:54.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:56:54.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:54.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:56:54.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:54.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:54.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889420+0000", 2024-09-18T04:56:54.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.181e-06 2024-09-18T04:56:54.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:54.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889426+0000", 2024-09-18T04:56:54.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8599999999999998e-06 2024-09-18T04:56:54.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:54.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889430+0000", 2024-09-18T04:56:54.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4399999999999998e-06 2024-09-18T04:56:54.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:54.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930020+0000", 2024-09-18T04:56:54.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040589698 2024-09-18T04:56:54.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:54.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930105+0000", 2024-09-18T04:56:54.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4812e-05 2024-09-18T04:56:54.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:54.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930187+0000", 2024-09-18T04:56:54.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2540999999999995e-05 2024-09-18T04:56:54.706 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.706 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:54.706 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.706 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11156 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2064384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:54.706 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:56:54.706 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.785049421, 2024-09-18T04:56:54.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040934419, 2024-09-18T04:56:54.707 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:54.707 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:54.707 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:54.707 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:54.707 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:54.707 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11156 2024-09-18T04:56:54.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.707 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:54.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:54.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:56:54.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:54.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:56:54.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:54.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889457+0000", 2024-09-18T04:56:54.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:54.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:54.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889462+0000", 2024-09-18T04:56:54.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8350000000000003e-06 2024-09-18T04:56:54.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:54.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889481+0000", 2024-09-18T04:56:54.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8845999999999999e-05 2024-09-18T04:56:54.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:54.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889495+0000", 2024-09-18T04:56:54.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4229e-05 2024-09-18T04:56:54.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:54.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930216+0000", 2024-09-18T04:56:54.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040721081999999999 2024-09-18T04:56:54.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:54.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930299+0000", 2024-09-18T04:56:54.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2995000000000001e-05 2024-09-18T04:56:54.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:54.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930393+0000", 2024-09-18T04:56:54.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3185999999999999e-05 2024-09-18T04:56:54.710 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.710 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:54.710 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.711 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11402 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1495040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:54.711 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896220+0000", 2024-09-18T04:56:54.711 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.778287451000001, 2024-09-18T04:56:54.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040050388999999999, 2024-09-18T04:56:54.711 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:54.711 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:54.711 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:54.711 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:54.711 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:54.711 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11402 2024-09-18T04:56:54.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.711 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:54.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:54.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896220+0000", 2024-09-18T04:56:54.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:54.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896220+0000", 2024-09-18T04:56:54.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:54.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896220+0000", 2024-09-18T04:56:54.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:54.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:54.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896222+0000", 2024-09-18T04:56:54.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0049999999999999e-06 2024-09-18T04:56:54.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:54.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896227+0000", 2024-09-18T04:56:54.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6130000000000003e-06 2024-09-18T04:56:54.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:54.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896232+0000", 2024-09-18T04:56:54.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4100000000000001e-06 2024-09-18T04:56:54.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:54.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936072+0000", 2024-09-18T04:56:54.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039840253999999999 2024-09-18T04:56:54.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:54.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936175+0000", 2024-09-18T04:56:54.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103037 2024-09-18T04:56:54.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:54.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936271+0000", 2024-09-18T04:56:54.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5383999999999993e-05 2024-09-18T04:56:54.715 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.715 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:54.715 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.715 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11403 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1544192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:54.715 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896250+0000", 2024-09-18T04:56:54.715 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.778258072, 2024-09-18T04:56:54.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040244017999999999, 2024-09-18T04:56:54.715 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:54.715 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:54.715 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:54.716 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:54.716 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:54.716 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11403 2024-09-18T04:56:54.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.716 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:54.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:54.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896250+0000", 2024-09-18T04:56:54.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:54.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896250+0000", 2024-09-18T04:56:54.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:54.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896249+0000", 2024-09-18T04:56:54.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:54.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:54.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896251+0000", 2024-09-18T04:56:54.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.136e-06 2024-09-18T04:56:54.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:54.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896256+0000", 2024-09-18T04:56:54.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9350000000000002e-06 2024-09-18T04:56:54.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:54.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896259+0000", 2024-09-18T04:56:54.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8839999999999998e-06 2024-09-18T04:56:54.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:54.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936320+0000", 2024-09-18T04:56:54.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040061173999999998 2024-09-18T04:56:54.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:54.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936401+0000", 2024-09-18T04:56:54.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0987000000000002e-05 2024-09-18T04:56:54.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:54.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936494+0000", 2024-09-18T04:56:54.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2281999999999995e-05 2024-09-18T04:56:54.719 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.719 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:54.719 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.720 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11404 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1609728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:54.720 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:56:54.720 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.778245258999998, 2024-09-18T04:56:54.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040669012999999997, 2024-09-18T04:56:54.720 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:54.720 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:54.720 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:54.720 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:54.720 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:54.720 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11404 2024-09-18T04:56:54.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.720 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:54.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:54.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:56:54.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:54.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:56:54.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:54.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:56:54.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:56:54.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:54.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896263+0000", 2024-09-18T04:56:54.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2729999999999999e-06 2024-09-18T04:56:54.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:54.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896266+0000", 2024-09-18T04:56:54.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8200000000000001e-06 2024-09-18T04:56:54.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:54.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896269+0000", 2024-09-18T04:56:54.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3199999999999998e-06 2024-09-18T04:56:54.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:54.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936538+0000", 2024-09-18T04:56:54.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040269394999999999 2024-09-18T04:56:54.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:54.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936657+0000", 2024-09-18T04:56:54.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119498 2024-09-18T04:56:54.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:54.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936931+0000", 2024-09-18T04:56:54.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027391900000000002 2024-09-18T04:56:54.724 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.724 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:54.724 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.724 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11405 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1658880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:54.724 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:56:54.724 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.778237001000001, 2024-09-18T04:56:54.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040891034999999999, 2024-09-18T04:56:54.724 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:54.724 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:54.724 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:54.724 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:54.724 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:54.725 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11405 2024-09-18T04:56:54.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.725 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:54.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:54.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:56:54.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:54.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:56:54.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:54.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896270+0000", 2024-09-18T04:56:54.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:56:54.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:54.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896272+0000", 2024-09-18T04:56:54.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.124e-06 2024-09-18T04:56:54.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:54.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896274+0000", 2024-09-18T04:56:54.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8660000000000002e-06 2024-09-18T04:56:54.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:54.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896285+0000", 2024-09-18T04:56:54.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0101e-05 2024-09-18T04:56:54.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:54.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936986+0000", 2024-09-18T04:56:54.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040701213999999999 2024-09-18T04:56:54.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:54.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937070+0000", 2024-09-18T04:56:54.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4211999999999999e-05 2024-09-18T04:56:54.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:54.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937162+0000", 2024-09-18T04:56:54.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1728e-05 2024-09-18T04:56:54.728 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.728 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:54.728 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.728 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11406 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1708032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:54.728 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:56:54.728 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.778220831999999, 2024-09-18T04:56:54.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041085251000000003, 2024-09-18T04:56:54.729 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:54.729 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:54.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:54.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:54.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:54.729 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11406 2024-09-18T04:56:54.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.729 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:54.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:54.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:56:54.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:54.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:56:54.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:54.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:56:54.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:56:54.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:54.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896288+0000", 2024-09-18T04:56:54.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2440000000000001e-06 2024-09-18T04:56:54.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:54.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896291+0000", 2024-09-18T04:56:54.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8289999999999999e-06 2024-09-18T04:56:54.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:54.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896293+0000", 2024-09-18T04:56:54.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5399999999999998e-06 2024-09-18T04:56:54.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:54.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937201+0000", 2024-09-18T04:56:54.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040907418000000001 2024-09-18T04:56:54.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:54.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937283+0000", 2024-09-18T04:56:54.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2460999999999995e-05 2024-09-18T04:56:54.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:54.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937372+0000", 2024-09-18T04:56:54.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8987999999999996e-05 2024-09-18T04:56:54.732 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.732 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:54.732 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.733 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11407 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:54.733 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:56:54.733 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.778211012, 2024-09-18T04:56:54.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041438580000000003, 2024-09-18T04:56:54.733 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:54.733 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:54.733 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:54.733 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:54.733 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:54.733 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11407 2024-09-18T04:56:54.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.733 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:54.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:54.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:56:54.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:54.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:56:54.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:54.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896296+0000", 2024-09-18T04:56:54.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:54.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:54.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896298+0000", 2024-09-18T04:56:54.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:56:54.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:54.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896304+0000", 2024-09-18T04:56:54.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6570000000000002e-06 2024-09-18T04:56:54.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:54.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896308+0000", 2024-09-18T04:56:54.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0940000000000001e-06 2024-09-18T04:56:54.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:54.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937540+0000", 2024-09-18T04:56:54.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041231469 2024-09-18T04:56:54.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:54.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937625+0000", 2024-09-18T04:56:54.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5767999999999996e-05 2024-09-18T04:56:54.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:54.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937735+0000", 2024-09-18T04:56:54.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010974 2024-09-18T04:56:54.737 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.737 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:54.737 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.737 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11408 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:54.737 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:56:54.737 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.778195372999999, 2024-09-18T04:56:54.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041653055000000001, 2024-09-18T04:56:54.737 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:54.737 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:54.737 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:54.738 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:54.738 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:54.738 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11408 2024-09-18T04:56:54.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.738 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:54.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:54.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:56:54.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:54.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:56:54.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:54.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:56:54.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:54.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:54.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896314+0000", 2024-09-18T04:56:54.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0140000000000001e-06 2024-09-18T04:56:54.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:54.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896320+0000", 2024-09-18T04:56:54.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-18T04:56:54.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:54.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896324+0000", 2024-09-18T04:56:54.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4410000000000003e-06 2024-09-18T04:56:54.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:54.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937769+0000", 2024-09-18T04:56:54.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041445243 2024-09-18T04:56:54.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:54.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937861+0000", 2024-09-18T04:56:54.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1471000000000006e-05 2024-09-18T04:56:54.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:54.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937965+0000", 2024-09-18T04:56:54.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104516 2024-09-18T04:56:54.742 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.742 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:54.742 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.742 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11409 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:54.742 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:56:54.742 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.778179316999999, 2024-09-18T04:56:54.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041840467999999999, 2024-09-18T04:56:54.742 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:54.742 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:54.742 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:54.742 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:54.742 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:54.743 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11409 2024-09-18T04:56:54.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.743 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:54.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:54.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:56:54.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:54.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:56:54.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:54.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:56:54.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:54.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:54.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896331+0000", 2024-09-18T04:56:54.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5950000000000001e-06 2024-09-18T04:56:54.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:54.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896336+0000", 2024-09-18T04:56:54.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7259999999999999e-06 2024-09-18T04:56:54.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:54.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896341+0000", 2024-09-18T04:56:54.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5619999999999997e-06 2024-09-18T04:56:54.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:54.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937995+0000", 2024-09-18T04:56:54.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041654060999999999 2024-09-18T04:56:54.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:54.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938078+0000", 2024-09-18T04:56:54.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2869999999999998e-05 2024-09-18T04:56:54.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:54.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938169+0000", 2024-09-18T04:56:54.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1040999999999998e-05 2024-09-18T04:56:54.746 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.746 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:54.746 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.746 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11410 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1921024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:54.746 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:56:54.746 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.778162834, 2024-09-18T04:56:54.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042060342000000001, 2024-09-18T04:56:54.747 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:54.747 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:54.747 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:54.747 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:54.747 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:54.747 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11410 2024-09-18T04:56:54.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.747 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:54.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:54.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:56:54.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:54.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:56:54.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:54.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896344+0000", 2024-09-18T04:56:54.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:54.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:54.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896360+0000", 2024-09-18T04:56:54.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5068e-05 2024-09-18T04:56:54.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:54.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896366+0000", 2024-09-18T04:56:54.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5119999999999997e-06 2024-09-18T04:56:54.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:54.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896371+0000", 2024-09-18T04:56:54.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:56:54.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:54.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938198+0000", 2024-09-18T04:56:54.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041826579000000003 2024-09-18T04:56:54.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:54.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938297+0000", 2024-09-18T04:56:54.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9067000000000005e-05 2024-09-18T04:56:54.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:54.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938405+0000", 2024-09-18T04:56:54.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108492 2024-09-18T04:56:54.750 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.750 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:54.750 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.751 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11411 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1970176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:54.751 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:56:54.751 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.778131991999999, 2024-09-18T04:56:54.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042259191000000002, 2024-09-18T04:56:54.751 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:54.751 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:54.751 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:54.751 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:54.751 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:54.751 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11411 2024-09-18T04:56:54.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.751 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:54.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:54.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:56:54.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:54.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:56:54.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:54.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896375+0000", 2024-09-18T04:56:54.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:54.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:54.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896377+0000", 2024-09-18T04:56:54.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2699999999999999e-06 2024-09-18T04:56:54.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:54.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896383+0000", 2024-09-18T04:56:54.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6740000000000002e-06 2024-09-18T04:56:54.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:54.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896388+0000", 2024-09-18T04:56:54.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:56:54.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:54.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938435+0000", 2024-09-18T04:56:54.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042047169000000002 2024-09-18T04:56:54.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:54.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938534+0000", 2024-09-18T04:56:54.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8939999999999998e-05 2024-09-18T04:56:54.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:54.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938635+0000", 2024-09-18T04:56:54.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101208 2024-09-18T04:56:54.754 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.755 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:54.755 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.755 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11412 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:54.755 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:56:54.755 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.778116010000002, 2024-09-18T04:56:54.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042828079999999998, 2024-09-18T04:56:54.755 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:54.755 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:54.755 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:54.755 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:54.755 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:54.755 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11412 2024-09-18T04:56:54.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.756 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:54.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:54.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:56:54.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:54.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:56:54.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:54.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896391+0000", 2024-09-18T04:56:54.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:54.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:54.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896393+0000", 2024-09-18T04:56:54.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0810000000000001e-06 2024-09-18T04:56:54.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:54.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896399+0000", 2024-09-18T04:56:54.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7230000000000001e-06 2024-09-18T04:56:54.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:54.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896403+0000", 2024-09-18T04:56:54.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3470000000000001e-06 2024-09-18T04:56:54.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:54.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939043+0000", 2024-09-18T04:56:54.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042639829999999997 2024-09-18T04:56:54.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:54.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939124+0000", 2024-09-18T04:56:54.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0809000000000005e-05 2024-09-18T04:56:54.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:54.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939220+0000", 2024-09-18T04:56:54.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5659e-05 2024-09-18T04:56:54.759 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.759 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:54.759 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.759 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11413 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2068480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:54.759 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:56:54.759 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.778100315, 2024-09-18T04:56:54.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043042108000000003, 2024-09-18T04:56:54.759 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:54.759 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:54.760 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:54.760 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:54.760 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:54.760 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11413 2024-09-18T04:56:54.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.760 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:54.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:54.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:56:54.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:54.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:56:54.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:54.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:56:54.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:54.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:54.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896409+0000", 2024-09-18T04:56:54.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.165e-06 2024-09-18T04:56:54.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:54.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896415+0000", 2024-09-18T04:56:54.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6149999999999996e-06 2024-09-18T04:56:54.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:54.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896419+0000", 2024-09-18T04:56:54.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2649999999999998e-06 2024-09-18T04:56:54.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:54.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939262+0000", 2024-09-18T04:56:54.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042843444000000001 2024-09-18T04:56:54.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:54.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939351+0000", 2024-09-18T04:56:54.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8944999999999996e-05 2024-09-18T04:56:54.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:54.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939449+0000", 2024-09-18T04:56:54.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8029000000000006e-05 2024-09-18T04:56:54.763 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.763 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:54.763 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.763 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11669 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2023424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:54.763 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.907990+0000", 2024-09-18T04:56:54.764 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.766517421, 2024-09-18T04:56:54.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040045942000000001, 2024-09-18T04:56:54.764 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:54.764 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:54.764 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:54.764 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:54.764 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:54.764 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11669 2024-09-18T04:56:54.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.764 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:54.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:54.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.907990+0000", 2024-09-18T04:56:54.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:54.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.907990+0000", 2024-09-18T04:56:54.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:54.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.907989+0000", 2024-09-18T04:56:54.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:54.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:54.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.907993+0000", 2024-09-18T04:56:54.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6109999999999998e-06 2024-09-18T04:56:54.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:54.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908003+0000", 2024-09-18T04:56:54.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7610000000000002e-06 2024-09-18T04:56:54.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:54.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908009+0000", 2024-09-18T04:56:54.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7660000000000004e-06 2024-09-18T04:56:54.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:54.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.947929+0000", 2024-09-18T04:56:54.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039920297 2024-09-18T04:56:54.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:54.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.947982+0000", 2024-09-18T04:56:54.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3176e-05 2024-09-18T04:56:54.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:54.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.948036+0000", 2024-09-18T04:56:54.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4052999999999999e-05 2024-09-18T04:56:54.767 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.767 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:54.767 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.768 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11670 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2072576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:56:54.768 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.908014+0000", 2024-09-18T04:56:54.768 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.766493539999999, 2024-09-18T04:56:54.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040179074000000002, 2024-09-18T04:56:54.768 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:54.768 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:54.768 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:54.768 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:54.768 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:54.768 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11670 2024-09-18T04:56:54.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.768 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:54.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:54.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908014+0000", 2024-09-18T04:56:54.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:54.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908014+0000", 2024-09-18T04:56:54.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:54.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:54.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908014+0000", 2024-09-18T04:56:54.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:54.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:54.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908016+0000", 2024-09-18T04:56:54.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3889999999999999e-06 2024-09-18T04:56:54.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:54.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908023+0000", 2024-09-18T04:56:54.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5599999999999999e-06 2024-09-18T04:56:54.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:54.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908028+0000", 2024-09-18T04:56:54.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.242e-06 2024-09-18T04:56:54.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:54.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.948075+0000", 2024-09-18T04:56:54.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040047117 2024-09-18T04:56:54.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:54.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.948128+0000", 2024-09-18T04:56:54.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3127999999999997e-05 2024-09-18T04:56:54.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:54.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:54.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:56:54.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.948193+0000", 2024-09-18T04:56:54.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5042000000000005e-05 2024-09-18T04:56:54.771 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.772 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:54.772 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.772 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:54.772 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:54.772 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:54.772 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:56:54.773 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:56:54.774 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:56:54.774 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:56:54.775 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:56:54.776 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:54.774+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:54.876 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:54.874+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:54.878 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:54.878 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:54.878 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:56:54.878 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:54.886 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:56:54.977 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:54.975+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:55.028 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:55.029 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:55.029 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:56:55.029 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:56:55.029 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:55.037 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:56:55.078 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:55.076+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:55.179 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:55.177+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:55.181 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:55.181 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:56:55.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:56:55.181 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:56:55.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.182 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:5082 3.1a 3:5d34c0ec:::rbd_data.11ac14fc1d07.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 26797226~22514] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-09-18T04:56:55.182 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:32.444662+0000", 2024-09-18T04:56:55.182 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.733921354, 2024-09-18T04:56:55.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048360470000000003, 2024-09-18T04:56:55.182 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.182 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:55.182 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.182 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.182 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.182 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 5082 2024-09-18T04:56:55.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.182 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.444662+0000", 2024-09-18T04:56:55.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.444662+0000", 2024-09-18T04:56:55.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.444661+0000", 2024-09-18T04:56:55.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:55.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.469347+0000", 2024-09-18T04:56:55.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024685830999999998 2024-09-18T04:56:55.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.469368+0000", 2024-09-18T04:56:55.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1166999999999999e-05 2024-09-18T04:56:55.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.469377+0000", 2024-09-18T04:56:55.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5790000000000004e-06 2024-09-18T04:56:55.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.469420+0000", 2024-09-18T04:56:55.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3238999999999997e-05 2024-09-18T04:56:55.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.469526+0000", 2024-09-18T04:56:55.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010607699999999999 2024-09-18T04:56:55.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:55.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.469895+0000", 2024-09-18T04:56:55.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00036904 2024-09-18T04:56:55.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:55.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.481413+0000", 2024-09-18T04:56:55.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011517583 2024-09-18T04:56:55.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:55.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.492856+0000", 2024-09-18T04:56:55.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.011443511999999999 2024-09-18T04:56:55.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:55.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.492944+0000", 2024-09-18T04:56:55.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7835000000000002e-05 2024-09-18T04:56:55.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:32.493023+0000", 2024-09-18T04:56:55.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8617999999999998e-05 2024-09-18T04:56:55.191 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.191 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.191 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.191 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61086 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 1538048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.191 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.108826+0000", 2024-09-18T04:56:55.191 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0697574169999999, 2024-09-18T04:56:55.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043247130000000002, 2024-09-18T04:56:55.191 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.191 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.192 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.192 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.192 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.192 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61086 2024-09-18T04:56:55.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.192 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108826+0000", 2024-09-18T04:56:55.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108826+0000", 2024-09-18T04:56:55.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108826+0000", 2024-09-18T04:56:55.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108828+0000", 2024-09-18T04:56:55.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.035e-06 2024-09-18T04:56:55.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108833+0000", 2024-09-18T04:56:55.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6489999999999996e-06 2024-09-18T04:56:55.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108838+0000", 2024-09-18T04:56:55.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3220000000000003e-06 2024-09-18T04:56:55.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.151968+0000", 2024-09-18T04:56:55.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043130773999999997 2024-09-18T04:56:55.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.152017+0000", 2024-09-18T04:56:55.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8439999999999997e-05 2024-09-18T04:56:55.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.152073+0000", 2024-09-18T04:56:55.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6270000000000002e-05 2024-09-18T04:56:55.198 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.198 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.198 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.199 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61087 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 1553408~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.199 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.108842+0000", 2024-09-18T04:56:55.199 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0697415780000004, 2024-09-18T04:56:55.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043416264000000003, 2024-09-18T04:56:55.199 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.199 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.199 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.199 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.199 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.200 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61087 2024-09-18T04:56:55.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.200 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108842+0000", 2024-09-18T04:56:55.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108842+0000", 2024-09-18T04:56:55.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108842+0000", 2024-09-18T04:56:55.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108844+0000", 2024-09-18T04:56:55.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9929999999999998e-06 2024-09-18T04:56:55.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108849+0000", 2024-09-18T04:56:55.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6760000000000004e-06 2024-09-18T04:56:55.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108854+0000", 2024-09-18T04:56:55.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8180000000000003e-06 2024-09-18T04:56:55.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.152098+0000", 2024-09-18T04:56:55.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043243990000000003 2024-09-18T04:56:55.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.152179+0000", 2024-09-18T04:56:55.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1043000000000003e-05 2024-09-18T04:56:55.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.152258+0000", 2024-09-18T04:56:55.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9060000000000005e-05 2024-09-18T04:56:55.203 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.203 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.203 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.204 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61088 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 1568768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.204 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.108858+0000", 2024-09-18T04:56:55.204 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0697253929999997, 2024-09-18T04:56:55.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043569878999999999, 2024-09-18T04:56:55.204 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.204 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.204 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.204 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.204 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.204 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61088 2024-09-18T04:56:55.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.204 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108858+0000", 2024-09-18T04:56:55.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108858+0000", 2024-09-18T04:56:55.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108858+0000", 2024-09-18T04:56:55.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108860+0000", 2024-09-18T04:56:55.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.88e-06 2024-09-18T04:56:55.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108865+0000", 2024-09-18T04:56:55.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5310000000000001e-06 2024-09-18T04:56:55.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.108870+0000", 2024-09-18T04:56:55.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8640000000000004e-06 2024-09-18T04:56:55.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.152285+0000", 2024-09-18T04:56:55.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043415020999999998 2024-09-18T04:56:55.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.152347+0000", 2024-09-18T04:56:55.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2459000000000006e-05 2024-09-18T04:56:55.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.152428+0000", 2024-09-18T04:56:55.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0433999999999996e-05 2024-09-18T04:56:55.210 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.210 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.210 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.210 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61130 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1413120~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.210 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.110351+0000", 2024-09-18T04:56:55.210 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0682319849999997, 2024-09-18T04:56:55.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043155852000000001, 2024-09-18T04:56:55.210 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.210 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.211 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.211 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.211 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.211 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61130 2024-09-18T04:56:55.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.211 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110351+0000", 2024-09-18T04:56:55.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110351+0000", 2024-09-18T04:56:55.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110351+0000", 2024-09-18T04:56:55.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110353+0000", 2024-09-18T04:56:55.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2539999999999999e-06 2024-09-18T04:56:55.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110360+0000", 2024-09-18T04:56:55.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3119999999999999e-06 2024-09-18T04:56:55.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110365+0000", 2024-09-18T04:56:55.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2270000000000001e-06 2024-09-18T04:56:55.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.153376+0000", 2024-09-18T04:56:55.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043011484000000003 2024-09-18T04:56:55.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.153443+0000", 2024-09-18T04:56:55.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7187000000000006e-05 2024-09-18T04:56:55.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.153507+0000", 2024-09-18T04:56:55.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3825000000000006e-05 2024-09-18T04:56:55.218 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.218 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61131 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1428480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.110394+0000", 2024-09-18T04:56:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0681894569999999, 2024-09-18T04:56:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043272860000000003, 2024-09-18T04:56:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.219 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61131 2024-09-18T04:56:55.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.220 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110394+0000", 2024-09-18T04:56:55.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110394+0000", 2024-09-18T04:56:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110393+0000", 2024-09-18T04:56:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110396+0000", 2024-09-18T04:56:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.999e-06 2024-09-18T04:56:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110404+0000", 2024-09-18T04:56:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.661e-06 2024-09-18T04:56:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110410+0000", 2024-09-18T04:56:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5130000000000004e-06 2024-09-18T04:56:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.153533+0000", 2024-09-18T04:56:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043123131000000002 2024-09-18T04:56:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.153600+0000", 2024-09-18T04:56:55.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7361999999999996e-05 2024-09-18T04:56:55.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.153667+0000", 2024-09-18T04:56:55.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6762999999999998e-05 2024-09-18T04:56:55.223 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.223 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.223 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61132 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1443840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.110414+0000", 2024-09-18T04:56:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0681690599999998, 2024-09-18T04:56:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043417771000000001, 2024-09-18T04:56:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61132 2024-09-18T04:56:55.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110414+0000", 2024-09-18T04:56:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110414+0000", 2024-09-18T04:56:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110414+0000", 2024-09-18T04:56:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110416+0000", 2024-09-18T04:56:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3089999999999998e-06 2024-09-18T04:56:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110422+0000", 2024-09-18T04:56:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9540000000000001e-06 2024-09-18T04:56:55.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110427+0000", 2024-09-18T04:56:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1100000000000002e-06 2024-09-18T04:56:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.153694+0000", 2024-09-18T04:56:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043266616000000001 2024-09-18T04:56:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.153765+0000", 2024-09-18T04:56:55.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1557999999999995e-05 2024-09-18T04:56:55.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.153832+0000", 2024-09-18T04:56:55.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6674999999999994e-05 2024-09-18T04:56:55.228 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.228 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.228 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.228 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61133 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1459200~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.110438+0000", 2024-09-18T04:56:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0681458920000004, 2024-09-18T04:56:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043566147999999999, 2024-09-18T04:56:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61133 2024-09-18T04:56:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110438+0000", 2024-09-18T04:56:55.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110438+0000", 2024-09-18T04:56:55.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110437+0000", 2024-09-18T04:56:55.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110440+0000", 2024-09-18T04:56:55.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5639999999999999e-06 2024-09-18T04:56:55.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110447+0000", 2024-09-18T04:56:55.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0639999999999996e-06 2024-09-18T04:56:55.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110452+0000", 2024-09-18T04:56:55.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0649999999999998e-06 2024-09-18T04:56:55.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.153861+0000", 2024-09-18T04:56:55.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043409534 2024-09-18T04:56:55.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.153933+0000", 2024-09-18T04:56:55.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1730999999999995e-05 2024-09-18T04:56:55.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154004+0000", 2024-09-18T04:56:55.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0610000000000003e-05 2024-09-18T04:56:55.234 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.234 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.234 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.235 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61134 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1474560~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.235 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.110456+0000", 2024-09-18T04:56:55.235 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0681273610000002, 2024-09-18T04:56:55.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043713114999999997, 2024-09-18T04:56:55.236 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.236 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.236 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.236 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.236 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.237 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61134 2024-09-18T04:56:55.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.237 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110456+0000", 2024-09-18T04:56:55.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110456+0000", 2024-09-18T04:56:55.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110456+0000", 2024-09-18T04:56:55.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110458+0000", 2024-09-18T04:56:55.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1380000000000001e-06 2024-09-18T04:56:55.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110464+0000", 2024-09-18T04:56:55.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.212e-06 2024-09-18T04:56:55.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110469+0000", 2024-09-18T04:56:55.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7779999999999998e-06 2024-09-18T04:56:55.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154033+0000", 2024-09-18T04:56:55.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043563892999999999 2024-09-18T04:56:55.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154104+0000", 2024-09-18T04:56:55.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1699000000000006e-05 2024-09-18T04:56:55.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154169+0000", 2024-09-18T04:56:55.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4775000000000002e-05 2024-09-18T04:56:55.244 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.244 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.245 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.245 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61135 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1489920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.245 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.110473+0000", 2024-09-18T04:56:55.245 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0681102979999997, 2024-09-18T04:56:55.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.04384685, 2024-09-18T04:56:55.245 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.245 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.245 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.245 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61135 2024-09-18T04:56:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110473+0000", 2024-09-18T04:56:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110473+0000", 2024-09-18T04:56:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110473+0000", 2024-09-18T04:56:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110475+0000", 2024-09-18T04:56:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1129999999999999e-06 2024-09-18T04:56:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110487+0000", 2024-09-18T04:56:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2507000000000001e-05 2024-09-18T04:56:55.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110494+0000", 2024-09-18T04:56:55.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1310000000000003e-06 2024-09-18T04:56:55.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154197+0000", 2024-09-18T04:56:55.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043702986999999999 2024-09-18T04:56:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154262+0000", 2024-09-18T04:56:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5236999999999999e-05 2024-09-18T04:56:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154320+0000", 2024-09-18T04:56:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8241e-05 2024-09-18T04:56:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61136 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1505280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.110499+0000", 2024-09-18T04:56:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0680848879999996, 2024-09-18T04:56:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043976113999999997, 2024-09-18T04:56:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61136 2024-09-18T04:56:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.250 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110499+0000", 2024-09-18T04:56:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110499+0000", 2024-09-18T04:56:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110498+0000", 2024-09-18T04:56:55.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110501+0000", 2024-09-18T04:56:55.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6910000000000002e-06 2024-09-18T04:56:55.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110507+0000", 2024-09-18T04:56:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.516e-06 2024-09-18T04:56:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110512+0000", 2024-09-18T04:56:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9409999999999998e-06 2024-09-18T04:56:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154348+0000", 2024-09-18T04:56:55.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043836075000000002 2024-09-18T04:56:55.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154415+0000", 2024-09-18T04:56:55.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6748999999999994e-05 2024-09-18T04:56:55.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154475+0000", 2024-09-18T04:56:55.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9635000000000001e-05 2024-09-18T04:56:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61137 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1520640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.110516+0000", 2024-09-18T04:56:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0680669549999999, 2024-09-18T04:56:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044112431000000001, 2024-09-18T04:56:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.255 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61137 2024-09-18T04:56:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110516+0000", 2024-09-18T04:56:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110516+0000", 2024-09-18T04:56:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110516+0000", 2024-09-18T04:56:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110518+0000", 2024-09-18T04:56:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3e-06 2024-09-18T04:56:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110524+0000", 2024-09-18T04:56:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8320000000000002e-06 2024-09-18T04:56:55.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110529+0000", 2024-09-18T04:56:55.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7559999999999999e-06 2024-09-18T04:56:55.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154507+0000", 2024-09-18T04:56:55.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043978189000000001 2024-09-18T04:56:55.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154571+0000", 2024-09-18T04:56:55.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3614000000000005e-05 2024-09-18T04:56:55.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154629+0000", 2024-09-18T04:56:55.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8118999999999997e-05 2024-09-18T04:56:55.260 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.260 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.260 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.261 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61138 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1536000~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.261 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.110533+0000", 2024-09-18T04:56:55.261 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0680499929999998, 2024-09-18T04:56:55.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044287528999999999, 2024-09-18T04:56:55.261 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.262 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.262 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.262 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.262 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.262 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61138 2024-09-18T04:56:55.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.263 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110533+0000", 2024-09-18T04:56:55.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110533+0000", 2024-09-18T04:56:55.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110533+0000", 2024-09-18T04:56:55.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110535+0000", 2024-09-18T04:56:55.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1679999999999998e-06 2024-09-18T04:56:55.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110541+0000", 2024-09-18T04:56:55.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8900000000000004e-06 2024-09-18T04:56:55.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110546+0000", 2024-09-18T04:56:55.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6759999999999998e-06 2024-09-18T04:56:55.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154669+0000", 2024-09-18T04:56:55.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044122863999999998 2024-09-18T04:56:55.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154752+0000", 2024-09-18T04:56:55.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3008000000000002e-05 2024-09-18T04:56:55.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154821+0000", 2024-09-18T04:56:55.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9283999999999996e-05 2024-09-18T04:56:55.268 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.268 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.268 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.268 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61139 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1551360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.268 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.110582+0000", 2024-09-18T04:56:55.268 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0680012420000002, 2024-09-18T04:56:55.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044399427999999998, 2024-09-18T04:56:55.268 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.268 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.268 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.269 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.269 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.269 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61139 2024-09-18T04:56:55.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.269 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110582+0000", 2024-09-18T04:56:55.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110582+0000", 2024-09-18T04:56:55.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110582+0000", 2024-09-18T04:56:55.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110585+0000", 2024-09-18T04:56:55.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.286e-06 2024-09-18T04:56:55.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110593+0000", 2024-09-18T04:56:55.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4260000000000003e-06 2024-09-18T04:56:55.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110600+0000", 2024-09-18T04:56:55.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1970000000000001e-06 2024-09-18T04:56:55.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154854+0000", 2024-09-18T04:56:55.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044254427999999998 2024-09-18T04:56:55.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154921+0000", 2024-09-18T04:56:55.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7453999999999995e-05 2024-09-18T04:56:55.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.154982+0000", 2024-09-18T04:56:55.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0191e-05 2024-09-18T04:56:55.272 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.272 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.272 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.273 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61140 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1566720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.273 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.110605+0000", 2024-09-18T04:56:55.273 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0679786519999999, 2024-09-18T04:56:55.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044541944999999999, 2024-09-18T04:56:55.273 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.273 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.273 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.273 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.273 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.273 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61140 2024-09-18T04:56:55.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.273 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110605+0000", 2024-09-18T04:56:55.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110605+0000", 2024-09-18T04:56:55.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110604+0000", 2024-09-18T04:56:55.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110607+0000", 2024-09-18T04:56:55.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.21e-06 2024-09-18T04:56:55.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110613+0000", 2024-09-18T04:56:55.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3670000000000002e-06 2024-09-18T04:56:55.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.110618+0000", 2024-09-18T04:56:55.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7829999999999998e-06 2024-09-18T04:56:55.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.155013+0000", 2024-09-18T04:56:55.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044395028000000003 2024-09-18T04:56:55.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.155085+0000", 2024-09-18T04:56:55.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1768999999999997e-05 2024-09-18T04:56:55.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.155147+0000", 2024-09-18T04:56:55.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2197000000000001e-05 2024-09-18T04:56:55.277 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.277 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.277 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.277 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61390 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1460224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.277 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.119198+0000", 2024-09-18T04:56:55.277 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.059385185, 2024-09-18T04:56:55.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043152569000000002, 2024-09-18T04:56:55.277 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.277 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.277 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.278 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.278 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.278 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61390 2024-09-18T04:56:55.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.278 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119198+0000", 2024-09-18T04:56:55.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119198+0000", 2024-09-18T04:56:55.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119198+0000", 2024-09-18T04:56:55.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119200+0000", 2024-09-18T04:56:55.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3640000000000002e-06 2024-09-18T04:56:55.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119206+0000", 2024-09-18T04:56:55.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9610000000000002e-06 2024-09-18T04:56:55.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119211+0000", 2024-09-18T04:56:55.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6059999999999996e-06 2024-09-18T04:56:55.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162225+0000", 2024-09-18T04:56:55.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043014189000000001 2024-09-18T04:56:55.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162289+0000", 2024-09-18T04:56:55.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4218000000000001e-05 2024-09-18T04:56:55.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162351+0000", 2024-09-18T04:56:55.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1773999999999995e-05 2024-09-18T04:56:55.281 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.281 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.281 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.281 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61394 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1521664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.282 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.119713+0000", 2024-09-18T04:56:55.282 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0588704739999999, 2024-09-18T04:56:55.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043168156999999999, 2024-09-18T04:56:55.282 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.282 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.282 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.282 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.282 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.282 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61394 2024-09-18T04:56:55.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.282 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119713+0000", 2024-09-18T04:56:55.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119713+0000", 2024-09-18T04:56:55.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119712+0000", 2024-09-18T04:56:55.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119715+0000", 2024-09-18T04:56:55.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7010000000000001e-06 2024-09-18T04:56:55.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119722+0000", 2024-09-18T04:56:55.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1330000000000001e-06 2024-09-18T04:56:55.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119728+0000", 2024-09-18T04:56:55.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.186e-06 2024-09-18T04:56:55.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162751+0000", 2024-09-18T04:56:55.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043023790999999999 2024-09-18T04:56:55.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162819+0000", 2024-09-18T04:56:55.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7970000000000001e-05 2024-09-18T04:56:55.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162881+0000", 2024-09-18T04:56:55.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1841999999999995e-05 2024-09-18T04:56:55.286 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.286 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.286 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.286 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61395 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1537024~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.286 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.119735+0000", 2024-09-18T04:56:55.286 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.058848309, 2024-09-18T04:56:55.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043266315, 2024-09-18T04:56:55.286 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.286 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.287 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61395 2024-09-18T04:56:55.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.287 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119735+0000", 2024-09-18T04:56:55.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119735+0000", 2024-09-18T04:56:55.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119732+0000", 2024-09-18T04:56:55.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999971 2024-09-18T04:56:55.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119737+0000", 2024-09-18T04:56:55.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9150000000000004e-06 2024-09-18T04:56:55.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119744+0000", 2024-09-18T04:56:55.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4189999999999999e-06 2024-09-18T04:56:55.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119748+0000", 2024-09-18T04:56:55.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8659999999999997e-06 2024-09-18T04:56:55.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162913+0000", 2024-09-18T04:56:55.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043164730999999998 2024-09-18T04:56:55.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162959+0000", 2024-09-18T04:56:55.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6190999999999999e-05 2024-09-18T04:56:55.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.163001+0000", 2024-09-18T04:56:55.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2089000000000003e-05 2024-09-18T04:56:55.291 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.291 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.291 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.291 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61396 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1552384~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.291 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.119753+0000", 2024-09-18T04:56:55.292 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0588305360000003, 2024-09-18T04:56:55.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043432158999999998, 2024-09-18T04:56:55.292 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.292 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.292 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.292 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.292 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.292 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61396 2024-09-18T04:56:55.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.292 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119753+0000", 2024-09-18T04:56:55.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119753+0000", 2024-09-18T04:56:55.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119753+0000", 2024-09-18T04:56:55.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119755+0000", 2024-09-18T04:56:55.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0980000000000001e-06 2024-09-18T04:56:55.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119771+0000", 2024-09-18T04:56:55.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6056999999999999e-05 2024-09-18T04:56:55.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119776+0000", 2024-09-18T04:56:55.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0719999999999999e-06 2024-09-18T04:56:55.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.163037+0000", 2024-09-18T04:56:55.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043261130000000002 2024-09-18T04:56:55.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.163114+0000", 2024-09-18T04:56:55.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7545000000000004e-05 2024-09-18T04:56:55.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.163185+0000", 2024-09-18T04:56:55.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0598000000000004e-05 2024-09-18T04:56:55.296 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.296 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.296 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.296 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:61397 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 1567744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.296 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.119782+0000", 2024-09-18T04:56:55.296 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.0588014399999999, 2024-09-18T04:56:55.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043533611999999999, 2024-09-18T04:56:55.297 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.297 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.297 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.297 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.297 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.297 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 61397 2024-09-18T04:56:55.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.297 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119782+0000", 2024-09-18T04:56:55.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119782+0000", 2024-09-18T04:56:55.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119782+0000", 2024-09-18T04:56:55.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119784+0000", 2024-09-18T04:56:55.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7690000000000001e-06 2024-09-18T04:56:55.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119791+0000", 2024-09-18T04:56:55.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1960000000000002e-06 2024-09-18T04:56:55.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.119798+0000", 2024-09-18T04:56:55.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0310000000000004e-06 2024-09-18T04:56:55.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.163230+0000", 2024-09-18T04:56:55.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043432456000000001 2024-09-18T04:56:55.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.163273+0000", 2024-09-18T04:56:55.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3234e-05 2024-09-18T04:56:55.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.163316+0000", 2024-09-18T04:56:55.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2382000000000002e-05 2024-09-18T04:56:55.301 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.301 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.301 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.301 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.301 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.301 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:55.301 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T04:56:55.303 INFO:teuthology.orchestra.run.smithi002.stdout:2332 read 0x5200c7b thru 0x5202912 (0x1c98 bytes) 2024-09-18T04:56:55.303 INFO:teuthology.orchestra.run.smithi002.stdout:2334 read 0x9d7f9d5 thru 0x9d829f8 (0x3024 bytes) 2024-09-18T04:56:55.303 INFO:teuthology.orchestra.run.smithi002.stdout:2335 writesame 0xe723a1a thru 0xe723dfc data_size 0x1(0x3e3 bytes) 2024-09-18T04:56:55.303 INFO:teuthology.orchestra.run.smithi002.stdout:2336 trunc from 0xe723dfd to 0xbedfaa7 2024-09-18T04:56:55.304 INFO:teuthology.orchestra.run.smithi002.stdout:2339 write 0xafaae9d thru 0xafb9ba6 (0xed0a bytes) 2024-09-18T04:56:55.304 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:55.300+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:55.304 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:56:55.318 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:55.334 INFO:teuthology.orchestra.run.smithi002.stdout:2341 trunc from 0xbedfaa7 to 0x124b133 2024-09-18T04:56:55.374 INFO:teuthology.orchestra.run.smithi002.stdout:2342 read 0x1048e29 thru 0x104be7e (0x3056 bytes) 2024-09-18T04:56:55.374 INFO:teuthology.orchestra.run.smithi002.stdout:2343 read 0x4c3613 thru 0x4c3ebb (0x8a9 bytes) 2024-09-18T04:56:55.374 INFO:teuthology.orchestra.run.smithi002.stdout:2344 read 0x3ed91e thru 0x3f7826 (0x9f09 bytes) 2024-09-18T04:56:55.380 INFO:teuthology.orchestra.run.smithi002.stdout:2346 punch from 0x163d19 to 0x16ab7b, (0x6e62 bytes) 2024-09-18T04:56:55.380 INFO:teuthology.orchestra.run.smithi002.stdout:2347 punch from 0x4a63f7 to 0x4b124b, (0xae54 bytes) 2024-09-18T04:56:55.393 INFO:teuthology.orchestra.run.smithi002.stdout:2349 write 0x72ebe0b thru 0x72f2d6d (0x6f63 bytes) 2024-09-18T04:56:55.404 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:55.402+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:55.411 INFO:teuthology.orchestra.run.smithi002.stdout:2351 punch from 0x6b50f05 to 0x6b5ef1c, (0xe017 bytes) 2024-09-18T04:56:55.411 INFO:teuthology.orchestra.run.smithi002.stdout:2352 read 0x44a733f thru 0x44ad2ee (0x5fb0 bytes) 2024-09-18T04:56:55.422 INFO:teuthology.orchestra.run.smithi002.stdout:2354 write 0xc3a67f8 thru 0xc3b00c9 (0x98d2 bytes) 2024-09-18T04:56:55.431 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:55.432 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:56:55.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.432 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:66944 3.1d 3:be521b93:::rbd_data.11ac8b4c71f8.000000000000030f:head [set-alloc-hint object_size 262144 write_size 262144,write 46080~2048 in=2048b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e50)", 2024-09-18T04:56:55.432 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:55.429901+0000", 2024-09-18T04:56:55.432 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.6471999999999996e-05, 2024-09-18T04:56:55.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9498000000000002e-05, 2024-09-18T04:56:55.432 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.432 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:55.432 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.432 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.432 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.433 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 66944 2024-09-18T04:56:55.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.433 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:55.429901+0000", 2024-09-18T04:56:55.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:55.429901+0000", 2024-09-18T04:56:55.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:55.429899+0000", 2024-09-18T04:56:55.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:56:55.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:55.429906+0000", 2024-09-18T04:56:55.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3459999999999999e-06 2024-09-18T04:56:55.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:55.429918+0000", 2024-09-18T04:56:55.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1848000000000001e-05 2024-09-18T04:56:55.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:55.429924+0000", 2024-09-18T04:56:55.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3119999999999999e-06 2024-09-18T04:56:55.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:55.429960+0000", 2024-09-18T04:56:55.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6118999999999998e-05 2024-09-18T04:56:55.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:55.430013+0000", 2024-09-18T04:56:55.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3263000000000001e-05 2024-09-18T04:56:55.436 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.436 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.436 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.436 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.436 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-18T04:56:55.436 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 1 2024-09-18T04:56:55.436 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:55.439 INFO:teuthology.orchestra.run.smithi002.stdout:2357 read 0xb9fabae thru 0xb9fb316 (0x769 bytes) 2024-09-18T04:56:55.439 INFO:teuthology.orchestra.run.smithi002.stdout:2361 read 0xc38bc7a thru 0xc39acbe (0xf045 bytes) 2024-09-18T04:56:55.440 INFO:teuthology.orchestra.run.smithi002.stdout:2362 punch from 0x657d7ac to 0x6586d37, (0x958b bytes) 2024-09-18T04:56:55.440 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:55.440 INFO:teuthology.orchestra.run.smithi002.stdout:2367 writesame 0x42447ed thru 0x42536e9 data_size 0x1(0xeefd bytes) 2024-09-18T04:56:55.445 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:56:55.463 INFO:teuthology.orchestra.run.smithi002.stdout:2369 punch from 0x4f8287 to 0x505ec8, (0xdc41 bytes) 2024-09-18T04:56:55.464 INFO:teuthology.orchestra.run.smithi002.stdout:2372 write 0xe89dd9 thru 0xe8b111 (0x1339 bytes) 2024-09-18T04:56:55.466 INFO:teuthology.orchestra.run.smithi002.stdout:2373 write 0x5a5c727 thru 0x5a64626 (0x7f00 bytes) 2024-09-18T04:56:55.496 INFO:teuthology.orchestra.run.smithi002.stdout:2374 trunc from 0xc3b00ca to 0x39961cc 2024-09-18T04:56:55.505 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:55.503+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:55.526 INFO:teuthology.orchestra.run.smithi002.stdout:2375 write 0x66f8c21 thru 0x6704e51 (0xc231 bytes) 2024-09-18T04:56:55.547 INFO:teuthology.orchestra.run.smithi002.stdout:2377 read 0x3a60353 thru 0x3a691cd (0x8e7b bytes) 2024-09-18T04:56:55.547 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:55.558 INFO:teuthology.orchestra.run.smithi002.stdout:2379 write 0xa95de16 thru 0xa95e776 (0x961 bytes) 2024-09-18T04:56:55.567 INFO:teuthology.orchestra.run.smithi002.stdout:2380 write 0x3c9e4f2 thru 0x3ca2a8a (0x4599 bytes) 2024-09-18T04:56:55.570 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:55.571 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:55.571 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:56:55.571 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:56:55.571 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:55.584 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:56:55.590 INFO:teuthology.orchestra.run.smithi002.stdout:2382 trunc from 0xa95e777 to 0x776a331 2024-09-18T04:56:55.613 INFO:teuthology.orchestra.run.smithi002.stdout:2383 write 0xd29ba43 thru 0xd29e978 (0x2f36 bytes) 2024-09-18T04:56:55.628 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:55.626+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:55.630 INFO:teuthology.orchestra.run.smithi002.stdout:2384 clone 21 order 23 su 131072 sc 7 2024-09-18T04:56:55.692 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:55.693 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:56:55.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:56:55.693 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:56:55.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.693 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60344 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1456128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.693 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:56:55.693 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.5774311340000002, 2024-09-18T04:56:55.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044026324999999998, 2024-09-18T04:56:55.693 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.693 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.693 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.693 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.693 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.694 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60344 2024-09-18T04:56:55.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.694 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:56:55.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:56:55.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:56:55.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112224+0000", 2024-09-18T04:56:55.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1320000000000001e-06 2024-09-18T04:56:55.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112230+0000", 2024-09-18T04:56:55.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9619999999999998e-06 2024-09-18T04:56:55.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112234+0000", 2024-09-18T04:56:55.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7890000000000002e-06 2024-09-18T04:56:55.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156105+0000", 2024-09-18T04:56:55.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043870531999999997 2024-09-18T04:56:55.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156170+0000", 2024-09-18T04:56:55.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5036000000000005e-05 2024-09-18T04:56:55.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156248+0000", 2024-09-18T04:56:55.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8295999999999997e-05 2024-09-18T04:56:55.697 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.697 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.697 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.697 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60345 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1471488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.697 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:56:55.698 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.5774143689999995, 2024-09-18T04:56:55.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044180090999999998, 2024-09-18T04:56:55.698 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.698 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.698 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.698 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.698 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.698 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60345 2024-09-18T04:56:55.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.698 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:56:55.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:56:55.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112238+0000", 2024-09-18T04:56:55.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112240+0000", 2024-09-18T04:56:55.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9910000000000001e-06 2024-09-18T04:56:55.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112246+0000", 2024-09-18T04:56:55.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0859999999999998e-06 2024-09-18T04:56:55.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112251+0000", 2024-09-18T04:56:55.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5020000000000003e-06 2024-09-18T04:56:55.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156274+0000", 2024-09-18T04:56:55.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044023063000000001 2024-09-18T04:56:55.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156337+0000", 2024-09-18T04:56:55.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3077000000000006e-05 2024-09-18T04:56:55.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156419+0000", 2024-09-18T04:56:55.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1699000000000005e-05 2024-09-18T04:56:55.701 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.701 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.702 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.702 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60346 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1486848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.702 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:56:55.702 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.577396588, 2024-09-18T04:56:55.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044336331999999999, 2024-09-18T04:56:55.702 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.702 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.702 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.702 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.702 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.702 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60346 2024-09-18T04:56:55.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.703 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:56:55.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:56:55.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112256+0000", 2024-09-18T04:56:55.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112258+0000", 2024-09-18T04:56:55.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1720000000000001e-06 2024-09-18T04:56:55.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112265+0000", 2024-09-18T04:56:55.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4280000000000001e-06 2024-09-18T04:56:55.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112269+0000", 2024-09-18T04:56:55.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.741e-06 2024-09-18T04:56:55.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156445+0000", 2024-09-18T04:56:55.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044175341 2024-09-18T04:56:55.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156512+0000", 2024-09-18T04:56:55.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6723000000000005e-05 2024-09-18T04:56:55.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156593+0000", 2024-09-18T04:56:55.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1284000000000003e-05 2024-09-18T04:56:55.706 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.706 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.706 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.706 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60347 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1502208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.706 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:56:55.706 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.5773681770000003, 2024-09-18T04:56:55.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044482560999999997, 2024-09-18T04:56:55.706 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.706 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.707 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.707 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.707 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.707 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60347 2024-09-18T04:56:55.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.707 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:56:55.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:56:55.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112284+0000", 2024-09-18T04:56:55.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112286+0000", 2024-09-18T04:56:55.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9709999999999998e-06 2024-09-18T04:56:55.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112290+0000", 2024-09-18T04:56:55.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9199999999999997e-06 2024-09-18T04:56:55.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112293+0000", 2024-09-18T04:56:55.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870000000000001e-06 2024-09-18T04:56:55.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156617+0000", 2024-09-18T04:56:55.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044323960000000003 2024-09-18T04:56:55.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156687+0000", 2024-09-18T04:56:55.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0141000000000005e-05 2024-09-18T04:56:55.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156767+0000", 2024-09-18T04:56:55.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0140000000000002e-05 2024-09-18T04:56:55.710 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.710 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.710 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.711 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60348 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1517568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.711 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:56:55.711 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.5773571730000002, 2024-09-18T04:56:55.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044645562, 2024-09-18T04:56:55.711 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.711 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.711 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.711 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.711 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.711 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60348 2024-09-18T04:56:55.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.711 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:56:55.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:56:55.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:56:55.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112297+0000", 2024-09-18T04:56:55.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.384e-06 2024-09-18T04:56:55.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112301+0000", 2024-09-18T04:56:55.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0570000000000003e-06 2024-09-18T04:56:55.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112304+0000", 2024-09-18T04:56:55.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6000000000000001e-06 2024-09-18T04:56:55.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156793+0000", 2024-09-18T04:56:55.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044489494999999997 2024-09-18T04:56:55.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156859+0000", 2024-09-18T04:56:55.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5483999999999998e-05 2024-09-18T04:56:55.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156941+0000", 2024-09-18T04:56:55.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2856999999999996e-05 2024-09-18T04:56:55.715 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.715 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.715 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.715 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60349 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1532928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.715 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:56:55.715 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.5773471969999999, 2024-09-18T04:56:55.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044832167999999999, 2024-09-18T04:56:55.715 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.715 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.715 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.715 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.715 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.716 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60349 2024-09-18T04:56:55.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.716 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:56:55.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:56:55.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:56:55.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112307+0000", 2024-09-18T04:56:55.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.308e-06 2024-09-18T04:56:55.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112310+0000", 2024-09-18T04:56:55.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-18T04:56:55.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112312+0000", 2024-09-18T04:56:55.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4889999999999998e-06 2024-09-18T04:56:55.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156977+0000", 2024-09-18T04:56:55.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044664598999999999 2024-09-18T04:56:55.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157053+0000", 2024-09-18T04:56:55.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6014999999999996e-05 2024-09-18T04:56:55.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157138+0000", 2024-09-18T04:56:55.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5162999999999998e-05 2024-09-18T04:56:55.719 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.719 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.719 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.720 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60350 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1548288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.720 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:56:55.720 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.5773382829999996, 2024-09-18T04:56:55.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045011858000000002, 2024-09-18T04:56:55.720 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.720 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.720 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.720 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.720 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.720 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60350 2024-09-18T04:56:55.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.721 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:56:55.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:56:55.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:56:55.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112316+0000", 2024-09-18T04:56:55.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7579999999999999e-06 2024-09-18T04:56:55.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112321+0000", 2024-09-18T04:56:55.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1649999999999997e-06 2024-09-18T04:56:55.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112325+0000", 2024-09-18T04:56:55.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.726e-06 2024-09-18T04:56:55.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157172+0000", 2024-09-18T04:56:55.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.04484718 2024-09-18T04:56:55.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157242+0000", 2024-09-18T04:56:55.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9675e-05 2024-09-18T04:56:55.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157327+0000", 2024-09-18T04:56:55.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4614e-05 2024-09-18T04:56:55.729 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.729 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.729 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.729 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60351 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1563648~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.729 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:56:55.729 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.5773248669999997, 2024-09-18T04:56:55.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045132113000000001, 2024-09-18T04:56:55.729 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.729 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.729 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.730 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.730 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.730 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60351 2024-09-18T04:56:55.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.730 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:56:55.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:56:55.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:56:55.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112330+0000", 2024-09-18T04:56:55.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1660000000000001e-06 2024-09-18T04:56:55.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112336+0000", 2024-09-18T04:56:55.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5940000000000001e-06 2024-09-18T04:56:55.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112340+0000", 2024-09-18T04:56:55.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5589999999999999e-06 2024-09-18T04:56:55.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157365+0000", 2024-09-18T04:56:55.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045024675 2024-09-18T04:56:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157409+0000", 2024-09-18T04:56:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4356e-05 2024-09-18T04:56:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157460+0000", 2024-09-18T04:56:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1072000000000002e-05 2024-09-18T04:56:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60501 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1494016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:56:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.5712641710000002, 2024-09-18T04:56:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044007728000000003, 2024-09-18T04:56:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60501 2024-09-18T04:56:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:56:55.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:56:55.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:56:55.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118391+0000", 2024-09-18T04:56:55.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:56:55.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118396+0000", 2024-09-18T04:56:55.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2680000000000002e-06 2024-09-18T04:56:55.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118400+0000", 2024-09-18T04:56:55.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2080000000000002e-06 2024-09-18T04:56:55.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162317+0000", 2024-09-18T04:56:55.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043916753000000003 2024-09-18T04:56:55.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162351+0000", 2024-09-18T04:56:55.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4372000000000002e-05 2024-09-18T04:56:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162397+0000", 2024-09-18T04:56:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5207999999999998e-05 2024-09-18T04:56:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60502 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1509376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:56:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.5712486940000003, 2024-09-18T04:56:55.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044092438999999997, 2024-09-18T04:56:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60502 2024-09-18T04:56:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:56:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:56:55.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:56:55.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118406+0000", 2024-09-18T04:56:55.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:56:55.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118412+0000", 2024-09-18T04:56:55.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.524e-06 2024-09-18T04:56:55.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118416+0000", 2024-09-18T04:56:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:56:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162418+0000", 2024-09-18T04:56:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001508000000002 2024-09-18T04:56:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162451+0000", 2024-09-18T04:56:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3662999999999999e-05 2024-09-18T04:56:55.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162497+0000", 2024-09-18T04:56:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5550000000000003e-05 2024-09-18T04:56:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60503 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1524736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:56:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.5712325360000001, 2024-09-18T04:56:55.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044178780000000001, 2024-09-18T04:56:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60503 2024-09-18T04:56:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:56:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:56:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118420+0000", 2024-09-18T04:56:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118422+0000", 2024-09-18T04:56:55.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.08e-06 2024-09-18T04:56:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118428+0000", 2024-09-18T04:56:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.964e-06 2024-09-18T04:56:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118433+0000", 2024-09-18T04:56:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.425e-06 2024-09-18T04:56:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162519+0000", 2024-09-18T04:56:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044086082999999998 2024-09-18T04:56:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162552+0000", 2024-09-18T04:56:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3719e-05 2024-09-18T04:56:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162599+0000", 2024-09-18T04:56:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6873999999999999e-05 2024-09-18T04:56:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.747 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.748 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.748 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60504 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1540096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.748 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:56:55.748 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.5712164309999999, 2024-09-18T04:56:55.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044261921000000003, 2024-09-18T04:56:55.748 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.748 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.748 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.748 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.748 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.748 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60504 2024-09-18T04:56:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:56:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:56:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118436+0000", 2024-09-18T04:56:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118438+0000", 2024-09-18T04:56:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0250000000000001e-06 2024-09-18T04:56:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118444+0000", 2024-09-18T04:56:55.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4360000000000001e-06 2024-09-18T04:56:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118448+0000", 2024-09-18T04:56:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.357e-06 2024-09-18T04:56:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162620+0000", 2024-09-18T04:56:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044171821 2024-09-18T04:56:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162653+0000", 2024-09-18T04:56:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3479999999999998e-05 2024-09-18T04:56:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162699+0000", 2024-09-18T04:56:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5113e-05 2024-09-18T04:56:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.752 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60584 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1196032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:56:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.5694234140000001, 2024-09-18T04:56:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044048727000000003, 2024-09-18T04:56:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60584 2024-09-18T04:56:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:56:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:56:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120229+0000", 2024-09-18T04:56:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120231+0000", 2024-09-18T04:56:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:56:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120237+0000", 2024-09-18T04:56:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8379999999999998e-06 2024-09-18T04:56:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120242+0000", 2024-09-18T04:56:55.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3889999999999998e-06 2024-09-18T04:56:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164187+0000", 2024-09-18T04:56:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043945045000000002 2024-09-18T04:56:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164227+0000", 2024-09-18T04:56:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0627999999999998e-05 2024-09-18T04:56:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164278+0000", 2024-09-18T04:56:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1146000000000001e-05 2024-09-18T04:56:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60585 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1211392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:56:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.5694073060000004, 2024-09-18T04:56:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044389814, 2024-09-18T04:56:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.757 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60585 2024-09-18T04:56:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:56:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:56:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120245+0000", 2024-09-18T04:56:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120247+0000", 2024-09-18T04:56:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0150000000000002e-06 2024-09-18T04:56:55.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120255+0000", 2024-09-18T04:56:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.858e-06 2024-09-18T04:56:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120260+0000", 2024-09-18T04:56:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6530000000000002e-06 2024-09-18T04:56:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164489+0000", 2024-09-18T04:56:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044229483999999999 2024-09-18T04:56:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164555+0000", 2024-09-18T04:56:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.533e-05 2024-09-18T04:56:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164636+0000", 2024-09-18T04:56:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0819e-05 2024-09-18T04:56:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60586 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1226752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:56:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.5693888549999997, 2024-09-18T04:56:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044578214999999997, 2024-09-18T04:56:55.763 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60586 2024-09-18T04:56:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:56:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:56:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:56:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120266+0000", 2024-09-18T04:56:55.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0660000000000002e-06 2024-09-18T04:56:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120272+0000", 2024-09-18T04:56:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-18T04:56:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120276+0000", 2024-09-18T04:56:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.476e-06 2024-09-18T04:56:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164673+0000", 2024-09-18T04:56:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044396672999999998 2024-09-18T04:56:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164749+0000", 2024-09-18T04:56:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5811999999999999e-05 2024-09-18T04:56:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164842+0000", 2024-09-18T04:56:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3560000000000005e-05 2024-09-18T04:56:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.767 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60587 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1242112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:56:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.5693397559999998, 2024-09-18T04:56:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044634794999999998, 2024-09-18T04:56:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60587 2024-09-18T04:56:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:56:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:56:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120280+0000", 2024-09-18T04:56:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999671 2024-09-18T04:56:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120316+0000", 2024-09-18T04:56:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5728000000000001e-05 2024-09-18T04:56:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120329+0000", 2024-09-18T04:56:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2914e-05 2024-09-18T04:56:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120335+0000", 2024-09-18T04:56:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8289999999999996e-06 2024-09-18T04:56:55.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164858+0000", 2024-09-18T04:56:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044523417000000003 2024-09-18T04:56:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164896+0000", 2024-09-18T04:56:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7969999999999997e-05 2024-09-18T04:56:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164948+0000", 2024-09-18T04:56:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1996000000000002e-05 2024-09-18T04:56:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60595 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1364992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:56:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.5674965400000005, 2024-09-18T04:56:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001360000000003, 2024-09-18T04:56:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60595 2024-09-18T04:56:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:56:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:56:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122156+0000", 2024-09-18T04:56:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122158+0000", 2024-09-18T04:56:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0040000000000002e-06 2024-09-18T04:56:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122164+0000", 2024-09-18T04:56:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8719999999999999e-06 2024-09-18T04:56:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122169+0000", 2024-09-18T04:56:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5090000000000004e-06 2024-09-18T04:56:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166040+0000", 2024-09-18T04:56:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043871226999999999 2024-09-18T04:56:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166096+0000", 2024-09-18T04:56:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6495e-05 2024-09-18T04:56:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166158+0000", 2024-09-18T04:56:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1626999999999998e-05 2024-09-18T04:56:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60596 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1380352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:56:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.5674802190000001, 2024-09-18T04:56:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044120800000000002, 2024-09-18T04:56:55.778 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60596 2024-09-18T04:56:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:56:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:56:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:56:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122175+0000", 2024-09-18T04:56:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:56:55.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122182+0000", 2024-09-18T04:56:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3509999999999996e-06 2024-09-18T04:56:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122187+0000", 2024-09-18T04:56:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7940000000000002e-06 2024-09-18T04:56:55.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166184+0000", 2024-09-18T04:56:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043997236000000002 2024-09-18T04:56:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166227+0000", 2024-09-18T04:56:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2561000000000001e-05 2024-09-18T04:56:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166294+0000", 2024-09-18T04:56:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7068000000000002e-05 2024-09-18T04:56:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60597 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1395712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:56:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.5674621909999997, 2024-09-18T04:56:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044242166999999999, 2024-09-18T04:56:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60597 2024-09-18T04:56:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:56:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:56:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:56:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122193+0000", 2024-09-18T04:56:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:56:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122199+0000", 2024-09-18T04:56:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6449999999999999e-06 2024-09-18T04:56:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122204+0000", 2024-09-18T04:56:55.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4819999999999996e-06 2024-09-18T04:56:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166329+0000", 2024-09-18T04:56:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044125059000000001 2024-09-18T04:56:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166370+0000", 2024-09-18T04:56:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1012e-05 2024-09-18T04:56:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166433+0000", 2024-09-18T04:56:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3178999999999999e-05 2024-09-18T04:56:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60608 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1564672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:56:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:56:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.5654643290000001, 2024-09-18T04:56:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044082723999999997, 2024-09-18T04:56:55.788 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60608 2024-09-18T04:56:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:56:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:56:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124188+0000", 2024-09-18T04:56:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124190+0000", 2024-09-18T04:56:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0329999999999998e-06 2024-09-18T04:56:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124196+0000", 2024-09-18T04:56:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.575e-06 2024-09-18T04:56:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124201+0000", 2024-09-18T04:56:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5000000000000001e-06 2024-09-18T04:56:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168166+0000", 2024-09-18T04:56:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043965763999999997 2024-09-18T04:56:55.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168218+0000", 2024-09-18T04:56:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1785e-05 2024-09-18T04:56:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168271+0000", 2024-09-18T04:56:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3429e-05 2024-09-18T04:56:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.792 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:55.793 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:55.793 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:55.794 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:56:55.794 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:55.792+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:55.895 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:55.893+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:55.938 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:55.938 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:55.938 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:56:55.938 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:55.946 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:56:55.996 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:55.993+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:56.048 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:56.048 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:56.048 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:56:56.048 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:56:56.048 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:56.056 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:56:56.097 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:56.095+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:56.198 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:56.195+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:56.199 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:56.199 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:56:56.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:56:56.199 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:56:56.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.199 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26189 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 237568~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:56.199 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:56:56.199 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.556469474, 2024-09-18T04:56:56.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022263431, 2024-09-18T04:56:56.200 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.200 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.200 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.200 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.200 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.200 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26189 2024-09-18T04:56:56.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.200 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:56:56.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:56:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:56:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639787+0000", 2024-09-18T04:56:56.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9759999999999998e-06 2024-09-18T04:56:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639792+0000", 2024-09-18T04:56:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5829999999999997e-06 2024-09-18T04:56:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639796+0000", 2024-09-18T04:56:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0690000000000003e-06 2024-09-18T04:56:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.661944+0000", 2024-09-18T04:56:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022147463999999999 2024-09-18T04:56:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.661985+0000", 2024-09-18T04:56:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1360999999999999e-05 2024-09-18T04:56:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662048+0000", 2024-09-18T04:56:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3298000000000003e-05 2024-09-18T04:56:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.203 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.204 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.204 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26199 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 4096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:56.204 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:56:56.204 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.556454188, 2024-09-18T04:56:56.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022348776000000001, 2024-09-18T04:56:56.204 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.204 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.204 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.204 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.204 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.204 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26199 2024-09-18T04:56:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:56:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:56:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:56:56.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639806+0000", 2024-09-18T04:56:56.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1539999999999999e-06 2024-09-18T04:56:56.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639812+0000", 2024-09-18T04:56:56.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5520000000000004e-06 2024-09-18T04:56:56.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639816+0000", 2024-09-18T04:56:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1840000000000001e-06 2024-09-18T04:56:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662063+0000", 2024-09-18T04:56:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022247125999999999 2024-09-18T04:56:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662098+0000", 2024-09-18T04:56:56.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4968000000000001e-05 2024-09-18T04:56:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662149+0000", 2024-09-18T04:56:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1122000000000003e-05 2024-09-18T04:56:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26200 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 32768~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:56:56.208 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.556434662999999, 2024-09-18T04:56:56.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022501118000000001, 2024-09-18T04:56:56.209 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.209 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.209 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.209 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.209 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.209 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26200 2024-09-18T04:56:56.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.209 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:56:56.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:56:56.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:56:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639822+0000", 2024-09-18T04:56:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.007e-06 2024-09-18T04:56:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639827+0000", 2024-09-18T04:56:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5709999999999997e-06 2024-09-18T04:56:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639831+0000", 2024-09-18T04:56:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2819999999999998e-06 2024-09-18T04:56:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662181+0000", 2024-09-18T04:56:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022349878 2024-09-18T04:56:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662252+0000", 2024-09-18T04:56:56.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0542000000000003e-05 2024-09-18T04:56:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662321+0000", 2024-09-18T04:56:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9167999999999999e-05 2024-09-18T04:56:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26201 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 61440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:56:56.214 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.556419372000001, 2024-09-18T04:56:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022569783, 2024-09-18T04:56:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26201 2024-09-18T04:56:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:56:56.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:56:56.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:56:56.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639837+0000", 2024-09-18T04:56:56.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.928e-06 2024-09-18T04:56:56.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639843+0000", 2024-09-18T04:56:56.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8130000000000001e-06 2024-09-18T04:56:56.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639847+0000", 2024-09-18T04:56:56.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0999999999999997e-06 2024-09-18T04:56:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662335+0000", 2024-09-18T04:56:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022488207999999999 2024-09-18T04:56:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662370+0000", 2024-09-18T04:56:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5292e-05 2024-09-18T04:56:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662405+0000", 2024-09-18T04:56:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4765999999999999e-05 2024-09-18T04:56:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26202 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 90112~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:56:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.556403955, 2024-09-18T04:56:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022655937000000001, 2024-09-18T04:56:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.219 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26202 2024-09-18T04:56:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:56:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:56:56.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639850+0000", 2024-09-18T04:56:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639852+0000", 2024-09-18T04:56:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9700000000000002e-06 2024-09-18T04:56:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639858+0000", 2024-09-18T04:56:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.434e-06 2024-09-18T04:56:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639862+0000", 2024-09-18T04:56:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3499999999999999e-06 2024-09-18T04:56:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662429+0000", 2024-09-18T04:56:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022567409 2024-09-18T04:56:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662469+0000", 2024-09-18T04:56:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9801000000000001e-05 2024-09-18T04:56:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662507+0000", 2024-09-18T04:56:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7311000000000002e-05 2024-09-18T04:56:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.223 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26203 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 118784~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:56:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.556388800000001, 2024-09-18T04:56:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022730358999999999, 2024-09-18T04:56:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26203 2024-09-18T04:56:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:56:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:56:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639865+0000", 2024-09-18T04:56:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639867+0000", 2024-09-18T04:56:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0889999999999998e-06 2024-09-18T04:56:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639873+0000", 2024-09-18T04:56:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4979999999999997e-06 2024-09-18T04:56:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639877+0000", 2024-09-18T04:56:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4510000000000002e-06 2024-09-18T04:56:56.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662528+0000", 2024-09-18T04:56:56.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022650791999999999 2024-09-18T04:56:56.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662563+0000", 2024-09-18T04:56:56.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4530000000000003e-05 2024-09-18T04:56:56.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662596+0000", 2024-09-18T04:56:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3463000000000001e-05 2024-09-18T04:56:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26204 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 147456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:56:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.556373408000001, 2024-09-18T04:56:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022803233999999999, 2024-09-18T04:56:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.228 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.229 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.229 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26204 2024-09-18T04:56:56.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.229 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:56:56.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:56:56.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:56:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639883+0000", 2024-09-18T04:56:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6520000000000002e-06 2024-09-18T04:56:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639888+0000", 2024-09-18T04:56:56.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2870000000000004e-06 2024-09-18T04:56:56.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639893+0000", 2024-09-18T04:56:56.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3839999999999999e-06 2024-09-18T04:56:56.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662617+0000", 2024-09-18T04:56:56.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022723876 2024-09-18T04:56:56.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662651+0000", 2024-09-18T04:56:56.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3797000000000002e-05 2024-09-18T04:56:56.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662684+0000", 2024-09-18T04:56:56.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3828999999999997e-05 2024-09-18T04:56:56.232 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.232 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.232 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.232 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26205 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 176128~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:56:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.556350447, 2024-09-18T04:56:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022919108000000001, 2024-09-18T04:56:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26205 2024-09-18T04:56:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:56:56.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:56:56.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:56:56.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639906+0000", 2024-09-18T04:56:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.531e-06 2024-09-18T04:56:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639913+0000", 2024-09-18T04:56:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.331e-06 2024-09-18T04:56:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639920+0000", 2024-09-18T04:56:56.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5399999999999998e-06 2024-09-18T04:56:56.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662706+0000", 2024-09-18T04:56:56.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.02278587 2024-09-18T04:56:56.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662764+0000", 2024-09-18T04:56:56.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7630000000000002e-05 2024-09-18T04:56:56.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662823+0000", 2024-09-18T04:56:56.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9648000000000002e-05 2024-09-18T04:56:56.237 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.237 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.237 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.237 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26206 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 204800~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:56.237 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:56:56.237 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.556330048, 2024-09-18T04:56:56.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023047517, 2024-09-18T04:56:56.237 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.237 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.238 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.238 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.238 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.238 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26206 2024-09-18T04:56:56.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.238 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:56:56.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:56:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:56:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639926+0000", 2024-09-18T04:56:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3120000000000001e-06 2024-09-18T04:56:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639932+0000", 2024-09-18T04:56:56.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4890000000000003e-06 2024-09-18T04:56:56.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639936+0000", 2024-09-18T04:56:56.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.07e-06 2024-09-18T04:56:56.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662850+0000", 2024-09-18T04:56:56.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022913579999999999 2024-09-18T04:56:56.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662903+0000", 2024-09-18T04:56:56.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3176e-05 2024-09-18T04:56:56.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662972+0000", 2024-09-18T04:56:56.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9270999999999995e-05 2024-09-18T04:56:56.241 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.241 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.241 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.242 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26207 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 233472~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:56.242 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:56:56.242 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.556314904000001, 2024-09-18T04:56:56.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023139541999999999, 2024-09-18T04:56:56.242 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.242 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.242 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.242 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.242 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.242 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26207 2024-09-18T04:56:56.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.243 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:56:56.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:56:56.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639939+0000", 2024-09-18T04:56:56.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639941+0000", 2024-09-18T04:56:56.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0080000000000001e-06 2024-09-18T04:56:56.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639947+0000", 2024-09-18T04:56:56.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.485e-06 2024-09-18T04:56:56.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639951+0000", 2024-09-18T04:56:56.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4549999999999997e-06 2024-09-18T04:56:56.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663005+0000", 2024-09-18T04:56:56.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023053569999999999 2024-09-18T04:56:56.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663042+0000", 2024-09-18T04:56:56.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7245e-05 2024-09-18T04:56:56.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663079+0000", 2024-09-18T04:56:56.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7119000000000002e-05 2024-09-18T04:56:56.246 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.247 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.247 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26208 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 524288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:56.247 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:56:56.247 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.556298156, 2024-09-18T04:56:56.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023283564999999999, 2024-09-18T04:56:56.247 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.248 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.248 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26208 2024-09-18T04:56:56.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.248 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:56:56.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:56:56.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:56:56.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639958+0000", 2024-09-18T04:56:56.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0899999999999999e-06 2024-09-18T04:56:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639963+0000", 2024-09-18T04:56:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2209999999999996e-06 2024-09-18T04:56:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639967+0000", 2024-09-18T04:56:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1169999999999997e-06 2024-09-18T04:56:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663114+0000", 2024-09-18T04:56:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023146102000000002 2024-09-18T04:56:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663183+0000", 2024-09-18T04:56:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9686999999999999e-05 2024-09-18T04:56:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663240+0000", 2024-09-18T04:56:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6728000000000003e-05 2024-09-18T04:56:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.251 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.252 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.252 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26209 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 552960~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:56.252 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:56:56.252 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.55628038, 2024-09-18T04:56:56.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023440966000000001, 2024-09-18T04:56:56.252 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.252 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.252 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.252 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.252 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26209 2024-09-18T04:56:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:56:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:56:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:56:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639976+0000", 2024-09-18T04:56:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4940000000000002e-06 2024-09-18T04:56:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639982+0000", 2024-09-18T04:56:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0560000000000001e-06 2024-09-18T04:56:56.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639987+0000", 2024-09-18T04:56:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-18T04:56:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663278+0000", 2024-09-18T04:56:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023290789999999999 2024-09-18T04:56:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663348+0000", 2024-09-18T04:56:56.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0837e-05 2024-09-18T04:56:56.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663415+0000", 2024-09-18T04:56:56.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6736000000000006e-05 2024-09-18T04:56:56.256 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.256 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.256 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.256 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26210 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 581632~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:56.256 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:56:56.257 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.556263372, 2024-09-18T04:56:56.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023616437000000001, 2024-09-18T04:56:56.257 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.257 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.257 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.257 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.257 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.257 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26210 2024-09-18T04:56:56.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.257 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:56:56.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:56:56.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:56:56.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639993+0000", 2024-09-18T04:56:56.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.419e-06 2024-09-18T04:56:56.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640006+0000", 2024-09-18T04:56:56.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2918000000000001e-05 2024-09-18T04:56:56.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640011+0000", 2024-09-18T04:56:56.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9320000000000004e-06 2024-09-18T04:56:56.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663454+0000", 2024-09-18T04:56:56.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023443035000000001 2024-09-18T04:56:56.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663532+0000", 2024-09-18T04:56:56.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7443999999999999e-05 2024-09-18T04:56:56.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663608+0000", 2024-09-18T04:56:56.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6093000000000005e-05 2024-09-18T04:56:56.261 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.261 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.261 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.261 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26211 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 610304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:56.261 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:56:56.261 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.556238820000001, 2024-09-18T04:56:56.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023774337999999999, 2024-09-18T04:56:56.261 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.261 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.261 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.261 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.261 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.262 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26211 2024-09-18T04:56:56.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.262 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:56:56.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:56:56.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640015+0000", 2024-09-18T04:56:56.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640023+0000", 2024-09-18T04:56:56.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3900000000000004e-06 2024-09-18T04:56:56.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640036+0000", 2024-09-18T04:56:56.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3091999999999999e-05 2024-09-18T04:56:56.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640041+0000", 2024-09-18T04:56:56.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6219999999999997e-06 2024-09-18T04:56:56.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663644+0000", 2024-09-18T04:56:56.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023602643999999999 2024-09-18T04:56:56.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663718+0000", 2024-09-18T04:56:56.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4481999999999996e-05 2024-09-18T04:56:56.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663790+0000", 2024-09-18T04:56:56.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1589999999999997e-05 2024-09-18T04:56:56.265 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.265 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.265 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.266 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26212 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 638976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:56.266 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:56:56.266 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.556205779999999, 2024-09-18T04:56:56.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023924748999999999, 2024-09-18T04:56:56.266 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.266 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.266 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.266 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.266 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.266 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26212 2024-09-18T04:56:56.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.267 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:56:56.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:56:56.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640048+0000", 2024-09-18T04:56:56.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640051+0000", 2024-09-18T04:56:56.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.095e-06 2024-09-18T04:56:56.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640060+0000", 2024-09-18T04:56:56.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8929999999999994e-06 2024-09-18T04:56:56.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640065+0000", 2024-09-18T04:56:56.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0270000000000003e-06 2024-09-18T04:56:56.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663824+0000", 2024-09-18T04:56:56.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023758873999999999 2024-09-18T04:56:56.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663904+0000", 2024-09-18T04:56:56.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9856000000000003e-05 2024-09-18T04:56:56.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663974+0000", 2024-09-18T04:56:56.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9450999999999996e-05 2024-09-18T04:56:56.270 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.270 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.270 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.270 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26213 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 667648~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:56.270 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:56:56.270 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.556184609000001, 2024-09-18T04:56:56.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024059812999999999, 2024-09-18T04:56:56.270 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.270 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.271 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26213 2024-09-18T04:56:56.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.271 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:56:56.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:56:56.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:56:56.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640072+0000", 2024-09-18T04:56:56.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4200000000000001e-06 2024-09-18T04:56:56.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640078+0000", 2024-09-18T04:56:56.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0179999999999998e-06 2024-09-18T04:56:56.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640083+0000", 2024-09-18T04:56:56.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:56:56.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664008+0000", 2024-09-18T04:56:56.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023925287 2024-09-18T04:56:56.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664059+0000", 2024-09-18T04:56:56.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0730999999999999e-05 2024-09-18T04:56:56.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664130+0000", 2024-09-18T04:56:56.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0801000000000002e-05 2024-09-18T04:56:56.274 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.274 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.274 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.275 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26214 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 696320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:56.275 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:56:56.275 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.556167639, 2024-09-18T04:56:56.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024223738000000002, 2024-09-18T04:56:56.275 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.275 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.275 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26214 2024-09-18T04:56:56.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.275 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:56:56.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:56:56.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:56:56.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640089+0000", 2024-09-18T04:56:56.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0849999999999999e-06 2024-09-18T04:56:56.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640095+0000", 2024-09-18T04:56:56.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9279999999999998e-06 2024-09-18T04:56:56.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640099+0000", 2024-09-18T04:56:56.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:56:56.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664166+0000", 2024-09-18T04:56:56.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024066513000000001 2024-09-18T04:56:56.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664239+0000", 2024-09-18T04:56:56.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3201999999999995e-05 2024-09-18T04:56:56.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664311+0000", 2024-09-18T04:56:56.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1434999999999996e-05 2024-09-18T04:56:56.279 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.279 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.279 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.279 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26215 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 724992~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:56.279 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:56:56.279 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.556150878, 2024-09-18T04:56:56.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024302284, 2024-09-18T04:56:56.279 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.279 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.280 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.280 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.280 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.280 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26215 2024-09-18T04:56:56.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.280 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:56:56.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:56:56.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640103+0000", 2024-09-18T04:56:56.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640106+0000", 2024-09-18T04:56:56.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2120000000000002e-06 2024-09-18T04:56:56.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640111+0000", 2024-09-18T04:56:56.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7880000000000003e-06 2024-09-18T04:56:56.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640116+0000", 2024-09-18T04:56:56.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6099999999999999e-06 2024-09-18T04:56:56.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664327+0000", 2024-09-18T04:56:56.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024210941999999999 2024-09-18T04:56:56.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664367+0000", 2024-09-18T04:56:56.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9666000000000003e-05 2024-09-18T04:56:56.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664406+0000", 2024-09-18T04:56:56.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9402e-05 2024-09-18T04:56:56.283 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.283 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.284 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.284 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26216 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 753664~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:56.284 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:56:56.284 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.556134384, 2024-09-18T04:56:56.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024452854999999999, 2024-09-18T04:56:56.284 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.284 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.284 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.284 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.284 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.284 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26216 2024-09-18T04:56:56.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.285 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:56:56.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:56:56.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:56:56.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640122+0000", 2024-09-18T04:56:56.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3599999999999999e-06 2024-09-18T04:56:56.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640128+0000", 2024-09-18T04:56:56.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6339999999999998e-06 2024-09-18T04:56:56.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640132+0000", 2024-09-18T04:56:56.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:56:56.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664436+0000", 2024-09-18T04:56:56.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024303585999999999 2024-09-18T04:56:56.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664507+0000", 2024-09-18T04:56:56.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1046999999999999e-05 2024-09-18T04:56:56.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664573+0000", 2024-09-18T04:56:56.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6217000000000007e-05 2024-09-18T04:56:56.288 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.288 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.288 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.288 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26217 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 782336~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:56:56.288 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:56:56.288 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.55611815, 2024-09-18T04:56:56.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024526287000000001, 2024-09-18T04:56:56.288 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.289 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.289 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.289 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.289 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.289 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26217 2024-09-18T04:56:56.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.289 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:56:56.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:56:56.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:56:56.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640140+0000", 2024-09-18T04:56:56.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2989999999999998e-06 2024-09-18T04:56:56.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640146+0000", 2024-09-18T04:56:56.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9569999999999999e-06 2024-09-18T04:56:56.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640151+0000", 2024-09-18T04:56:56.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:56:56.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664590+0000", 2024-09-18T04:56:56.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0244389 2024-09-18T04:56:56.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664628+0000", 2024-09-18T04:56:56.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7910999999999997e-05 2024-09-18T04:56:56.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664663+0000", 2024-09-18T04:56:56.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5024999999999998e-05 2024-09-18T04:56:56.292 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.292 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.292 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.293 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.293 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.293 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:56.293 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T04:56:56.293 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T04:56:56.293 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-18T04:56:56.294 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T04:56:56.296 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-18T04:56:56.297 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:56:56.337 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:56.335+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:56.341 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone20 from 0xd29e979 (overlap 0xe1a2f8) to 0xb48260 2024-09-18T04:56:56.399 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:56.397+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:56.454 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-09-18T04:56:56.455 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:56.455 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:56.455 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:56:56.455 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:56.463 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:56:56.508 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:56.506+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:56.568 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:56.568 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:56.569 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:56:56.569 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:56:56.569 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:56.577 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:56:56.600 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:56.598+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:56.643 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:56:56.643 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:56.643 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 51, 2024-09-18T04:56:56.643 INFO:teuthology.orchestra.run.smithi002.stdout: "fsid": "c0761d0c-69d5-4b64-b8b9-c7d098b2d06d", 2024-09-18T04:56:56.643 INFO:teuthology.orchestra.run.smithi002.stdout: "created": "2024-09-18T04:55:58.216538+0000", 2024-09-18T04:56:56.643 INFO:teuthology.orchestra.run.smithi002.stdout: "modified": "2024-09-18T04:56:56.255478+0000", 2024-09-18T04:56:56.643 INFO:teuthology.orchestra.run.smithi002.stdout: "last_up_change": "2024-09-18T04:56:08.369924+0000", 2024-09-18T04:56:56.643 INFO:teuthology.orchestra.run.smithi002.stdout: "last_in_change": "2024-09-18T04:56:00.333800+0000", 2024-09-18T04:56:56.643 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T04:56:56.644 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_num": 5801984, 2024-09-18T04:56:56.644 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_set": [ 2024-09-18T04:56:56.644 INFO:teuthology.orchestra.run.smithi002.stdout: "noscrub", 2024-09-18T04:56:56.644 INFO:teuthology.orchestra.run.smithi002.stdout: "pglog_hardlimit", 2024-09-18T04:56:56.644 INFO:teuthology.orchestra.run.smithi002.stdout: "purged_snapdirs", 2024-09-18T04:56:56.644 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_deletes", 2024-09-18T04:56:56.645 INFO:teuthology.orchestra.run.smithi002.stdout: "sortbitwise" 2024-09-18T04:56:56.645 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:56:56.645 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_version": 4, 2024-09-18T04:56:56.645 INFO:teuthology.orchestra.run.smithi002.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T04:56:56.646 INFO:teuthology.orchestra.run.smithi002.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T04:56:56.646 INFO:teuthology.orchestra.run.smithi002.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T04:56:56.646 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_snapshot": "", 2024-09-18T04:56:56.646 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_max": 3, 2024-09-18T04:56:56.646 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd": 8, 2024-09-18T04:56:56.646 INFO:teuthology.orchestra.run.smithi002.stdout: "require_min_compat_client": "luminous", 2024-09-18T04:56:56.646 INFO:teuthology.orchestra.run.smithi002.stdout: "min_compat_client": "luminous", 2024-09-18T04:56:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: "require_osd_release": "reef", 2024-09-18T04:56:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_crimson": false, 2024-09-18T04:56:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: "pools": [ 2024-09-18T04:56:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 1, 2024-09-18T04:56:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "rbd", 2024-09-18T04:56:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-18T04:56:08.983344+0000", 2024-09-18T04:56:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-18T04:56:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T04:56:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-18T04:56:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-18T04:56:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-18T04:56:56.647 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-18T04:56:56.648 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-18T04:56:56.648 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-18T04:56:56.648 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T04:56:56.648 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T04:56:56.648 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-18T04:56:56.648 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-18T04:56:56.648 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 8, 2024-09-18T04:56:56.648 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 8, 2024-09-18T04:56:56.648 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 8, 2024-09-18T04:56:56.648 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 8, 2024-09-18T04:56:56.649 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 8, 2024-09-18T04:56:56.649 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-18T04:56:56.649 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-18T04:56:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-18T04:56:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-18T04:56:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-18T04:56:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-18T04:56:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-18T04:56:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "17", 2024-09-18T04:56:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-18T04:56:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T04:56:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T04:56:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-18T04:56:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-18T04:56:56.650 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 3, 2024-09-18T04:56:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 17, 2024-09-18T04:56:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-18T04:56:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-18T04:56:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-18T04:56:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-18T04:56:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-18T04:56:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-18T04:56:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-18T04:56:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-18T04:56:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-18T04:56:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-18T04:56:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-18T04:56:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T04:56:56.651 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T04:56:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T04:56:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-18T04:56:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-18T04:56:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-18T04:56:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-18T04:56:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-18T04:56:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-18T04:56:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-18T04:56:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-18T04:56:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-18T04:56:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-18T04:56:56.652 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T04:56:56.653 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-18T04:56:56.653 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-18T04:56:56.653 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-18T04:56:56.653 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-18T04:56:56.653 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-18T04:56:56.653 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-18T04:56:56.653 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-18T04:56:56.653 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-18T04:56:56.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.653 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-18T04:56:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 3, 2024-09-18T04:56:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 3, 2024-09-18T04:56:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 1, 2024-09-18T04:56:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 3, 2024-09-18T04:56:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 3, 2024-09-18T04:56:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-18T04:56:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-18T04:56:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-18T04:56:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 2, 2024-09-18T04:56:56.654 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": ".mgr", 2024-09-18T04:56:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-18T04:56:10.313988+0000", 2024-09-18T04:56:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-18T04:56:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-18T04:56:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-18T04:56:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-18T04:56:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-18T04:56:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-18T04:56:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-18T04:56:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-18T04:56:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T04:56:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T04:56:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-18T04:56:56.655 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-18T04:56:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 1, 2024-09-18T04:56:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 1, 2024-09-18T04:56:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 1, 2024-09-18T04:56:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 1, 2024-09-18T04:56:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 1, 2024-09-18T04:56:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-18T04:56:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-18T04:56:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-18T04:56:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-18T04:56:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-18T04:56:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-18T04:56:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-18T04:56:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.656 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "17", 2024-09-18T04:56:56.657 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-18T04:56:56.657 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T04:56:56.657 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T04:56:56.657 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-18T04:56:56.657 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-18T04:56:56.657 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-18T04:56:56.657 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-18T04:56:56.657 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-18T04:56:56.657 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-18T04:56:56.657 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-18T04:56:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-18T04:56:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-18T04:56:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-18T04:56:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-18T04:56:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-18T04:56:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-18T04:56:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-18T04:56:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-18T04:56:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T04:56:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T04:56:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T04:56:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-18T04:56:56.658 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-18T04:56:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-18T04:56:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-18T04:56:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-18T04:56:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-18T04:56:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-18T04:56:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-18T04:56:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-18T04:56:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-18T04:56:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T04:56:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-18T04:56:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-18T04:56:56.659 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-18T04:56:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-18T04:56:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-18T04:56:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-18T04:56:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_max": 32, 2024-09-18T04:56:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 1 2024-09-18T04:56:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-18T04:56:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": {} 2024-09-18T04:56:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-18T04:56:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 8, 2024-09-18T04:56:56.660 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 8, 2024-09-18T04:56:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.25, 2024-09-18T04:56:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 2, 2024-09-18T04:56:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 2, 2024-09-18T04:56:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-18T04:56:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-18T04:56:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-18T04:56:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-18T04:56:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "pool_client.0", 2024-09-18T04:56:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-18T04:56:19.562579+0000", 2024-09-18T04:56:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-18T04:56:56.661 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T04:56:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-18T04:56:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-18T04:56:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-18T04:56:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-18T04:56:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-18T04:56:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-18T04:56:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T04:56:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T04:56:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-18T04:56:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-18T04:56:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 32, 2024-09-18T04:56:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 32, 2024-09-18T04:56:56.662 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 32, 2024-09-18T04:56:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 32, 2024-09-18T04:56:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 32, 2024-09-18T04:56:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-18T04:56:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-18T04:56:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-18T04:56:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-18T04:56:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-18T04:56:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-18T04:56:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-18T04:56:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "51", 2024-09-18T04:56:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-18T04:56:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T04:56:56.663 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T04:56:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-18T04:56:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-18T04:56:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 24, 2024-09-18T04:56:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 51, 2024-09-18T04:56:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-18T04:56:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-18T04:56:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-18T04:56:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-18T04:56:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-18T04:56:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-18T04:56:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-18T04:56:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-18T04:56:56.664 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-18T04:56:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-18T04:56:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-18T04:56:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T04:56:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T04:56:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T04:56:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-18T04:56:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-18T04:56:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-18T04:56:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-18T04:56:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-18T04:56:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-18T04:56:56.665 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-18T04:56:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-18T04:56:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-18T04:56:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-18T04:56:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T04:56:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-18T04:56:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-18T04:56:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-18T04:56:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-18T04:56:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-18T04:56:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-18T04:56:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-18T04:56:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-18T04:56:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.666 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-18T04:56:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 1.5, 2024-09-18T04:56:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 1.5, 2024-09-18T04:56:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 1, 2024-09-18T04:56:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 1.5, 2024-09-18T04:56:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 1.5, 2024-09-18T04:56:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 1, 2024-09-18T04:56:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-18T04:56:56.667 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 1 2024-09-18T04:56:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:56:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-18T04:56:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-18T04:56:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "c43f8024-3a3e-424e-aa54-398310ceb70d", 2024-09-18T04:56:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:56.668 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 19, 2024-09-18T04:56:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6824", 2024-09-18T04:56:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6825", 2024-09-18T04:56:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6826", 2024-09-18T04:56:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6827", 2024-09-18T04:56:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6830", 2024-09-18T04:56:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6831", 2024-09-18T04:56:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6828", 2024-09-18T04:56:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:56.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6829", 2024-09-18T04:56:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:56:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6825/4106704527", 2024-09-18T04:56:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6827/4106704527", 2024-09-18T04:56:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6831/4106704527", 2024-09-18T04:56:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6829/4106704527", 2024-09-18T04:56:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:56.673 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-18T04:56:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "c664b970-145c-4b6f-8852-2f6dcf1fa30d", 2024-09-18T04:56:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:56.674 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 19, 2024-09-18T04:56:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6808", 2024-09-18T04:56:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:56.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6809", 2024-09-18T04:56:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6810", 2024-09-18T04:56:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:56.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6811", 2024-09-18T04:56:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6814", 2024-09-18T04:56:56.677 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6815", 2024-09-18T04:56:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6812", 2024-09-18T04:56:56.678 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6813", 2024-09-18T04:56:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:56:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6809/1781959259", 2024-09-18T04:56:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6811/1781959259", 2024-09-18T04:56:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6815/1781959259", 2024-09-18T04:56:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6813/1781959259", 2024-09-18T04:56:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:56.679 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.680 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-18T04:56:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "0609ea8c-d975-490e-8844-04a3e4512efd", 2024-09-18T04:56:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 0, 2024-09-18T04:56:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:56.681 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6800", 2024-09-18T04:56:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6801", 2024-09-18T04:56:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.682 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6802", 2024-09-18T04:56:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6803", 2024-09-18T04:56:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.683 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6806", 2024-09-18T04:56:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6807", 2024-09-18T04:56:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:56.684 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6804", 2024-09-18T04:56:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6805", 2024-09-18T04:56:56.685 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:56:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6801/1153570802", 2024-09-18T04:56:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6803/1153570802", 2024-09-18T04:56:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6807/1153570802", 2024-09-18T04:56:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6805/1153570802", 2024-09-18T04:56:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.686 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-18T04:56:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "c6bd5ac8-ab89-4fa2-b843-3b166d1b238b", 2024-09-18T04:56:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 48, 2024-09-18T04:56:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:56.687 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6816", 2024-09-18T04:56:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6817", 2024-09-18T04:56:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6818", 2024-09-18T04:56:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6819", 2024-09-18T04:56:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6822", 2024-09-18T04:56:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6823", 2024-09-18T04:56:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6820", 2024-09-18T04:56:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6821", 2024-09-18T04:56:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:56:56.691 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6817/2008321375", 2024-09-18T04:56:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6819/2008321375", 2024-09-18T04:56:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6823/2008321375", 2024-09-18T04:56:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6821/2008321375", 2024-09-18T04:56:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-18T04:56:56.692 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "89d6a371-4f4e-465e-a023-d5083cce6e6c", 2024-09-18T04:56:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 19, 2024-09-18T04:56:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6824", 2024-09-18T04:56:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6825", 2024-09-18T04:56:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6826", 2024-09-18T04:56:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6827", 2024-09-18T04:56:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6830", 2024-09-18T04:56:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6831", 2024-09-18T04:56:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:56.696 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6828", 2024-09-18T04:56:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6829", 2024-09-18T04:56:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:56:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6825/2939896598", 2024-09-18T04:56:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6827/2939896598", 2024-09-18T04:56:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6831/2939896598", 2024-09-18T04:56:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6829/2939896598", 2024-09-18T04:56:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-18T04:56:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "da99686b-d979-4753-b4aa-a3b26e9a0497", 2024-09-18T04:56:56.698 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 19, 2024-09-18T04:56:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6816", 2024-09-18T04:56:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6817", 2024-09-18T04:56:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6818", 2024-09-18T04:56:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6819", 2024-09-18T04:56:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:56.701 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6822", 2024-09-18T04:56:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6823", 2024-09-18T04:56:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:56.702 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6820", 2024-09-18T04:56:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6821", 2024-09-18T04:56:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:56:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6817/3440879618", 2024-09-18T04:56:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6819/3440879618", 2024-09-18T04:56:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6823/3440879618", 2024-09-18T04:56:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6821/3440879618", 2024-09-18T04:56:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-18T04:56:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "ceec6aa6-603d-4706-bfea-3c07dbdaed3a", 2024-09-18T04:56:56.704 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 41, 2024-09-18T04:56:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.705 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6800", 2024-09-18T04:56:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6801", 2024-09-18T04:56:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.706 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6802", 2024-09-18T04:56:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6803", 2024-09-18T04:56:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6806", 2024-09-18T04:56:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6807", 2024-09-18T04:56:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6804", 2024-09-18T04:56:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6805", 2024-09-18T04:56:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:56:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6801/2493830504", 2024-09-18T04:56:56.709 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6803/2493830504", 2024-09-18T04:56:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6807/2493830504", 2024-09-18T04:56:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6805/2493830504", 2024-09-18T04:56:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.710 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-18T04:56:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "6779dfad-f60b-41f4-b666-a99db6dc1814", 2024-09-18T04:56:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:56:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:56:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:56:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:56:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:56:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:56:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:56:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 49, 2024-09-18T04:56:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:56:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:56:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:56:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6808", 2024-09-18T04:56:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6809", 2024-09-18T04:56:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:56.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:56:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6810", 2024-09-18T04:56:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.714 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6811", 2024-09-18T04:56:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:56:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6814", 2024-09-18T04:56:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.715 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6815", 2024-09-18T04:56:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:56:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:56:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:56:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6812", 2024-09-18T04:56:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:56:56.716 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6813", 2024-09-18T04:56:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:56:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6809/91038858", 2024-09-18T04:56:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6811/91038858", 2024-09-18T04:56:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6815/91038858", 2024-09-18T04:56:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6813/91038858", 2024-09-18T04:56:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:56:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:56:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:56:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:56:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_xinfo": [ 2024-09-18T04:56:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-18T04:56:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.468213+0000", 2024-09-18T04:56:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.718 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-18T04:56:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.538610+0000", 2024-09-18T04:56:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-18T04:56:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:56.719 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.500881+0000", 2024-09-18T04:56:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-18T04:56:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.474179+0000", 2024-09-18T04:56:56.720 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-18T04:56:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.483737+0000", 2024-09-18T04:56:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-18T04:56:56.721 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.460693+0000", 2024-09-18T04:56:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-18T04:56:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:56.722 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.475970+0000", 2024-09-18T04:56:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-18T04:56:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:56:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:56:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:56:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:56:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:56:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.519287+0000", 2024-09-18T04:56:56.723 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:56:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:56:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap": [ 2024-09-18T04:56:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "1.4", 2024-09-18T04:56:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-18T04:56:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: 6, 2024-09-18T04:56:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: 7 2024-09-18T04:56:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:56:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_items": [], 2024-09-18T04:56:56.724 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_primaries": [], 2024-09-18T04:56:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_temp": [], 2024-09-18T04:56:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_temp": [], 2024-09-18T04:56:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: "blocklist": {}, 2024-09-18T04:56:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: "range_blocklist": {}, 2024-09-18T04:56:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profiles": { 2024-09-18T04:56:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: "default": { 2024-09-18T04:56:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-failure-domain": "osd", 2024-09-18T04:56:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: "k": "2", 2024-09-18T04:56:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: "m": "1", 2024-09-18T04:56:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: "plugin": "jerasure", 2024-09-18T04:56:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: "technique": "reed_sol_van" 2024-09-18T04:56:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps_queue": [], 2024-09-18T04:56:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "new_removed_snaps": [], 2024-09-18T04:56:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "new_purged_snaps": [], 2024-09-18T04:56:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_node_flags": {}, 2024-09-18T04:56:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "device_class_flags": {}, 2024-09-18T04:56:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode": { 2024-09-18T04:56:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_enabled": false, 2024-09-18T04:56:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_bucket_count": 0, 2024-09-18T04:56:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "degraded_stretch_mode": 0, 2024-09-18T04:56:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "recovering_stretch_mode": 0, 2024-09-18T04:56:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_bucket": 0 2024-09-18T04:56:56.726 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:56.726 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:56.728 INFO:tasks.thrashosds.thrasher:j is {'epoch': 51, 'fsid': 'c0761d0c-69d5-4b64-b8b9-c7d098b2d06d', 'created': '2024-09-18T04:55:58.216538+0000', 'modified': '2024-09-18T04:56:56.255478+0000', 'last_up_change': '2024-09-18T04:56:08.369924+0000', 'last_in_change': '2024-09-18T04:56:00.333800+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-18T04:56:08.983344+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 3, 'score_stable': 3, 'optimal_score': 1, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-18T04:56:10.313988+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T04:56:19.562579+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '51', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 24, 'snap_epoch': 51, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': 'c43f8024-3a3e-424e-aa54-398310ceb70d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6824', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6825', 'nonce': 4106704527}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6826', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6827', 'nonce': 4106704527}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6830', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6831', 'nonce': 4106704527}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6828', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6829', 'nonce': 4106704527}]}, 'public_addr': '172.21.15.2:6825/4106704527', 'cluster_addr': '172.21.15.2:6827/4106704527', 'heartbeat_back_addr': '172.21.15.2:6831/4106704527', 'heartbeat_front_addr': '172.21.15.2:6829/4106704527', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c664b970-145c-4b6f-8852-2f6dcf1fa30d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6808', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6809', 'nonce': 1781959259}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6810', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6811', 'nonce': 1781959259}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6814', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6815', 'nonce': 1781959259}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6812', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6813', 'nonce': 1781959259}]}, 'public_addr': '172.21.15.2:6809/1781959259', 'cluster_addr': '172.21.15.2:6811/1781959259', 'heartbeat_back_addr': '172.21.15.2:6815/1781959259', 'heartbeat_front_addr': '172.21.15.2:6813/1781959259', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0609ea8c-d975-490e-8844-04a3e4512efd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6800', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6801', 'nonce': 1153570802}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6802', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6803', 'nonce': 1153570802}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6806', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6807', 'nonce': 1153570802}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6804', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6805', 'nonce': 1153570802}]}, 'public_addr': '172.21.15.2:6801/1153570802', 'cluster_addr': '172.21.15.2:6803/1153570802', 'heartbeat_back_addr': '172.21.15.2:6807/1153570802', 'heartbeat_front_addr': '172.21.15.2:6805/1153570802', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'c6bd5ac8-ab89-4fa2-b843-3b166d1b238b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6816', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6817', 'nonce': 2008321375}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6818', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6819', 'nonce': 2008321375}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6822', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6823', 'nonce': 2008321375}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6820', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6821', 'nonce': 2008321375}]}, 'public_addr': '172.21.15.2:6817/2008321375', 'cluster_addr': '172.21.15.2:6819/2008321375', 'heartbeat_back_addr': '172.21.15.2:6823/2008321375', 'heartbeat_front_addr': '172.21.15.2:6821/2008321375', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '89d6a371-4f4e-465e-a023-d5083cce6e6c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6824', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6825', 'nonce': 2939896598}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6826', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6827', 'nonce': 2939896598}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6830', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6831', 'nonce': 2939896598}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6828', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6829', 'nonce': 2939896598}]}, 'public_addr': '172.21.15.99:6825/2939896598', 'cluster_addr': '172.21.15.99:6827/2939896598', 'heartbeat_back_addr': '172.21.15.99:6831/2939896598', 'heartbeat_front_addr': '172.21.15.99:6829/2939896598', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'da99686b-d979-4753-b4aa-a3b26e9a0497', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6816', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6817', 'nonce': 3440879618}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6818', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6819', 'nonce': 3440879618}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6822', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6823', 'nonce': 3440879618}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6820', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6821', 'nonce': 3440879618}]}, 'public_addr': '172.21.15.99:6817/3440879618', 'cluster_addr': '172.21.15.99:6819/3440879618', 'heartbeat_back_addr': '172.21.15.99:6823/3440879618', 'heartbeat_front_addr': '172.21.15.99:6821/3440879618', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ceec6aa6-603d-4706-bfea-3c07dbdaed3a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 41, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6800', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6801', 'nonce': 2493830504}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6802', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6803', 'nonce': 2493830504}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6806', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6807', 'nonce': 2493830504}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6804', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6805', 'nonce': 2493830504}]}, 'public_addr': '172.21.15.99:6801/2493830504', 'cluster_addr': '172.21.15.99:6803/2493830504', 'heartbeat_back_addr': '172.21.15.99:6807/2493830504', 'heartbeat_front_addr': '172.21.15.99:6805/2493830504', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '6779dfad-f60b-41f4-b666-a99db6dc1814', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 49, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6808', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6809', 'nonce': 91038858}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6810', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6811', 'nonce': 91038858}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6814', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6815', 'nonce': 91038858}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6812', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6813', 'nonce': 91038858}]}, 'public_addr': '172.21.15.99:6809/91038858', 'cluster_addr': '172.21.15.99:6811/91038858', 'heartbeat_back_addr': '172.21.15.99:6815/91038858', 'heartbeat_front_addr': '172.21.15.99:6813/91038858', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.468213+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.538610+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.500881+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.474179+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.483737+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.460693+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.475970+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.519287+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '1.4', 'osds': [6, 7]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T04:56:56.729 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T04:56:56.729 INFO:teuthology.orchestra.run.smithi002.stdout:2385 writesame 0x7dbd140 thru 0x7dc0bc3 data_size 0xa(0x3a84 bytes) 2024-09-18T04:56:56.729 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:56.729 INFO:teuthology.orchestra.run.smithi002.stdout:2388 write 0x6a48a8 thru 0x6ab4dd (0x6c36 bytes) 2024-09-18T04:56:56.730 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:56.725+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:56.740 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:56.740 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:56:56.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:56:56.740 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:56:56.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.740 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:56:56.740 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:56:56.741 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.344116342, 2024-09-18T04:56:56.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037718511000000003, 2024-09-18T04:56:56.741 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.741 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:56:56.741 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.741 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:56:56.741 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:56:56.741 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-18T04:56:56.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.741 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:56:56.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:56:56.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393125+0000", 2024-09-18T04:56:56.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:56.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393133+0000", 2024-09-18T04:56:56.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.225e-06 2024-09-18T04:56:56.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393144+0000", 2024-09-18T04:56:56.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1279e-05 2024-09-18T04:56:56.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393153+0000", 2024-09-18T04:56:56.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4540000000000007e-06 2024-09-18T04:56:56.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393203+0000", 2024-09-18T04:56:56.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9963000000000003e-05 2024-09-18T04:56:56.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:56:56.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393210+0000", 2024-09-18T04:56:56.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5689999999999999e-06 2024-09-18T04:56:56.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427399+0000", 2024-09-18T04:56:56.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034188784999999999 2024-09-18T04:56:56.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427532+0000", 2024-09-18T04:56:56.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013264700000000001 2024-09-18T04:56:56.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:56:56.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427685+0000", 2024-09-18T04:56:56.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015300599999999999 2024-09-18T04:56:56.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:56:56.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.428520+0000", 2024-09-18T04:56:56.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00083502499999999998 2024-09-18T04:56:56.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:56:56.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430772+0000", 2024-09-18T04:56:56.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022518880000000001 2024-09-18T04:56:56.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:56:56.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430817+0000", 2024-09-18T04:56:56.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5077000000000002e-05 2024-09-18T04:56:56.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430845+0000", 2024-09-18T04:56:56.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8036999999999999e-05 2024-09-18T04:56:56.748 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.748 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.748 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.748 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54065 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3453952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:56:56.748 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221852+0000", 2024-09-18T04:56:56.749 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.5153909849999998, 2024-09-18T04:56:56.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029151789000000001, 2024-09-18T04:56:56.749 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.749 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.749 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54065 2024-09-18T04:56:56.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.750 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221852+0000", 2024-09-18T04:56:56.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221852+0000", 2024-09-18T04:56:56.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221851+0000", 2024-09-18T04:56:56.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221853+0000", 2024-09-18T04:56:56.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9570000000000001e-06 2024-09-18T04:56:56.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221859+0000", 2024-09-18T04:56:56.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0410000000000003e-06 2024-09-18T04:56:56.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221870+0000", 2024-09-18T04:56:56.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0801e-05 2024-09-18T04:56:56.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.250810+0000", 2024-09-18T04:56:56.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.028940006000000001 2024-09-18T04:56:56.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.250882+0000", 2024-09-18T04:56:56.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1737999999999997e-05 2024-09-18T04:56:56.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251003+0000", 2024-09-18T04:56:56.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121605 2024-09-18T04:56:56.753 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.753 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.753 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.753 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54066 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3469312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:56:56.753 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221875+0000", 2024-09-18T04:56:56.754 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.5153678060000004, 2024-09-18T04:56:56.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029331864999999999, 2024-09-18T04:56:56.754 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.754 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.754 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54066 2024-09-18T04:56:56.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.754 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221875+0000", 2024-09-18T04:56:56.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221875+0000", 2024-09-18T04:56:56.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221874+0000", 2024-09-18T04:56:56.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221877+0000", 2024-09-18T04:56:56.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1280000000000002e-06 2024-09-18T04:56:56.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221883+0000", 2024-09-18T04:56:56.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7109999999999998e-06 2024-09-18T04:56:56.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221888+0000", 2024-09-18T04:56:56.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.668e-06 2024-09-18T04:56:56.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251030+0000", 2024-09-18T04:56:56.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029142208999999999 2024-09-18T04:56:56.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251138+0000", 2024-09-18T04:56:56.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107518 2024-09-18T04:56:56.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251207+0000", 2024-09-18T04:56:56.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8978000000000003e-05 2024-09-18T04:56:56.758 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.758 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.758 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.758 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54067 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3484672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:56:56.758 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221893+0000", 2024-09-18T04:56:56.758 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.5153496610000001, 2024-09-18T04:56:56.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029508817999999999, 2024-09-18T04:56:56.759 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.759 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.759 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.759 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.759 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.759 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54067 2024-09-18T04:56:56.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.759 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221893+0000", 2024-09-18T04:56:56.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221893+0000", 2024-09-18T04:56:56.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221893+0000", 2024-09-18T04:56:56.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221895+0000", 2024-09-18T04:56:56.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9709999999999998e-06 2024-09-18T04:56:56.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221902+0000", 2024-09-18T04:56:56.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4379999999999998e-06 2024-09-18T04:56:56.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221907+0000", 2024-09-18T04:56:56.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0019999999999997e-06 2024-09-18T04:56:56.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251260+0000", 2024-09-18T04:56:56.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029353338999999999 2024-09-18T04:56:56.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251336+0000", 2024-09-18T04:56:56.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5591000000000002e-05 2024-09-18T04:56:56.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251402+0000", 2024-09-18T04:56:56.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5752000000000003e-05 2024-09-18T04:56:56.763 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.763 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.763 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.763 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54068 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3500032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:56:56.763 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221912+0000", 2024-09-18T04:56:56.763 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.5153309720000001, 2024-09-18T04:56:56.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029654553, 2024-09-18T04:56:56.763 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.763 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.764 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.764 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.764 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54068 2024-09-18T04:56:56.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.764 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221912+0000", 2024-09-18T04:56:56.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221912+0000", 2024-09-18T04:56:56.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221911+0000", 2024-09-18T04:56:56.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221916+0000", 2024-09-18T04:56:56.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0130000000000001e-06 2024-09-18T04:56:56.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221922+0000", 2024-09-18T04:56:56.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.105e-06 2024-09-18T04:56:56.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221928+0000", 2024-09-18T04:56:56.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0430000000000004e-06 2024-09-18T04:56:56.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251427+0000", 2024-09-18T04:56:56.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029498586 2024-09-18T04:56:56.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251495+0000", 2024-09-18T04:56:56.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8125000000000003e-05 2024-09-18T04:56:56.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251566+0000", 2024-09-18T04:56:56.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1145999999999999e-05 2024-09-18T04:56:56.767 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.767 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.767 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.768 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54069 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3515392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:56:56.768 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221945+0000", 2024-09-18T04:56:56.768 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.5152973630000002, 2024-09-18T04:56:56.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029786185999999999, 2024-09-18T04:56:56.768 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.768 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.768 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54069 2024-09-18T04:56:56.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.769 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221945+0000", 2024-09-18T04:56:56.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221945+0000", 2024-09-18T04:56:56.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221945+0000", 2024-09-18T04:56:56.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221947+0000", 2024-09-18T04:56:56.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6189999999999998e-06 2024-09-18T04:56:56.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221955+0000", 2024-09-18T04:56:56.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6249999999999998e-06 2024-09-18T04:56:56.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221968+0000", 2024-09-18T04:56:56.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2846e-05 2024-09-18T04:56:56.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251595+0000", 2024-09-18T04:56:56.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029627158000000001 2024-09-18T04:56:56.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251666+0000", 2024-09-18T04:56:56.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1437000000000001e-05 2024-09-18T04:56:56.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251731+0000", 2024-09-18T04:56:56.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5071000000000001e-05 2024-09-18T04:56:56.772 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.772 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.772 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.772 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54070 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:56:56.772 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221973+0000", 2024-09-18T04:56:56.772 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.5152693800000003, 2024-09-18T04:56:56.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029891219, 2024-09-18T04:56:56.773 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.773 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.773 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.773 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.773 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.773 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54070 2024-09-18T04:56:56.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.773 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221973+0000", 2024-09-18T04:56:56.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221973+0000", 2024-09-18T04:56:56.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221973+0000", 2024-09-18T04:56:56.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221985+0000", 2024-09-18T04:56:56.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1888999999999999e-05 2024-09-18T04:56:56.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221992+0000", 2024-09-18T04:56:56.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2830000000000004e-06 2024-09-18T04:56:56.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222008+0000", 2024-09-18T04:56:56.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5608999999999999e-05 2024-09-18T04:56:56.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251757+0000", 2024-09-18T04:56:56.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029749652000000001 2024-09-18T04:56:56.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251826+0000", 2024-09-18T04:56:56.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9103000000000006e-05 2024-09-18T04:56:56.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251864+0000", 2024-09-18T04:56:56.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8179000000000002e-05 2024-09-18T04:56:56.777 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.777 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.777 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.777 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54071 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3546112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:56:56.777 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222014+0000", 2024-09-18T04:56:56.777 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.5152289489999999, 2024-09-18T04:56:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03002749, 2024-09-18T04:56:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54071 2024-09-18T04:56:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222014+0000", 2024-09-18T04:56:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222014+0000", 2024-09-18T04:56:56.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222013+0000", 2024-09-18T04:56:56.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222016+0000", 2024-09-18T04:56:56.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6639999999999998e-06 2024-09-18T04:56:56.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222023+0000", 2024-09-18T04:56:56.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.849e-06 2024-09-18T04:56:56.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222028+0000", 2024-09-18T04:56:56.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0030000000000002e-06 2024-09-18T04:56:56.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251895+0000", 2024-09-18T04:56:56.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029867482000000001 2024-09-18T04:56:56.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251963+0000", 2024-09-18T04:56:56.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8038000000000001e-05 2024-09-18T04:56:56.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252041+0000", 2024-09-18T04:56:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8028999999999994e-05 2024-09-18T04:56:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54072 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3561472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:56:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222034+0000", 2024-09-18T04:56:56.782 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.5152088790000002, 2024-09-18T04:56:56.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030190214, 2024-09-18T04:56:56.783 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.783 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.783 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54072 2024-09-18T04:56:56.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.783 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222034+0000", 2024-09-18T04:56:56.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222034+0000", 2024-09-18T04:56:56.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222033+0000", 2024-09-18T04:56:56.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222035+0000", 2024-09-18T04:56:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0490000000000002e-06 2024-09-18T04:56:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222051+0000", 2024-09-18T04:56:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5914e-05 2024-09-18T04:56:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222057+0000", 2024-09-18T04:56:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4269999999999999e-06 2024-09-18T04:56:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252079+0000", 2024-09-18T04:56:56.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030022338999999999 2024-09-18T04:56:56.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252157+0000", 2024-09-18T04:56:56.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7459999999999994e-05 2024-09-18T04:56:56.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252224+0000", 2024-09-18T04:56:56.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7415000000000004e-05 2024-09-18T04:56:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54073 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3576832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:56:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222062+0000", 2024-09-18T04:56:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.5151811019999997, 2024-09-18T04:56:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030280892, 2024-09-18T04:56:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.787 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.788 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.788 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54073 2024-09-18T04:56:56.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.788 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222062+0000", 2024-09-18T04:56:56.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222062+0000", 2024-09-18T04:56:56.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222061+0000", 2024-09-18T04:56:56.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222063+0000", 2024-09-18T04:56:56.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0729999999999999e-06 2024-09-18T04:56:56.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222070+0000", 2024-09-18T04:56:56.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3199999999999996e-06 2024-09-18T04:56:56.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222074+0000", 2024-09-18T04:56:56.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8659999999999997e-06 2024-09-18T04:56:56.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252255+0000", 2024-09-18T04:56:56.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030180350000000002 2024-09-18T04:56:56.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252305+0000", 2024-09-18T04:56:56.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0086000000000001e-05 2024-09-18T04:56:56.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252342+0000", 2024-09-18T04:56:56.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7594e-05 2024-09-18T04:56:56.791 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.791 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.791 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54074 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3592192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:56:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222079+0000", 2024-09-18T04:56:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.5151637109999996, 2024-09-18T04:56:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030370844000000001, 2024-09-18T04:56:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54074 2024-09-18T04:56:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222079+0000", 2024-09-18T04:56:56.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222079+0000", 2024-09-18T04:56:56.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222079+0000", 2024-09-18T04:56:56.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222081+0000", 2024-09-18T04:56:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0820000000000001e-06 2024-09-18T04:56:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222087+0000", 2024-09-18T04:56:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0789999999999997e-06 2024-09-18T04:56:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222097+0000", 2024-09-18T04:56:56.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0446e-05 2024-09-18T04:56:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252365+0000", 2024-09-18T04:56:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030267544 2024-09-18T04:56:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252402+0000", 2024-09-18T04:56:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7117999999999999e-05 2024-09-18T04:56:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252450+0000", 2024-09-18T04:56:56.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7930999999999999e-05 2024-09-18T04:56:56.796 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.796 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.796 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.796 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54075 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3607552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:56:56.796 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222103+0000", 2024-09-18T04:56:56.796 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.5151396589999999, 2024-09-18T04:56:56.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030531175000000001, 2024-09-18T04:56:56.796 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.796 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.797 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.797 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.797 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.797 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54075 2024-09-18T04:56:56.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.797 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222103+0000", 2024-09-18T04:56:56.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222103+0000", 2024-09-18T04:56:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222101+0000", 2024-09-18T04:56:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:56:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222105+0000", 2024-09-18T04:56:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4089999999999999e-06 2024-09-18T04:56:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222111+0000", 2024-09-18T04:56:56.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3720000000000001e-06 2024-09-18T04:56:56.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222116+0000", 2024-09-18T04:56:56.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.544e-06 2024-09-18T04:56:56.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252475+0000", 2024-09-18T04:56:56.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030358975999999999 2024-09-18T04:56:56.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252562+0000", 2024-09-18T04:56:56.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7257999999999997e-05 2024-09-18T04:56:56.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252634+0000", 2024-09-18T04:56:56.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2144999999999995e-05 2024-09-18T04:56:56.800 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.800 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.800 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54076 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3622912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:56:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222121+0000", 2024-09-18T04:56:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.5151217680000002, 2024-09-18T04:56:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030699046000000001, 2024-09-18T04:56:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54076 2024-09-18T04:56:56.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222121+0000", 2024-09-18T04:56:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222121+0000", 2024-09-18T04:56:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222120+0000", 2024-09-18T04:56:56.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222123+0000", 2024-09-18T04:56:56.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2199999999999999e-06 2024-09-18T04:56:56.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222129+0000", 2024-09-18T04:56:56.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5250000000000002e-06 2024-09-18T04:56:56.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222134+0000", 2024-09-18T04:56:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8180000000000003e-06 2024-09-18T04:56:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252677+0000", 2024-09-18T04:56:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030542969999999999 2024-09-18T04:56:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252750+0000", 2024-09-18T04:56:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2601000000000005e-05 2024-09-18T04:56:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252820+0000", 2024-09-18T04:56:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0409999999999998e-05 2024-09-18T04:56:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54077 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3638272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:56:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222138+0000", 2024-09-18T04:56:56.805 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.515104268, 2024-09-18T04:56:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031008865, 2024-09-18T04:56:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54077 2024-09-18T04:56:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222138+0000", 2024-09-18T04:56:56.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222138+0000", 2024-09-18T04:56:56.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222138+0000", 2024-09-18T04:56:56.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222140+0000", 2024-09-18T04:56:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9700000000000002e-06 2024-09-18T04:56:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222146+0000", 2024-09-18T04:56:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1460000000000001e-06 2024-09-18T04:56:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222151+0000", 2024-09-18T04:56:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.763e-06 2024-09-18T04:56:56.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252989+0000", 2024-09-18T04:56:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030838580000000001 2024-09-18T04:56:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253072+0000", 2024-09-18T04:56:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2204000000000001e-05 2024-09-18T04:56:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253147+0000", 2024-09-18T04:56:56.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5661000000000006e-05 2024-09-18T04:56:56.810 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.810 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.810 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.810 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54078 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:56:56.810 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222155+0000", 2024-09-18T04:56:56.810 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.5150876819999999, 2024-09-18T04:56:56.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031175118000000002, 2024-09-18T04:56:56.810 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.810 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54078 2024-09-18T04:56:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222155+0000", 2024-09-18T04:56:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222155+0000", 2024-09-18T04:56:56.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222155+0000", 2024-09-18T04:56:56.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222157+0000", 2024-09-18T04:56:56.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0660000000000002e-06 2024-09-18T04:56:56.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222163+0000", 2024-09-18T04:56:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2820000000000002e-06 2024-09-18T04:56:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222168+0000", 2024-09-18T04:56:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7979999999999996e-06 2024-09-18T04:56:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253174+0000", 2024-09-18T04:56:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031005721999999999 2024-09-18T04:56:56.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253251+0000", 2024-09-18T04:56:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7717000000000001e-05 2024-09-18T04:56:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253330+0000", 2024-09-18T04:56:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8858999999999998e-05 2024-09-18T04:56:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54079 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3668992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:56:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222172+0000", 2024-09-18T04:56:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.5150705320000002, 2024-09-18T04:56:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031327661, 2024-09-18T04:56:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54079 2024-09-18T04:56:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222172+0000", 2024-09-18T04:56:56.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222172+0000", 2024-09-18T04:56:56.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222172+0000", 2024-09-18T04:56:56.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222174+0000", 2024-09-18T04:56:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0159999999999998e-06 2024-09-18T04:56:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222180+0000", 2024-09-18T04:56:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9950000000000002e-06 2024-09-18T04:56:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222185+0000", 2024-09-18T04:56:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7679999999999999e-06 2024-09-18T04:56:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253360+0000", 2024-09-18T04:56:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031175423000000001 2024-09-18T04:56:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253430+0000", 2024-09-18T04:56:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9929000000000001e-05 2024-09-18T04:56:56.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253500+0000", 2024-09-18T04:56:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9896999999999998e-05 2024-09-18T04:56:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54332 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3623936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:56:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.229403+0000", 2024-09-18T04:56:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.5078398489999998, 2024-09-18T04:56:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029249332999999999, 2024-09-18T04:56:56.819 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54332 2024-09-18T04:56:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229403+0000", 2024-09-18T04:56:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229403+0000", 2024-09-18T04:56:56.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229402+0000", 2024-09-18T04:56:56.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229405+0000", 2024-09-18T04:56:56.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0150000000000002e-06 2024-09-18T04:56:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229412+0000", 2024-09-18T04:56:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6260000000000003e-06 2024-09-18T04:56:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229417+0000", 2024-09-18T04:56:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6199999999999998e-06 2024-09-18T04:56:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258523+0000", 2024-09-18T04:56:56.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029106396999999999 2024-09-18T04:56:56.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258589+0000", 2024-09-18T04:56:56.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6054000000000001e-05 2024-09-18T04:56:56.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258652+0000", 2024-09-18T04:56:56.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2917999999999996e-05 2024-09-18T04:56:56.823 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.823 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54333 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3639296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:56:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.229446+0000", 2024-09-18T04:56:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.5077968510000002, 2024-09-18T04:56:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029358565, 2024-09-18T04:56:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54333 2024-09-18T04:56:56.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.825 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229446+0000", 2024-09-18T04:56:56.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229446+0000", 2024-09-18T04:56:56.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229445+0000", 2024-09-18T04:56:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229448+0000", 2024-09-18T04:56:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8200000000000001e-06 2024-09-18T04:56:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229456+0000", 2024-09-18T04:56:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.371e-06 2024-09-18T04:56:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229473+0000", 2024-09-18T04:56:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6595e-05 2024-09-18T04:56:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258677+0000", 2024-09-18T04:56:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029203863 2024-09-18T04:56:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258742+0000", 2024-09-18T04:56:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5273999999999999e-05 2024-09-18T04:56:56.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258804+0000", 2024-09-18T04:56:56.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2176999999999998e-05 2024-09-18T04:56:56.828 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.828 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.828 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.828 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54334 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3654656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:56:56.828 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.229481+0000", 2024-09-18T04:56:56.828 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.5077615699999996, 2024-09-18T04:56:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029474845999999999, 2024-09-18T04:56:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:56:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54334 2024-09-18T04:56:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229481+0000", 2024-09-18T04:56:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229481+0000", 2024-09-18T04:56:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229481+0000", 2024-09-18T04:56:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229483+0000", 2024-09-18T04:56:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7580000000000001e-06 2024-09-18T04:56:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229490+0000", 2024-09-18T04:56:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7020000000000004e-06 2024-09-18T04:56:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229495+0000", 2024-09-18T04:56:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7770000000000002e-06 2024-09-18T04:56:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258830+0000", 2024-09-18T04:56:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029334828 2024-09-18T04:56:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:56:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258896+0000", 2024-09-18T04:56:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6384999999999995e-05 2024-09-18T04:56:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258956+0000", 2024-09-18T04:56:56.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9889999999999997e-05 2024-09-18T04:56:56.833 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.833 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.833 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.833 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:62520 3.13 3.84cdb893 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T04:56:56.833 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:56:56.833 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.4398064589999997, 2024-09-18T04:56:56.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94452134799999998, 2024-09-18T04:56:56.833 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:56:56.833 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:56:56.833 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:56:56.833 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:56:56.834 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:56.834 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 62520 2024-09-18T04:56:56.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.834 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:56:56.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:56:56.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:56:56.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:56:56.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:56:56.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:56:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:56:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:56:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:56:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297441+0000", 2024-09-18T04:56:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9840000000000001e-06 2024-09-18T04:56:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:56:56.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297449+0000", 2024-09-18T04:56:56.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3999999999999992e-06 2024-09-18T04:56:56.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:56:56.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297455+0000", 2024-09-18T04:56:56.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3350000000000003e-06 2024-09-18T04:56:56.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297552+0000", 2024-09-18T04:56:56.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6602999999999996e-05 2024-09-18T04:56:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:56:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297559+0000", 2024-09-18T04:56:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.52e-06 2024-09-18T04:56:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:56:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241943+0000", 2024-09-18T04:56:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94438357100000003 2024-09-18T04:56:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:56:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:56:56.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:56:56.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241958+0000", 2024-09-18T04:56:56.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4598e-05 2024-09-18T04:56:56.838 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.838 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.838 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.838 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:56:56.838 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:56:56.838 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:56.838 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:56:56.841 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:56:56.842 INFO:teuthology.orchestra.run.smithi002.stdout:2389 read 0x91cc4b8 thru 0x91d2e7c (0x69c5 bytes) 2024-09-18T04:56:56.842 INFO:teuthology.orchestra.run.smithi002.stdout:2390 read 0xdc3704 thru 0xdc8093 (0x4990 bytes) 2024-09-18T04:56:56.843 INFO:teuthology.orchestra.run.smithi002.stdout:2393 read 0x2521dca thru 0x252494d (0x2b84 bytes) 2024-09-18T04:56:56.843 INFO:teuthology.orchestra.run.smithi002.stdout:2395 punch from 0x94fb6fb to 0x9504e7b, (0x9780 bytes) 2024-09-18T04:56:56.843 INFO:teuthology.orchestra.run.smithi002.stdout:2396 read 0xe43cf1 thru 0xe48ff5 (0x5305 bytes) 2024-09-18T04:56:56.843 INFO:teuthology.orchestra.run.smithi002.stdout:2397 read 0x66fe097 thru 0x670d6ca (0xf634 bytes) 2024-09-18T04:56:56.843 INFO:teuthology.orchestra.run.smithi002.stdout:2399 punch from 0x7b960d8 to 0x7b9618e, (0xb6 bytes) 2024-09-18T04:56:56.843 INFO:teuthology.orchestra.run.smithi002.stdout:2401 punch from 0xb1731bc to 0xb17ce96, (0x9cda bytes) 2024-09-18T04:56:56.843 INFO:teuthology.orchestra.run.smithi002.stdout:2402 punch from 0x82b9011 to 0x82bcab5, (0x3aa4 bytes) 2024-09-18T04:56:56.843 INFO:teuthology.orchestra.run.smithi002.stdout:2403 writesame 0xe4e864f thru 0xe4eb594 data_size 0x1(0x2f46 bytes) 2024-09-18T04:56:56.843 INFO:teuthology.orchestra.run.smithi002.stdout:2404 write 0x80e7cf4 thru 0x80ec3f6 (0x4703 bytes) 2024-09-18T04:56:56.843 INFO:teuthology.orchestra.run.smithi002.stdout:2405 punch from 0x8af77b5 to 0x8afa3aa, (0x2bf5 bytes) 2024-09-18T04:56:56.843 INFO:teuthology.orchestra.run.smithi002.stdout:2406 trunc from 0xe4eb595 to 0xbc57419 2024-09-18T04:56:56.843 INFO:teuthology.orchestra.run.smithi002.stdout:2408 write 0x8008a54 thru 0x80109b1 (0x7f5e bytes) 2024-09-18T04:56:56.843 INFO:teuthology.orchestra.run.smithi002.stdout:2409 trunc from 0xbc57419 to 0xccd6b0f 2024-09-18T04:56:56.843 INFO:teuthology.orchestra.run.smithi002.stdout:2410 write 0x9039761 thru 0x9042470 (0x8d10 bytes) 2024-09-18T04:56:56.844 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:56.844 INFO:teuthology.orchestra.run.smithi002.stdout:2412 writesame 0x2f0c27a thru 0x2f18b36 data_size 0x3b(0xc8bd bytes) 2024-09-18T04:56:56.844 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:56.837+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:56.920 INFO:teuthology.orchestra.run.smithi002.stdout:2413 read 0x7e543d thru 0x7ea5e8 (0x51ac bytes) 2024-09-18T04:56:56.922 INFO:teuthology.orchestra.run.smithi002.stdout:2414 read 0x5450e13 thru 0x545538c (0x457a bytes) 2024-09-18T04:56:56.922 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:56.922 INFO:teuthology.orchestra.run.smithi002.stdout:2420 writesame 0x315f682 thru 0x316225a data_size 0x5(0x2bd9 bytes) 2024-09-18T04:56:56.945 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:56.939+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:57.007 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:56:57.008 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:56:57.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.008 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72341 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.008 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969431+0000", 2024-09-18T04:56:57.008 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024947871, 2024-09-18T04:56:57.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024973986, 2024-09-18T04:56:57.008 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.008 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.008 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.008 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.008 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.009 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72341 2024-09-18T04:56:57.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.009 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969431+0000", 2024-09-18T04:56:57.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969431+0000", 2024-09-18T04:56:57.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969431+0000", 2024-09-18T04:56:57.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969433+0000", 2024-09-18T04:56:57.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.278e-06 2024-09-18T04:56:57.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969439+0000", 2024-09-18T04:56:57.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0499999999999997e-06 2024-09-18T04:56:57.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969444+0000", 2024-09-18T04:56:57.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6670000000000004e-06 2024-09-18T04:56:57.011 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.011 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.011 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.012 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72504 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.012 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.970043+0000", 2024-09-18T04:56:57.012 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024336113999999999, 2024-09-18T04:56:57.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024428819000000001, 2024-09-18T04:56:57.012 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.012 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.012 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.012 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.012 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.012 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72504 2024-09-18T04:56:57.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.012 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970043+0000", 2024-09-18T04:56:57.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970043+0000", 2024-09-18T04:56:57.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970043+0000", 2024-09-18T04:56:57.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970045+0000", 2024-09-18T04:56:57.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5519999999999999e-06 2024-09-18T04:56:57.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970051+0000", 2024-09-18T04:56:57.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0499999999999997e-06 2024-09-18T04:56:57.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970056+0000", 2024-09-18T04:56:57.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6120000000000001e-06 2024-09-18T04:56:57.015 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.015 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.015 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.015 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72679 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.015 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.973429+0000", 2024-09-18T04:56:57.015 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.020949997000000001, 2024-09-18T04:56:57.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021103986000000002, 2024-09-18T04:56:57.016 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.016 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.016 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.016 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.016 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.016 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72679 2024-09-18T04:56:57.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.016 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973429+0000", 2024-09-18T04:56:57.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973429+0000", 2024-09-18T04:56:57.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973429+0000", 2024-09-18T04:56:57.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973431+0000", 2024-09-18T04:56:57.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5390000000000002e-06 2024-09-18T04:56:57.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973438+0000", 2024-09-18T04:56:57.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5209999999999999e-06 2024-09-18T04:56:57.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973442+0000", 2024-09-18T04:56:57.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.515e-06 2024-09-18T04:56:57.019 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.019 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.019 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.019 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72842 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.019 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.980643+0000", 2024-09-18T04:56:57.019 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.013735874, 2024-09-18T04:56:57.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013946391000000001, 2024-09-18T04:56:57.019 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.019 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.019 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.019 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.019 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.020 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72842 2024-09-18T04:56:57.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.020 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.980643+0000", 2024-09-18T04:56:57.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.980643+0000", 2024-09-18T04:56:57.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.980643+0000", 2024-09-18T04:56:57.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.980684+0000", 2024-09-18T04:56:57.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.108e-05 2024-09-18T04:56:57.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.980790+0000", 2024-09-18T04:56:57.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106395 2024-09-18T04:56:57.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.980797+0000", 2024-09-18T04:56:57.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7789999999999998e-06 2024-09-18T04:56:57.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.023 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.024 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72342 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.024 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969448+0000", 2024-09-18T04:56:57.024 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024930904, 2024-09-18T04:56:57.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025193384999999999, 2024-09-18T04:56:57.024 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.024 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.024 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72342 2024-09-18T04:56:57.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.025 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969448+0000", 2024-09-18T04:56:57.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969448+0000", 2024-09-18T04:56:57.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969448+0000", 2024-09-18T04:56:57.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969450+0000", 2024-09-18T04:56:57.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3290000000000001e-06 2024-09-18T04:56:57.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969456+0000", 2024-09-18T04:56:57.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.536e-06 2024-09-18T04:56:57.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969461+0000", 2024-09-18T04:56:57.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7859999999999996e-06 2024-09-18T04:56:57.027 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.028 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.028 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.028 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72505 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.028 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.970060+0000", 2024-09-18T04:56:57.028 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.02431939, 2024-09-18T04:56:57.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024636793000000001, 2024-09-18T04:56:57.028 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.028 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.028 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.028 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.028 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.029 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72505 2024-09-18T04:56:57.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.029 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970060+0000", 2024-09-18T04:56:57.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970060+0000", 2024-09-18T04:56:57.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970060+0000", 2024-09-18T04:56:57.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970062+0000", 2024-09-18T04:56:57.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4660000000000002e-06 2024-09-18T04:56:57.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970076+0000", 2024-09-18T04:56:57.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.418e-05 2024-09-18T04:56:57.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970082+0000", 2024-09-18T04:56:57.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5090000000000001e-06 2024-09-18T04:56:57.031 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.031 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.031 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.031 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72681 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.031 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.973446+0000", 2024-09-18T04:56:57.031 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.02093331, 2024-09-18T04:56:57.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021305542, 2024-09-18T04:56:57.032 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.032 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.032 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.032 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.032 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.032 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72681 2024-09-18T04:56:57.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.032 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973446+0000", 2024-09-18T04:56:57.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973446+0000", 2024-09-18T04:56:57.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973446+0000", 2024-09-18T04:56:57.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973448+0000", 2024-09-18T04:56:57.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0990000000000001e-06 2024-09-18T04:56:57.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973453+0000", 2024-09-18T04:56:57.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2769999999999996e-06 2024-09-18T04:56:57.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973457+0000", 2024-09-18T04:56:57.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8619999999999996e-06 2024-09-18T04:56:57.034 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.034 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.034 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.035 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72844 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.035 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.980804+0000", 2024-09-18T04:56:57.035 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.013574866999999999, 2024-09-18T04:56:57.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014009259, 2024-09-18T04:56:57.035 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.035 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.035 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.035 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.035 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.035 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72844 2024-09-18T04:56:57.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.035 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.980804+0000", 2024-09-18T04:56:57.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.980804+0000", 2024-09-18T04:56:57.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.980804+0000", 2024-09-18T04:56:57.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.980808+0000", 2024-09-18T04:56:57.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9419999999999997e-06 2024-09-18T04:56:57.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.980814+0000", 2024-09-18T04:56:57.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6109999999999999e-06 2024-09-18T04:56:57.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.980940+0000", 2024-09-18T04:56:57.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000126107 2024-09-18T04:56:57.038 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.038 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.038 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.038 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72343 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.038 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969465+0000", 2024-09-18T04:56:57.038 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024914209, 2024-09-18T04:56:57.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025404258999999998, 2024-09-18T04:56:57.038 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.038 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.038 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.039 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.039 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.039 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72343 2024-09-18T04:56:57.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.039 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969465+0000", 2024-09-18T04:56:57.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969465+0000", 2024-09-18T04:56:57.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969465+0000", 2024-09-18T04:56:57.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969467+0000", 2024-09-18T04:56:57.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4889999999999998e-06 2024-09-18T04:56:57.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969473+0000", 2024-09-18T04:56:57.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9009999999999999e-06 2024-09-18T04:56:57.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969478+0000", 2024-09-18T04:56:57.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5410000000000002e-06 2024-09-18T04:56:57.041 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.041 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.041 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.041 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72506 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.042 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.970088+0000", 2024-09-18T04:56:57.042 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024290998000000001, 2024-09-18T04:56:57.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024837264000000001, 2024-09-18T04:56:57.042 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.042 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.042 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.042 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.042 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.042 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72506 2024-09-18T04:56:57.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.042 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970088+0000", 2024-09-18T04:56:57.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970088+0000", 2024-09-18T04:56:57.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970088+0000", 2024-09-18T04:56:57.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970090+0000", 2024-09-18T04:56:57.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5600000000000001e-06 2024-09-18T04:56:57.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970097+0000", 2024-09-18T04:56:57.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.584e-06 2024-09-18T04:56:57.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970102+0000", 2024-09-18T04:56:57.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0150000000000003e-06 2024-09-18T04:56:57.045 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.045 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.045 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.045 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72683 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.045 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.973461+0000", 2024-09-18T04:56:57.045 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.020918642000000001, 2024-09-18T04:56:57.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021506115999999999, 2024-09-18T04:56:57.045 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.045 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.045 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.045 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.045 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.046 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72683 2024-09-18T04:56:57.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.046 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973461+0000", 2024-09-18T04:56:57.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973461+0000", 2024-09-18T04:56:57.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973460+0000", 2024-09-18T04:56:57.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973462+0000", 2024-09-18T04:56:57.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0600000000000002e-06 2024-09-18T04:56:57.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973470+0000", 2024-09-18T04:56:57.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1110000000000002e-06 2024-09-18T04:56:57.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973474+0000", 2024-09-18T04:56:57.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5990000000000004e-06 2024-09-18T04:56:57.048 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.048 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.048 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.048 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72846 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.048 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.980993+0000", 2024-09-18T04:56:57.049 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.013386415, 2024-09-18T04:56:57.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014002702000000001, 2024-09-18T04:56:57.049 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.049 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.049 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.049 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.049 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.049 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72846 2024-09-18T04:56:57.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.049 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.980993+0000", 2024-09-18T04:56:57.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.980993+0000", 2024-09-18T04:56:57.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.980992+0000", 2024-09-18T04:56:57.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.980996+0000", 2024-09-18T04:56:57.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8650000000000003e-06 2024-09-18T04:56:57.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981004+0000", 2024-09-18T04:56:57.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7489999999999998e-06 2024-09-18T04:56:57.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981068+0000", 2024-09-18T04:56:57.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3856999999999995e-05 2024-09-18T04:56:57.052 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.052 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.052 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.052 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72374 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.052 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969482+0000", 2024-09-18T04:56:57.052 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024897236999999999, 2024-09-18T04:56:57.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025541350000000001, 2024-09-18T04:56:57.052 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.053 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.053 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.053 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.053 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.053 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72374 2024-09-18T04:56:57.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.053 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969482+0000", 2024-09-18T04:56:57.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969482+0000", 2024-09-18T04:56:57.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969482+0000", 2024-09-18T04:56:57.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969484+0000", 2024-09-18T04:56:57.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.34e-06 2024-09-18T04:56:57.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969490+0000", 2024-09-18T04:56:57.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9970000000000004e-06 2024-09-18T04:56:57.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969495+0000", 2024-09-18T04:56:57.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6070000000000001e-06 2024-09-18T04:56:57.055 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.055 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.055 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.056 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72507 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.056 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.970106+0000", 2024-09-18T04:56:57.056 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024272961999999999, 2024-09-18T04:56:57.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.024957428, 2024-09-18T04:56:57.056 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.056 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.056 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72507 2024-09-18T04:56:57.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.056 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970106+0000", 2024-09-18T04:56:57.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970106+0000", 2024-09-18T04:56:57.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970106+0000", 2024-09-18T04:56:57.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970108+0000", 2024-09-18T04:56:57.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1069999999999999e-06 2024-09-18T04:56:57.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970114+0000", 2024-09-18T04:56:57.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4720000000000003e-06 2024-09-18T04:56:57.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970119+0000", 2024-09-18T04:56:57.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9080000000000003e-06 2024-09-18T04:56:57.059 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.059 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.059 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.059 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72685 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.059 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.973478+0000", 2024-09-18T04:56:57.059 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.020901141000000002, 2024-09-18T04:56:57.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021622927, 2024-09-18T04:56:57.059 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.059 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.060 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.060 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.060 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.060 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72685 2024-09-18T04:56:57.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.060 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973478+0000", 2024-09-18T04:56:57.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973478+0000", 2024-09-18T04:56:57.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973478+0000", 2024-09-18T04:56:57.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973480+0000", 2024-09-18T04:56:57.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.61e-06 2024-09-18T04:56:57.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973488+0000", 2024-09-18T04:56:57.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5159999999999997e-06 2024-09-18T04:56:57.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973493+0000", 2024-09-18T04:56:57.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6949999999999999e-06 2024-09-18T04:56:57.062 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.062 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.062 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.062 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72848 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.063 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.981075+0000", 2024-09-18T04:56:57.063 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.013303927, 2024-09-18T04:56:57.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014076470000000001, 2024-09-18T04:56:57.063 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.063 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.063 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.063 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.063 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.063 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72848 2024-09-18T04:56:57.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.063 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981075+0000", 2024-09-18T04:56:57.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981075+0000", 2024-09-18T04:56:57.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981075+0000", 2024-09-18T04:56:57.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981079+0000", 2024-09-18T04:56:57.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.721e-06 2024-09-18T04:56:57.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981087+0000", 2024-09-18T04:56:57.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0649999999999999e-06 2024-09-18T04:56:57.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981092+0000", 2024-09-18T04:56:57.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8340000000000004e-06 2024-09-18T04:56:57.065 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.066 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.066 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.066 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72376 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.066 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969499+0000", 2024-09-18T04:56:57.066 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024880254000000001, 2024-09-18T04:56:57.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025701983000000001, 2024-09-18T04:56:57.066 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.066 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.066 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.066 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.066 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.066 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72376 2024-09-18T04:56:57.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.067 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969499+0000", 2024-09-18T04:56:57.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969499+0000", 2024-09-18T04:56:57.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969499+0000", 2024-09-18T04:56:57.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969501+0000", 2024-09-18T04:56:57.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4650000000000001e-06 2024-09-18T04:56:57.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969507+0000", 2024-09-18T04:56:57.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1840000000000004e-06 2024-09-18T04:56:57.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969512+0000", 2024-09-18T04:56:57.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0989999999999998e-06 2024-09-18T04:56:57.069 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.069 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.069 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.069 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72508 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.069 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.970123+0000", 2024-09-18T04:56:57.069 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024256389999999999, 2024-09-18T04:56:57.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025122648000000001, 2024-09-18T04:56:57.069 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.070 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.070 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.070 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.070 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.070 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72508 2024-09-18T04:56:57.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.070 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970123+0000", 2024-09-18T04:56:57.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970123+0000", 2024-09-18T04:56:57.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970123+0000", 2024-09-18T04:56:57.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970125+0000", 2024-09-18T04:56:57.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3659999999999999e-06 2024-09-18T04:56:57.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970131+0000", 2024-09-18T04:56:57.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1059999999999996e-06 2024-09-18T04:56:57.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970136+0000", 2024-09-18T04:56:57.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0889999999999999e-06 2024-09-18T04:56:57.072 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.072 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.072 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.073 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72687 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.073 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.973497+0000", 2024-09-18T04:56:57.073 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.020882402000000001, 2024-09-18T04:56:57.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021797858, 2024-09-18T04:56:57.073 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.073 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.073 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.073 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.073 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.073 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72687 2024-09-18T04:56:57.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.073 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973497+0000", 2024-09-18T04:56:57.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973497+0000", 2024-09-18T04:56:57.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973497+0000", 2024-09-18T04:56:57.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973499+0000", 2024-09-18T04:56:57.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3240000000000001e-06 2024-09-18T04:56:57.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973507+0000", 2024-09-18T04:56:57.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7950000000000008e-06 2024-09-18T04:56:57.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973512+0000", 2024-09-18T04:56:57.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.147e-06 2024-09-18T04:56:57.076 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.076 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.076 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.077 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72850 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.077 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.981097+0000", 2024-09-18T04:56:57.077 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.013282225, 2024-09-18T04:56:57.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014243476, 2024-09-18T04:56:57.077 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.077 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.077 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.077 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.077 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.077 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72850 2024-09-18T04:56:57.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.077 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981097+0000", 2024-09-18T04:56:57.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981097+0000", 2024-09-18T04:56:57.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981097+0000", 2024-09-18T04:56:57.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981099+0000", 2024-09-18T04:56:57.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4370000000000001e-06 2024-09-18T04:56:57.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981105+0000", 2024-09-18T04:56:57.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6720000000000001e-06 2024-09-18T04:56:57.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981110+0000", 2024-09-18T04:56:57.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3850000000000001e-06 2024-09-18T04:56:57.080 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.080 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.080 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.080 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72378 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.080 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969517+0000", 2024-09-18T04:56:57.080 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024862355999999999, 2024-09-18T04:56:57.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025871600000000002, 2024-09-18T04:56:57.080 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.081 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.081 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.081 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.081 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.081 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72378 2024-09-18T04:56:57.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.081 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969517+0000", 2024-09-18T04:56:57.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969517+0000", 2024-09-18T04:56:57.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969517+0000", 2024-09-18T04:56:57.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969524+0000", 2024-09-18T04:56:57.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2189999999999998e-06 2024-09-18T04:56:57.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969530+0000", 2024-09-18T04:56:57.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1319999999999999e-06 2024-09-18T04:56:57.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969535+0000", 2024-09-18T04:56:57.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1669999999999998e-06 2024-09-18T04:56:57.083 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.083 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.083 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.084 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72509 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.084 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.970142+0000", 2024-09-18T04:56:57.084 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024237407999999998, 2024-09-18T04:56:57.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025306016000000001, 2024-09-18T04:56:57.084 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.084 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.084 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.084 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.084 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.084 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72509 2024-09-18T04:56:57.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.084 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970142+0000", 2024-09-18T04:56:57.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970142+0000", 2024-09-18T04:56:57.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970142+0000", 2024-09-18T04:56:57.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970144+0000", 2024-09-18T04:56:57.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1679999999999998e-06 2024-09-18T04:56:57.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970150+0000", 2024-09-18T04:56:57.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1410000000000002e-06 2024-09-18T04:56:57.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970155+0000", 2024-09-18T04:56:57.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7190000000000001e-06 2024-09-18T04:56:57.087 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.087 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.087 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.087 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72689 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.087 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.973516+0000", 2024-09-18T04:56:57.087 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.020863112999999999, 2024-09-18T04:56:57.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021989603999999999, 2024-09-18T04:56:57.087 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.087 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.087 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.088 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.088 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.088 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72689 2024-09-18T04:56:57.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.088 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.088 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.088 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973516+0000", 2024-09-18T04:56:57.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.088 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.088 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973516+0000", 2024-09-18T04:56:57.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973516+0000", 2024-09-18T04:56:57.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973519+0000", 2024-09-18T04:56:57.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.757e-06 2024-09-18T04:56:57.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973526+0000", 2024-09-18T04:56:57.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3719999999999999e-06 2024-09-18T04:56:57.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973533+0000", 2024-09-18T04:56:57.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7580000000000003e-06 2024-09-18T04:56:57.090 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.090 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.090 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.090 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72851 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.091 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.981115+0000", 2024-09-18T04:56:57.091 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.013264576, 2024-09-18T04:56:57.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014448859, 2024-09-18T04:56:57.091 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.091 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.091 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.091 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.091 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.091 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72851 2024-09-18T04:56:57.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.091 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981115+0000", 2024-09-18T04:56:57.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981115+0000", 2024-09-18T04:56:57.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981114+0000", 2024-09-18T04:56:57.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981117+0000", 2024-09-18T04:56:57.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3690000000000001e-06 2024-09-18T04:56:57.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981335+0000", 2024-09-18T04:56:57.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021812599999999999 2024-09-18T04:56:57.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.093 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.093 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981342+0000", 2024-09-18T04:56:57.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8460000000000002e-06 2024-09-18T04:56:57.093 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.093 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.094 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.094 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72380 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.094 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969540+0000", 2024-09-18T04:56:57.094 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024839097000000001, 2024-09-18T04:56:57.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026080827000000001, 2024-09-18T04:56:57.094 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.094 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.094 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.094 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.094 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.094 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72380 2024-09-18T04:56:57.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.095 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969540+0000", 2024-09-18T04:56:57.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969540+0000", 2024-09-18T04:56:57.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969540+0000", 2024-09-18T04:56:57.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969543+0000", 2024-09-18T04:56:57.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7609999999999999e-06 2024-09-18T04:56:57.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969549+0000", 2024-09-18T04:56:57.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3040000000000001e-06 2024-09-18T04:56:57.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969554+0000", 2024-09-18T04:56:57.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.237e-06 2024-09-18T04:56:57.097 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.097 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.097 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.097 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72542 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.097 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.970160+0000", 2024-09-18T04:56:57.097 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024219434000000001, 2024-09-18T04:56:57.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025517142999999999, 2024-09-18T04:56:57.097 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.097 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.098 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.098 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.098 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.098 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72542 2024-09-18T04:56:57.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.098 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970160+0000", 2024-09-18T04:56:57.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970160+0000", 2024-09-18T04:56:57.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970159+0000", 2024-09-18T04:56:57.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970162+0000", 2024-09-18T04:56:57.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2469999999999998e-06 2024-09-18T04:56:57.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970168+0000", 2024-09-18T04:56:57.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2369999999999997e-06 2024-09-18T04:56:57.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970173+0000", 2024-09-18T04:56:57.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5519999999999998e-06 2024-09-18T04:56:57.100 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.100 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.100 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.101 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72715 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.101 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.974411+0000", 2024-09-18T04:56:57.101 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.019968452000000001, 2024-09-18T04:56:57.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021322105000000001, 2024-09-18T04:56:57.101 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.101 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.101 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.101 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.101 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.101 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72715 2024-09-18T04:56:57.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.101 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974411+0000", 2024-09-18T04:56:57.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974411+0000", 2024-09-18T04:56:57.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974410+0000", 2024-09-18T04:56:57.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:57.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974415+0000", 2024-09-18T04:56:57.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0479999999999998e-06 2024-09-18T04:56:57.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974426+0000", 2024-09-18T04:56:57.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1124000000000001e-05 2024-09-18T04:56:57.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974434+0000", 2024-09-18T04:56:57.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4780000000000002e-06 2024-09-18T04:56:57.104 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.104 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.104 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.104 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72853 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.104 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.981349+0000", 2024-09-18T04:56:57.104 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.01303026, 2024-09-18T04:56:57.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014450876, 2024-09-18T04:56:57.104 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.104 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.104 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.104 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.104 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.105 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72853 2024-09-18T04:56:57.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.105 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981349+0000", 2024-09-18T04:56:57.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981349+0000", 2024-09-18T04:56:57.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981348+0000", 2024-09-18T04:56:57.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981352+0000", 2024-09-18T04:56:57.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9759999999999997e-06 2024-09-18T04:56:57.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981423+0000", 2024-09-18T04:56:57.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0889999999999994e-05 2024-09-18T04:56:57.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981430+0000", 2024-09-18T04:56:57.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4130000000000003e-06 2024-09-18T04:56:57.107 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.107 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.107 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.107 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72382 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.107 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969559+0000", 2024-09-18T04:56:57.107 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024820796999999999, 2024-09-18T04:56:57.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026300121999999999, 2024-09-18T04:56:57.107 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.108 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.108 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.108 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.108 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.108 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72382 2024-09-18T04:56:57.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.108 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969559+0000", 2024-09-18T04:56:57.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969559+0000", 2024-09-18T04:56:57.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969558+0000", 2024-09-18T04:56:57.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969560+0000", 2024-09-18T04:56:57.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.114e-06 2024-09-18T04:56:57.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969566+0000", 2024-09-18T04:56:57.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6300000000000003e-06 2024-09-18T04:56:57.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969571+0000", 2024-09-18T04:56:57.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0640000000000001e-06 2024-09-18T04:56:57.110 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.110 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.110 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.110 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72543 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.111 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.970176+0000", 2024-09-18T04:56:57.111 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024202929000000001, 2024-09-18T04:56:57.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025739407999999998, 2024-09-18T04:56:57.111 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.111 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.111 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.111 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.111 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.111 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72543 2024-09-18T04:56:57.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.111 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970176+0000", 2024-09-18T04:56:57.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970176+0000", 2024-09-18T04:56:57.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970176+0000", 2024-09-18T04:56:57.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970178+0000", 2024-09-18T04:56:57.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.097e-06 2024-09-18T04:56:57.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970184+0000", 2024-09-18T04:56:57.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9989999999999997e-06 2024-09-18T04:56:57.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970190+0000", 2024-09-18T04:56:57.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5620000000000003e-06 2024-09-18T04:56:57.114 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.114 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.114 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.114 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72716 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.114 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.974441+0000", 2024-09-18T04:56:57.114 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.019938589999999999, 2024-09-18T04:56:57.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021531630999999999, 2024-09-18T04:56:57.115 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.115 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.115 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.115 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.115 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.115 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72716 2024-09-18T04:56:57.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.115 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974441+0000", 2024-09-18T04:56:57.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974441+0000", 2024-09-18T04:56:57.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974440+0000", 2024-09-18T04:56:57.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974443+0000", 2024-09-18T04:56:57.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.948e-06 2024-09-18T04:56:57.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974450+0000", 2024-09-18T04:56:57.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0709999999999997e-06 2024-09-18T04:56:57.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974456+0000", 2024-09-18T04:56:57.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.237e-06 2024-09-18T04:56:57.117 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.118 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.118 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.118 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72855 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.118 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.981437+0000", 2024-09-18T04:56:57.118 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.012942594999999999, 2024-09-18T04:56:57.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014594121999999999, 2024-09-18T04:56:57.118 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.118 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.118 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.118 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.118 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.119 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72855 2024-09-18T04:56:57.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.119 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981437+0000", 2024-09-18T04:56:57.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981437+0000", 2024-09-18T04:56:57.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981436+0000", 2024-09-18T04:56:57.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981440+0000", 2024-09-18T04:56:57.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8510000000000001e-06 2024-09-18T04:56:57.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981510+0000", 2024-09-18T04:56:57.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0507999999999996e-05 2024-09-18T04:56:57.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.981518+0000", 2024-09-18T04:56:57.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.204e-06 2024-09-18T04:56:57.121 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.121 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.121 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.121 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72384 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.121 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969575+0000", 2024-09-18T04:56:57.122 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024803940999999999, 2024-09-18T04:56:57.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026512563999999999, 2024-09-18T04:56:57.122 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.122 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.122 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.122 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.122 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.122 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72384 2024-09-18T04:56:57.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.122 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969575+0000", 2024-09-18T04:56:57.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969575+0000", 2024-09-18T04:56:57.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969575+0000", 2024-09-18T04:56:57.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969578+0000", 2024-09-18T04:56:57.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9100000000000001e-06 2024-09-18T04:56:57.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969584+0000", 2024-09-18T04:56:57.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0700000000000003e-06 2024-09-18T04:56:57.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969589+0000", 2024-09-18T04:56:57.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6290000000000001e-06 2024-09-18T04:56:57.124 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.125 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.125 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.125 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72544 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.125 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.970194+0000", 2024-09-18T04:56:57.125 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024185268999999999, 2024-09-18T04:56:57.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025953642999999998, 2024-09-18T04:56:57.125 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.125 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.125 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.125 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.125 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.125 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72544 2024-09-18T04:56:57.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.126 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970194+0000", 2024-09-18T04:56:57.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970194+0000", 2024-09-18T04:56:57.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970194+0000", 2024-09-18T04:56:57.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970196+0000", 2024-09-18T04:56:57.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7099999999999999e-06 2024-09-18T04:56:57.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970202+0000", 2024-09-18T04:56:57.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1e-06 2024-09-18T04:56:57.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970207+0000", 2024-09-18T04:56:57.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5750000000000002e-06 2024-09-18T04:56:57.128 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.128 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.128 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.128 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72717 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.128 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.974460+0000", 2024-09-18T04:56:57.128 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.019919029000000001, 2024-09-18T04:56:57.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021743551, 2024-09-18T04:56:57.129 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.129 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.129 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.129 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.129 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.129 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72717 2024-09-18T04:56:57.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.129 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974460+0000", 2024-09-18T04:56:57.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974460+0000", 2024-09-18T04:56:57.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974460+0000", 2024-09-18T04:56:57.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974463+0000", 2024-09-18T04:56:57.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.056e-06 2024-09-18T04:56:57.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974469+0000", 2024-09-18T04:56:57.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5270000000000004e-06 2024-09-18T04:56:57.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974475+0000", 2024-09-18T04:56:57.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1619999999999999e-06 2024-09-18T04:56:57.131 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.131 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.131 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.132 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72889 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.132 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.982850+0000", 2024-09-18T04:56:57.132 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.011529665999999999, 2024-09-18T04:56:57.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013455333999999999, 2024-09-18T04:56:57.132 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.132 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.132 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.132 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.132 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.132 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72889 2024-09-18T04:56:57.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.133 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982850+0000", 2024-09-18T04:56:57.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982850+0000", 2024-09-18T04:56:57.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982849+0000", 2024-09-18T04:56:57.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:57.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982855+0000", 2024-09-18T04:56:57.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5640000000000002e-06 2024-09-18T04:56:57.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982868+0000", 2024-09-18T04:56:57.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2636999999999999e-05 2024-09-18T04:56:57.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982876+0000", 2024-09-18T04:56:57.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3289999999999994e-06 2024-09-18T04:56:57.135 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.135 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.135 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.135 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72386 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.135 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969593+0000", 2024-09-18T04:56:57.135 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024786488999999998, 2024-09-18T04:56:57.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026770525999999999, 2024-09-18T04:56:57.135 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.136 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.136 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.136 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.136 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.136 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72386 2024-09-18T04:56:57.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.136 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969593+0000", 2024-09-18T04:56:57.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969593+0000", 2024-09-18T04:56:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969592+0000", 2024-09-18T04:56:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969600+0000", 2024-09-18T04:56:57.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8849999999999999e-06 2024-09-18T04:56:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969607+0000", 2024-09-18T04:56:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8109999999999997e-06 2024-09-18T04:56:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969612+0000", 2024-09-18T04:56:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8119999999999998e-06 2024-09-18T04:56:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72545 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.970211+0000", 2024-09-18T04:56:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024168069, 2024-09-18T04:56:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026211834, 2024-09-18T04:56:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72545 2024-09-18T04:56:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.139 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970211+0000", 2024-09-18T04:56:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970211+0000", 2024-09-18T04:56:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970211+0000", 2024-09-18T04:56:57.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970219+0000", 2024-09-18T04:56:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1289999999999996e-06 2024-09-18T04:56:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970225+0000", 2024-09-18T04:56:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3910000000000003e-06 2024-09-18T04:56:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970230+0000", 2024-09-18T04:56:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8980000000000004e-06 2024-09-18T04:56:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72718 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.974479+0000", 2024-09-18T04:56:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.019900461000000001, 2024-09-18T04:56:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022003028000000001, 2024-09-18T04:56:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.142 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72718 2024-09-18T04:56:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974479+0000", 2024-09-18T04:56:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974479+0000", 2024-09-18T04:56:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974479+0000", 2024-09-18T04:56:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974481+0000", 2024-09-18T04:56:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.464e-06 2024-09-18T04:56:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974489+0000", 2024-09-18T04:56:57.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5650000000000004e-06 2024-09-18T04:56:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974494+0000", 2024-09-18T04:56:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9860000000000002e-06 2024-09-18T04:56:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72891 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.982883+0000", 2024-09-18T04:56:57.145 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.011495906, 2024-09-18T04:56:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013657842, 2024-09-18T04:56:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72891 2024-09-18T04:56:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982883+0000", 2024-09-18T04:56:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982883+0000", 2024-09-18T04:56:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982883+0000", 2024-09-18T04:56:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982886+0000", 2024-09-18T04:56:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.078e-06 2024-09-18T04:56:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982893+0000", 2024-09-18T04:56:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1720000000000001e-06 2024-09-18T04:56:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982899+0000", 2024-09-18T04:56:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7289999999999997e-06 2024-09-18T04:56:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.148 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72388 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969617+0000", 2024-09-18T04:56:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024762658, 2024-09-18T04:56:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026981891000000001, 2024-09-18T04:56:57.149 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72388 2024-09-18T04:56:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969617+0000", 2024-09-18T04:56:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969617+0000", 2024-09-18T04:56:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969616+0000", 2024-09-18T04:56:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969619+0000", 2024-09-18T04:56:57.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5600000000000001e-06 2024-09-18T04:56:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969625+0000", 2024-09-18T04:56:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4080000000000003e-06 2024-09-18T04:56:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969630+0000", 2024-09-18T04:56:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.527e-06 2024-09-18T04:56:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.152 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72546 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.970235+0000", 2024-09-18T04:56:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024144822999999999, 2024-09-18T04:56:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026420826000000001, 2024-09-18T04:56:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72546 2024-09-18T04:56:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.153 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970235+0000", 2024-09-18T04:56:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970235+0000", 2024-09-18T04:56:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970234+0000", 2024-09-18T04:56:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970236+0000", 2024-09-18T04:56:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3700000000000002e-06 2024-09-18T04:56:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970243+0000", 2024-09-18T04:56:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0739999999999998e-06 2024-09-18T04:56:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970247+0000", 2024-09-18T04:56:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8860000000000003e-06 2024-09-18T04:56:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72719 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.974498+0000", 2024-09-18T04:56:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.019881442999999999, 2024-09-18T04:56:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022217078000000001, 2024-09-18T04:56:57.156 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72719 2024-09-18T04:56:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974498+0000", 2024-09-18T04:56:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974498+0000", 2024-09-18T04:56:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974497+0000", 2024-09-18T04:56:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974500+0000", 2024-09-18T04:56:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.351e-06 2024-09-18T04:56:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974506+0000", 2024-09-18T04:56:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2380000000000002e-06 2024-09-18T04:56:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974511+0000", 2024-09-18T04:56:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.181e-06 2024-09-18T04:56:57.159 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72893 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.982904+0000", 2024-09-18T04:56:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.011475245, 2024-09-18T04:56:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.013872325, 2024-09-18T04:56:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.160 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72893 2024-09-18T04:56:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982904+0000", 2024-09-18T04:56:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982904+0000", 2024-09-18T04:56:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982903+0000", 2024-09-18T04:56:57.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982906+0000", 2024-09-18T04:56:57.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8270000000000002e-06 2024-09-18T04:56:57.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982913+0000", 2024-09-18T04:56:57.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3500000000000002e-06 2024-09-18T04:56:57.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982917+0000", 2024-09-18T04:56:57.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.763e-06 2024-09-18T04:56:57.163 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.163 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.163 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.163 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72390 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.163 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969634+0000", 2024-09-18T04:56:57.163 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024744859000000001, 2024-09-18T04:56:57.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027201724, 2024-09-18T04:56:57.164 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.164 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.164 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.164 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.164 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.164 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72390 2024-09-18T04:56:57.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.164 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969634+0000", 2024-09-18T04:56:57.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969634+0000", 2024-09-18T04:56:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969634+0000", 2024-09-18T04:56:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969637+0000", 2024-09-18T04:56:57.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6910000000000002e-06 2024-09-18T04:56:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969643+0000", 2024-09-18T04:56:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7690000000000002e-06 2024-09-18T04:56:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969647+0000", 2024-09-18T04:56:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6450000000000004e-06 2024-09-18T04:56:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72547 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.971457+0000", 2024-09-18T04:56:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022922767, 2024-09-18T04:56:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025438952000000001, 2024-09-18T04:56:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72547 2024-09-18T04:56:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.167 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971457+0000", 2024-09-18T04:56:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971457+0000", 2024-09-18T04:56:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971456+0000", 2024-09-18T04:56:57.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971460+0000", 2024-09-18T04:56:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5379999999999996e-06 2024-09-18T04:56:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971473+0000", 2024-09-18T04:56:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2302e-05 2024-09-18T04:56:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971480+0000", 2024-09-18T04:56:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0099999999999998e-06 2024-09-18T04:56:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72720 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.974515+0000", 2024-09-18T04:56:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.019863825000000002, 2024-09-18T04:56:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022439878, 2024-09-18T04:56:57.170 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72720 2024-09-18T04:56:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974515+0000", 2024-09-18T04:56:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974515+0000", 2024-09-18T04:56:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974515+0000", 2024-09-18T04:56:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974517+0000", 2024-09-18T04:56:57.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3240000000000001e-06 2024-09-18T04:56:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974524+0000", 2024-09-18T04:56:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0619999999999997e-06 2024-09-18T04:56:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974529+0000", 2024-09-18T04:56:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1880000000000001e-06 2024-09-18T04:56:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72895 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.982922+0000", 2024-09-18T04:56:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.011457263000000001, 2024-09-18T04:56:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014089927, 2024-09-18T04:56:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72895 2024-09-18T04:56:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982922+0000", 2024-09-18T04:56:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982922+0000", 2024-09-18T04:56:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982922+0000", 2024-09-18T04:56:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982924+0000", 2024-09-18T04:56:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.339e-06 2024-09-18T04:56:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982930+0000", 2024-09-18T04:56:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3330000000000002e-06 2024-09-18T04:56:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982935+0000", 2024-09-18T04:56:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9969999999999998e-06 2024-09-18T04:56:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72417 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969651+0000", 2024-09-18T04:56:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.02472794, 2024-09-18T04:56:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027419644, 2024-09-18T04:56:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.177 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72417 2024-09-18T04:56:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969651+0000", 2024-09-18T04:56:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969651+0000", 2024-09-18T04:56:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969651+0000", 2024-09-18T04:56:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969653+0000", 2024-09-18T04:56:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:56:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969659+0000", 2024-09-18T04:56:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8850000000000004e-06 2024-09-18T04:56:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969664+0000", 2024-09-18T04:56:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5569999999999997e-06 2024-09-18T04:56:57.180 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72548 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.971486+0000", 2024-09-18T04:56:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022893672, 2024-09-18T04:56:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025641944, 2024-09-18T04:56:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.181 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72548 2024-09-18T04:56:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971486+0000", 2024-09-18T04:56:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971486+0000", 2024-09-18T04:56:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971485+0000", 2024-09-18T04:56:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971488+0000", 2024-09-18T04:56:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.19e-06 2024-09-18T04:56:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971499+0000", 2024-09-18T04:56:57.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0213e-05 2024-09-18T04:56:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971504+0000", 2024-09-18T04:56:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.48e-06 2024-09-18T04:56:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72721 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.974542+0000", 2024-09-18T04:56:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.019837218, 2024-09-18T04:56:57.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022643307000000001, 2024-09-18T04:56:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72721 2024-09-18T04:56:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974542+0000", 2024-09-18T04:56:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974542+0000", 2024-09-18T04:56:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974533+0000", 2024-09-18T04:56:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999905 2024-09-18T04:56:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974545+0000", 2024-09-18T04:56:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2271e-05 2024-09-18T04:56:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974555+0000", 2024-09-18T04:56:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0397999999999999e-05 2024-09-18T04:56:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974561+0000", 2024-09-18T04:56:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6609999999999997e-06 2024-09-18T04:56:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72897 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.982940+0000", 2024-09-18T04:56:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.011439675999999999, 2024-09-18T04:56:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014301566, 2024-09-18T04:56:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72897 2024-09-18T04:56:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.188 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982940+0000", 2024-09-18T04:56:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982940+0000", 2024-09-18T04:56:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982939+0000", 2024-09-18T04:56:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982942+0000", 2024-09-18T04:56:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7080000000000002e-06 2024-09-18T04:56:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982949+0000", 2024-09-18T04:56:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2819999999999999e-06 2024-09-18T04:56:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982954+0000", 2024-09-18T04:56:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8709999999999996e-06 2024-09-18T04:56:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72418 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969668+0000", 2024-09-18T04:56:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.02471133, 2024-09-18T04:56:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027629326999999999, 2024-09-18T04:56:57.191 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72418 2024-09-18T04:56:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969668+0000", 2024-09-18T04:56:57.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969668+0000", 2024-09-18T04:56:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969668+0000", 2024-09-18T04:56:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969670+0000", 2024-09-18T04:56:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5270000000000001e-06 2024-09-18T04:56:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969676+0000", 2024-09-18T04:56:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6690000000000003e-06 2024-09-18T04:56:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969680+0000", 2024-09-18T04:56:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5329999999999996e-06 2024-09-18T04:56:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72549 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.971509+0000", 2024-09-18T04:56:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022870231000000001, 2024-09-18T04:56:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.025850115999999999, 2024-09-18T04:56:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.195 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72549 2024-09-18T04:56:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971509+0000", 2024-09-18T04:56:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971509+0000", 2024-09-18T04:56:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971509+0000", 2024-09-18T04:56:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971512+0000", 2024-09-18T04:56:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2399999999999999e-06 2024-09-18T04:56:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971521+0000", 2024-09-18T04:56:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3389999999999999e-06 2024-09-18T04:56:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971527+0000", 2024-09-18T04:56:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6710000000000004e-06 2024-09-18T04:56:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.198 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72722 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.974566+0000", 2024-09-18T04:56:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.019813186999999999, 2024-09-18T04:56:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022850591999999999, 2024-09-18T04:56:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72722 2024-09-18T04:56:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974566+0000", 2024-09-18T04:56:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974566+0000", 2024-09-18T04:56:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974566+0000", 2024-09-18T04:56:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974568+0000", 2024-09-18T04:56:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4449999999999999e-06 2024-09-18T04:56:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974575+0000", 2024-09-18T04:56:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3459999999999999e-06 2024-09-18T04:56:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.974580+0000", 2024-09-18T04:56:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9910000000000002e-06 2024-09-18T04:56:57.201 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72899 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.982958+0000", 2024-09-18T04:56:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.011420972, 2024-09-18T04:56:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014518243, 2024-09-18T04:56:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.202 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72899 2024-09-18T04:56:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982958+0000", 2024-09-18T04:56:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982958+0000", 2024-09-18T04:56:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982958+0000", 2024-09-18T04:56:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982960+0000", 2024-09-18T04:56:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.362e-06 2024-09-18T04:56:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982967+0000", 2024-09-18T04:56:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2060000000000004e-06 2024-09-18T04:56:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982971+0000", 2024-09-18T04:56:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9409999999999998e-06 2024-09-18T04:56:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72419 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969684+0000", 2024-09-18T04:56:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024695132000000002, 2024-09-18T04:56:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027851272999999999, 2024-09-18T04:56:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.205 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72419 2024-09-18T04:56:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969684+0000", 2024-09-18T04:56:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969684+0000", 2024-09-18T04:56:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969684+0000", 2024-09-18T04:56:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969686+0000", 2024-09-18T04:56:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.216e-06 2024-09-18T04:56:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969692+0000", 2024-09-18T04:56:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9320000000000001e-06 2024-09-18T04:56:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969697+0000", 2024-09-18T04:56:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.51e-06 2024-09-18T04:56:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.208 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72587 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.971532+0000", 2024-09-18T04:56:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022847765999999999, 2024-09-18T04:56:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026062887999999999, 2024-09-18T04:56:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72587 2024-09-18T04:56:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971532+0000", 2024-09-18T04:56:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971532+0000", 2024-09-18T04:56:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971531+0000", 2024-09-18T04:56:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971534+0000", 2024-09-18T04:56:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7429999999999998e-06 2024-09-18T04:56:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971543+0000", 2024-09-18T04:56:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0019999999999995e-06 2024-09-18T04:56:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971548+0000", 2024-09-18T04:56:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4700000000000001e-06 2024-09-18T04:56:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.211 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72757 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.975975+0000", 2024-09-18T04:56:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.018404093999999999, 2024-09-18T04:56:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021675313000000002, 2024-09-18T04:56:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.212 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72757 2024-09-18T04:56:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.975975+0000", 2024-09-18T04:56:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.975975+0000", 2024-09-18T04:56:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.975974+0000", 2024-09-18T04:56:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.975979+0000", 2024-09-18T04:56:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1540000000000001e-06 2024-09-18T04:56:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.975990+0000", 2024-09-18T04:56:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0088999999999999e-05 2024-09-18T04:56:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.975996+0000", 2024-09-18T04:56:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1129999999999997e-06 2024-09-18T04:56:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.215 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72901 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.982976+0000", 2024-09-18T04:56:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.011403594, 2024-09-18T04:56:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014732574, 2024-09-18T04:56:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.216 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72901 2024-09-18T04:56:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982976+0000", 2024-09-18T04:56:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982976+0000", 2024-09-18T04:56:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982975+0000", 2024-09-18T04:56:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982978+0000", 2024-09-18T04:56:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2529999999999999e-06 2024-09-18T04:56:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982984+0000", 2024-09-18T04:56:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0399999999999998e-06 2024-09-18T04:56:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982988+0000", 2024-09-18T04:56:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6110000000000004e-06 2024-09-18T04:56:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.219 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.220 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.220 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72420 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.220 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969701+0000", 2024-09-18T04:56:57.220 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024678451000000001, 2024-09-18T04:56:57.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028067567000000002, 2024-09-18T04:56:57.220 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.220 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.220 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.220 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.220 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.221 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72420 2024-09-18T04:56:57.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.221 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969701+0000", 2024-09-18T04:56:57.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969701+0000", 2024-09-18T04:56:57.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969701+0000", 2024-09-18T04:56:57.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969703+0000", 2024-09-18T04:56:57.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3039999999999999e-06 2024-09-18T04:56:57.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969709+0000", 2024-09-18T04:56:57.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4869999999999999e-06 2024-09-18T04:56:57.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969714+0000", 2024-09-18T04:56:57.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9830000000000004e-06 2024-09-18T04:56:57.224 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.224 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.224 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.224 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72588 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.224 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.971553+0000", 2024-09-18T04:56:57.224 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022825952, 2024-09-18T04:56:57.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026274737999999999, 2024-09-18T04:56:57.225 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.225 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.225 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.225 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.225 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.225 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72588 2024-09-18T04:56:57.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.225 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971553+0000", 2024-09-18T04:56:57.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971553+0000", 2024-09-18T04:56:57.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971553+0000", 2024-09-18T04:56:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971556+0000", 2024-09-18T04:56:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8600000000000001e-06 2024-09-18T04:56:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971562+0000", 2024-09-18T04:56:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5540000000000003e-06 2024-09-18T04:56:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971567+0000", 2024-09-18T04:56:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7740000000000004e-06 2024-09-18T04:56:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72759 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.228 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.976506+0000", 2024-09-18T04:56:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.017873686999999999, 2024-09-18T04:56:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021379387999999999, 2024-09-18T04:56:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72759 2024-09-18T04:56:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.976506+0000", 2024-09-18T04:56:57.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.976506+0000", 2024-09-18T04:56:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.976505+0000", 2024-09-18T04:56:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.976510+0000", 2024-09-18T04:56:57.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8289999999999998e-06 2024-09-18T04:56:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.976520+0000", 2024-09-18T04:56:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8970000000000003e-06 2024-09-18T04:56:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.976526+0000", 2024-09-18T04:56:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7839999999999998e-06 2024-09-18T04:56:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72903 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.982992+0000", 2024-09-18T04:56:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.011386944, 2024-09-18T04:56:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014950147, 2024-09-18T04:56:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72903 2024-09-18T04:56:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.232 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982992+0000", 2024-09-18T04:56:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982992+0000", 2024-09-18T04:56:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982992+0000", 2024-09-18T04:56:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.982994+0000", 2024-09-18T04:56:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1749999999999999e-06 2024-09-18T04:56:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.983000+0000", 2024-09-18T04:56:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.993e-06 2024-09-18T04:56:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.983005+0000", 2024-09-18T04:56:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7960000000000003e-06 2024-09-18T04:56:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72421 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969718+0000", 2024-09-18T04:56:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024661234000000001, 2024-09-18T04:56:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028280337999999999, 2024-09-18T04:56:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.235 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72421 2024-09-18T04:56:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969718+0000", 2024-09-18T04:56:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969718+0000", 2024-09-18T04:56:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969718+0000", 2024-09-18T04:56:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969720+0000", 2024-09-18T04:56:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1150000000000001e-06 2024-09-18T04:56:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969726+0000", 2024-09-18T04:56:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9150000000000001e-06 2024-09-18T04:56:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969731+0000", 2024-09-18T04:56:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.882e-06 2024-09-18T04:56:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72589 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.971571+0000", 2024-09-18T04:56:57.238 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022807997, 2024-09-18T04:56:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026483374, 2024-09-18T04:56:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72589 2024-09-18T04:56:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971571+0000", 2024-09-18T04:56:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971571+0000", 2024-09-18T04:56:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971571+0000", 2024-09-18T04:56:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971573+0000", 2024-09-18T04:56:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3949999999999999e-06 2024-09-18T04:56:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971582+0000", 2024-09-18T04:56:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0119999999999994e-06 2024-09-18T04:56:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971588+0000", 2024-09-18T04:56:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3879999999999999e-06 2024-09-18T04:56:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.241 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72761 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.976533+0000", 2024-09-18T04:56:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.017846391, 2024-09-18T04:56:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021580281999999999, 2024-09-18T04:56:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72761 2024-09-18T04:56:57.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.976533+0000", 2024-09-18T04:56:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.976533+0000", 2024-09-18T04:56:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.976532+0000", 2024-09-18T04:56:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.976535+0000", 2024-09-18T04:56:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7099999999999999e-06 2024-09-18T04:56:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.976543+0000", 2024-09-18T04:56:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5920000000000003e-06 2024-09-18T04:56:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.976548+0000", 2024-09-18T04:56:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5450000000000003e-06 2024-09-18T04:56:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72905 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.983009+0000", 2024-09-18T04:56:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.011370289, 2024-09-18T04:56:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015159107, 2024-09-18T04:56:57.245 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.246 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.246 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.246 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.246 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.246 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72905 2024-09-18T04:56:57.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.246 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.983009+0000", 2024-09-18T04:56:57.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.983009+0000", 2024-09-18T04:56:57.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.983009+0000", 2024-09-18T04:56:57.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.983011+0000", 2024-09-18T04:56:57.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.187e-06 2024-09-18T04:56:57.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.983017+0000", 2024-09-18T04:56:57.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1120000000000001e-06 2024-09-18T04:56:57.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.983022+0000", 2024-09-18T04:56:57.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0880000000000002e-06 2024-09-18T04:56:57.248 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.248 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.248 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.249 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72422 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.249 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969735+0000", 2024-09-18T04:56:57.249 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024643993999999999, 2024-09-18T04:56:57.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028491242999999999, 2024-09-18T04:56:57.249 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.249 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.249 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.249 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.249 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72422 2024-09-18T04:56:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969735+0000", 2024-09-18T04:56:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969735+0000", 2024-09-18T04:56:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969735+0000", 2024-09-18T04:56:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969741+0000", 2024-09-18T04:56:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1059999999999996e-06 2024-09-18T04:56:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969747+0000", 2024-09-18T04:56:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2149999999999998e-06 2024-09-18T04:56:57.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969752+0000", 2024-09-18T04:56:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9239999999999998e-06 2024-09-18T04:56:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72590 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.971593+0000", 2024-09-18T04:56:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.02278678, 2024-09-18T04:56:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026689127999999999, 2024-09-18T04:56:57.252 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.253 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.253 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.253 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.253 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.253 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72590 2024-09-18T04:56:57.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.253 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971593+0000", 2024-09-18T04:56:57.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971593+0000", 2024-09-18T04:56:57.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971592+0000", 2024-09-18T04:56:57.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971595+0000", 2024-09-18T04:56:57.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9220000000000001e-06 2024-09-18T04:56:57.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971603+0000", 2024-09-18T04:56:57.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4030000000000007e-06 2024-09-18T04:56:57.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971609+0000", 2024-09-18T04:56:57.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1889999999999998e-06 2024-09-18T04:56:57.256 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.256 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.256 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.256 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72765 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.256 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.976553+0000", 2024-09-18T04:56:57.257 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.017826654000000001, 2024-09-18T04:56:57.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021863257000000001, 2024-09-18T04:56:57.257 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.257 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.257 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.257 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.257 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.257 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72765 2024-09-18T04:56:57.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.257 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.976553+0000", 2024-09-18T04:56:57.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.976553+0000", 2024-09-18T04:56:57.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.976552+0000", 2024-09-18T04:56:57.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.976555+0000", 2024-09-18T04:56:57.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3590000000000002e-06 2024-09-18T04:56:57.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.976561+0000", 2024-09-18T04:56:57.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2129999999999996e-06 2024-09-18T04:56:57.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.976566+0000", 2024-09-18T04:56:57.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.797e-06 2024-09-18T04:56:57.260 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.260 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.260 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.261 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72926 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.261 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.983906+0000", 2024-09-18T04:56:57.261 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.010473549, 2024-09-18T04:56:57.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014565929, 2024-09-18T04:56:57.261 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.261 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.261 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.261 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.261 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72926 2024-09-18T04:56:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.983906+0000", 2024-09-18T04:56:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.983906+0000", 2024-09-18T04:56:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.983905+0000", 2024-09-18T04:56:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.983910+0000", 2024-09-18T04:56:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.558e-06 2024-09-18T04:56:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.983920+0000", 2024-09-18T04:56:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4919999999999999e-06 2024-09-18T04:56:57.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.983926+0000", 2024-09-18T04:56:57.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4649999999999999e-06 2024-09-18T04:56:57.264 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.264 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.264 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.264 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72423 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.264 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969757+0000", 2024-09-18T04:56:57.264 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024622398, 2024-09-18T04:56:57.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028771327999999999, 2024-09-18T04:56:57.265 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.265 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.265 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.265 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.265 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.265 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72423 2024-09-18T04:56:57.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.265 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969757+0000", 2024-09-18T04:56:57.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969757+0000", 2024-09-18T04:56:57.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969756+0000", 2024-09-18T04:56:57.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969759+0000", 2024-09-18T04:56:57.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.401e-06 2024-09-18T04:56:57.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969770+0000", 2024-09-18T04:56:57.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1240000000000001e-05 2024-09-18T04:56:57.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969775+0000", 2024-09-18T04:56:57.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9459999999999997e-06 2024-09-18T04:56:57.267 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.267 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.267 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.267 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72591 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.268 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.971613+0000", 2024-09-18T04:56:57.268 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022765974000000001, 2024-09-18T04:56:57.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.026971663, 2024-09-18T04:56:57.268 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.268 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.268 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.268 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.268 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.268 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72591 2024-09-18T04:56:57.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.268 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971613+0000", 2024-09-18T04:56:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971613+0000", 2024-09-18T04:56:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971613+0000", 2024-09-18T04:56:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971616+0000", 2024-09-18T04:56:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7049999999999999e-06 2024-09-18T04:56:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971624+0000", 2024-09-18T04:56:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6200000000000005e-06 2024-09-18T04:56:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971629+0000", 2024-09-18T04:56:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1390000000000003e-06 2024-09-18T04:56:57.270 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72767 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.976570+0000", 2024-09-18T04:56:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.017809465, 2024-09-18T04:56:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022071819999999999, 2024-09-18T04:56:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.271 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72767 2024-09-18T04:56:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.976570+0000", 2024-09-18T04:56:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.976570+0000", 2024-09-18T04:56:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.976569+0000", 2024-09-18T04:56:57.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.976572+0000", 2024-09-18T04:56:57.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1600000000000001e-06 2024-09-18T04:56:57.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.976578+0000", 2024-09-18T04:56:57.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.049e-06 2024-09-18T04:56:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.976582+0000", 2024-09-18T04:56:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4850000000000003e-06 2024-09-18T04:56:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72938 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.983933+0000", 2024-09-18T04:56:57.274 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.010446735, 2024-09-18T04:56:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014764492000000001, 2024-09-18T04:56:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72938 2024-09-18T04:56:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.983933+0000", 2024-09-18T04:56:57.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.983933+0000", 2024-09-18T04:56:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.983932+0000", 2024-09-18T04:56:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.983935+0000", 2024-09-18T04:56:57.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6649999999999999e-06 2024-09-18T04:56:57.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.983941+0000", 2024-09-18T04:56:57.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2230000000000004e-06 2024-09-18T04:56:57.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.983946+0000", 2024-09-18T04:56:57.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1159999999999998e-06 2024-09-18T04:56:57.277 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72424 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969780+0000", 2024-09-18T04:56:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024599539, 2024-09-18T04:56:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028975695999999999, 2024-09-18T04:56:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.278 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72424 2024-09-18T04:56:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969780+0000", 2024-09-18T04:56:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969780+0000", 2024-09-18T04:56:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969779+0000", 2024-09-18T04:56:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969782+0000", 2024-09-18T04:56:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.632e-06 2024-09-18T04:56:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969794+0000", 2024-09-18T04:56:57.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2310000000000001e-05 2024-09-18T04:56:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969800+0000", 2024-09-18T04:56:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.48e-06 2024-09-18T04:56:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72592 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.971634+0000", 2024-09-18T04:56:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022744923, 2024-09-18T04:56:57.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027182187, 2024-09-18T04:56:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72592 2024-09-18T04:56:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971634+0000", 2024-09-18T04:56:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971634+0000", 2024-09-18T04:56:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971634+0000", 2024-09-18T04:56:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971637+0000", 2024-09-18T04:56:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5799999999999999e-06 2024-09-18T04:56:57.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971643+0000", 2024-09-18T04:56:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3860000000000004e-06 2024-09-18T04:56:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971648+0000", 2024-09-18T04:56:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8219999999999997e-06 2024-09-18T04:56:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72768 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.977063+0000", 2024-09-18T04:56:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.017316772000000001, 2024-09-18T04:56:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021810183, 2024-09-18T04:56:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72768 2024-09-18T04:56:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.285 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.977063+0000", 2024-09-18T04:56:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.977063+0000", 2024-09-18T04:56:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.977062+0000", 2024-09-18T04:56:57.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.977067+0000", 2024-09-18T04:56:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2809999999999999e-06 2024-09-18T04:56:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.977077+0000", 2024-09-18T04:56:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0387000000000001e-05 2024-09-18T04:56:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.977084+0000", 2024-09-18T04:56:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.725e-06 2024-09-18T04:56:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72947 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.983950+0000", 2024-09-18T04:56:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.010428873999999999, 2024-09-18T04:56:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014981097000000001, 2024-09-18T04:56:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.288 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.289 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.289 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.289 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72947 2024-09-18T04:56:57.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.289 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.983950+0000", 2024-09-18T04:56:57.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.983950+0000", 2024-09-18T04:56:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.983950+0000", 2024-09-18T04:56:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.983952+0000", 2024-09-18T04:56:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1280000000000002e-06 2024-09-18T04:56:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.983958+0000", 2024-09-18T04:56:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.818e-06 2024-09-18T04:56:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.983965+0000", 2024-09-18T04:56:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8639999999999998e-06 2024-09-18T04:56:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72425 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.291 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969804+0000", 2024-09-18T04:56:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024575034999999999, 2024-09-18T04:56:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029183145000000001, 2024-09-18T04:56:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72425 2024-09-18T04:56:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969804+0000", 2024-09-18T04:56:57.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969804+0000", 2024-09-18T04:56:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969804+0000", 2024-09-18T04:56:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969807+0000", 2024-09-18T04:56:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5959999999999998e-06 2024-09-18T04:56:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969813+0000", 2024-09-18T04:56:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1689999999999997e-06 2024-09-18T04:56:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969817+0000", 2024-09-18T04:56:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3579999999999996e-06 2024-09-18T04:56:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.294 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.295 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.295 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72593 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.295 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.971652+0000", 2024-09-18T04:56:57.295 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022727239999999999, 2024-09-18T04:56:57.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027397847999999999, 2024-09-18T04:56:57.295 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.295 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.295 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.295 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.295 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.295 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72593 2024-09-18T04:56:57.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.296 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971652+0000", 2024-09-18T04:56:57.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971652+0000", 2024-09-18T04:56:57.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971652+0000", 2024-09-18T04:56:57.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971654+0000", 2024-09-18T04:56:57.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2400000000000002e-06 2024-09-18T04:56:57.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971663+0000", 2024-09-18T04:56:57.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1789999999999997e-06 2024-09-18T04:56:57.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971668+0000", 2024-09-18T04:56:57.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9790000000000001e-06 2024-09-18T04:56:57.298 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.298 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.298 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.299 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72769 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.299 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.977376+0000", 2024-09-18T04:56:57.299 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.017003338999999999, 2024-09-18T04:56:57.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021730309999999999, 2024-09-18T04:56:57.299 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.299 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.299 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.299 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.299 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.299 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72769 2024-09-18T04:56:57.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.299 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.977376+0000", 2024-09-18T04:56:57.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.977376+0000", 2024-09-18T04:56:57.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.977375+0000", 2024-09-18T04:56:57.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:57.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.977380+0000", 2024-09-18T04:56:57.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2959999999999998e-06 2024-09-18T04:56:57.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.977393+0000", 2024-09-18T04:56:57.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2982e-05 2024-09-18T04:56:57.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.977405+0000", 2024-09-18T04:56:57.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1661999999999999e-05 2024-09-18T04:56:57.302 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.302 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.302 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.302 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72951 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.302 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.984822+0000", 2024-09-18T04:56:57.302 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0095571249999999996, 2024-09-18T04:56:57.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014343404000000001, 2024-09-18T04:56:57.302 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.302 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.303 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.303 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.303 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.303 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72951 2024-09-18T04:56:57.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.303 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984822+0000", 2024-09-18T04:56:57.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984822+0000", 2024-09-18T04:56:57.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984821+0000", 2024-09-18T04:56:57.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:57.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984866+0000", 2024-09-18T04:56:57.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4477000000000001e-05 2024-09-18T04:56:57.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984876+0000", 2024-09-18T04:56:57.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8130000000000007e-06 2024-09-18T04:56:57.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984882+0000", 2024-09-18T04:56:57.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6789999999999999e-06 2024-09-18T04:56:57.305 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.305 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.305 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.305 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72459 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 531456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.306 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969822+0000", 2024-09-18T04:56:57.306 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024557694000000001, 2024-09-18T04:56:57.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029407104999999999, 2024-09-18T04:56:57.306 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.306 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:57.306 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.306 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.306 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.306 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72459 2024-09-18T04:56:57.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.306 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969822+0000", 2024-09-18T04:56:57.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969822+0000", 2024-09-18T04:56:57.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969821+0000", 2024-09-18T04:56:57.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969824+0000", 2024-09-18T04:56:57.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5770000000000001e-06 2024-09-18T04:56:57.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969830+0000", 2024-09-18T04:56:57.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7599999999999999e-06 2024-09-18T04:56:57.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969834+0000", 2024-09-18T04:56:57.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7589999999999997e-06 2024-09-18T04:56:57.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:57.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.999167+0000", 2024-09-18T04:56:57.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029333095999999999 2024-09-18T04:56:57.309 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.309 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.309 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.309 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72594 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.309 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.971673+0000", 2024-09-18T04:56:57.309 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022706786, 2024-09-18T04:56:57.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027623068000000001, 2024-09-18T04:56:57.309 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.309 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.310 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.310 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.310 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.310 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72594 2024-09-18T04:56:57.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.310 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971673+0000", 2024-09-18T04:56:57.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971673+0000", 2024-09-18T04:56:57.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971672+0000", 2024-09-18T04:56:57.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971685+0000", 2024-09-18T04:56:57.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3332999999999999e-05 2024-09-18T04:56:57.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971696+0000", 2024-09-18T04:56:57.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0346999999999999e-05 2024-09-18T04:56:57.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971702+0000", 2024-09-18T04:56:57.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0639999999999999e-06 2024-09-18T04:56:57.312 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.312 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.312 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.312 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72770 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.313 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.977413+0000", 2024-09-18T04:56:57.313 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.016966716, 2024-09-18T04:56:57.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021937610999999999, 2024-09-18T04:56:57.313 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.313 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.313 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.313 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.313 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.313 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72770 2024-09-18T04:56:57.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.313 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.977413+0000", 2024-09-18T04:56:57.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.977413+0000", 2024-09-18T04:56:57.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.977412+0000", 2024-09-18T04:56:57.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.977415+0000", 2024-09-18T04:56:57.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2660000000000002e-06 2024-09-18T04:56:57.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.977423+0000", 2024-09-18T04:56:57.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5100000000000001e-06 2024-09-18T04:56:57.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.977429+0000", 2024-09-18T04:56:57.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7169999999999996e-06 2024-09-18T04:56:57.316 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.316 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.316 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.316 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72953 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.316 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.984889+0000", 2024-09-18T04:56:57.316 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0094901840000000005, 2024-09-18T04:56:57.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014520557, 2024-09-18T04:56:57.316 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.317 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.317 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.317 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.317 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.317 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72953 2024-09-18T04:56:57.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.317 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984889+0000", 2024-09-18T04:56:57.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984889+0000", 2024-09-18T04:56:57.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984889+0000", 2024-09-18T04:56:57.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984892+0000", 2024-09-18T04:56:57.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.675e-06 2024-09-18T04:56:57.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984899+0000", 2024-09-18T04:56:57.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6359999999999997e-06 2024-09-18T04:56:57.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984904+0000", 2024-09-18T04:56:57.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8130000000000003e-06 2024-09-18T04:56:57.320 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.320 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.320 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.320 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72460 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 546816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.320 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969839+0000", 2024-09-18T04:56:57.321 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024540387, 2024-09-18T04:56:57.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029631228999999999, 2024-09-18T04:56:57.321 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.321 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:57.321 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.321 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.321 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.321 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72460 2024-09-18T04:56:57.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.321 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969839+0000", 2024-09-18T04:56:57.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969839+0000", 2024-09-18T04:56:57.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969839+0000", 2024-09-18T04:56:57.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969841+0000", 2024-09-18T04:56:57.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.413e-06 2024-09-18T04:56:57.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969847+0000", 2024-09-18T04:56:57.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5100000000000004e-06 2024-09-18T04:56:57.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969852+0000", 2024-09-18T04:56:57.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3590000000000001e-06 2024-09-18T04:56:57.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:57.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.999374+0000", 2024-09-18T04:56:57.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029522518000000001 2024-09-18T04:56:57.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:57.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.999446+0000", 2024-09-18T04:56:57.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2133999999999997e-05 2024-09-18T04:56:57.324 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.324 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.324 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.325 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72624 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.325 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.971707+0000", 2024-09-18T04:56:57.325 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022671922000000001, 2024-09-18T04:56:57.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027833429999999999, 2024-09-18T04:56:57.325 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.325 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.325 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.325 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.325 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.325 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72624 2024-09-18T04:56:57.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.326 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971707+0000", 2024-09-18T04:56:57.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971707+0000", 2024-09-18T04:56:57.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971707+0000", 2024-09-18T04:56:57.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971710+0000", 2024-09-18T04:56:57.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6900000000000001e-06 2024-09-18T04:56:57.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971718+0000", 2024-09-18T04:56:57.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9990000000000008e-06 2024-09-18T04:56:57.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971722+0000", 2024-09-18T04:56:57.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6110000000000004e-06 2024-09-18T04:56:57.328 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.328 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.328 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.329 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72796 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.329 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.978397+0000", 2024-09-18T04:56:57.329 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.015981984000000001, 2024-09-18T04:56:57.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021200289000000001, 2024-09-18T04:56:57.329 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.329 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.329 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.329 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.330 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.330 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72796 2024-09-18T04:56:57.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.330 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978397+0000", 2024-09-18T04:56:57.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978397+0000", 2024-09-18T04:56:57.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978396+0000", 2024-09-18T04:56:57.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:57.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978402+0000", 2024-09-18T04:56:57.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4049999999999999e-06 2024-09-18T04:56:57.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978416+0000", 2024-09-18T04:56:57.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4600000000000001e-05 2024-09-18T04:56:57.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978424+0000", 2024-09-18T04:56:57.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1709999999999996e-06 2024-09-18T04:56:57.333 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.333 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.333 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.333 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72955 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.334 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.984908+0000", 2024-09-18T04:56:57.334 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0094708550000000002, 2024-09-18T04:56:57.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014747369, 2024-09-18T04:56:57.334 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.334 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.334 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.334 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.334 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.334 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72955 2024-09-18T04:56:57.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.334 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984908+0000", 2024-09-18T04:56:57.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984908+0000", 2024-09-18T04:56:57.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984908+0000", 2024-09-18T04:56:57.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984911+0000", 2024-09-18T04:56:57.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4830000000000002e-06 2024-09-18T04:56:57.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984917+0000", 2024-09-18T04:56:57.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0959999999999997e-06 2024-09-18T04:56:57.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984922+0000", 2024-09-18T04:56:57.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1039999999999998e-06 2024-09-18T04:56:57.337 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.337 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.337 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.337 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72461 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 562176~1024 out=1024b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.337 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969857+0000", 2024-09-18T04:56:57.337 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024522783999999999, 2024-09-18T04:56:57.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029863529, 2024-09-18T04:56:57.337 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.337 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:56:57.337 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.337 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.337 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.338 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72461 2024-09-18T04:56:57.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.338 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969857+0000", 2024-09-18T04:56:57.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969857+0000", 2024-09-18T04:56:57.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969856+0000", 2024-09-18T04:56:57.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969859+0000", 2024-09-18T04:56:57.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4169999999999999e-06 2024-09-18T04:56:57.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969865+0000", 2024-09-18T04:56:57.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2829999999999998e-06 2024-09-18T04:56:57.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969870+0000", 2024-09-18T04:56:57.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8180000000000003e-06 2024-09-18T04:56:57.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:57.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.999604+0000", 2024-09-18T04:56:57.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.02973396 2024-09-18T04:56:57.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:56:57.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.999670+0000", 2024-09-18T04:56:57.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6497999999999999e-05 2024-09-18T04:56:57.341 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.341 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.341 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.341 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72626 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.341 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.971727+0000", 2024-09-18T04:56:57.341 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022652482000000002, 2024-09-18T04:56:57.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028079415, 2024-09-18T04:56:57.341 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.341 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.341 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.341 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.342 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.342 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72626 2024-09-18T04:56:57.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.342 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971727+0000", 2024-09-18T04:56:57.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971727+0000", 2024-09-18T04:56:57.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971726+0000", 2024-09-18T04:56:57.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971729+0000", 2024-09-18T04:56:57.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7499999999999999e-06 2024-09-18T04:56:57.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971735+0000", 2024-09-18T04:56:57.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1500000000000004e-06 2024-09-18T04:56:57.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971740+0000", 2024-09-18T04:56:57.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8010000000000003e-06 2024-09-18T04:56:57.344 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.344 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.344 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.344 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72797 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.344 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.978430+0000", 2024-09-18T04:56:57.344 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.015948980000000001, 2024-09-18T04:56:57.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021433747999999999, 2024-09-18T04:56:57.345 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.345 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.345 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.345 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.345 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.345 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72797 2024-09-18T04:56:57.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.345 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978430+0000", 2024-09-18T04:56:57.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978430+0000", 2024-09-18T04:56:57.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978430+0000", 2024-09-18T04:56:57.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978433+0000", 2024-09-18T04:56:57.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1130000000000001e-06 2024-09-18T04:56:57.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978440+0000", 2024-09-18T04:56:57.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3409999999999997e-06 2024-09-18T04:56:57.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978446+0000", 2024-09-18T04:56:57.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1989999999999997e-06 2024-09-18T04:56:57.347 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.347 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.347 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.348 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72957 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.348 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.984926+0000", 2024-09-18T04:56:57.348 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.009453085, 2024-09-18T04:56:57.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014996971, 2024-09-18T04:56:57.348 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.348 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.348 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.348 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.348 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.348 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72957 2024-09-18T04:56:57.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.348 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984926+0000", 2024-09-18T04:56:57.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984926+0000", 2024-09-18T04:56:57.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984926+0000", 2024-09-18T04:56:57.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984928+0000", 2024-09-18T04:56:57.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3379999999999999e-06 2024-09-18T04:56:57.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984934+0000", 2024-09-18T04:56:57.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1009999999999997e-06 2024-09-18T04:56:57.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984939+0000", 2024-09-18T04:56:57.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.69e-06 2024-09-18T04:56:57.351 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.351 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.351 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.351 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72462 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 577536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.351 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969874+0000", 2024-09-18T04:56:57.351 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024505460999999999, 2024-09-18T04:56:57.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.03011177, 2024-09-18T04:56:57.352 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.352 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:57.352 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.352 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.352 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.352 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72462 2024-09-18T04:56:57.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.352 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969874+0000", 2024-09-18T04:56:57.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969874+0000", 2024-09-18T04:56:57.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969873+0000", 2024-09-18T04:56:57.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969876+0000", 2024-09-18T04:56:57.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3149999999999999e-06 2024-09-18T04:56:57.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969884+0000", 2024-09-18T04:56:57.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1089999999999998e-06 2024-09-18T04:56:57.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969890+0000", 2024-09-18T04:56:57.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9959999999999999e-06 2024-09-18T04:56:57.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:57.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.999938+0000", 2024-09-18T04:56:57.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030047763000000002 2024-09-18T04:56:57.355 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.355 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.355 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.356 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72627 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.356 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.971745+0000", 2024-09-18T04:56:57.356 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022634665000000002, 2024-09-18T04:56:57.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028307954999999999, 2024-09-18T04:56:57.356 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.356 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.356 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.356 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.356 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.356 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72627 2024-09-18T04:56:57.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.357 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971745+0000", 2024-09-18T04:56:57.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971745+0000", 2024-09-18T04:56:57.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971744+0000", 2024-09-18T04:56:57.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971747+0000", 2024-09-18T04:56:57.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5730000000000002e-06 2024-09-18T04:56:57.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971753+0000", 2024-09-18T04:56:57.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7169999999999996e-06 2024-09-18T04:56:57.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971758+0000", 2024-09-18T04:56:57.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1669999999999998e-06 2024-09-18T04:56:57.359 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.359 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.359 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.359 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72798 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.360 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.978451+0000", 2024-09-18T04:56:57.360 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.015928680000000001, 2024-09-18T04:56:57.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021658652, 2024-09-18T04:56:57.360 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.360 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.360 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.360 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.360 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.360 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72798 2024-09-18T04:56:57.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.360 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978451+0000", 2024-09-18T04:56:57.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978451+0000", 2024-09-18T04:56:57.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978450+0000", 2024-09-18T04:56:57.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978453+0000", 2024-09-18T04:56:57.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.615e-06 2024-09-18T04:56:57.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978461+0000", 2024-09-18T04:56:57.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1769999999999998e-06 2024-09-18T04:56:57.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978466+0000", 2024-09-18T04:56:57.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5160000000000002e-06 2024-09-18T04:56:57.363 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.363 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.363 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.364 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72959 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.364 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.984943+0000", 2024-09-18T04:56:57.364 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0094362579999999995, 2024-09-18T04:56:57.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015221126, 2024-09-18T04:56:57.364 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.364 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.364 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.364 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.365 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.365 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72959 2024-09-18T04:56:57.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.365 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984943+0000", 2024-09-18T04:56:57.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984943+0000", 2024-09-18T04:56:57.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984943+0000", 2024-09-18T04:56:57.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984945+0000", 2024-09-18T04:56:57.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2029999999999999e-06 2024-09-18T04:56:57.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984951+0000", 2024-09-18T04:56:57.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8390000000000003e-06 2024-09-18T04:56:57.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984958+0000", 2024-09-18T04:56:57.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2969999999999998e-06 2024-09-18T04:56:57.368 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.368 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.368 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.368 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72463 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 592896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.368 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969895+0000", 2024-09-18T04:56:57.368 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024484793000000001, 2024-09-18T04:56:57.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030333344000000002, 2024-09-18T04:56:57.369 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.369 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:57.369 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.369 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.369 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.369 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72463 2024-09-18T04:56:57.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.369 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969895+0000", 2024-09-18T04:56:57.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969895+0000", 2024-09-18T04:56:57.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969894+0000", 2024-09-18T04:56:57.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969897+0000", 2024-09-18T04:56:57.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7149999999999998e-06 2024-09-18T04:56:57.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969904+0000", 2024-09-18T04:56:57.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9800000000000001e-06 2024-09-18T04:56:57.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969909+0000", 2024-09-18T04:56:57.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1950000000000002e-06 2024-09-18T04:56:57.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:57.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:57.000205+0000", 2024-09-18T04:56:57.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030295483000000002 2024-09-18T04:56:57.372 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.372 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.372 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.372 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72629 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.372 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.971766+0000", 2024-09-18T04:56:57.372 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022613172000000001, 2024-09-18T04:56:57.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028525425, 2024-09-18T04:56:57.372 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.372 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.372 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.372 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.373 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.373 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72629 2024-09-18T04:56:57.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.373 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971766+0000", 2024-09-18T04:56:57.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971766+0000", 2024-09-18T04:56:57.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971766+0000", 2024-09-18T04:56:57.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971769+0000", 2024-09-18T04:56:57.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9500000000000001e-06 2024-09-18T04:56:57.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971775+0000", 2024-09-18T04:56:57.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2960000000000004e-06 2024-09-18T04:56:57.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971780+0000", 2024-09-18T04:56:57.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8239999999999999e-06 2024-09-18T04:56:57.375 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.375 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.375 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.375 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72799 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.375 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.978472+0000", 2024-09-18T04:56:57.375 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.015907257000000001, 2024-09-18T04:56:57.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.021877079000000001, 2024-09-18T04:56:57.376 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.376 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.376 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.376 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.376 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.376 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72799 2024-09-18T04:56:57.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.376 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978472+0000", 2024-09-18T04:56:57.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978472+0000", 2024-09-18T04:56:57.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978472+0000", 2024-09-18T04:56:57.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978474+0000", 2024-09-18T04:56:57.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.751e-06 2024-09-18T04:56:57.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978482+0000", 2024-09-18T04:56:57.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0530000000000007e-06 2024-09-18T04:56:57.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978487+0000", 2024-09-18T04:56:57.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9949999999999996e-06 2024-09-18T04:56:57.378 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.378 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.379 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.379 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72961 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.379 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.984963+0000", 2024-09-18T04:56:57.379 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0094165559999999995, 2024-09-18T04:56:57.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015440967, 2024-09-18T04:56:57.379 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.379 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.379 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.379 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.379 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.379 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72961 2024-09-18T04:56:57.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.380 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984963+0000", 2024-09-18T04:56:57.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984963+0000", 2024-09-18T04:56:57.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984962+0000", 2024-09-18T04:56:57.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984965+0000", 2024-09-18T04:56:57.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3719999999999999e-06 2024-09-18T04:56:57.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984971+0000", 2024-09-18T04:56:57.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2639999999999997e-06 2024-09-18T04:56:57.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.984976+0000", 2024-09-18T04:56:57.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9919999999999998e-06 2024-09-18T04:56:57.382 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.382 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.382 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.382 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72464 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 608256~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.382 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969913+0000", 2024-09-18T04:56:57.382 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024465837000000001, 2024-09-18T04:56:57.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030551443000000001, 2024-09-18T04:56:57.382 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.383 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:56:57.383 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.383 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.383 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.383 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72464 2024-09-18T04:56:57.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.383 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969913+0000", 2024-09-18T04:56:57.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969913+0000", 2024-09-18T04:56:57.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969913+0000", 2024-09-18T04:56:57.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969915+0000", 2024-09-18T04:56:57.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2699999999999999e-06 2024-09-18T04:56:57.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969929+0000", 2024-09-18T04:56:57.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3378e-05 2024-09-18T04:56:57.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969935+0000", 2024-09-18T04:56:57.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.801e-06 2024-09-18T04:56:57.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:56:57.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:57.000408+0000", 2024-09-18T04:56:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030473706 2024-09-18T04:56:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72631 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.971785+0000", 2024-09-18T04:56:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022594507999999999, 2024-09-18T04:56:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028742243000000001, 2024-09-18T04:56:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.386 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72631 2024-09-18T04:56:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971785+0000", 2024-09-18T04:56:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971785+0000", 2024-09-18T04:56:57.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971784+0000", 2024-09-18T04:56:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971787+0000", 2024-09-18T04:56:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6680000000000001e-06 2024-09-18T04:56:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971795+0000", 2024-09-18T04:56:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7980000000000006e-06 2024-09-18T04:56:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971800+0000", 2024-09-18T04:56:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0420000000000002e-06 2024-09-18T04:56:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72800 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.389 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.978492+0000", 2024-09-18T04:56:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.015887206000000001, 2024-09-18T04:56:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022092805, 2024-09-18T04:56:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72800 2024-09-18T04:56:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978492+0000", 2024-09-18T04:56:57.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978492+0000", 2024-09-18T04:56:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978492+0000", 2024-09-18T04:56:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978494+0000", 2024-09-18T04:56:57.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4200000000000001e-06 2024-09-18T04:56:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978500+0000", 2024-09-18T04:56:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1500000000000004e-06 2024-09-18T04:56:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978505+0000", 2024-09-18T04:56:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0699999999999997e-06 2024-09-18T04:56:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72465 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969940+0000", 2024-09-18T04:56:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024439183, 2024-09-18T04:56:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030704149, 2024-09-18T04:56:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72465 2024-09-18T04:56:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969940+0000", 2024-09-18T04:56:57.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969940+0000", 2024-09-18T04:56:57.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969940+0000", 2024-09-18T04:56:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969942+0000", 2024-09-18T04:56:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.886e-06 2024-09-18T04:56:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969950+0000", 2024-09-18T04:56:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0700000000000007e-06 2024-09-18T04:56:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969956+0000", 2024-09-18T04:56:57.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1309999999999997e-06 2024-09-18T04:56:57.396 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.396 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.396 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.396 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72633 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.396 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.971804+0000", 2024-09-18T04:56:57.396 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022574975000000001, 2024-09-18T04:56:57.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028901429999999999, 2024-09-18T04:56:57.397 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.397 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.397 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.397 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.397 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.397 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72633 2024-09-18T04:56:57.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.397 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971804+0000", 2024-09-18T04:56:57.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971804+0000", 2024-09-18T04:56:57.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971804+0000", 2024-09-18T04:56:57.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971806+0000", 2024-09-18T04:56:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.48e-06 2024-09-18T04:56:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971813+0000", 2024-09-18T04:56:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1789999999999996e-06 2024-09-18T04:56:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971817+0000", 2024-09-18T04:56:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4920000000000004e-06 2024-09-18T04:56:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.399 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.400 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.400 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72801 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.400 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.978511+0000", 2024-09-18T04:56:57.400 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.015868611000000001, 2024-09-18T04:56:57.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022262041, 2024-09-18T04:56:57.400 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.400 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.400 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.400 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.401 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.401 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72801 2024-09-18T04:56:57.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.401 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978511+0000", 2024-09-18T04:56:57.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978511+0000", 2024-09-18T04:56:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978510+0000", 2024-09-18T04:56:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978513+0000", 2024-09-18T04:56:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2790000000000001e-06 2024-09-18T04:56:57.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978519+0000", 2024-09-18T04:56:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0990000000000004e-06 2024-09-18T04:56:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978523+0000", 2024-09-18T04:56:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6310000000000002e-06 2024-09-18T04:56:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.404 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72466 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.404 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969962+0000", 2024-09-18T04:56:57.404 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024416938999999999, 2024-09-18T04:56:57.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.030873011999999998, 2024-09-18T04:56:57.404 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.404 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.404 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.404 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.404 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.404 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72466 2024-09-18T04:56:57.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.404 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969962+0000", 2024-09-18T04:56:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969962+0000", 2024-09-18T04:56:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969962+0000", 2024-09-18T04:56:57.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969965+0000", 2024-09-18T04:56:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9809999999999999e-06 2024-09-18T04:56:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969974+0000", 2024-09-18T04:56:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9109999999999999e-06 2024-09-18T04:56:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969979+0000", 2024-09-18T04:56:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.215e-06 2024-09-18T04:56:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72636 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.971833+0000", 2024-09-18T04:56:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.022546243000000001, 2024-09-18T04:56:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.029059234, 2024-09-18T04:56:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.407 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72636 2024-09-18T04:56:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971833+0000", 2024-09-18T04:56:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971833+0000", 2024-09-18T04:56:57.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971832+0000", 2024-09-18T04:56:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971836+0000", 2024-09-18T04:56:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3239999999999999e-06 2024-09-18T04:56:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971845+0000", 2024-09-18T04:56:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3440000000000007e-06 2024-09-18T04:56:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.971851+0000", 2024-09-18T04:56:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5709999999999997e-06 2024-09-18T04:56:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72802 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.410 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.978528+0000", 2024-09-18T04:56:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.015851746999999999, 2024-09-18T04:56:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022428057000000001, 2024-09-18T04:56:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72802 2024-09-18T04:56:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978528+0000", 2024-09-18T04:56:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978528+0000", 2024-09-18T04:56:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978527+0000", 2024-09-18T04:56:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978529+0000", 2024-09-18T04:56:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1069999999999999e-06 2024-09-18T04:56:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978535+0000", 2024-09-18T04:56:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7420000000000003e-06 2024-09-18T04:56:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978540+0000", 2024-09-18T04:56:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7910000000000004e-06 2024-09-18T04:56:57.413 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72467 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.969984+0000", 2024-09-18T04:56:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024395460000000001, 2024-09-18T04:56:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.031029185000000001, 2024-09-18T04:56:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.414 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72467 2024-09-18T04:56:57.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.415 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969984+0000", 2024-09-18T04:56:57.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969984+0000", 2024-09-18T04:56:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969983+0000", 2024-09-18T04:56:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969990+0000", 2024-09-18T04:56:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0419999999999997e-06 2024-09-18T04:56:57.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.969997+0000", 2024-09-18T04:56:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7560000000000002e-06 2024-09-18T04:56:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970002+0000", 2024-09-18T04:56:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9470000000000002e-06 2024-09-18T04:56:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72638 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.973341+0000", 2024-09-18T04:56:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.021037977999999999, 2024-09-18T04:56:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027730621, 2024-09-18T04:56:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72638 2024-09-18T04:56:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973341+0000", 2024-09-18T04:56:57.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973341+0000", 2024-09-18T04:56:57.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973340+0000", 2024-09-18T04:56:57.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:57.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973346+0000", 2024-09-18T04:56:57.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5659999999999997e-06 2024-09-18T04:56:57.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973358+0000", 2024-09-18T04:56:57.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2201000000000001e-05 2024-09-18T04:56:57.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973367+0000", 2024-09-18T04:56:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4659999999999999e-06 2024-09-18T04:56:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72803 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.978544+0000", 2024-09-18T04:56:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.015835318000000001, 2024-09-18T04:56:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022585072000000001, 2024-09-18T04:56:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.421 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72803 2024-09-18T04:56:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978544+0000", 2024-09-18T04:56:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978544+0000", 2024-09-18T04:56:57.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978544+0000", 2024-09-18T04:56:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978546+0000", 2024-09-18T04:56:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1220000000000002e-06 2024-09-18T04:56:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978551+0000", 2024-09-18T04:56:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7370000000000003e-06 2024-09-18T04:56:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978561+0000", 2024-09-18T04:56:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2399999999999996e-06 2024-09-18T04:56:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72502 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.424 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.970007+0000", 2024-09-18T04:56:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024372195999999999, 2024-09-18T04:56:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.031179136, 2024-09-18T04:56:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72502 2024-09-18T04:56:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970007+0000", 2024-09-18T04:56:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970007+0000", 2024-09-18T04:56:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970007+0000", 2024-09-18T04:56:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970010+0000", 2024-09-18T04:56:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8959999999999999e-06 2024-09-18T04:56:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970016+0000", 2024-09-18T04:56:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1510000000000001e-06 2024-09-18T04:56:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970021+0000", 2024-09-18T04:56:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9350000000000002e-06 2024-09-18T04:56:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.427 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72675 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.973374+0000", 2024-09-18T04:56:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.021005382, 2024-09-18T04:56:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.027872109999999999, 2024-09-18T04:56:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.428 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72675 2024-09-18T04:56:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973374+0000", 2024-09-18T04:56:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973374+0000", 2024-09-18T04:56:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973373+0000", 2024-09-18T04:56:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973376+0000", 2024-09-18T04:56:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.993e-06 2024-09-18T04:56:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973385+0000", 2024-09-18T04:56:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6149999999999997e-06 2024-09-18T04:56:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973402+0000", 2024-09-18T04:56:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7220000000000001e-05 2024-09-18T04:56:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72804 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.978566+0000", 2024-09-18T04:56:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.015813427000000001, 2024-09-18T04:56:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.022736975, 2024-09-18T04:56:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.431 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72804 2024-09-18T04:56:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978566+0000", 2024-09-18T04:56:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978566+0000", 2024-09-18T04:56:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978566+0000", 2024-09-18T04:56:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978568+0000", 2024-09-18T04:56:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2879999999999999e-06 2024-09-18T04:56:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978574+0000", 2024-09-18T04:56:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7819999999999999e-06 2024-09-18T04:56:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.978578+0000", 2024-09-18T04:56:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5979999999999999e-06 2024-09-18T04:56:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.434 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72503 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.970025+0000", 2024-09-18T04:56:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.024354377999999999, 2024-09-18T04:56:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.031336320000000001, 2024-09-18T04:56:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72503 2024-09-18T04:56:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970025+0000", 2024-09-18T04:56:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970025+0000", 2024-09-18T04:56:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970025+0000", 2024-09-18T04:56:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970027+0000", 2024-09-18T04:56:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4260000000000002e-06 2024-09-18T04:56:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970034+0000", 2024-09-18T04:56:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5520000000000001e-06 2024-09-18T04:56:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.970039+0000", 2024-09-18T04:56:57.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1209999999999998e-06 2024-09-18T04:56:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72677 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.973410+0000", 2024-09-18T04:56:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.020969623, 2024-09-18T04:56:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.028008568000000001, 2024-09-18T04:56:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.438 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72677 2024-09-18T04:56:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973410+0000", 2024-09-18T04:56:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973410+0000", 2024-09-18T04:56:57.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973409+0000", 2024-09-18T04:56:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:56:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973412+0000", 2024-09-18T04:56:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.999e-06 2024-09-18T04:56:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973420+0000", 2024-09-18T04:56:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5109999999999997e-06 2024-09-18T04:56:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.973425+0000", 2024-09-18T04:56:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2680000000000002e-06 2024-09-18T04:56:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:72840 3.1 3.77c08a81 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:56:57.441 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.980556+0000", 2024-09-18T04:56:57.442 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.013823675000000001, 2024-09-18T04:56:57.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.020921249999999999, 2024-09-18T04:56:57.442 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:56:57.442 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:56:57.442 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:56:57.442 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:56:57.442 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:56:57.442 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 72840 2024-09-18T04:56:57.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.442 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:56:57.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:56:57.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.980556+0000", 2024-09-18T04:56:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:56:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.980556+0000", 2024-09-18T04:56:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:56:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:56:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.980555+0000", 2024-09-18T04:56:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:56:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:56:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.980560+0000", 2024-09-18T04:56:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7409999999999998e-06 2024-09-18T04:56:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:56:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.980572+0000", 2024-09-18T04:56:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1995e-05 2024-09-18T04:56:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:56:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:56:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:56:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.980580+0000", 2024-09-18T04:56:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3379999999999999e-06 2024-09-18T04:56:57.444 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.445 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:56:57.445 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.445 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:56:57.445 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:56:57.445 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 122 2024-09-18T04:56:57.445 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:56:57.447 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:56:57.447 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:56:57.447 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:56:57.447 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:56:57.448 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:56:57.448 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:56:57.448 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:56:57.449 INFO:teuthology.orchestra.run.smithi002.stdout:2421 write 0x14f14a3 thru 0x14fcfe7 (0xbb45 bytes) 2024-09-18T04:56:57.449 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:56:57.449 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:56:57.449 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:56:57.450 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:56:57.450 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:56:57.450 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:56:57.451 INFO:teuthology.orchestra.run.smithi002.stdout:2423 read 0x57b91dd thru 0x57bcdd6 (0x3bfa bytes) 2024-09-18T04:56:57.451 INFO:teuthology.orchestra.run.smithi002.stdout:2425 trunc from 0xccd6b0f to 0x8a375e7 2024-09-18T04:56:57.451 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:56:57.451 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:56:57.451 INFO:teuthology.orchestra.run.smithi002.stdout:2426 write 0x28f4e8f thru 0x28f5b31 (0xca3 bytes) 2024-09-18T04:56:57.452 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:56:57.452 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:56:57.452 INFO:teuthology.orchestra.run.smithi002.stdout:2429 write 0xa7a26e0 thru 0xa7b1471 (0xed92 bytes) 2024-09-18T04:56:57.452 INFO:teuthology.orchestra.run.smithi002.stdout:2431 trunc from 0xa7b1472 to 0x47f52fe 2024-09-18T04:56:57.454 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:56:57.455 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-18T04:56:57.455 INFO:teuthology.orchestra.run.smithi002.stdout:2433 write 0xca96766 thru 0xca9bee0 (0x577b bytes) 2024-09-18T04:56:57.456 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-09-18T04:56:56.275603+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":808160403,"num_objects":793,"num_object_clones":110,"num_object_copies":1586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":793,"num_whiteouts":104,"num_read":83872,"num_read_kb":268852,"num_write":8507,"num_write_kb":182736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":4916923,"num_keys_recovered":29,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6459,"ondisk_log_size":6459,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":83,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":307268,"kb_used_data":29660,"kb_used_omap":12,"kb_used_meta":277491,"kb_avail":754667452,"statfs":{"total":773094113280,"available":772779470848,"internally_reserved":0,"allocated":30371840,"data_stored":192489861,"data_compressed":4770432,"data_compressed_allocated":14757888,"data_compressed_original":181345114,"omap_allocated":12742,"internal_metadata":284151354},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24085528,"num_objects":182,"num_object_clones":0,"num_object_copies":364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":182,"num_whiteouts":0,"num_read":25477,"num_read_kb":28998,"num_write":1545,"num_write_kb":4551,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":1661384,"num_keys_recovered":20,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000524"},"pg_stats":[{"pgid":"3.1f","version":"48'204","reported_seq":1673,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.141412+0000","last_change":"2024-09-18T04:56:23.130295+0000","last_active":"2024-09-18T04:56:54.141412+0000","last_peered":"2024-09-18T04:56:54.141412+0000","last_clean":"2024-09-18T04:56:54.141412+0000","last_became_active":"2024-09-18T04:56:21.132342+0000","last_became_peered":"2024-09-18T04:56:21.132342+0000","last_unstale":"2024-09-18T04:56:54.141412+0000","last_undegraded":"2024-09-18T04:56:54.141412+0000","last_fullsized":"2024-09-18T04:56:54.141412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:43:47.128460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011070279999999999,"stat_sum":{"num_bytes":31101394,"num_objects":33,"num_object_clones":3,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":3,"num_read":2457,"num_read_kb":7152,"num_write":202,"num_write_kb":4328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"48'302","reported_seq":417,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.150201+0000","last_change":"2024-09-18T04:56:23.582447+0000","last_active":"2024-09-18T04:56:54.150201+0000","last_peered":"2024-09-18T04:56:54.150201+0000","last_clean":"2024-09-18T04:56:54.150201+0000","last_became_active":"2024-09-18T04:56:21.126431+0000","last_became_peered":"2024-09-18T04:56:21.126431+0000","last_unstale":"2024-09-18T04:56:54.150201+0000","last_undegraded":"2024-09-18T04:56:54.150201+0000","last_fullsized":"2024-09-18T04:56:54.150201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:59.671751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062644600000000001,"stat_sum":{"num_bytes":21996943,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":3,"num_read":1122,"num_read_kb":4828,"num_write":461,"num_write_kb":4345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"48'157","reported_seq":4926,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.288773+0000","last_change":"2024-09-18T04:56:23.137064+0000","last_active":"2024-09-18T04:56:54.288773+0000","last_peered":"2024-09-18T04:56:54.288773+0000","last_clean":"2024-09-18T04:56:54.288773+0000","last_became_active":"2024-09-18T04:56:21.131613+0000","last_became_peered":"2024-09-18T04:56:21.131613+0000","last_unstale":"2024-09-18T04:56:54.288773+0000","last_undegraded":"2024-09-18T04:56:54.288773+0000","last_fullsized":"2024-09-18T04:56:54.288773+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:15:27.419409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000455248,"stat_sum":{"num_bytes":51908295,"num_objects":28,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":1,"num_read":4818,"num_read_kb":17684,"num_write":172,"num_write_kb":17365,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"48'369","reported_seq":3463,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.081398+0000","last_change":"2024-09-18T04:56:23.137422+0000","last_active":"2024-09-18T04:56:54.081398+0000","last_peered":"2024-09-18T04:56:54.081398+0000","last_clean":"2024-09-18T04:56:54.081398+0000","last_became_active":"2024-09-18T04:56:21.129551+0000","last_became_peered":"2024-09-18T04:56:21.129551+0000","last_unstale":"2024-09-18T04:56:54.081398+0000","last_undegraded":"2024-09-18T04:56:54.081398+0000","last_fullsized":"2024-09-18T04:56:54.081398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:38:38.231233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081678499999999999,"stat_sum":{"num_bytes":10318971,"num_objects":18,"num_object_clones":1,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":2,"num_read":4485,"num_read_kb":3970,"num_write":550,"num_write_kb":3338,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"48'110","reported_seq":206,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:53.993990+0000","last_change":"2024-09-18T04:56:23.130841+0000","last_active":"2024-09-18T04:56:53.993990+0000","last_peered":"2024-09-18T04:56:53.993990+0000","last_clean":"2024-09-18T04:56:53.993990+0000","last_became_active":"2024-09-18T04:56:21.129660+0000","last_became_peered":"2024-09-18T04:56:21.129660+0000","last_unstale":"2024-09-18T04:56:53.993990+0000","last_undegraded":"2024-09-18T04:56:53.993990+0000","last_fullsized":"2024-09-18T04:56:53.993990+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":110,"log_dups_size":0,"ondisk_log_size":110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:43.007677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014550170000000001,"stat_sum":{"num_bytes":3275804,"num_objects":20,"num_object_clones":1,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":3,"num_read":96,"num_read_kb":684,"num_write":118,"num_write_kb":1218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"48'118","reported_seq":952,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.147059+0000","last_change":"2024-09-18T04:56:23.133311+0000","last_active":"2024-09-18T04:56:54.147059+0000","last_peered":"2024-09-18T04:56:54.147059+0000","last_clean":"2024-09-18T04:56:54.147059+0000","last_became_active":"2024-09-18T04:56:21.127888+0000","last_became_peered":"2024-09-18T04:56:21.127888+0000","last_unstale":"2024-09-18T04:56:54.147059+0000","last_undegraded":"2024-09-18T04:56:54.147059+0000","last_fullsized":"2024-09-18T04:56:54.147059+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:35:20.652713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087073300000000005,"stat_sum":{"num_bytes":11554148,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":4,"num_read":843,"num_read_kb":4774,"num_write":130,"num_write_kb":14322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"48'109","reported_seq":2510,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:53.994451+0000","last_change":"2024-09-18T04:56:23.130039+0000","last_active":"2024-09-18T04:56:53.994451+0000","last_peered":"2024-09-18T04:56:53.994451+0000","last_clean":"2024-09-18T04:56:53.994451+0000","last_became_active":"2024-09-18T04:56:21.125153+0000","last_became_peered":"2024-09-18T04:56:21.125153+0000","last_unstale":"2024-09-18T04:56:53.994451+0000","last_undegraded":"2024-09-18T04:56:53.994451+0000","last_fullsized":"2024-09-18T04:56:53.994451+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:54:02.903679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012370669999999999,"stat_sum":{"num_bytes":30585493,"num_objects":32,"num_object_clones":5,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":3,"num_read":2711,"num_read_kb":19227,"num_write":144,"num_write_kb":8892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"48'158","reported_seq":1167,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.255851+0000","last_change":"2024-09-18T04:56:23.582074+0000","last_active":"2024-09-18T04:56:54.255851+0000","last_peered":"2024-09-18T04:56:54.255851+0000","last_clean":"2024-09-18T04:56:54.255851+0000","last_became_active":"2024-09-18T04:56:21.127959+0000","last_became_peered":"2024-09-18T04:56:21.127959+0000","last_unstale":"2024-09-18T04:56:54.255851+0000","last_undegraded":"2024-09-18T04:56:54.255851+0000","last_fullsized":"2024-09-18T04:56:54.255851+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":158,"log_dups_size":0,"ondisk_log_size":158,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:17:32.019845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053040699999999999,"stat_sum":{"num_bytes":24291507,"num_objects":27,"num_object_clones":2,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":5,"num_read":1287,"num_read_kb":5834,"num_write":226,"num_write_kb":6975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"48'102","reported_seq":469,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.033949+0000","last_change":"2024-09-18T04:56:23.130832+0000","last_active":"2024-09-18T04:56:54.033949+0000","last_peered":"2024-09-18T04:56:54.033949+0000","last_clean":"2024-09-18T04:56:54.033949+0000","last_became_active":"2024-09-18T04:56:21.130316+0000","last_became_peered":"2024-09-18T04:56:21.130316+0000","last_unstale":"2024-09-18T04:56:54.033949+0000","last_undegraded":"2024-09-18T04:56:54.033949+0000","last_fullsized":"2024-09-18T04:56:54.033949+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:12:05.411040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001612023,"stat_sum":{"num_bytes":8368055,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":2,"num_read":976,"num_read_kb":2020,"num_write":124,"num_write_kb":1371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"49'455","reported_seq":2871,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.253398+0000","last_change":"2024-09-18T04:56:23.136846+0000","last_active":"2024-09-18T04:56:54.253398+0000","last_peered":"2024-09-18T04:56:54.253398+0000","last_clean":"2024-09-18T04:56:54.253398+0000","last_became_active":"2024-09-18T04:56:21.131133+0000","last_became_peered":"2024-09-18T04:56:21.131133+0000","last_unstale":"2024-09-18T04:56:54.253398+0000","last_undegraded":"2024-09-18T04:56:54.253398+0000","last_fullsized":"2024-09-18T04:56:54.253398+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:01:57.036904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025847199999999999,"stat_sum":{"num_bytes":55934869,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":8,"num_read":4306,"num_read_kb":12795,"num_write":760,"num_write_kb":6792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"48'202","reported_seq":2374,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.143286+0000","last_change":"2024-09-18T04:56:23.142657+0000","last_active":"2024-09-18T04:56:54.143286+0000","last_peered":"2024-09-18T04:56:54.143286+0000","last_clean":"2024-09-18T04:56:54.143286+0000","last_became_active":"2024-09-18T04:56:21.131979+0000","last_became_peered":"2024-09-18T04:56:21.131979+0000","last_unstale":"2024-09-18T04:56:54.143286+0000","last_undegraded":"2024-09-18T04:56:54.143286+0000","last_fullsized":"2024-09-18T04:56:54.143286+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:16:43.303511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029355000000000001,"stat_sum":{"num_bytes":61608263,"num_objects":31,"num_object_clones":5,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":5,"num_read":3286,"num_read_kb":16999,"num_write":201,"num_write_kb":2173,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"48'129","reported_seq":444,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:53.994599+0000","last_change":"2024-09-18T04:56:23.132876+0000","last_active":"2024-09-18T04:56:53.994599+0000","last_peered":"2024-09-18T04:56:53.994599+0000","last_clean":"2024-09-18T04:56:53.994599+0000","last_became_active":"2024-09-18T04:56:21.134979+0000","last_became_peered":"2024-09-18T04:56:21.134979+0000","last_unstale":"2024-09-18T04:56:53.994599+0000","last_undegraded":"2024-09-18T04:56:53.994599+0000","last_fullsized":"2024-09-18T04:56:53.994599+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:36:24.881782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038004100000000002,"stat_sum":{"num_bytes":4521370,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":560,"num_read_kb":3778,"num_write":183,"num_write_kb":2299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":47,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-18T04:56:52.320262+0000","last_change":"2024-09-18T04:56:12.390222+0000","last_active":"2024-09-18T04:56:52.320262+0000","last_peered":"2024-09-18T04:56:52.320262+0000","last_clean":"2024-09-18T04:56:52.320262+0000","last_became_active":"2024-09-18T04:56:10.401272+0000","last_became_peered":"2024-09-18T04:56:10.401272+0000","last_unstale":"2024-09-18T04:56:52.320262+0000","last_undegraded":"2024-09-18T04:56:52.320262+0000","last_fullsized":"2024-09-18T04:56:52.320262+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:33:50.692096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063436799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"48'195","reported_seq":4146,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.289660+0000","last_change":"2024-09-18T04:56:23.137022+0000","last_active":"2024-09-18T04:56:54.289660+0000","last_peered":"2024-09-18T04:56:54.289660+0000","last_clean":"2024-09-18T04:56:54.289660+0000","last_became_active":"2024-09-18T04:56:21.129964+0000","last_became_peered":"2024-09-18T04:56:21.129964+0000","last_unstale":"2024-09-18T04:56:54.289660+0000","last_undegraded":"2024-09-18T04:56:54.289660+0000","last_fullsized":"2024-09-18T04:56:54.289660+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":195,"log_dups_size":0,"ondisk_log_size":195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:29:32.843883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043204099999999998,"stat_sum":{"num_bytes":29471589,"num_objects":27,"num_object_clones":2,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":3,"num_read":4647,"num_read_kb":13708,"num_write":312,"num_write_kb":10280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":47,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-18T04:56:52.320287+0000","last_change":"2024-09-18T04:56:12.389971+0000","last_active":"2024-09-18T04:56:52.320287+0000","last_peered":"2024-09-18T04:56:52.320287+0000","last_clean":"2024-09-18T04:56:52.320287+0000","last_became_active":"2024-09-18T04:56:10.412595+0000","last_became_peered":"2024-09-18T04:56:10.412595+0000","last_unstale":"2024-09-18T04:56:52.320287+0000","last_undegraded":"2024-09-18T04:56:52.320287+0000","last_fullsized":"2024-09-18T04:56:52.320287+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:45.638420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038576799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"48'397","reported_seq":3989,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.122324+0000","last_change":"2024-09-18T04:56:23.129727+0000","last_active":"2024-09-18T04:56:54.122324+0000","last_peered":"2024-09-18T04:56:54.122324+0000","last_clean":"2024-09-18T04:56:54.122324+0000","last_became_active":"2024-09-18T04:56:21.122488+0000","last_became_peered":"2024-09-18T04:56:21.122488+0000","last_unstale":"2024-09-18T04:56:54.122324+0000","last_undegraded":"2024-09-18T04:56:54.122324+0000","last_fullsized":"2024-09-18T04:56:54.122324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":397,"log_dups_size":0,"ondisk_log_size":397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:06:03.886036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072197,"stat_sum":{"num_bytes":32947096,"num_objects":34,"num_object_clones":5,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":4,"num_read":5119,"num_read_kb":16084,"num_write":660,"num_write_kb":4960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":48,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:53.254586+0000","last_change":"2024-09-18T04:56:12.412298+0000","last_active":"2024-09-18T04:56:53.254586+0000","last_peered":"2024-09-18T04:56:53.254586+0000","last_clean":"2024-09-18T04:56:53.254586+0000","last_became_active":"2024-09-18T04:56:10.399718+0000","last_became_peered":"2024-09-18T04:56:10.399718+0000","last_unstale":"2024-09-18T04:56:53.254586+0000","last_undegraded":"2024-09-18T04:56:53.254586+0000","last_fullsized":"2024-09-18T04:56:53.254586+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:41:31.956685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024059099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"48'331","reported_seq":2582,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.138328+0000","last_change":"2024-09-18T04:56:23.582182+0000","last_active":"2024-09-18T04:56:54.138328+0000","last_peered":"2024-09-18T04:56:54.138328+0000","last_clean":"2024-09-18T04:56:54.138328+0000","last_became_active":"2024-09-18T04:56:21.125785+0000","last_became_peered":"2024-09-18T04:56:21.125785+0000","last_unstale":"2024-09-18T04:56:54.138328+0000","last_undegraded":"2024-09-18T04:56:54.138328+0000","last_fullsized":"2024-09-18T04:56:54.138328+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":331,"log_dups_size":0,"ondisk_log_size":331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:35:40.684580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048263400000000001,"stat_sum":{"num_bytes":23693862,"num_objects":31,"num_object_clones":2,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":6,"num_read":3641,"num_read_kb":13586,"num_write":460,"num_write_kb":1236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-18T04:56:52.320228+0000","last_change":"2024-09-18T04:56:46.254241+0000","last_active":"2024-09-18T04:56:52.320228+0000","last_peered":"2024-09-18T04:56:52.320228+0000","last_clean":"2024-09-18T04:56:52.320228+0000","last_became_active":"2024-09-18T04:56:46.253894+0000","last_became_peered":"2024-09-18T04:56:46.253894+0000","last_unstale":"2024-09-18T04:56:52.320228+0000","last_undegraded":"2024-09-18T04:56:52.320228+0000","last_fullsized":"2024-09-18T04:56:52.320228+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:58:42.098099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045144100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"48'122","reported_seq":3663,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:53.994497+0000","last_change":"2024-09-18T04:56:23.129577+0000","last_active":"2024-09-18T04:56:53.994497+0000","last_peered":"2024-09-18T04:56:53.994497+0000","last_clean":"2024-09-18T04:56:53.994497+0000","last_became_active":"2024-09-18T04:56:21.127750+0000","last_became_peered":"2024-09-18T04:56:21.127750+0000","last_unstale":"2024-09-18T04:56:53.994497+0000","last_undegraded":"2024-09-18T04:56:53.994497+0000","last_fullsized":"2024-09-18T04:56:53.994497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T04:58:18.075339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039168900000000002,"stat_sum":{"num_bytes":44832536,"num_objects":21,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":3594,"num_read_kb":21008,"num_write":148,"num_write_kb":29174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"48'84","reported_seq":2517,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.066276+0000","last_change":"2024-09-18T04:56:23.130445+0000","last_active":"2024-09-18T04:56:54.066276+0000","last_peered":"2024-09-18T04:56:54.066276+0000","last_clean":"2024-09-18T04:56:54.066276+0000","last_became_active":"2024-09-18T04:56:21.130514+0000","last_became_peered":"2024-09-18T04:56:21.130514+0000","last_unstale":"2024-09-18T04:56:54.066276+0000","last_undegraded":"2024-09-18T04:56:54.066276+0000","last_fullsized":"2024-09-18T04:56:54.066276+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:06:37.491009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001360497,"stat_sum":{"num_bytes":24468790,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":3,"num_read":2432,"num_read_kb":7289,"num_write":93,"num_write_kb":6508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":192,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:53.622670+0000","last_change":"2024-09-18T04:56:11.428051+0000","last_active":"2024-09-18T04:56:53.622670+0000","last_peered":"2024-09-18T04:56:53.622670+0000","last_clean":"2024-09-18T04:56:53.622670+0000","last_became_active":"2024-09-18T04:56:11.427170+0000","last_became_peered":"2024-09-18T04:56:11.427170+0000","last_unstale":"2024-09-18T04:56:53.622670+0000","last_undegraded":"2024-09-18T04:56:53.622670+0000","last_fullsized":"2024-09-18T04:56:53.622670+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_clean_scrub_stamp":"2024-09-18T04:56:10.373785+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:16:08.102627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":48,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-18T04:56:52.320554+0000","last_change":"2024-09-18T04:56:12.413125+0000","last_active":"2024-09-18T04:56:52.320554+0000","last_peered":"2024-09-18T04:56:52.320554+0000","last_clean":"2024-09-18T04:56:52.320554+0000","last_became_active":"2024-09-18T04:56:10.401641+0000","last_became_peered":"2024-09-18T04:56:10.401641+0000","last_unstale":"2024-09-18T04:56:52.320554+0000","last_undegraded":"2024-09-18T04:56:52.320554+0000","last_fullsized":"2024-09-18T04:56:52.320554+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:19:46.375505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000622431,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"48'155","reported_seq":5700,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.057283+0000","last_change":"2024-09-18T04:56:23.129808+0000","last_active":"2024-09-18T04:56:54.057283+0000","last_peered":"2024-09-18T04:56:54.057283+0000","last_clean":"2024-09-18T04:56:54.057283+0000","last_became_active":"2024-09-18T04:56:21.122785+0000","last_became_peered":"2024-09-18T04:56:21.122785+0000","last_unstale":"2024-09-18T04:56:54.057283+0000","last_undegraded":"2024-09-18T04:56:54.057283+0000","last_fullsized":"2024-09-18T04:56:54.057283+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:05:30.391004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010536390000000001,"stat_sum":{"num_bytes":34784736,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":5,"num_read":6347,"num_read_kb":13923,"num_write":144,"num_write_kb":3181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":50,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:53.622787+0000","last_change":"2024-09-18T04:56:12.398631+0000","last_active":"2024-09-18T04:56:53.622787+0000","last_peered":"2024-09-18T04:56:53.622787+0000","last_clean":"2024-09-18T04:56:53.622787+0000","last_became_active":"2024-09-18T04:56:10.412614+0000","last_became_peered":"2024-09-18T04:56:10.412614+0000","last_unstale":"2024-09-18T04:56:53.622787+0000","last_undegraded":"2024-09-18T04:56:53.622787+0000","last_fullsized":"2024-09-18T04:56:53.622787+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:44:13.401320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075126299999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"48'100","reported_seq":210,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.123080+0000","last_change":"2024-09-18T04:56:23.133345+0000","last_active":"2024-09-18T04:56:54.123080+0000","last_peered":"2024-09-18T04:56:54.123080+0000","last_clean":"2024-09-18T04:56:54.123080+0000","last_became_active":"2024-09-18T04:56:21.128793+0000","last_became_peered":"2024-09-18T04:56:21.128793+0000","last_unstale":"2024-09-18T04:56:54.123080+0000","last_undegraded":"2024-09-18T04:56:54.123080+0000","last_fullsized":"2024-09-18T04:56:54.123080+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:10:36.658665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067161900000000004,"stat_sum":{"num_bytes":6059382,"num_objects":24,"num_object_clones":1,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":3,"num_read":94,"num_read_kb":637,"num_write":100,"num_write_kb":963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":48,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:53.633906+0000","last_change":"2024-09-18T04:56:12.390044+0000","last_active":"2024-09-18T04:56:53.633906+0000","last_peered":"2024-09-18T04:56:53.633906+0000","last_clean":"2024-09-18T04:56:53.633906+0000","last_became_active":"2024-09-18T04:56:10.413042+0000","last_became_peered":"2024-09-18T04:56:10.413042+0000","last_unstale":"2024-09-18T04:56:53.633906+0000","last_undegraded":"2024-09-18T04:56:53.633906+0000","last_fullsized":"2024-09-18T04:56:53.633906+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:46:11.074243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"48'124","reported_seq":1811,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.197460+0000","last_change":"2024-09-18T04:56:23.581937+0000","last_active":"2024-09-18T04:56:54.197460+0000","last_peered":"2024-09-18T04:56:54.197460+0000","last_clean":"2024-09-18T04:56:54.197460+0000","last_became_active":"2024-09-18T04:56:21.130465+0000","last_became_peered":"2024-09-18T04:56:21.130465+0000","last_unstale":"2024-09-18T04:56:54.197460+0000","last_undegraded":"2024-09-18T04:56:54.197460+0000","last_fullsized":"2024-09-18T04:56:54.197460+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:38:38.231233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037357100000000001,"stat_sum":{"num_bytes":26333788,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":3,"num_read":1985,"num_read_kb":4505,"num_write":117,"num_write_kb":1605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":48,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:53.622665+0000","last_change":"2024-09-18T04:56:12.398299+0000","last_active":"2024-09-18T04:56:53.622665+0000","last_peered":"2024-09-18T04:56:53.622665+0000","last_clean":"2024-09-18T04:56:53.622665+0000","last_became_active":"2024-09-18T04:56:10.413140+0000","last_became_peered":"2024-09-18T04:56:10.413140+0000","last_unstale":"2024-09-18T04:56:53.622665+0000","last_undegraded":"2024-09-18T04:56:53.622665+0000","last_fullsized":"2024-09-18T04:56:53.622665+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:37:11.125495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041592799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"48'106","reported_seq":1291,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.232417+0000","last_change":"2024-09-18T04:56:23.582318+0000","last_active":"2024-09-18T04:56:54.232417+0000","last_peered":"2024-09-18T04:56:54.232417+0000","last_clean":"2024-09-18T04:56:54.232417+0000","last_became_active":"2024-09-18T04:56:21.125295+0000","last_became_peered":"2024-09-18T04:56:21.125295+0000","last_unstale":"2024-09-18T04:56:54.232417+0000","last_undegraded":"2024-09-18T04:56:54.232417+0000","last_fullsized":"2024-09-18T04:56:54.232417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:29:59.427218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053349799999999998,"stat_sum":{"num_bytes":20699906,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":6,"num_read":1172,"num_read_kb":3976,"num_write":106,"num_write_kb":2866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"49'206","reported_seq":6084,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.276026+0000","last_change":"2024-09-18T04:56:23.133113+0000","last_active":"2024-09-18T04:56:54.276026+0000","last_peered":"2024-09-18T04:56:54.276026+0000","last_clean":"2024-09-18T04:56:54.276026+0000","last_became_active":"2024-09-18T04:56:21.128135+0000","last_became_peered":"2024-09-18T04:56:21.128135+0000","last_unstale":"2024-09-18T04:56:54.276026+0000","last_undegraded":"2024-09-18T04:56:54.276026+0000","last_fullsized":"2024-09-18T04:56:54.276026+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:46:04.362749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000513702,"stat_sum":{"num_bytes":29115184,"num_objects":28,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":3,"num_read":7068,"num_read_kb":13998,"num_write":205,"num_write_kb":10925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"48'122","reported_seq":572,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.063396+0000","last_change":"2024-09-18T04:56:23.641225+0000","last_active":"2024-09-18T04:56:54.063396+0000","last_peered":"2024-09-18T04:56:54.063396+0000","last_clean":"2024-09-18T04:56:54.063396+0000","last_became_active":"2024-09-18T04:56:21.127832+0000","last_became_peered":"2024-09-18T04:56:21.127832+0000","last_unstale":"2024-09-18T04:56:54.063396+0000","last_undegraded":"2024-09-18T04:56:54.063396+0000","last_fullsized":"2024-09-18T04:56:54.063396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":122,"log_dups_size":0,"ondisk_log_size":122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:23:15.821734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029947799999999999,"stat_sum":{"num_bytes":5855451,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":2,"num_read":545,"num_read_kb":1910,"num_write":138,"num_write_kb":1433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"48'234","reported_seq":2453,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.080117+0000","last_change":"2024-09-18T04:56:23.582528+0000","last_active":"2024-09-18T04:56:54.080117+0000","last_peered":"2024-09-18T04:56:54.080117+0000","last_clean":"2024-09-18T04:56:54.080117+0000","last_became_active":"2024-09-18T04:56:21.127444+0000","last_became_peered":"2024-09-18T04:56:21.127444+0000","last_unstale":"2024-09-18T04:56:54.080117+0000","last_undegraded":"2024-09-18T04:56:54.080117+0000","last_fullsized":"2024-09-18T04:56:54.080117+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":234,"log_dups_size":0,"ondisk_log_size":234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:15:41.251274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060817200000000003,"stat_sum":{"num_bytes":14166252,"num_objects":23,"num_object_clones":4,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":4,"num_read":2871,"num_read_kb":8218,"num_write":330,"num_write_kb":2981,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"48'146","reported_seq":222,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.071741+0000","last_change":"2024-09-18T04:56:23.136791+0000","last_active":"2024-09-18T04:56:54.071741+0000","last_peered":"2024-09-18T04:56:54.071741+0000","last_clean":"2024-09-18T04:56:54.071741+0000","last_became_active":"2024-09-18T04:56:21.131676+0000","last_became_peered":"2024-09-18T04:56:21.131676+0000","last_unstale":"2024-09-18T04:56:54.071741+0000","last_undegraded":"2024-09-18T04:56:54.071741+0000","last_fullsized":"2024-09-18T04:56:54.071741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":146,"log_dups_size":0,"ondisk_log_size":146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:34:36.563239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000302343,"stat_sum":{"num_bytes":8047196,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":219,"num_read_kb":524,"num_write":170,"num_write_kb":1389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"49'201","reported_seq":2967,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.256274+0000","last_change":"2024-09-18T04:56:23.143279+0000","last_active":"2024-09-18T04:56:54.256274+0000","last_peered":"2024-09-18T04:56:54.256274+0000","last_clean":"2024-09-18T04:56:54.256274+0000","last_became_active":"2024-09-18T04:56:21.133686+0000","last_became_peered":"2024-09-18T04:56:21.133686+0000","last_unstale":"2024-09-18T04:56:54.256274+0000","last_undegraded":"2024-09-18T04:56:54.256274+0000","last_fullsized":"2024-09-18T04:56:54.256274+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:50:51.011660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087309599999999996,"stat_sum":{"num_bytes":30906923,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":5,"num_read":4100,"num_read_kb":7836,"num_write":200,"num_write_kb":4311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"49'520","reported_seq":724,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.279244+0000","last_change":"2024-09-18T04:56:23.143532+0000","last_active":"2024-09-18T04:56:54.279244+0000","last_peered":"2024-09-18T04:56:54.279244+0000","last_clean":"2024-09-18T04:56:54.279244+0000","last_became_active":"2024-09-18T04:56:21.132364+0000","last_became_peered":"2024-09-18T04:56:21.132364+0000","last_unstale":"2024-09-18T04:56:54.279244+0000","last_undegraded":"2024-09-18T04:56:54.279244+0000","last_fullsized":"2024-09-18T04:56:54.279244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:25:44.430438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010190329999999999,"stat_sum":{"num_bytes":13948704,"num_objects":21,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":3,"num_read":2726,"num_read_kb":4777,"num_write":938,"num_write_kb":1652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"48'154","reported_seq":2071,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.099918+0000","last_change":"2024-09-18T04:56:23.143500+0000","last_active":"2024-09-18T04:56:54.099918+0000","last_peered":"2024-09-18T04:56:54.099918+0000","last_clean":"2024-09-18T04:56:54.099918+0000","last_became_active":"2024-09-18T04:56:21.130105+0000","last_became_peered":"2024-09-18T04:56:21.130105+0000","last_unstale":"2024-09-18T04:56:54.099918+0000","last_undegraded":"2024-09-18T04:56:54.099918+0000","last_fullsized":"2024-09-18T04:56:54.099918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":154,"log_dups_size":0,"ondisk_log_size":154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:31:55.392378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010487090000000001,"stat_sum":{"num_bytes":33696645,"num_objects":26,"num_object_clones":7,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":6,"num_read":2412,"num_read_kb":13697,"num_write":152,"num_write_kb":1246,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"48'220","reported_seq":499,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.042797+0000","last_change":"2024-09-18T04:56:23.641201+0000","last_active":"2024-09-18T04:56:54.042797+0000","last_peered":"2024-09-18T04:56:54.042797+0000","last_clean":"2024-09-18T04:56:54.042797+0000","last_became_active":"2024-09-18T04:56:21.131202+0000","last_became_peered":"2024-09-18T04:56:21.131202+0000","last_unstale":"2024-09-18T04:56:54.042797+0000","last_undegraded":"2024-09-18T04:56:54.042797+0000","last_fullsized":"2024-09-18T04:56:54.042797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":220,"log_dups_size":0,"ondisk_log_size":220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:17:35.438723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000304567,"stat_sum":{"num_bytes":46534640,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":2,"num_read":850,"num_read_kb":3648,"num_write":250,"num_write_kb":3041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"49'289","reported_seq":3447,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.255975+0000","last_change":"2024-09-18T04:56:23.142612+0000","last_active":"2024-09-18T04:56:54.255975+0000","last_peered":"2024-09-18T04:56:54.255975+0000","last_clean":"2024-09-18T04:56:54.255975+0000","last_became_active":"2024-09-18T04:56:21.130594+0000","last_became_peered":"2024-09-18T04:56:21.130594+0000","last_unstale":"2024-09-18T04:56:54.255975+0000","last_undegraded":"2024-09-18T04:56:54.255975+0000","last_fullsized":"2024-09-18T04:56:54.255975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":289,"log_dups_size":0,"ondisk_log_size":289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:46:34.324509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030763399999999999,"stat_sum":{"num_bytes":42789723,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":2,"num_read":4804,"num_read_kb":11100,"num_write":336,"num_write_kb":15662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"48'98","reported_seq":1213,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-18T04:56:54.081526+0000","last_change":"2024-09-18T04:56:23.129633+0000","last_active":"2024-09-18T04:56:54.081526+0000","last_peered":"2024-09-18T04:56:54.081526+0000","last_clean":"2024-09-18T04:56:54.081526+0000","last_became_active":"2024-09-18T04:56:21.129597+0000","last_became_peered":"2024-09-18T04:56:21.129597+0000","last_unstale":"2024-09-18T04:56:54.081526+0000","last_undegraded":"2024-09-18T04:56:54.081526+0000","last_fullsized":"2024-09-18T04:56:54.081526+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:50.665769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059848499999999995,"stat_sum":{"num_bytes":11195853,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":4,"num_read":1091,"num_read_kb":7595,"num_write":97,"num_write_kb":1604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"48'120","reported_seq":330,"reported_epoch":49,"state":"creating+peering","last_fresh":"2024-09-18T04:56:54.148909+0000","last_change":"2024-09-18T04:56:54.293919+0000","last_active":"2024-09-18T04:56:54.293919+0000","last_peered":"2024-09-18T04:56:54.148909+0000","last_clean":"2024-09-18T04:56:51.001826+0000","last_became_active":"2024-09-18T04:56:53.625562+0000","last_became_peered":"2024-09-18T04:56:53.625562+0000","last_unstale":"2024-09-18T04:56:54.148909+0000","last_undegraded":"2024-09-18T04:56:54.148909+0000","last_fullsized":"2024-09-18T04:56:54.148909+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":120,"log_dups_size":0,"ondisk_log_size":120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:01:08.639453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085320600000000002,"stat_sum":{"num_bytes":12556648,"num_objects":17,"num_object_clones":2,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":1092,"num_read_kb":988,"num_write":119,"num_write_kb":1883,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":4916923,"num_keys_recovered":29,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":807570016,"num_objects":789,"num_object_clones":110,"num_object_copies":1578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":789,"num_whiteouts":104,"num_read":83756,"num_read_kb":268752,"num_write":8346,"num_write_kb":180318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":4916923,"num_keys_recovered":29,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":26615808,"data_stored":192011768,"data_compressed":4767534,"data_compressed_allocated":14790656,"data_compressed_original":182569818,"omap_allocated":0,"internal_metadata":0},"log_size":6339,"ondisk_log_size":6339,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17398,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574861,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38972,"kb_used_data":3760,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94332868,"statfs":{"total":96636764160,"available":96596856832,"internally_reserved":0,"allocated":3850240,"data_stored":21372951,"data_compressed":455569,"data_compressed_allocated":1617920,"data_compressed_original":19760688,"omap_allocated":1589,"internal_metadata":36043211},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.314}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574860,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32704,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94339136,"statfs":{"total":96636764160,"available":96603275264,"internally_reserved":0,"allocated":1617920,"data_stored":9533736,"data_compressed":113463,"data_compressed_allocated":536576,"data_compressed_original":8814371,"omap_allocated":1593,"internal_metadata":31848903},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.1319999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.0410000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.248}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574860,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39580,"kb_used_data":4368,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94332260,"statfs":{"total":96636764160,"available":96596234240,"internally_reserved":0,"allocated":4472832,"data_stored":26006485,"data_compressed":659936,"data_compressed_allocated":2056192,"data_compressed_original":24212531,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]}]},{"osd":4,"up_from":13,"seq":55834574860,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39972,"kb_used_data":4764,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94331868,"statfs":{"total":96636764160,"available":96595832832,"internally_reserved":0,"allocated":4878336,"data_stored":32372911,"data_compressed":832459,"data_compressed_allocated":2342912,"data_compressed_original":30441353,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574860,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40104,"kb_used_data":4896,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94331736,"statfs":{"total":96636764160,"available":96595697664,"internally_reserved":0,"allocated":5013504,"data_stored":32518901,"data_compressed":637549,"data_compressed_allocated":2445312,"data_compressed_original":30658324,"omap_allocated":1592,"internal_metadata":36043208},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574860,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37152,"kb_used_data":1936,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334688,"statfs":{"total":96636764160,"available":96598720512,"internally_reserved":0,"allocated":1982464,"data_stored":10410893,"data_compressed":415733,"data_compressed_allocated":884736,"data_compressed_original":9734291,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26500000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39096,"kb_used_data":3880,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94332744,"statfs":{"total":96636764160,"available":96596729856,"internally_reserved":0,"allocated":3973120,"data_stored":28185040,"data_compressed":790362,"data_compressed_allocated":2318336,"data_compressed_original":26963752,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.253}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]}]},{"osd":0,"up_from":13,"seq":55834574861,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39688,"kb_used_data":4476,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94332152,"statfs":{"total":96636764160,"available":96596123648,"internally_reserved":0,"allocated":4583424,"data_stored":32088944,"data_compressed":865361,"data_compressed_allocated":2555904,"data_compressed_original":30759804,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61799999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4038656,"data_stored":31848648,"data_compressed":865361,"data_compressed_allocated":2555904,"data_compressed_original":30759804,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3543040,"data_stored":27440491,"data_compressed":781663,"data_compressed_allocated":2281472,"data_compressed_original":26390312,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1482752,"data_stored":10070699,"data_compressed":415733,"data_compressed_allocated":884736,"data_compressed_original":9617555,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4558848,"data_stored":31961048,"data_compressed":637549,"data_compressed_allocated":2445312,"data_compressed_original":30305044,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4415488,"data_stored":31988624,"data_compressed":832459,"data_compressed_allocated":2342912,"data_compressed_original":30265225,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3952640,"data_stored":25780127,"data_compressed":659936,"data_compressed_allocated":2056192,"data_compressed_original":24212531,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1101824,"data_stored":9177571,"data_compressed":113463,"data_compressed_allocated":536576,"data_compressed_original":8696611,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3522560,"data_stored":23744560,"data_compressed":461370,"data_compressed_allocated":1687552,"data_compressed_original":22322736,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T04:56:57.459 INFO:tasks.thrashosds.thrasher:Setting 1.0 to [6, 4] 2024-09-18T04:56:57.459 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '1.0', '6', '4'] 2024-09-18T04:56:57.459 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 1.0 6 4 2024-09-18T04:56:57.461 INFO:teuthology.orchestra.run.smithi002.stdout:2434 read 0x3841842 thru 0x38422a0 (0xa5f bytes) 2024-09-18T04:56:57.461 INFO:teuthology.orchestra.run.smithi002.stdout:2435 write 0x687b0f5 thru 0x687bdec (0xcf8 bytes) 2024-09-18T04:56:57.461 INFO:teuthology.orchestra.run.smithi002.stdout:2436 read 0x8a2a thru 0x1410d (0xb6e4 bytes) 2024-09-18T04:56:57.461 INFO:teuthology.orchestra.run.smithi002.stdout:2437 writesame 0xc18736 thru 0xc1fa74 data_size 0x1(0x733f bytes) 2024-09-18T04:56:57.461 INFO:teuthology.orchestra.run.smithi002.stdout:2438 read 0xa49fcad thru 0xa4a5f65 (0x62b9 bytes) 2024-09-18T04:56:57.461 INFO:teuthology.orchestra.run.smithi002.stdout:2439 writesame 0x956d3ba thru 0x956dbca data_size 0x1(0x811 bytes) 2024-09-18T04:56:57.462 INFO:teuthology.orchestra.run.smithi002.stdout:2440 write 0x502299b thru 0x502d0fa (0xa760 bytes) 2024-09-18T04:56:57.462 INFO:teuthology.orchestra.run.smithi002.stdout:2442 write 0xd54dff0 thru 0xd5542d4 (0x62e5 bytes) 2024-09-18T04:56:57.462 INFO:teuthology.orchestra.run.smithi002.stdout:2443 write 0x69452e9 thru 0x6947763 (0x247b bytes) 2024-09-18T04:56:57.462 INFO:teuthology.orchestra.run.smithi002.stdout:2444 writesame 0xc972ca9 thru 0xc97f1dc data_size 0x2a(0xc534 bytes) 2024-09-18T04:56:57.462 INFO:teuthology.orchestra.run.smithi002.stdout:2445 read 0x3fa209 thru 0x40387d (0x9675 bytes) 2024-09-18T04:56:57.462 INFO:teuthology.orchestra.run.smithi002.stdout:2448 writesame 0xbc7640d thru 0xbc768e8 data_size 0x1(0x4dc bytes) 2024-09-18T04:56:57.462 INFO:teuthology.orchestra.run.smithi002.stdout:2450 trunc from 0xd5542d5 to 0x5b1f9af 2024-09-18T04:56:57.462 INFO:teuthology.orchestra.run.smithi002.stdout:2452 read 0x2cb3221 thru 0x2cb789e (0x467e bytes) 2024-09-18T04:56:57.462 INFO:teuthology.orchestra.run.smithi002.stdout:2453 write 0x5d3ba23 thru 0x5d481dc (0xc7ba bytes) 2024-09-18T04:56:57.462 INFO:teuthology.orchestra.run.smithi002.stdout:2454 punch from 0x2adfd37 to 0x2ae7434, (0x76fd bytes) 2024-09-18T04:56:57.462 INFO:teuthology.orchestra.run.smithi002.stdout:2455 read 0x50d0682 thru 0x50da4b4 (0x9e33 bytes) 2024-09-18T04:56:57.462 INFO:teuthology.orchestra.run.smithi002.stdout:2456 write 0x10d2296 thru 0x10e1370 (0xf0db bytes) 2024-09-18T04:56:57.462 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:57.462 INFO:teuthology.orchestra.run.smithi002.stdout:2459 writesame 0x3e0d6ed thru 0x3e0f5e1 data_size 0x5(0x1ef5 bytes) 2024-09-18T04:56:57.463 INFO:teuthology.orchestra.run.smithi002.stdout:2460 read 0x2cd753e thru 0x2cdc1a6 (0x4c69 bytes) 2024-09-18T04:56:57.463 INFO:teuthology.orchestra.run.smithi002.stdout:2461 trunc from 0x5d481dd to 0x8ad41cf 2024-09-18T04:56:57.463 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:57.444+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:57.464 INFO:teuthology.orchestra.run.smithi002.stdout:2462 write 0x9eec170 thru 0x9ef950d (0xd39e bytes) 2024-09-18T04:56:57.464 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:57.468 INFO:teuthology.orchestra.run.smithi002.stdout:2465 writesame 0xbcf6485 thru 0xbd03cec data_size 0x32(0xd868 bytes) 2024-09-18T04:56:57.469 INFO:teuthology.orchestra.run.smithi002.stdout:2466 read 0x2223e9a thru 0x2225f4d (0x20b4 bytes) 2024-09-18T04:56:57.469 INFO:teuthology.orchestra.run.smithi002.stdout:2467 read 0x1ee81e0 thru 0x1eed728 (0x5549 bytes) 2024-09-18T04:56:57.469 INFO:teuthology.orchestra.run.smithi002.stdout:2470 read 0x8464845 thru 0x8471dde (0xd59a bytes) 2024-09-18T04:56:57.477 INFO:teuthology.orchestra.run.smithi002.stdout:2471 writesame 0xd895506 thru 0xd89ea17 data_size 0x1(0x9512 bytes) 2024-09-18T04:56:57.477 INFO:teuthology.orchestra.run.smithi002.stdout:2472 read 0xbd3f2ed thru 0xbd44a9e (0x57b2 bytes) 2024-09-18T04:56:57.477 INFO:teuthology.orchestra.run.smithi002.stdout:2473 punch from 0x6d30b7e to 0x6d39f4f, (0x93d1 bytes) 2024-09-18T04:56:57.477 INFO:teuthology.orchestra.run.smithi002.stdout:2474 read 0xcd9f759 thru 0xcda61f0 (0x6a98 bytes) 2024-09-18T04:56:57.477 INFO:teuthology.orchestra.run.smithi002.stdout:2475 write 0xca3826b thru 0xca448bb (0xc651 bytes) 2024-09-18T04:56:57.485 INFO:teuthology.orchestra.run.smithi002.stdout:2476 writesame 0xda1749e thru 0xda1b274 data_size 0x3(0x3dd7 bytes) 2024-09-18T04:56:57.486 INFO:teuthology.orchestra.run.smithi002.stdout:2477 writesame 0x80f6501 thru 0x80fbced data_size 0x3d(0x57ed bytes) 2024-09-18T04:56:57.488 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:57.488 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:57.489 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:57.489 INFO:teuthology.orchestra.run.smithi002.stdout:2482 read 0x6116e69 thru 0x6125264 (0xe3fc bytes) 2024-09-18T04:56:57.489 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:57.489 INFO:teuthology.orchestra.run.smithi002.stdout:2484 write 0x8d965ce thru 0x8d9d554 (0x6f87 bytes) 2024-09-18T04:56:57.492 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:57.493 INFO:teuthology.orchestra.run.smithi002.stdout:2486 write 0xa57dbd7 thru 0xa58a959 (0xcd83 bytes) 2024-09-18T04:56:57.498 INFO:teuthology.orchestra.run.smithi002.stdout:2487 trunc from 0xda1b275 to 0xd6e0780 2024-09-18T04:56:57.504 INFO:teuthology.orchestra.run.smithi002.stdout:2490 punch from 0x9df131f to 0x9dfe039, (0xcd1a bytes) 2024-09-18T04:56:57.504 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:57.504 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:57.505 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:57.505 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:57.505 INFO:teuthology.orchestra.run.smithi002.stdout:2495 read 0x85e4deb thru 0x85efe92 (0xb0a8 bytes) 2024-09-18T04:56:57.515 INFO:teuthology.orchestra.run.smithi002.stdout:2496 trunc from 0xd6e0780 to 0x44f66d1 2024-09-18T04:56:57.530 INFO:teuthology.orchestra.run.smithi002.stdout:2497 read 0x3abefe3 thru 0x3acb130 (0xc14e bytes) 2024-09-18T04:56:57.531 INFO:teuthology.orchestra.run.smithi002.stdout:2498 read 0xb554a5 thru 0xb5b3e1 (0x5f3d bytes) 2024-09-18T04:56:57.532 INFO:teuthology.orchestra.run.smithi002.stdout:2500 punch from 0xddf22 to 0xe6a7a, (0x8b58 bytes) 2024-09-18T04:56:57.541 INFO:teuthology.orchestra.run.smithi002.stdout:2501 trunc from 0x44f66d1 to 0xbb40c68 2024-09-18T04:56:57.547 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:57.545+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:57.548 INFO:teuthology.orchestra.run.smithi002.stdout:2502 read 0xb4d8351 thru 0xb4e1a9a (0x974a bytes) 2024-09-18T04:56:57.548 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:57.548 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:57.548 INFO:teuthology.orchestra.run.smithi002.stdout:2507 punch from 0x34029e9 to 0x340ab5e, (0x8175 bytes) 2024-09-18T04:56:57.548 INFO:teuthology.orchestra.run.smithi002.stdout:2508 punch from 0x3c9143b to 0x3c9b651, (0xa216 bytes) 2024-09-18T04:56:57.548 INFO:teuthology.orchestra.run.smithi002.stdout:2509 writesame 0x608879b thru 0x608f556 data_size 0x6(0x6dbc bytes) 2024-09-18T04:56:57.550 INFO:teuthology.orchestra.run.smithi002.stdout:2510 read 0x74ef17e thru 0x74f60c9 (0x6f4c bytes) 2024-09-18T04:56:57.558 INFO:teuthology.orchestra.run.smithi002.stdout:2511 trunc from 0xbb40c68 to 0x607d9ae 2024-09-18T04:56:57.565 INFO:teuthology.orchestra.run.smithi002.stdout:2512 write 0x3010711 thru 0x3014686 (0x3f76 bytes) 2024-09-18T04:56:57.567 INFO:teuthology.orchestra.run.smithi002.stdout:2513 read 0x533ef12 thru 0x5340cf7 (0x1de6 bytes) 2024-09-18T04:56:57.567 INFO:teuthology.orchestra.run.smithi002.stdout:2514 read 0xe14fa8 thru 0xe160ed (0x1146 bytes) 2024-09-18T04:56:57.574 INFO:teuthology.orchestra.run.smithi002.stdout:2516 writesame 0x6531d64 thru 0x653eb23 data_size 0x20(0xcdc0 bytes) 2024-09-18T04:56:57.577 INFO:teuthology.orchestra.run.smithi002.stdout:2517 read 0x3c7e52d thru 0x3c83db1 (0x5885 bytes) 2024-09-18T04:56:57.577 INFO:teuthology.orchestra.run.smithi002.stdout:2518 punch from 0x3e8443e to 0x3e93afa, (0xf6bc bytes) 2024-09-18T04:56:57.577 INFO:teuthology.orchestra.run.smithi002.stdout:2519 punch from 0x3b29674 to 0x3b2bc4e, (0x25da bytes) 2024-09-18T04:56:57.577 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:57.577 INFO:teuthology.orchestra.run.smithi002.stdout:2522 read 0x19c62c9 thru 0x19cff8e (0x9cc6 bytes) 2024-09-18T04:56:57.577 INFO:teuthology.orchestra.run.smithi002.stdout:2523 write 0x75235a thru 0x759205 (0x6eac bytes) 2024-09-18T04:56:57.578 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:57.578 INFO:teuthology.orchestra.run.smithi002.stdout:2525 clone 22 order 19 su 32768 sc 14 2024-09-18T04:56:57.587 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:57.587 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:57.597 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:56:57.648 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:57.646+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:57.745 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:57.745 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:57.749 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:57.747+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:57.754 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:56:57.850 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:57.848+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:57.861 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:57.861 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:57.869 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:56:57.954 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:57.952+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:58.017 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:58.017 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:58.025 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:56:58.052 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:58.050+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:58.153 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:58.150+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:58.172 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:58.173 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:58.181 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:56:58.253 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:58.251+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:58.274 INFO:teuthology.orchestra.run.smithi002.stderr:set 1.0 pg_upmap mapping to [6,4] 2024-09-18T04:56:58.291 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:58.291 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:58.298 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T04:56:58.301 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:56:58.351 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone21 from 0x653eb24 (overlap 0x44f66d1) to 0x0 2024-09-18T04:56:58.354 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:56:58.352+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:56:58.455 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:58.453+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:58.477 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:58.477 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:58.487 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:56:58.549 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-09-18T04:56:58.556 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:58.554+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:58.657 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:56:58.654+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:56:58.694 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:58.694 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:58.708 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:56:58.758 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:58.755+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:58.858 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:58.856+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:58.890 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:58.890 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:58.899 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:56:58.959 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:58.957+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:58.972 INFO:teuthology.orchestra.run.smithi002.stdout:2526 write 0x13c1875 thru 0x13c6c75 (0x5401 bytes) 2024-09-18T04:56:58.991 INFO:teuthology.orchestra.run.smithi002.stdout:2527 write 0x8b0bdec thru 0x8b11c80 (0x5e95 bytes) 2024-09-18T04:56:58.993 INFO:teuthology.orchestra.run.smithi002.stdout:2528 write 0x327c6ed thru 0x3286681 (0x9f95 bytes) 2024-09-18T04:56:58.999 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:58.999 INFO:teuthology.orchestra.run.smithi002.stdout:2531 punch from 0x1a68d0b to 0x1a7270f, (0x9a04 bytes) 2024-09-18T04:56:59.001 INFO:teuthology.orchestra.run.smithi002.stdout:2534 trunc from 0x8b11c81 to 0x6c78f09 2024-09-18T04:56:59.010 INFO:teuthology.orchestra.run.smithi002.stdout:2535 read 0x47381fe thru 0x473fe2b (0x7c2e bytes) 2024-09-18T04:56:59.016 INFO:teuthology.orchestra.run.smithi002.stdout:2536 write 0x72448fe thru 0x724fad9 (0xb1dc bytes) 2024-09-18T04:56:59.016 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:59.016 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:59.020 INFO:teuthology.orchestra.run.smithi002.stdout:2537 read 0x589da73 thru 0x589fad2 (0x2060 bytes) 2024-09-18T04:56:59.020 INFO:teuthology.orchestra.run.smithi002.stdout:2538 punch from 0x1094be5 to 0x109567f, (0xa9a bytes) 2024-09-18T04:56:59.020 INFO:teuthology.orchestra.run.smithi002.stdout:2543 punch from 0x487f618 to 0x48821ae, (0x2b96 bytes) 2024-09-18T04:56:59.026 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:56:59.033 INFO:teuthology.orchestra.run.smithi002.stdout:2546 write 0xcde724b thru 0xcdf3732 (0xc4e8 bytes) 2024-09-18T04:56:59.036 INFO:teuthology.orchestra.run.smithi002.stdout:2547 writesame 0x5a9aeec thru 0x5aa4c09 data_size 0x77(0x9d1e bytes) 2024-09-18T04:56:59.043 INFO:teuthology.orchestra.run.smithi002.stdout:2548 read 0x21cd092 thru 0x21d91b6 (0xc125 bytes) 2024-09-18T04:56:59.044 INFO:teuthology.orchestra.run.smithi002.stdout:2549 write 0x1ba97db thru 0x1bb7965 (0xe18b bytes) 2024-09-18T04:56:59.049 INFO:teuthology.orchestra.run.smithi002.stdout:2550 read 0x9a479b1 thru 0x9a48094 (0x6e4 bytes) 2024-09-18T04:56:59.049 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:59.049 INFO:teuthology.orchestra.run.smithi002.stdout:2552 read 0x2f261dd thru 0x2f306f1 (0xa515 bytes) 2024-09-18T04:56:59.049 INFO:teuthology.orchestra.run.smithi002.stdout:2553 read 0xcb0a864 thru 0xcb0eefc (0x4699 bytes) 2024-09-18T04:56:59.060 INFO:teuthology.orchestra.run.smithi002.stdout:2557 trunc from 0xcdf3733 to 0x37f5b6f 2024-09-18T04:56:59.061 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:56:59.058+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:56:59.079 INFO:teuthology.orchestra.run.smithi002.stdout:2559 punch from 0x2e865f6 to 0x2e8d45f, (0x6e69 bytes) 2024-09-18T04:56:59.079 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:59.090 INFO:teuthology.orchestra.run.smithi002.stdout:2561 trunc from 0x37f5b6f to 0xd466026 2024-09-18T04:56:59.095 INFO:teuthology.orchestra.run.smithi002.stdout:2562 write 0x929e803 thru 0x92a4c8c (0x648a bytes) 2024-09-18T04:56:59.100 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:59.100 INFO:teuthology.orchestra.run.smithi002.stdout:2568 punch from 0xd2e571b to 0xd2f397d, (0xe262 bytes) 2024-09-18T04:56:59.100 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:59.101 INFO:teuthology.orchestra.run.smithi002.stdout:2572 flatten 2024-09-18T04:56:59.162 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:59.160+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:59.166 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:59.167 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:59.177 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:56:59.263 INFO:teuthology.orchestra.run.smithi002.stdout:2575 read 0xa54f58f thru 0xa555b55 (0x65c7 bytes) 2024-09-18T04:56:59.263 INFO:teuthology.orchestra.run.smithi002.stdout:2578 read 0x155ab2c thru 0x15606cb (0x5ba0 bytes) 2024-09-18T04:56:59.263 INFO:teuthology.orchestra.run.smithi002.stdout:2579 read 0x7090154 thru 0x7099f00 (0x9dad bytes) 2024-09-18T04:56:59.263 INFO:teuthology.orchestra.run.smithi002.stdout:2580 read 0x125f44c thru 0x126ea05 (0xf5ba bytes) 2024-09-18T04:56:59.263 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:59.261+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:59.264 INFO:teuthology.orchestra.run.smithi002.stdout:2581 trunc from 0xd466026 to 0xe02755d 2024-09-18T04:56:59.269 INFO:teuthology.orchestra.run.smithi002.stdout:2582 read 0x287f57f thru 0x2880bec (0x166e bytes) 2024-09-18T04:56:59.270 INFO:teuthology.orchestra.run.smithi002.stdout:2583 punch from 0x2942ae0 to 0x294facc, (0xcfec bytes) 2024-09-18T04:56:59.275 INFO:teuthology.orchestra.run.smithi002.stdout:2584 write 0xe0f6f07 thru 0xe0fe74e (0x7848 bytes) 2024-09-18T04:56:59.278 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-18T04:56:59.285 INFO:teuthology.orchestra.run.smithi002.stdout:2586 read 0x518d75e thru 0x5191009 (0x38ac bytes) 2024-09-18T04:56:59.285 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:59.285 INFO:teuthology.orchestra.run.smithi002.stdout:2588 read 0x7e78f92 thru 0x7e827e9 (0x9858 bytes) 2024-09-18T04:56:59.285 INFO:teuthology.orchestra.run.smithi002.stdout:2589 write 0x23ce761 thru 0x23da3a6 (0xbc46 bytes) 2024-09-18T04:56:59.290 INFO:teuthology.orchestra.run.smithi002.stdout:2591 write 0x516d979 thru 0x5170aa5 (0x312d bytes) 2024-09-18T04:56:59.305 INFO:teuthology.orchestra.run.smithi002.stdout:2592 writesame 0xb85ccc5 thru 0xb865707 data_size 0x1(0x8a43 bytes) 2024-09-18T04:56:59.307 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:59.307 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:59.309 INFO:teuthology.orchestra.run.smithi002.stdout:2593 read 0xc442fdf thru 0xc44d532 (0xa554 bytes) 2024-09-18T04:56:59.309 INFO:teuthology.orchestra.run.smithi002.stdout:2594 read 0xd9b9346 thru 0xd9b9ebf (0xb7a bytes) 2024-09-18T04:56:59.309 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:59.309 INFO:teuthology.orchestra.run.smithi002.stdout:2597 write 0xfdc26c thru 0xfdfd49 (0x3ade bytes) 2024-09-18T04:56:59.310 INFO:teuthology.orchestra.run.smithi002.stdout:2598 write 0x4079e6a thru 0x407c986 (0x2b1d bytes) 2024-09-18T04:56:59.314 INFO:teuthology.orchestra.run.smithi002.stdout:2599 read 0x70a87f2 thru 0x70b4f68 (0xc777 bytes) 2024-09-18T04:56:59.314 INFO:teuthology.orchestra.run.smithi002.stdout:2601 read 0x17e747a thru 0x17f4340 (0xcec7 bytes) 2024-09-18T04:56:59.314 INFO:teuthology.orchestra.run.smithi002.stdout:2602 write 0x2ee7344 thru 0x2ef3889 (0xc546 bytes) 2024-09-18T04:56:59.316 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:56:59.318 INFO:teuthology.orchestra.run.smithi002.stdout:2603 writesame 0x13eaa0a thru 0x13f3dec data_size 0x11(0x93e3 bytes) 2024-09-18T04:56:59.326 INFO:teuthology.orchestra.run.smithi002.stdout:2604 writesame 0xea6d068 thru 0xea751c1 data_size 0x3(0x815a bytes) 2024-09-18T04:56:59.327 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:59.327 INFO:teuthology.orchestra.run.smithi002.stdout:2606 write 0x4920936 thru 0x493032c (0xf9f7 bytes) 2024-09-18T04:56:59.331 INFO:teuthology.orchestra.run.smithi002.stdout:2608 read 0x2fe6669 thru 0x2ff051e (0x9eb6 bytes) 2024-09-18T04:56:59.331 INFO:teuthology.orchestra.run.smithi002.stdout:2611 read 0xb3b2892 thru 0xb3b3531 (0xca0 bytes) 2024-09-18T04:56:59.332 INFO:teuthology.orchestra.run.smithi002.stdout:2612 read 0x2b10b35 thru 0x2b16fd6 (0x64a2 bytes) 2024-09-18T04:56:59.332 INFO:teuthology.orchestra.run.smithi002.stdout:2613 read 0xd113175 thru 0xd11edb1 (0xbc3d bytes) 2024-09-18T04:56:59.332 INFO:teuthology.orchestra.run.smithi002.stdout:2615 writesame 0x59db451 thru 0x59e52a4 data_size 0x2(0x9e54 bytes) 2024-09-18T04:56:59.341 INFO:teuthology.orchestra.run.smithi002.stdout:2617 read 0x4c97588 thru 0x4ca3eb9 (0xc932 bytes) 2024-09-18T04:56:59.341 INFO:teuthology.orchestra.run.smithi002.stdout:2618 read 0xe80ab68 thru 0xe817bc4 (0xd05d bytes) 2024-09-18T04:56:59.342 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:59.342 INFO:teuthology.orchestra.run.smithi002.stdout:2622 punch from 0x3116450 to 0x312170a, (0xb2ba bytes) 2024-09-18T04:56:59.342 INFO:teuthology.orchestra.run.smithi002.stdout:2624 read 0x999fb53 thru 0x99aee5f (0xf30d bytes) 2024-09-18T04:56:59.342 INFO:teuthology.orchestra.run.smithi002.stdout:2625 writesame 0xaefefbe thru 0xaf072c9 data_size 0x2(0x830c bytes) 2024-09-18T04:56:59.352 INFO:teuthology.orchestra.run.smithi002.stdout:2626 read 0x745d103 thru 0x7462758 (0x5656 bytes) 2024-09-18T04:56:59.352 INFO:teuthology.orchestra.run.smithi002.stdout:2627 read 0x897b28c thru 0x897c65c (0x13d1 bytes) 2024-09-18T04:56:59.354 INFO:teuthology.orchestra.run.smithi002.stdout:2628 trunc from 0xea751c2 to 0xd6bb57a 2024-09-18T04:56:59.364 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:59.362+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:59.364 INFO:teuthology.orchestra.run.smithi002.stdout:2632 read 0x3a80f82 thru 0x3a8a947 (0x99c6 bytes) 2024-09-18T04:56:59.365 INFO:teuthology.orchestra.run.smithi002.stdout:2634 writesame 0xcd2b0aa thru 0xcd391f0 data_size 0x1(0xe147 bytes) 2024-09-18T04:56:59.395 INFO:teuthology.orchestra.run.smithi002.stdout:2635 trunc from 0xd6bb57a to 0x29a1508 2024-09-18T04:56:59.411 INFO:teuthology.orchestra.run.smithi002.stdout:2636 read 0xcba70e thru 0xcbf853 (0x5146 bytes) 2024-09-18T04:56:59.429 INFO:teuthology.orchestra.run.smithi002.stdout:2638 write 0xdcdce21 thru 0xdce2707 (0x58e7 bytes) 2024-09-18T04:56:59.431 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:59.431 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:59.433 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:59.433 INFO:teuthology.orchestra.run.smithi002.stdout:2641 writesame 0xa0e8fb1 thru 0xa0f3777 data_size 0x3(0xa7c7 bytes) 2024-09-18T04:56:59.440 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:56:59.441 INFO:teuthology.orchestra.run.smithi002.stdout:2642 write 0x9bf0db4 thru 0x9bfbb2b (0xad78 bytes) 2024-09-18T04:56:59.445 INFO:teuthology.orchestra.run.smithi002.stdout:2643 write 0xc7629ef thru 0xc76b5d2 (0x8be4 bytes) 2024-09-18T04:56:59.454 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:56:59.457 INFO:teuthology.orchestra.run.smithi002.stdout:2645 trunc from 0xdce2708 to 0x6b72ebe 2024-09-18T04:56:59.468 INFO:teuthology.orchestra.run.smithi002.stdout:2646 trunc from 0x6b72ebe to 0x5c19150 2024-09-18T04:56:59.469 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:56:59.467+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:56:59.473 INFO:teuthology.orchestra.run.smithi002.stdout:2647 read 0x1a1a979 thru 0x1a1d7bd (0x2e45 bytes) 2024-09-18T04:56:59.473 INFO:teuthology.orchestra.run.smithi002.stdout:2649 writesame 0x1e7743 thru 0x1eb870 data_size 0x51(0x412e bytes) 2024-09-18T04:56:59.475 INFO:teuthology.orchestra.run.smithi002.stdout:2650 read 0x58b86d7 thru 0x58c0016 (0x7940 bytes) 2024-09-18T04:56:59.490 INFO:teuthology.orchestra.run.smithi002.stdout:2652 writesame 0xe365b80 thru 0xe36c7d8 data_size 0x1(0x6c59 bytes) 2024-09-18T04:56:59.490 INFO:teuthology.orchestra.run.smithi002.stdout:2655 writesame 0x68488df thru 0x684c6b4 data_size 0x5(0x3dd6 bytes) 2024-09-18T04:56:59.493 INFO:teuthology.orchestra.run.smithi002.stdout:2656 read 0x73e50b5 thru 0x73e77b8 (0x2704 bytes) 2024-09-18T04:56:59.493 INFO:teuthology.orchestra.run.smithi002.stdout:2657 write 0x7230a26 thru 0x723627c (0x5857 bytes) 2024-09-18T04:56:59.496 INFO:teuthology.orchestra.run.smithi002.stdout:2658 write 0x7b8dbd3 thru 0x7b9954c (0xb97a bytes) 2024-09-18T04:56:59.500 INFO:teuthology.orchestra.run.smithi002.stdout:2661 write 0x3d0e21 thru 0x3d4c08 (0x3de8 bytes) 2024-09-18T04:56:59.503 INFO:teuthology.orchestra.run.smithi002.stdout:2662 writesame 0x4a8d25f thru 0x4a95948 data_size 0x7(0x86ea bytes) 2024-09-18T04:56:59.507 INFO:teuthology.orchestra.run.smithi002.stdout:2663 punch from 0x4943a63 to 0x4948237, (0x47d4 bytes) 2024-09-18T04:56:59.507 INFO:teuthology.orchestra.run.smithi002.stdout:2665 write 0x2d14dbb thru 0x2d14dfc (0x42 bytes) 2024-09-18T04:56:59.510 INFO:teuthology.orchestra.run.smithi002.stdout:2666 punch from 0x62d0e52 to 0x62d1346, (0x4f4 bytes) 2024-09-18T04:56:59.510 INFO:teuthology.orchestra.run.smithi002.stdout:2667 write 0x46ea92b thru 0x46f486b (0x9f41 bytes) 2024-09-18T04:56:59.513 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:59.513 INFO:teuthology.orchestra.run.smithi002.stdout:2672 punch from 0xe7b846 to 0xe8462a, (0x8de4 bytes) 2024-09-18T04:56:59.513 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:59.513 INFO:teuthology.orchestra.run.smithi002.stdout:2674 read 0xa1ee65d thru 0xa1fc0e7 (0xda8b bytes) 2024-09-18T04:56:59.514 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:59.514 INFO:teuthology.orchestra.run.smithi002.stdout:2676 read 0xbddef44 thru 0xbde43f0 (0x54ad bytes) 2024-09-18T04:56:59.518 INFO:teuthology.orchestra.run.smithi002.stdout:2677 trunc from 0xe36c7d9 to 0x9d6e2d2 2024-09-18T04:56:59.528 INFO:teuthology.orchestra.run.smithi002.stdout:2681 trunc from 0x9d6e2d2 to 0xe3cea22 2024-09-18T04:56:59.534 INFO:teuthology.orchestra.run.smithi002.stdout:2682 read 0x26a4702 thru 0x26a78e6 (0x31e5 bytes) 2024-09-18T04:56:59.534 INFO:teuthology.orchestra.run.smithi002.stdout:2683 write 0xd8a2f61 thru 0xd8a4263 (0x1303 bytes) 2024-09-18T04:56:59.537 INFO:teuthology.orchestra.run.smithi002.stdout:2684 punch from 0xc30e34e to 0xc319e26, (0xbad8 bytes) 2024-09-18T04:56:59.537 INFO:teuthology.orchestra.run.smithi002.stdout:2685 read 0x86d61de thru 0x86d8152 (0x1f75 bytes) 2024-09-18T04:56:59.538 INFO:teuthology.orchestra.run.smithi002.stdout:2686 trunc from 0xe3cea22 to 0xd07a1b5 2024-09-18T04:56:59.548 INFO:teuthology.orchestra.run.smithi002.stdout:2687 punch from 0x85e5a10 to 0x85ed0db, (0x76cb bytes) 2024-09-18T04:56:59.548 INFO:teuthology.orchestra.run.smithi002.stdout:2688 read 0x86b802a thru 0x86c6294 (0xe26b bytes) 2024-09-18T04:56:59.548 INFO:teuthology.orchestra.run.smithi002.stdout:2690 read 0x1cbe482 thru 0x1cc8b6a (0xa6e9 bytes) 2024-09-18T04:56:59.548 INFO:teuthology.orchestra.run.smithi002.stdout:2691 read 0x7908a2c thru 0x791101d (0x85f2 bytes) 2024-09-18T04:56:59.548 INFO:teuthology.orchestra.run.smithi002.stdout:truncating to largest ever: 0xee0ba01 2024-09-18T04:56:59.550 INFO:teuthology.orchestra.run.smithi002.stdout:2692 trunc from 0xd07a1b5 to 0xee0ba01 2024-09-18T04:56:59.555 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:59.556 INFO:teuthology.orchestra.run.smithi002.stdout:2695 write 0xe3437be thru 0xe34ef9a (0xb7dd bytes) 2024-09-18T04:56:59.556 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:59.556 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:59.558 INFO:teuthology.orchestra.run.smithi002.stdout:2697 punch from 0x5d46bc9 to 0x5d4d813, (0x6c4a bytes) 2024-09-18T04:56:59.561 INFO:teuthology.orchestra.run.smithi002.stdout:2699 trunc from 0xee0ba01 to 0xc98fc4a 2024-09-18T04:56:59.565 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:56:59.566 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:56:59.564+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:56:59.571 INFO:teuthology.orchestra.run.smithi002.stdout:2700 write 0xb002390 thru 0xb00edb1 (0xca22 bytes) 2024-09-18T04:56:59.581 INFO:teuthology.orchestra.run.smithi002.stdout:2702 write 0xd83a673 thru 0xd8445c8 (0x9f56 bytes) 2024-09-18T04:56:59.585 INFO:teuthology.orchestra.run.smithi002.stdout:2703 read 0xaf1cd5 thru 0xaf5924 (0x3c50 bytes) 2024-09-18T04:56:59.585 INFO:teuthology.orchestra.run.smithi002.stdout:2705 writesame 0xc4c6785 thru 0xc4d17fd data_size 0x25(0xb079 bytes) 2024-09-18T04:56:59.589 INFO:teuthology.orchestra.run.smithi002.stdout:2706 write 0x5f5082e thru 0x5f5650a (0x5cdd bytes) 2024-09-18T04:56:59.592 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:56:59.592 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:56:59.593 INFO:teuthology.orchestra.run.smithi002.stdout:2707 trunc from 0xd8445c9 to 0xcd96dcb 2024-09-18T04:56:59.604 INFO:teuthology.orchestra.run.smithi002.stdout:2708 punch from 0x8e6200f to 0x8e63c9d, (0x1c8e bytes) 2024-09-18T04:56:59.604 INFO:teuthology.orchestra.run.smithi002.stdout:2710 write 0x4f97d2c thru 0x4f9df24 (0x61f9 bytes) 2024-09-18T04:56:59.606 INFO:teuthology.orchestra.run.smithi002.stdout:2711 writesame 0x3c19f85 thru 0x3c1afcf data_size 0x1(0x104b bytes) 2024-09-18T04:56:59.609 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:56:59.610 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:56:59.610 INFO:teuthology.orchestra.run.smithi002.stdout:2712 read 0xc39221b thru 0xc398da2 (0x6b88 bytes) 2024-09-18T04:56:59.612 INFO:teuthology.orchestra.run.smithi002.stdout:2713 trunc from 0xcd96dcb to 0xe0bcefd 2024-09-18T04:56:59.626 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:56:59.627 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:56:59.628 INFO:teuthology.orchestra.run.smithi002.stdout:2714 trunc from 0xe0bcefd to 0x40900ef 2024-09-18T04:56:59.647 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:56:59.647 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:56:59.659 INFO:teuthology.orchestra.run.smithi002.stdout:2716 writesame 0xd94616c thru 0xd950a60 data_size 0x25(0xa8f5 bytes) 2024-09-18T04:56:59.659 INFO:teuthology.orchestra.run.smithi002.stdout:2717 read 0x232202c thru 0x232fd65 (0xdd3a bytes) 2024-09-18T04:56:59.659 INFO:teuthology.orchestra.run.smithi002.stdout:2718 write 0xa9466c0 thru 0xa9516fa (0xb03b bytes) 2024-09-18T04:56:59.663 INFO:teuthology.orchestra.run.smithi002.stdout:2719 writesame 0xc90190a thru 0xc9118d9 data_size 0x8(0xffd0 bytes) 2024-09-18T04:56:59.666 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:56:59.666 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:56:59.666 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:56:59.664+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:56:59.670 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:59.671 INFO:teuthology.orchestra.run.smithi002.stdout:2723 trunc from 0xd950a61 to 0xe7b1edc 2024-09-18T04:56:59.677 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:59.677 INFO:teuthology.orchestra.run.smithi002.stdout:2725 punch from 0xbb6c490 to 0xbb717ff, (0x536f bytes) 2024-09-18T04:56:59.677 INFO:teuthology.orchestra.run.smithi002.stdout:2727 write 0x6a68c3d thru 0x6a725f9 (0x99bd bytes) 2024-09-18T04:56:59.678 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:56:59.679 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:56:59.681 INFO:teuthology.orchestra.run.smithi002.stdout:2729 punch from 0xd6e6033 to 0xd6e6e42, (0xe0f bytes) 2024-09-18T04:56:59.681 INFO:teuthology.orchestra.run.smithi002.stdout:2731 punch from 0x7633f27 to 0x7637016, (0x30ef bytes) 2024-09-18T04:56:59.681 INFO:teuthology.orchestra.run.smithi002.stdout:2732 punch from 0xb81ba54 to 0xb8207c9, (0x4d75 bytes) 2024-09-18T04:56:59.685 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:56:59.685 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:56:59.688 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:56:59.690 INFO:teuthology.orchestra.run.smithi002.stdout:2733 trunc from 0xe7b1edc to 0x6db93f1 2024-09-18T04:56:59.705 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:56:59.705 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:56:59.706 INFO:teuthology.orchestra.run.smithi002.stdout:2734 trunc from 0x6db93f1 to 0x26a7f09 2024-09-18T04:56:59.718 INFO:teuthology.orchestra.run.smithi002.stdout:2735 punch from 0x1ebc817 to 0x1ecb9a4, (0xf18d bytes) 2024-09-18T04:56:59.718 INFO:teuthology.orchestra.run.smithi002.stdout:2736 read 0x23495c3 thru 0x234ebe8 (0x5626 bytes) 2024-09-18T04:56:59.720 INFO:teuthology.orchestra.run.smithi002.stdout:2738 trunc from 0x26a7f09 to 0x492dcca 2024-09-18T04:56:59.723 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:56:59.723 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:56:59.725 INFO:teuthology.orchestra.run.smithi002.stdout:2740 read 0x2752b5 thru 0x281de5 (0xcb31 bytes) 2024-09-18T04:56:59.735 INFO:teuthology.orchestra.run.smithi002.stdout:2741 trunc from 0x492dcca to 0xc6c95dd 2024-09-18T04:56:59.741 INFO:teuthology.orchestra.run.smithi002.stdout:2742 write 0x38dbe28 thru 0x38eaf3f (0xf118 bytes) 2024-09-18T04:56:59.745 INFO:teuthology.orchestra.run.smithi002.stdout:2743 write 0xa31d371 thru 0xa31d95e (0x5ee bytes) 2024-09-18T04:56:59.748 INFO:teuthology.orchestra.run.smithi002.stdout:2745 write 0x6dda48a thru 0x6de6f8f (0xcb06 bytes) 2024-09-18T04:56:59.752 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:59.754 INFO:teuthology.orchestra.run.smithi002.stdout:2747 trunc from 0xc6c95dd to 0xdaebd04 2024-09-18T04:56:59.759 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:59.759 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:59.759 INFO:teuthology.orchestra.run.smithi002.stdout:2751 read 0x743bd16 thru 0x744a725 (0xea10 bytes) 2024-09-18T04:56:59.762 INFO:teuthology.orchestra.run.smithi002.stdout:2752 trunc from 0xdaebd04 to 0xb42e975 2024-09-18T04:56:59.768 INFO:teuthology.orchestra.run.smithi002.stdout:2753 write 0x891001c thru 0x891267a (0x265f bytes) 2024-09-18T04:56:59.773 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:59.771+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:59.781 INFO:teuthology.orchestra.run.smithi002.stdout:2754 write 0xedcfc90 thru 0xedd066c (0x9dd bytes) 2024-09-18T04:56:59.787 INFO:teuthology.orchestra.run.smithi002.stdout:2755 read 0xd22ce4 thru 0xd31e00 (0xf11d bytes) 2024-09-18T04:56:59.787 INFO:teuthology.orchestra.run.smithi002.stdout:2756 writesame 0xc1be2a1 thru 0xc1c627e data_size 0xd(0x7fde bytes) 2024-09-18T04:56:59.793 INFO:teuthology.orchestra.run.smithi002.stdout:2757 read 0x65c9445 thru 0x65d1ed1 (0x8a8d bytes) 2024-09-18T04:56:59.808 INFO:teuthology.orchestra.run.smithi002.stdout:2758 trunc from 0xedd066d to 0x6db9d2 2024-09-18T04:56:59.825 INFO:teuthology.orchestra.run.smithi002.stdout:2759 read 0x4e5b19 thru 0x4f1c50 (0xc138 bytes) 2024-09-18T04:56:59.826 INFO:teuthology.orchestra.run.smithi002.stdout:2760 read 0x4f4dc2 thru 0x501492 (0xc6d1 bytes) 2024-09-18T04:56:59.827 INFO:teuthology.orchestra.run.smithi002.stdout:2761 punch from 0x55ac16 to 0x567c8d, (0xd077 bytes) 2024-09-18T04:56:59.827 INFO:teuthology.orchestra.run.smithi002.stdout:2763 read 0x352180 thru 0x3567fd (0x467e bytes) 2024-09-18T04:56:59.838 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:56:59.839 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:56:59.839 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:56:59.839 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:56:59.839 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:56:59.847 INFO:teuthology.orchestra.run.smithi002.stdout:2764 write 0xcc18068 thru 0xcc1b12c (0x30c5 bytes) 2024-09-18T04:56:59.848 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:56:59.850 INFO:teuthology.orchestra.run.smithi002.stdout:2766 punch from 0x6960482 to 0x69647e1, (0x435f bytes) 2024-09-18T04:56:59.850 INFO:teuthology.orchestra.run.smithi002.stdout:2767 write 0x741b1be thru 0x742b08e (0xfed1 bytes) 2024-09-18T04:56:59.854 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:59.854 INFO:teuthology.orchestra.run.smithi002.stdout:2770 write 0x5fe0d1a thru 0x5fe92ab (0x8592 bytes) 2024-09-18T04:56:59.857 INFO:teuthology.orchestra.run.smithi002.stdout:2771 writesame 0x7310f26 thru 0x7319ab7 data_size 0x12(0x8b92 bytes) 2024-09-18T04:56:59.861 INFO:teuthology.orchestra.run.smithi002.stdout:2772 punch from 0xa81e69f to 0xa81ebe5, (0x546 bytes) 2024-09-18T04:56:59.861 INFO:teuthology.orchestra.run.smithi002.stdout:2773 write 0xa5825b4 thru 0xa58665c (0x40a9 bytes) 2024-09-18T04:56:59.864 INFO:teuthology.orchestra.run.smithi002.stdout:2774 write 0x1ffe139 thru 0x200bfcc (0xde94 bytes) 2024-09-18T04:56:59.877 INFO:teuthology.orchestra.run.smithi002.stdout:2775 write 0xe318ccc thru 0xe321ffc (0x9331 bytes) 2024-09-18T04:56:59.882 INFO:teuthology.orchestra.run.smithi002.stdout:2776 punch from 0x142dcb to 0x151f6e, (0xf1a3 bytes) 2024-09-18T04:56:59.882 INFO:teuthology.orchestra.run.smithi002.stdout:2777 write 0xcf3e082 thru 0xcf4825d (0xa1dc bytes) 2024-09-18T04:56:59.886 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:59.887 INFO:teuthology.orchestra.run.smithi002.stdout:2779 read 0x9e2be8f thru 0x9e2e937 (0x2aa9 bytes) 2024-09-18T04:56:59.887 INFO:teuthology.orchestra.run.smithi002.stdout:2780 read 0xcc0478a thru 0xcc11c27 (0xd49e bytes) 2024-09-18T04:56:59.889 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:56:59.887+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:56:59.900 INFO:teuthology.orchestra.run.smithi002.stdout:2782 trunc from 0xe321ffd to 0x2398262 2024-09-18T04:56:59.914 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:59.914 INFO:teuthology.orchestra.run.smithi002.stdout:2785 punch from 0xee4214 to 0xee50a9, (0xe95 bytes) 2024-09-18T04:56:59.914 INFO:teuthology.orchestra.run.smithi002.stdout:2787 read 0x78e964 thru 0x7980d8 (0x9775 bytes) 2024-09-18T04:56:59.927 INFO:teuthology.orchestra.run.smithi002.stdout:2788 writesame 0x9093a0d thru 0x9094535 data_size 0x1(0xb29 bytes) 2024-09-18T04:56:59.927 INFO:teuthology.orchestra.run.smithi002.stdout:2790 punch from 0x4af0fb7 to 0x4af8e98, (0x7ee1 bytes) 2024-09-18T04:56:59.934 INFO:teuthology.orchestra.run.smithi002.stdout:2791 write 0x9a8149a thru 0x9a8ef32 (0xda99 bytes) 2024-09-18T04:56:59.937 INFO:teuthology.orchestra.run.smithi002.stdout:2793 punch from 0x99ea3ad to 0x99ebf3f, (0x1b92 bytes) 2024-09-18T04:56:59.942 INFO:teuthology.orchestra.run.smithi002.stdout:2794 write 0x9c947eb thru 0x9c9ce76 (0x868c bytes) 2024-09-18T04:56:59.946 INFO:teuthology.orchestra.run.smithi002.stdout:2796 punch from 0x7583681 to 0x7592a14, (0xf393 bytes) 2024-09-18T04:56:59.946 INFO:teuthology.orchestra.run.smithi002.stdout:2797 write 0x357a71a thru 0x3587e14 (0xd6fb bytes) 2024-09-18T04:56:59.950 INFO:teuthology.orchestra.run.smithi002.stdout:2800 write 0x2daaf10 thru 0x2db82e8 (0xd3d9 bytes) 2024-09-18T04:56:59.954 INFO:teuthology.orchestra.run.smithi002.stdout:2801 read 0x39ead3c thru 0x39fa868 (0xfb2d bytes) 2024-09-18T04:56:59.954 INFO:teuthology.orchestra.run.smithi002.stdout:2802 punch from 0x2eb5a4 to 0x2ef3df, (0x3e3b bytes) 2024-09-18T04:56:59.955 INFO:teuthology.orchestra.run.smithi002.stdout:2803 trunc from 0x9c9ce77 to 0x8b1761b 2024-09-18T04:56:59.965 INFO:teuthology.orchestra.run.smithi002.stdout:2804 read 0x6a8bda3 thru 0x6a9935b (0xd5b9 bytes) 2024-09-18T04:56:59.965 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:59.965 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:59.965 INFO:teuthology.orchestra.run.smithi002.stdout:2807 write 0x323efb9 thru 0x324eb57 (0xfb9f bytes) 2024-09-18T04:56:59.968 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:56:59.966+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:56:59.969 INFO:teuthology.orchestra.run.smithi002.stdout:2809 punch from 0x5540de6 to 0x554d28d, (0xc4a7 bytes) 2024-09-18T04:56:59.969 INFO:teuthology.orchestra.run.smithi002.stdout:2811 read 0x126aa0e thru 0x1275182 (0xa775 bytes) 2024-09-18T04:56:59.971 INFO:teuthology.orchestra.run.smithi002.stdout:2812 trunc from 0x8b1761b to 0x6f9f155 2024-09-18T04:56:59.977 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:59.985 INFO:teuthology.orchestra.run.smithi002.stdout:2814 trunc from 0x6f9f155 to 0xed073e2 2024-09-18T04:56:59.991 INFO:teuthology.orchestra.run.smithi002.stdout:2816 read 0xd0ce6f6 thru 0xd0de4cd (0xfdd8 bytes) 2024-09-18T04:56:59.992 INFO:teuthology.orchestra.run.smithi002.stdout:2817 write 0xb93d914 thru 0xb947aee (0xa1db bytes) 2024-09-18T04:56:59.994 INFO:teuthology.orchestra.run.smithi002.stdout:2818 punch from 0x14d254a to 0x14d5ea8, (0x395e bytes) 2024-09-18T04:56:59.995 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:56:59.995 INFO:teuthology.orchestra.run.smithi002.stdout:2822 punch from 0x92280f7 to 0x9237a17, (0xf920 bytes) 2024-09-18T04:56:59.996 INFO:teuthology.orchestra.run.smithi002.stdout:2823 trunc from 0xed073e2 to 0xdd55bbd 2024-09-18T04:57:00.001 INFO:teuthology.orchestra.run.smithi002.stdout:2824 clone 23 order 25 su 262144 sc 13 2024-09-18T04:57:00.007 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:00.008 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:00.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:00.008 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:00.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.008 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60344 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1456128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:00.008 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:00.008 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8923664819999999, 2024-09-18T04:57:00.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044026324999999998, 2024-09-18T04:57:00.008 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.009 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.009 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.009 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.009 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.009 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60344 2024-09-18T04:57:00.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.009 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:00.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:00.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:00.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112224+0000", 2024-09-18T04:57:00.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1320000000000001e-06 2024-09-18T04:57:00.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112230+0000", 2024-09-18T04:57:00.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9619999999999998e-06 2024-09-18T04:57:00.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112234+0000", 2024-09-18T04:57:00.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7890000000000002e-06 2024-09-18T04:57:00.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156105+0000", 2024-09-18T04:57:00.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043870531999999997 2024-09-18T04:57:00.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156170+0000", 2024-09-18T04:57:00.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5036000000000005e-05 2024-09-18T04:57:00.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156248+0000", 2024-09-18T04:57:00.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8295999999999997e-05 2024-09-18T04:57:00.014 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.014 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.014 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.014 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60345 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1471488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:00.014 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:57:00.015 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8923497170000001, 2024-09-18T04:57:00.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044180090999999998, 2024-09-18T04:57:00.015 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.015 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.015 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60345 2024-09-18T04:57:00.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.015 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:57:00.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:57:00.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112238+0000", 2024-09-18T04:57:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112240+0000", 2024-09-18T04:57:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9910000000000001e-06 2024-09-18T04:57:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112246+0000", 2024-09-18T04:57:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0859999999999998e-06 2024-09-18T04:57:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112251+0000", 2024-09-18T04:57:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5020000000000003e-06 2024-09-18T04:57:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156274+0000", 2024-09-18T04:57:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044023063000000001 2024-09-18T04:57:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156337+0000", 2024-09-18T04:57:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3077000000000006e-05 2024-09-18T04:57:00.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156419+0000", 2024-09-18T04:57:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1699000000000005e-05 2024-09-18T04:57:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60346 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1486848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:57:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8923319359999997, 2024-09-18T04:57:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044336331999999999, 2024-09-18T04:57:00.019 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60346 2024-09-18T04:57:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:57:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:57:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112256+0000", 2024-09-18T04:57:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112258+0000", 2024-09-18T04:57:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1720000000000001e-06 2024-09-18T04:57:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112265+0000", 2024-09-18T04:57:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4280000000000001e-06 2024-09-18T04:57:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112269+0000", 2024-09-18T04:57:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.741e-06 2024-09-18T04:57:00.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156445+0000", 2024-09-18T04:57:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044175341 2024-09-18T04:57:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156512+0000", 2024-09-18T04:57:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6723000000000005e-05 2024-09-18T04:57:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156593+0000", 2024-09-18T04:57:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1284000000000003e-05 2024-09-18T04:57:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.024 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60347 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1502208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:00.025 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:57:00.025 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8923035249999991, 2024-09-18T04:57:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044482560999999997, 2024-09-18T04:57:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60347 2024-09-18T04:57:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:57:00.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:57:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112284+0000", 2024-09-18T04:57:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112286+0000", 2024-09-18T04:57:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9709999999999998e-06 2024-09-18T04:57:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112290+0000", 2024-09-18T04:57:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9199999999999997e-06 2024-09-18T04:57:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112293+0000", 2024-09-18T04:57:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870000000000001e-06 2024-09-18T04:57:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156617+0000", 2024-09-18T04:57:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044323960000000003 2024-09-18T04:57:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156687+0000", 2024-09-18T04:57:00.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0141000000000005e-05 2024-09-18T04:57:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156767+0000", 2024-09-18T04:57:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0140000000000002e-05 2024-09-18T04:57:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60348 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1517568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8922925209999999, 2024-09-18T04:57:00.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044645562, 2024-09-18T04:57:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60348 2024-09-18T04:57:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112297+0000", 2024-09-18T04:57:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.384e-06 2024-09-18T04:57:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112301+0000", 2024-09-18T04:57:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0570000000000003e-06 2024-09-18T04:57:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112304+0000", 2024-09-18T04:57:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6000000000000001e-06 2024-09-18T04:57:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156793+0000", 2024-09-18T04:57:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044489494999999997 2024-09-18T04:57:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156859+0000", 2024-09-18T04:57:00.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5483999999999998e-05 2024-09-18T04:57:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156941+0000", 2024-09-18T04:57:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2856999999999996e-05 2024-09-18T04:57:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60349 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1532928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:00.035 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8922825450000005, 2024-09-18T04:57:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044832167999999999, 2024-09-18T04:57:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60349 2024-09-18T04:57:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:00.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112307+0000", 2024-09-18T04:57:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.308e-06 2024-09-18T04:57:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112310+0000", 2024-09-18T04:57:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-18T04:57:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112312+0000", 2024-09-18T04:57:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4889999999999998e-06 2024-09-18T04:57:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156977+0000", 2024-09-18T04:57:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044664598999999999 2024-09-18T04:57:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157053+0000", 2024-09-18T04:57:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6014999999999996e-05 2024-09-18T04:57:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157138+0000", 2024-09-18T04:57:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5162999999999998e-05 2024-09-18T04:57:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60350 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1548288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8922736310000001, 2024-09-18T04:57:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045011858000000002, 2024-09-18T04:57:00.040 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60350 2024-09-18T04:57:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112316+0000", 2024-09-18T04:57:00.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7579999999999999e-06 2024-09-18T04:57:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112321+0000", 2024-09-18T04:57:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1649999999999997e-06 2024-09-18T04:57:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112325+0000", 2024-09-18T04:57:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.726e-06 2024-09-18T04:57:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157172+0000", 2024-09-18T04:57:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.04484718 2024-09-18T04:57:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157242+0000", 2024-09-18T04:57:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9675e-05 2024-09-18T04:57:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157327+0000", 2024-09-18T04:57:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4614e-05 2024-09-18T04:57:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.044 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60351 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1563648~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8922602150000003, 2024-09-18T04:57:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045132113000000001, 2024-09-18T04:57:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.045 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60351 2024-09-18T04:57:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112330+0000", 2024-09-18T04:57:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1660000000000001e-06 2024-09-18T04:57:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112336+0000", 2024-09-18T04:57:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5940000000000001e-06 2024-09-18T04:57:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112340+0000", 2024-09-18T04:57:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5589999999999999e-06 2024-09-18T04:57:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157365+0000", 2024-09-18T04:57:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045024675 2024-09-18T04:57:00.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157409+0000", 2024-09-18T04:57:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4356e-05 2024-09-18T04:57:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157460+0000", 2024-09-18T04:57:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1072000000000002e-05 2024-09-18T04:57:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60501 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1494016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8861995189999998, 2024-09-18T04:57:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044007728000000003, 2024-09-18T04:57:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60501 2024-09-18T04:57:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118391+0000", 2024-09-18T04:57:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:57:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118396+0000", 2024-09-18T04:57:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2680000000000002e-06 2024-09-18T04:57:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118400+0000", 2024-09-18T04:57:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2080000000000002e-06 2024-09-18T04:57:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162317+0000", 2024-09-18T04:57:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043916753000000003 2024-09-18T04:57:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162351+0000", 2024-09-18T04:57:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4372000000000002e-05 2024-09-18T04:57:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162397+0000", 2024-09-18T04:57:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5207999999999998e-05 2024-09-18T04:57:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60502 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1509376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.886184042, 2024-09-18T04:57:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044092438999999997, 2024-09-18T04:57:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.054 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60502 2024-09-18T04:57:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:00.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118406+0000", 2024-09-18T04:57:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:57:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118412+0000", 2024-09-18T04:57:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.524e-06 2024-09-18T04:57:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118416+0000", 2024-09-18T04:57:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:57:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162418+0000", 2024-09-18T04:57:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001508000000002 2024-09-18T04:57:00.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162451+0000", 2024-09-18T04:57:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3662999999999999e-05 2024-09-18T04:57:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162497+0000", 2024-09-18T04:57:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5550000000000003e-05 2024-09-18T04:57:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60503 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1524736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:57:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8861678840000007, 2024-09-18T04:57:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044178780000000001, 2024-09-18T04:57:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60503 2024-09-18T04:57:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:57:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:57:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118420+0000", 2024-09-18T04:57:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118422+0000", 2024-09-18T04:57:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.08e-06 2024-09-18T04:57:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118428+0000", 2024-09-18T04:57:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.964e-06 2024-09-18T04:57:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118433+0000", 2024-09-18T04:57:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.425e-06 2024-09-18T04:57:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162519+0000", 2024-09-18T04:57:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044086082999999998 2024-09-18T04:57:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162552+0000", 2024-09-18T04:57:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3719e-05 2024-09-18T04:57:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162599+0000", 2024-09-18T04:57:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6873999999999999e-05 2024-09-18T04:57:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60504 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1540096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:57:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8861517790000004, 2024-09-18T04:57:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044261921000000003, 2024-09-18T04:57:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.063 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60504 2024-09-18T04:57:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:57:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:57:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118436+0000", 2024-09-18T04:57:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118438+0000", 2024-09-18T04:57:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0250000000000001e-06 2024-09-18T04:57:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118444+0000", 2024-09-18T04:57:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4360000000000001e-06 2024-09-18T04:57:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118448+0000", 2024-09-18T04:57:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.357e-06 2024-09-18T04:57:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162620+0000", 2024-09-18T04:57:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044171821 2024-09-18T04:57:00.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162653+0000", 2024-09-18T04:57:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3479999999999998e-05 2024-09-18T04:57:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162699+0000", 2024-09-18T04:57:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5113e-05 2024-09-18T04:57:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60584 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1196032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:57:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8843587619999997, 2024-09-18T04:57:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044048727000000003, 2024-09-18T04:57:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60584 2024-09-18T04:57:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:57:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:57:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120229+0000", 2024-09-18T04:57:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120231+0000", 2024-09-18T04:57:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:57:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120237+0000", 2024-09-18T04:57:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8379999999999998e-06 2024-09-18T04:57:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120242+0000", 2024-09-18T04:57:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3889999999999998e-06 2024-09-18T04:57:00.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164187+0000", 2024-09-18T04:57:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043945045000000002 2024-09-18T04:57:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164227+0000", 2024-09-18T04:57:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0627999999999998e-05 2024-09-18T04:57:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164278+0000", 2024-09-18T04:57:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1146000000000001e-05 2024-09-18T04:57:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60585 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1211392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:57:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8843426539999992, 2024-09-18T04:57:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044389814, 2024-09-18T04:57:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.072 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60585 2024-09-18T04:57:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:57:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:57:00.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120245+0000", 2024-09-18T04:57:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120247+0000", 2024-09-18T04:57:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0150000000000002e-06 2024-09-18T04:57:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120255+0000", 2024-09-18T04:57:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.858e-06 2024-09-18T04:57:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120260+0000", 2024-09-18T04:57:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6530000000000002e-06 2024-09-18T04:57:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164489+0000", 2024-09-18T04:57:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044229483999999999 2024-09-18T04:57:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164555+0000", 2024-09-18T04:57:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.533e-05 2024-09-18T04:57:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164636+0000", 2024-09-18T04:57:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0819e-05 2024-09-18T04:57:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.076 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60586 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1226752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8843242030000003, 2024-09-18T04:57:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044578214999999997, 2024-09-18T04:57:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60586 2024-09-18T04:57:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120266+0000", 2024-09-18T04:57:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0660000000000002e-06 2024-09-18T04:57:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120272+0000", 2024-09-18T04:57:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-18T04:57:00.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120276+0000", 2024-09-18T04:57:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.476e-06 2024-09-18T04:57:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164673+0000", 2024-09-18T04:57:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044396672999999998 2024-09-18T04:57:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164749+0000", 2024-09-18T04:57:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5811999999999999e-05 2024-09-18T04:57:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164842+0000", 2024-09-18T04:57:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3560000000000005e-05 2024-09-18T04:57:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60587 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1242112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:57:00.081 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8842751040000003, 2024-09-18T04:57:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044634794999999998, 2024-09-18T04:57:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60587 2024-09-18T04:57:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:57:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:57:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120280+0000", 2024-09-18T04:57:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999671 2024-09-18T04:57:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120316+0000", 2024-09-18T04:57:00.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5728000000000001e-05 2024-09-18T04:57:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120329+0000", 2024-09-18T04:57:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2914e-05 2024-09-18T04:57:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120335+0000", 2024-09-18T04:57:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8289999999999996e-06 2024-09-18T04:57:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164858+0000", 2024-09-18T04:57:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044523417000000003 2024-09-18T04:57:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164896+0000", 2024-09-18T04:57:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7969999999999997e-05 2024-09-18T04:57:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164948+0000", 2024-09-18T04:57:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1996000000000002e-05 2024-09-18T04:57:00.085 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60595 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1364992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:57:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8824318879999993, 2024-09-18T04:57:00.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001360000000003, 2024-09-18T04:57:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60595 2024-09-18T04:57:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:57:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:57:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122156+0000", 2024-09-18T04:57:00.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122158+0000", 2024-09-18T04:57:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0040000000000002e-06 2024-09-18T04:57:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122164+0000", 2024-09-18T04:57:00.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8719999999999999e-06 2024-09-18T04:57:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122169+0000", 2024-09-18T04:57:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5090000000000004e-06 2024-09-18T04:57:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166040+0000", 2024-09-18T04:57:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043871226999999999 2024-09-18T04:57:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166096+0000", 2024-09-18T04:57:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6495e-05 2024-09-18T04:57:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166158+0000", 2024-09-18T04:57:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1626999999999998e-05 2024-09-18T04:57:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.091 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60596 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1380352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8824155670000007, 2024-09-18T04:57:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044120800000000002, 2024-09-18T04:57:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60596 2024-09-18T04:57:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122175+0000", 2024-09-18T04:57:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:57:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122182+0000", 2024-09-18T04:57:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3509999999999996e-06 2024-09-18T04:57:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122187+0000", 2024-09-18T04:57:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7940000000000002e-06 2024-09-18T04:57:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166184+0000", 2024-09-18T04:57:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043997236000000002 2024-09-18T04:57:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166227+0000", 2024-09-18T04:57:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2561000000000001e-05 2024-09-18T04:57:00.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166294+0000", 2024-09-18T04:57:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7068000000000002e-05 2024-09-18T04:57:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60597 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1395712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8823975389999994, 2024-09-18T04:57:00.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044242166999999999, 2024-09-18T04:57:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60597 2024-09-18T04:57:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122193+0000", 2024-09-18T04:57:00.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:57:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122199+0000", 2024-09-18T04:57:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6449999999999999e-06 2024-09-18T04:57:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122204+0000", 2024-09-18T04:57:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4819999999999996e-06 2024-09-18T04:57:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166329+0000", 2024-09-18T04:57:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044125059000000001 2024-09-18T04:57:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166370+0000", 2024-09-18T04:57:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1012e-05 2024-09-18T04:57:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166433+0000", 2024-09-18T04:57:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3178999999999999e-05 2024-09-18T04:57:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.100 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60608 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1564672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:57:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.8803996769999998, 2024-09-18T04:57:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044082723999999997, 2024-09-18T04:57:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.101 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60608 2024-09-18T04:57:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:57:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:57:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124188+0000", 2024-09-18T04:57:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124190+0000", 2024-09-18T04:57:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0329999999999998e-06 2024-09-18T04:57:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124196+0000", 2024-09-18T04:57:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.575e-06 2024-09-18T04:57:00.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124201+0000", 2024-09-18T04:57:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5000000000000001e-06 2024-09-18T04:57:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168166+0000", 2024-09-18T04:57:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043965763999999997 2024-09-18T04:57:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168218+0000", 2024-09-18T04:57:00.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1785e-05 2024-09-18T04:57:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168271+0000", 2024-09-18T04:57:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3429e-05 2024-09-18T04:57:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.105 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.105 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:00.107 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:57:00.107 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:00.105+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:00.207 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:00.206+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:00.209 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:00.210 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:00.210 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:57:00.210 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:00.218 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:57:00.308 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:00.306+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:00.325 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:00.325 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:00.326 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:57:00.326 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:57:00.326 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:00.335 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:57:00.409 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:00.407+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:00.473 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-09-18T04:57:00.473 INFO:teuthology.orchestra.run.smithi002.stdout:2825 read 0xca9848e thru 0xca9cb23 (0x4696 bytes) 2024-09-18T04:57:00.474 INFO:teuthology.orchestra.run.smithi002.stdout:2826 write 0x432079f thru 0x4328098 (0x78fa bytes) 2024-09-18T04:57:00.481 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26189 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 237568~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.838604266000001, 2024-09-18T04:57:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022263431, 2024-09-18T04:57:00.482 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.483 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26189 2024-09-18T04:57:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639787+0000", 2024-09-18T04:57:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9759999999999998e-06 2024-09-18T04:57:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639792+0000", 2024-09-18T04:57:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5829999999999997e-06 2024-09-18T04:57:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639796+0000", 2024-09-18T04:57:00.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0690000000000003e-06 2024-09-18T04:57:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.661944+0000", 2024-09-18T04:57:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022147463999999999 2024-09-18T04:57:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.661985+0000", 2024-09-18T04:57:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1360999999999999e-05 2024-09-18T04:57:00.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662048+0000", 2024-09-18T04:57:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3298000000000003e-05 2024-09-18T04:57:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.488 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26199 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 4096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.838588980000001, 2024-09-18T04:57:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022348776000000001, 2024-09-18T04:57:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26199 2024-09-18T04:57:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.489 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639806+0000", 2024-09-18T04:57:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1539999999999999e-06 2024-09-18T04:57:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639812+0000", 2024-09-18T04:57:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5520000000000004e-06 2024-09-18T04:57:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639816+0000", 2024-09-18T04:57:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1840000000000001e-06 2024-09-18T04:57:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662063+0000", 2024-09-18T04:57:00.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022247125999999999 2024-09-18T04:57:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662098+0000", 2024-09-18T04:57:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4968000000000001e-05 2024-09-18T04:57:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662149+0000", 2024-09-18T04:57:00.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1122000000000003e-05 2024-09-18T04:57:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26200 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 32768~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.838569454999998, 2024-09-18T04:57:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022501118000000001, 2024-09-18T04:57:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26200 2024-09-18T04:57:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639822+0000", 2024-09-18T04:57:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.007e-06 2024-09-18T04:57:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639827+0000", 2024-09-18T04:57:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5709999999999997e-06 2024-09-18T04:57:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639831+0000", 2024-09-18T04:57:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2819999999999998e-06 2024-09-18T04:57:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662181+0000", 2024-09-18T04:57:00.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022349878 2024-09-18T04:57:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662252+0000", 2024-09-18T04:57:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0542000000000003e-05 2024-09-18T04:57:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662321+0000", 2024-09-18T04:57:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9167999999999999e-05 2024-09-18T04:57:00.498 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26201 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 61440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.838554164000001, 2024-09-18T04:57:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022569783, 2024-09-18T04:57:00.499 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26201 2024-09-18T04:57:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639837+0000", 2024-09-18T04:57:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.928e-06 2024-09-18T04:57:00.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639843+0000", 2024-09-18T04:57:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8130000000000001e-06 2024-09-18T04:57:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639847+0000", 2024-09-18T04:57:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0999999999999997e-06 2024-09-18T04:57:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662335+0000", 2024-09-18T04:57:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022488207999999999 2024-09-18T04:57:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662370+0000", 2024-09-18T04:57:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5292e-05 2024-09-18T04:57:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662405+0000", 2024-09-18T04:57:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4765999999999999e-05 2024-09-18T04:57:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.503 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26202 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 90112~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:57:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.838538747000001, 2024-09-18T04:57:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022655937000000001, 2024-09-18T04:57:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26202 2024-09-18T04:57:00.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:57:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:57:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639850+0000", 2024-09-18T04:57:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639852+0000", 2024-09-18T04:57:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9700000000000002e-06 2024-09-18T04:57:00.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639858+0000", 2024-09-18T04:57:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.434e-06 2024-09-18T04:57:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639862+0000", 2024-09-18T04:57:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3499999999999999e-06 2024-09-18T04:57:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662429+0000", 2024-09-18T04:57:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022567409 2024-09-18T04:57:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662469+0000", 2024-09-18T04:57:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9801000000000001e-05 2024-09-18T04:57:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662507+0000", 2024-09-18T04:57:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7311000000000002e-05 2024-09-18T04:57:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26203 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 118784~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:57:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.838523592000001, 2024-09-18T04:57:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022730358999999999, 2024-09-18T04:57:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.509 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26203 2024-09-18T04:57:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:57:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:57:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639865+0000", 2024-09-18T04:57:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639867+0000", 2024-09-18T04:57:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0889999999999998e-06 2024-09-18T04:57:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639873+0000", 2024-09-18T04:57:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4979999999999997e-06 2024-09-18T04:57:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639877+0000", 2024-09-18T04:57:00.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4510000000000002e-06 2024-09-18T04:57:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662528+0000", 2024-09-18T04:57:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022650791999999999 2024-09-18T04:57:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662563+0000", 2024-09-18T04:57:00.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4530000000000003e-05 2024-09-18T04:57:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662596+0000", 2024-09-18T04:57:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3463000000000001e-05 2024-09-18T04:57:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.514 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26204 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 147456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.8385082, 2024-09-18T04:57:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022803233999999999, 2024-09-18T04:57:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26204 2024-09-18T04:57:00.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639883+0000", 2024-09-18T04:57:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6520000000000002e-06 2024-09-18T04:57:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639888+0000", 2024-09-18T04:57:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2870000000000004e-06 2024-09-18T04:57:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639893+0000", 2024-09-18T04:57:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3839999999999999e-06 2024-09-18T04:57:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662617+0000", 2024-09-18T04:57:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022723876 2024-09-18T04:57:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662651+0000", 2024-09-18T04:57:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3797000000000002e-05 2024-09-18T04:57:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662684+0000", 2024-09-18T04:57:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3828999999999997e-05 2024-09-18T04:57:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26205 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 176128~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.838485239000001, 2024-09-18T04:57:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022919108000000001, 2024-09-18T04:57:00.521 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26205 2024-09-18T04:57:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639906+0000", 2024-09-18T04:57:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.531e-06 2024-09-18T04:57:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639913+0000", 2024-09-18T04:57:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.331e-06 2024-09-18T04:57:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639920+0000", 2024-09-18T04:57:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5399999999999998e-06 2024-09-18T04:57:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662706+0000", 2024-09-18T04:57:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.02278587 2024-09-18T04:57:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662764+0000", 2024-09-18T04:57:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7630000000000002e-05 2024-09-18T04:57:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662823+0000", 2024-09-18T04:57:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9648000000000002e-05 2024-09-18T04:57:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26206 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 204800~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.83846484, 2024-09-18T04:57:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023047517, 2024-09-18T04:57:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26206 2024-09-18T04:57:00.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639926+0000", 2024-09-18T04:57:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3120000000000001e-06 2024-09-18T04:57:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639932+0000", 2024-09-18T04:57:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4890000000000003e-06 2024-09-18T04:57:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639936+0000", 2024-09-18T04:57:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.07e-06 2024-09-18T04:57:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662850+0000", 2024-09-18T04:57:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022913579999999999 2024-09-18T04:57:00.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662903+0000", 2024-09-18T04:57:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3176e-05 2024-09-18T04:57:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662972+0000", 2024-09-18T04:57:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9270999999999995e-05 2024-09-18T04:57:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.531 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26207 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 233472~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:57:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.838449696000001, 2024-09-18T04:57:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023139541999999999, 2024-09-18T04:57:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26207 2024-09-18T04:57:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.532 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:57:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:57:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639939+0000", 2024-09-18T04:57:00.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639941+0000", 2024-09-18T04:57:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0080000000000001e-06 2024-09-18T04:57:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639947+0000", 2024-09-18T04:57:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.485e-06 2024-09-18T04:57:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639951+0000", 2024-09-18T04:57:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4549999999999997e-06 2024-09-18T04:57:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663005+0000", 2024-09-18T04:57:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023053569999999999 2024-09-18T04:57:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663042+0000", 2024-09-18T04:57:00.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7245e-05 2024-09-18T04:57:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663079+0000", 2024-09-18T04:57:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7119000000000002e-05 2024-09-18T04:57:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26208 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 524288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.838432948000001, 2024-09-18T04:57:00.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023283564999999999, 2024-09-18T04:57:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26208 2024-09-18T04:57:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639958+0000", 2024-09-18T04:57:00.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0899999999999999e-06 2024-09-18T04:57:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639963+0000", 2024-09-18T04:57:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2209999999999996e-06 2024-09-18T04:57:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639967+0000", 2024-09-18T04:57:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1169999999999997e-06 2024-09-18T04:57:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663114+0000", 2024-09-18T04:57:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023146102000000002 2024-09-18T04:57:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663183+0000", 2024-09-18T04:57:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9686999999999999e-05 2024-09-18T04:57:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663240+0000", 2024-09-18T04:57:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6728000000000003e-05 2024-09-18T04:57:00.540 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.541 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26209 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 552960~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.838415172000001, 2024-09-18T04:57:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023440966000000001, 2024-09-18T04:57:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26209 2024-09-18T04:57:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:00.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639976+0000", 2024-09-18T04:57:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4940000000000002e-06 2024-09-18T04:57:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639982+0000", 2024-09-18T04:57:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0560000000000001e-06 2024-09-18T04:57:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639987+0000", 2024-09-18T04:57:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-18T04:57:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663278+0000", 2024-09-18T04:57:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023290789999999999 2024-09-18T04:57:00.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663348+0000", 2024-09-18T04:57:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0837e-05 2024-09-18T04:57:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663415+0000", 2024-09-18T04:57:00.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6736000000000006e-05 2024-09-18T04:57:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26210 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 581632~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.838398164000001, 2024-09-18T04:57:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023616437000000001, 2024-09-18T04:57:00.548 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26210 2024-09-18T04:57:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639993+0000", 2024-09-18T04:57:00.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.419e-06 2024-09-18T04:57:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640006+0000", 2024-09-18T04:57:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2918000000000001e-05 2024-09-18T04:57:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640011+0000", 2024-09-18T04:57:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9320000000000004e-06 2024-09-18T04:57:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663454+0000", 2024-09-18T04:57:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023443035000000001 2024-09-18T04:57:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663532+0000", 2024-09-18T04:57:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7443999999999999e-05 2024-09-18T04:57:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663608+0000", 2024-09-18T04:57:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6093000000000005e-05 2024-09-18T04:57:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26211 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 610304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:57:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.838373612000002, 2024-09-18T04:57:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023774337999999999, 2024-09-18T04:57:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26211 2024-09-18T04:57:00.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:57:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:57:00.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640015+0000", 2024-09-18T04:57:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640023+0000", 2024-09-18T04:57:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3900000000000004e-06 2024-09-18T04:57:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640036+0000", 2024-09-18T04:57:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3091999999999999e-05 2024-09-18T04:57:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640041+0000", 2024-09-18T04:57:00.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6219999999999997e-06 2024-09-18T04:57:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663644+0000", 2024-09-18T04:57:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023602643999999999 2024-09-18T04:57:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663718+0000", 2024-09-18T04:57:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4481999999999996e-05 2024-09-18T04:57:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663790+0000", 2024-09-18T04:57:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1589999999999997e-05 2024-09-18T04:57:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26212 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 638976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:57:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.838340572, 2024-09-18T04:57:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023924748999999999, 2024-09-18T04:57:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.560 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26212 2024-09-18T04:57:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:57:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:57:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640048+0000", 2024-09-18T04:57:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640051+0000", 2024-09-18T04:57:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.095e-06 2024-09-18T04:57:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640060+0000", 2024-09-18T04:57:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8929999999999994e-06 2024-09-18T04:57:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640065+0000", 2024-09-18T04:57:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0270000000000003e-06 2024-09-18T04:57:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663824+0000", 2024-09-18T04:57:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023758873999999999 2024-09-18T04:57:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663904+0000", 2024-09-18T04:57:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9856000000000003e-05 2024-09-18T04:57:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663974+0000", 2024-09-18T04:57:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9450999999999996e-05 2024-09-18T04:57:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26213 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 667648~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.838319401, 2024-09-18T04:57:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024059812999999999, 2024-09-18T04:57:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26213 2024-09-18T04:57:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640072+0000", 2024-09-18T04:57:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4200000000000001e-06 2024-09-18T04:57:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640078+0000", 2024-09-18T04:57:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0179999999999998e-06 2024-09-18T04:57:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640083+0000", 2024-09-18T04:57:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:57:00.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664008+0000", 2024-09-18T04:57:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023925287 2024-09-18T04:57:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664059+0000", 2024-09-18T04:57:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0730999999999999e-05 2024-09-18T04:57:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664130+0000", 2024-09-18T04:57:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0801000000000002e-05 2024-09-18T04:57:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26214 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 696320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.838302430999999, 2024-09-18T04:57:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024223738000000002, 2024-09-18T04:57:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.569 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26214 2024-09-18T04:57:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640089+0000", 2024-09-18T04:57:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0849999999999999e-06 2024-09-18T04:57:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640095+0000", 2024-09-18T04:57:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9279999999999998e-06 2024-09-18T04:57:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640099+0000", 2024-09-18T04:57:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:57:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664166+0000", 2024-09-18T04:57:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024066513000000001 2024-09-18T04:57:00.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664239+0000", 2024-09-18T04:57:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3201999999999995e-05 2024-09-18T04:57:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664311+0000", 2024-09-18T04:57:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1434999999999996e-05 2024-09-18T04:57:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.573 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26215 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 724992~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:57:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.838285670000001, 2024-09-18T04:57:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024302284, 2024-09-18T04:57:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.574 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26215 2024-09-18T04:57:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:57:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:57:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640103+0000", 2024-09-18T04:57:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640106+0000", 2024-09-18T04:57:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2120000000000002e-06 2024-09-18T04:57:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640111+0000", 2024-09-18T04:57:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7880000000000003e-06 2024-09-18T04:57:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640116+0000", 2024-09-18T04:57:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6099999999999999e-06 2024-09-18T04:57:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664327+0000", 2024-09-18T04:57:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024210941999999999 2024-09-18T04:57:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664367+0000", 2024-09-18T04:57:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9666000000000003e-05 2024-09-18T04:57:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664406+0000", 2024-09-18T04:57:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9402e-05 2024-09-18T04:57:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26216 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 753664~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.838269176000001, 2024-09-18T04:57:00.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024452854999999999, 2024-09-18T04:57:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26216 2024-09-18T04:57:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640122+0000", 2024-09-18T04:57:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3599999999999999e-06 2024-09-18T04:57:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640128+0000", 2024-09-18T04:57:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6339999999999998e-06 2024-09-18T04:57:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640132+0000", 2024-09-18T04:57:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:57:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664436+0000", 2024-09-18T04:57:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024303585999999999 2024-09-18T04:57:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664507+0000", 2024-09-18T04:57:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1046999999999999e-05 2024-09-18T04:57:00.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664573+0000", 2024-09-18T04:57:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6217000000000007e-05 2024-09-18T04:57:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.585 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26217 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 782336~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.838252942, 2024-09-18T04:57:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024526287000000001, 2024-09-18T04:57:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.586 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26217 2024-09-18T04:57:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640140+0000", 2024-09-18T04:57:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2989999999999998e-06 2024-09-18T04:57:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640146+0000", 2024-09-18T04:57:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9569999999999999e-06 2024-09-18T04:57:00.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640151+0000", 2024-09-18T04:57:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:57:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664590+0000", 2024-09-18T04:57:00.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0244389 2024-09-18T04:57:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664628+0000", 2024-09-18T04:57:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7910999999999997e-05 2024-09-18T04:57:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664663+0000", 2024-09-18T04:57:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5024999999999998e-05 2024-09-18T04:57:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.592 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.592 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:00.594 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:57:00.595 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:00.591+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:00.596 INFO:teuthology.orchestra.run.smithi002.stdout:2827 read 0x1d4c2ac thru 0x1d58043 (0xbd98 bytes) 2024-09-18T04:57:00.596 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:00.596 INFO:teuthology.orchestra.run.smithi002.stdout:2829 write 0x58cee2c thru 0x58de15b (0xf330 bytes) 2024-09-18T04:57:00.596 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:00.596 INFO:teuthology.orchestra.run.smithi002.stdout:2833 read 0x821ef0d thru 0x822965b (0xa74f bytes) 2024-09-18T04:57:00.596 INFO:teuthology.orchestra.run.smithi002.stdout:2834 write 0x5a6222c thru 0x5a6f25e (0xd033 bytes) 2024-09-18T04:57:00.596 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:00.597 INFO:teuthology.orchestra.run.smithi002.stdout:2838 writesame 0x12c0f76 thru 0x12cbe59 data_size 0x7b(0xaee4 bytes) 2024-09-18T04:57:00.597 INFO:teuthology.orchestra.run.smithi002.stdout:2839 write 0x536b157 thru 0x536f593 (0x443d bytes) 2024-09-18T04:57:00.597 INFO:teuthology.orchestra.run.smithi002.stdout:2840 writesame 0x3f89317 thru 0x3f8f17f data_size 0x1(0x5e69 bytes) 2024-09-18T04:57:00.597 INFO:teuthology.orchestra.run.smithi002.stdout:2842 punch from 0xa39e016 to 0xa3a89d7, (0xa9c1 bytes) 2024-09-18T04:57:00.597 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:00.597 INFO:teuthology.orchestra.run.smithi002.stdout:2844 writesame 0xa06f2c2 thru 0xa06f7e3 data_size 0x9(0x522 bytes) 2024-09-18T04:57:00.597 INFO:teuthology.orchestra.run.smithi002.stdout:2845 read 0xffa4f7 thru 0x10053da (0xaee4 bytes) 2024-09-18T04:57:00.597 INFO:teuthology.orchestra.run.smithi002.stdout:2846 punch from 0xc7676bf to 0xc76f4ae, (0x7def bytes) 2024-09-18T04:57:00.597 INFO:teuthology.orchestra.run.smithi002.stdout:2847 writesame 0x792641b thru 0x792fac4 data_size 0x46(0x96aa bytes) 2024-09-18T04:57:00.597 INFO:teuthology.orchestra.run.smithi002.stdout:2848 writesame 0xd544982 thru 0xd54e28f data_size 0x89(0x990e bytes) 2024-09-18T04:57:00.597 INFO:teuthology.orchestra.run.smithi002.stdout:2849 writesame 0xc11d2a3 thru 0xc120e0b data_size 0x1(0x3b69 bytes) 2024-09-18T04:57:00.597 INFO:teuthology.orchestra.run.smithi002.stdout:2850 punch from 0xca571aa to 0xca58858, (0x16ae bytes) 2024-09-18T04:57:00.597 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:00.598 INFO:teuthology.orchestra.run.smithi002.stdout:2852 trunc from 0xdd55bbd to 0x9892836 2024-09-18T04:57:00.598 INFO:teuthology.orchestra.run.smithi002.stdout:2853 trunc from 0x9892836 to 0xc8a910c 2024-09-18T04:57:00.598 INFO:teuthology.orchestra.run.smithi002.stdout:2854 read 0xc46b219 thru 0xc46bc77 (0xa5f bytes) 2024-09-18T04:57:00.598 INFO:teuthology.orchestra.run.smithi002.stdout:2855 clone 24 order 19 su 4096 sc 10 2024-09-18T04:57:00.694 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:00.693+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:00.737 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:57:00.738 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:00.746 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:57:00.795 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:00.793+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:00.847 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:00.847 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:00.848 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:57:00.848 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:57:00.848 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:00.855 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:57:00.899 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:00.897+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:00.960 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:00.960 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:00.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:00.960 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:00.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.960 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:57:00.961 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:57:00.961 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 49.562785634000001, 2024-09-18T04:57:00.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037718511000000003, 2024-09-18T04:57:00.961 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.961 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:00.961 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.961 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:57:00.961 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:57:00.961 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-18T04:57:00.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.961 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:57:00.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:57:00.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393125+0000", 2024-09-18T04:57:00.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:57:00.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393133+0000", 2024-09-18T04:57:00.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.225e-06 2024-09-18T04:57:00.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393144+0000", 2024-09-18T04:57:00.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1279e-05 2024-09-18T04:57:00.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393153+0000", 2024-09-18T04:57:00.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4540000000000007e-06 2024-09-18T04:57:00.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393203+0000", 2024-09-18T04:57:00.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9963000000000003e-05 2024-09-18T04:57:00.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:57:00.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393210+0000", 2024-09-18T04:57:00.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5689999999999999e-06 2024-09-18T04:57:00.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427399+0000", 2024-09-18T04:57:00.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034188784999999999 2024-09-18T04:57:00.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427532+0000", 2024-09-18T04:57:00.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013264700000000001 2024-09-18T04:57:00.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:57:00.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427685+0000", 2024-09-18T04:57:00.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015300599999999999 2024-09-18T04:57:00.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:00.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.428520+0000", 2024-09-18T04:57:00.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00083502499999999998 2024-09-18T04:57:00.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:57:00.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430772+0000", 2024-09-18T04:57:00.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022518880000000001 2024-09-18T04:57:00.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:57:00.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430817+0000", 2024-09-18T04:57:00.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5077000000000002e-05 2024-09-18T04:57:00.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430845+0000", 2024-09-18T04:57:00.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8036999999999999e-05 2024-09-18T04:57:00.967 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.967 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.967 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.967 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54065 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3453952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:00.967 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221852+0000", 2024-09-18T04:57:00.967 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.734060276999999, 2024-09-18T04:57:00.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029151789000000001, 2024-09-18T04:57:00.967 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.967 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.968 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.968 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.968 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.968 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54065 2024-09-18T04:57:00.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.968 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221852+0000", 2024-09-18T04:57:00.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221852+0000", 2024-09-18T04:57:00.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221851+0000", 2024-09-18T04:57:00.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221853+0000", 2024-09-18T04:57:00.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9570000000000001e-06 2024-09-18T04:57:00.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221859+0000", 2024-09-18T04:57:00.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0410000000000003e-06 2024-09-18T04:57:00.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221870+0000", 2024-09-18T04:57:00.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0801e-05 2024-09-18T04:57:00.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.250810+0000", 2024-09-18T04:57:00.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.028940006000000001 2024-09-18T04:57:00.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.250882+0000", 2024-09-18T04:57:00.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1737999999999997e-05 2024-09-18T04:57:00.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251003+0000", 2024-09-18T04:57:00.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121605 2024-09-18T04:57:00.972 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.972 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.972 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.972 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54066 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3469312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:00.972 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221875+0000", 2024-09-18T04:57:00.972 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.734037098, 2024-09-18T04:57:00.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029331864999999999, 2024-09-18T04:57:00.972 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.973 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.973 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.973 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.973 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.973 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54066 2024-09-18T04:57:00.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.973 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221875+0000", 2024-09-18T04:57:00.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221875+0000", 2024-09-18T04:57:00.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221874+0000", 2024-09-18T04:57:00.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221877+0000", 2024-09-18T04:57:00.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1280000000000002e-06 2024-09-18T04:57:00.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221883+0000", 2024-09-18T04:57:00.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7109999999999998e-06 2024-09-18T04:57:00.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221888+0000", 2024-09-18T04:57:00.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.668e-06 2024-09-18T04:57:00.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251030+0000", 2024-09-18T04:57:00.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029142208999999999 2024-09-18T04:57:00.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251138+0000", 2024-09-18T04:57:00.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107518 2024-09-18T04:57:00.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251207+0000", 2024-09-18T04:57:00.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8978000000000003e-05 2024-09-18T04:57:00.977 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.977 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.977 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.977 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54067 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3484672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:00.977 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221893+0000", 2024-09-18T04:57:00.977 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.734018953, 2024-09-18T04:57:00.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029508817999999999, 2024-09-18T04:57:00.977 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.977 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.977 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.977 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.978 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.978 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54067 2024-09-18T04:57:00.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.978 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221893+0000", 2024-09-18T04:57:00.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221893+0000", 2024-09-18T04:57:00.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221893+0000", 2024-09-18T04:57:00.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221895+0000", 2024-09-18T04:57:00.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9709999999999998e-06 2024-09-18T04:57:00.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221902+0000", 2024-09-18T04:57:00.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4379999999999998e-06 2024-09-18T04:57:00.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221907+0000", 2024-09-18T04:57:00.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0019999999999997e-06 2024-09-18T04:57:00.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251260+0000", 2024-09-18T04:57:00.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029353338999999999 2024-09-18T04:57:00.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251336+0000", 2024-09-18T04:57:00.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5591000000000002e-05 2024-09-18T04:57:00.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251402+0000", 2024-09-18T04:57:00.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5752000000000003e-05 2024-09-18T04:57:00.981 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.981 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.982 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.982 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54068 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3500032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:00.982 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221912+0000", 2024-09-18T04:57:00.982 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.734000264000001, 2024-09-18T04:57:00.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029654553, 2024-09-18T04:57:00.982 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.982 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.982 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.982 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.982 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.982 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54068 2024-09-18T04:57:00.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.983 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221912+0000", 2024-09-18T04:57:00.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221912+0000", 2024-09-18T04:57:00.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221911+0000", 2024-09-18T04:57:00.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221916+0000", 2024-09-18T04:57:00.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0130000000000001e-06 2024-09-18T04:57:00.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221922+0000", 2024-09-18T04:57:00.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.105e-06 2024-09-18T04:57:00.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221928+0000", 2024-09-18T04:57:00.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0430000000000004e-06 2024-09-18T04:57:00.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251427+0000", 2024-09-18T04:57:00.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029498586 2024-09-18T04:57:00.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251495+0000", 2024-09-18T04:57:00.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8125000000000003e-05 2024-09-18T04:57:00.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251566+0000", 2024-09-18T04:57:00.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1145999999999999e-05 2024-09-18T04:57:00.986 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.986 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.986 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.986 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54069 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3515392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:00.986 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221945+0000", 2024-09-18T04:57:00.986 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.733966655, 2024-09-18T04:57:00.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029786185999999999, 2024-09-18T04:57:00.986 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.987 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.987 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.987 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.987 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.987 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54069 2024-09-18T04:57:00.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.987 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221945+0000", 2024-09-18T04:57:00.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221945+0000", 2024-09-18T04:57:00.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221945+0000", 2024-09-18T04:57:00.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221947+0000", 2024-09-18T04:57:00.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6189999999999998e-06 2024-09-18T04:57:00.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221955+0000", 2024-09-18T04:57:00.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6249999999999998e-06 2024-09-18T04:57:00.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221968+0000", 2024-09-18T04:57:00.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2846e-05 2024-09-18T04:57:00.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251595+0000", 2024-09-18T04:57:00.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029627158000000001 2024-09-18T04:57:00.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251666+0000", 2024-09-18T04:57:00.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1437000000000001e-05 2024-09-18T04:57:00.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251731+0000", 2024-09-18T04:57:00.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5071000000000001e-05 2024-09-18T04:57:00.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.990 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.991 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54070 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:00.991 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221973+0000", 2024-09-18T04:57:00.991 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.733938672000001, 2024-09-18T04:57:00.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029891219, 2024-09-18T04:57:00.991 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.991 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.991 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.991 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.991 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.991 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54070 2024-09-18T04:57:00.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.991 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221973+0000", 2024-09-18T04:57:00.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221973+0000", 2024-09-18T04:57:00.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221973+0000", 2024-09-18T04:57:00.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221985+0000", 2024-09-18T04:57:00.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1888999999999999e-05 2024-09-18T04:57:00.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221992+0000", 2024-09-18T04:57:00.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2830000000000004e-06 2024-09-18T04:57:00.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222008+0000", 2024-09-18T04:57:00.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5608999999999999e-05 2024-09-18T04:57:00.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251757+0000", 2024-09-18T04:57:00.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029749652000000001 2024-09-18T04:57:00.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251826+0000", 2024-09-18T04:57:00.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9103000000000006e-05 2024-09-18T04:57:00.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:00.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251864+0000", 2024-09-18T04:57:00.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8179000000000002e-05 2024-09-18T04:57:00.995 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.995 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:00.995 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:00.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.996 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54071 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3546112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:00.996 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222014+0000", 2024-09-18T04:57:00.996 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.733898241, 2024-09-18T04:57:00.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03002749, 2024-09-18T04:57:00.996 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:00.996 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:00.996 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:00.996 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:00.996 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:00.996 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54071 2024-09-18T04:57:00.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.996 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:00.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:00.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222014+0000", 2024-09-18T04:57:00.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:00.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222014+0000", 2024-09-18T04:57:00.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:00.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:00.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222013+0000", 2024-09-18T04:57:00.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:00.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:00.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222016+0000", 2024-09-18T04:57:00.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6639999999999998e-06 2024-09-18T04:57:00.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:00.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222023+0000", 2024-09-18T04:57:00.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.849e-06 2024-09-18T04:57:00.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:00.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222028+0000", 2024-09-18T04:57:00.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0030000000000002e-06 2024-09-18T04:57:00.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:00.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251895+0000", 2024-09-18T04:57:00.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029867482000000001 2024-09-18T04:57:00.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:00.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251963+0000", 2024-09-18T04:57:00.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8038000000000001e-05 2024-09-18T04:57:00.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:00.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:00.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:01.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252041+0000", 2024-09-18T04:57:01.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8028999999999994e-05 2024-09-18T04:57:01.000 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.000 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:01.000 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.000 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54072 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3561472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:01.000 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222034+0000", 2024-09-18T04:57:01.000 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.733878171000001, 2024-09-18T04:57:01.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030190214, 2024-09-18T04:57:01.000 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:01.000 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:01.001 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:01.001 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:01.001 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.001 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54072 2024-09-18T04:57:01.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.001 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:01.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:01.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222034+0000", 2024-09-18T04:57:01.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:01.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:01.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222034+0000", 2024-09-18T04:57:01.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:01.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:01.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222033+0000", 2024-09-18T04:57:01.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:01.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222035+0000", 2024-09-18T04:57:01.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0490000000000002e-06 2024-09-18T04:57:01.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:01.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222051+0000", 2024-09-18T04:57:01.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5914e-05 2024-09-18T04:57:01.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222057+0000", 2024-09-18T04:57:01.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4269999999999999e-06 2024-09-18T04:57:01.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:01.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252079+0000", 2024-09-18T04:57:01.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030022338999999999 2024-09-18T04:57:01.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:01.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252157+0000", 2024-09-18T04:57:01.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7459999999999994e-05 2024-09-18T04:57:01.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:01.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252224+0000", 2024-09-18T04:57:01.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7415000000000004e-05 2024-09-18T04:57:01.005 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.005 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:01.005 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.005 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54073 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3576832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:01.005 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222062+0000", 2024-09-18T04:57:01.005 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.733850393999999, 2024-09-18T04:57:01.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030280892, 2024-09-18T04:57:01.006 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:01.006 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:01.006 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:01.006 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:01.006 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.006 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54073 2024-09-18T04:57:01.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.006 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:01.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:01.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222062+0000", 2024-09-18T04:57:01.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:01.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:01.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222062+0000", 2024-09-18T04:57:01.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:01.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:01.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222061+0000", 2024-09-18T04:57:01.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:01.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222063+0000", 2024-09-18T04:57:01.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0729999999999999e-06 2024-09-18T04:57:01.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:01.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222070+0000", 2024-09-18T04:57:01.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3199999999999996e-06 2024-09-18T04:57:01.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222074+0000", 2024-09-18T04:57:01.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8659999999999997e-06 2024-09-18T04:57:01.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:01.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252255+0000", 2024-09-18T04:57:01.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030180350000000002 2024-09-18T04:57:01.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:01.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252305+0000", 2024-09-18T04:57:01.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0086000000000001e-05 2024-09-18T04:57:01.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:01.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252342+0000", 2024-09-18T04:57:01.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7594e-05 2024-09-18T04:57:01.010 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.010 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:01.010 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.010 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54074 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3592192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:01.010 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222079+0000", 2024-09-18T04:57:01.010 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.733833003000001, 2024-09-18T04:57:01.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030370844000000001, 2024-09-18T04:57:01.010 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:01.010 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:01.010 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:01.010 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:01.010 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.011 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54074 2024-09-18T04:57:01.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.011 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:01.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:01.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222079+0000", 2024-09-18T04:57:01.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:01.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:01.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222079+0000", 2024-09-18T04:57:01.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:01.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:01.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222079+0000", 2024-09-18T04:57:01.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:01.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222081+0000", 2024-09-18T04:57:01.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0820000000000001e-06 2024-09-18T04:57:01.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:01.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222087+0000", 2024-09-18T04:57:01.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0789999999999997e-06 2024-09-18T04:57:01.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222097+0000", 2024-09-18T04:57:01.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0446e-05 2024-09-18T04:57:01.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:01.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252365+0000", 2024-09-18T04:57:01.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030267544 2024-09-18T04:57:01.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:01.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252402+0000", 2024-09-18T04:57:01.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7117999999999999e-05 2024-09-18T04:57:01.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:01.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252450+0000", 2024-09-18T04:57:01.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7930999999999999e-05 2024-09-18T04:57:01.014 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.014 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:01.014 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.015 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54075 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3607552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:01.015 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222103+0000", 2024-09-18T04:57:01.015 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.733808951, 2024-09-18T04:57:01.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030531175000000001, 2024-09-18T04:57:01.015 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:01.015 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:01.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:01.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:01.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.015 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54075 2024-09-18T04:57:01.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.015 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:01.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:01.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222103+0000", 2024-09-18T04:57:01.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:01.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:01.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222103+0000", 2024-09-18T04:57:01.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:01.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:01.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222101+0000", 2024-09-18T04:57:01.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:57:01.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:01.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222105+0000", 2024-09-18T04:57:01.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4089999999999999e-06 2024-09-18T04:57:01.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:01.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222111+0000", 2024-09-18T04:57:01.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3720000000000001e-06 2024-09-18T04:57:01.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222116+0000", 2024-09-18T04:57:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.544e-06 2024-09-18T04:57:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252475+0000", 2024-09-18T04:57:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030358975999999999 2024-09-18T04:57:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252562+0000", 2024-09-18T04:57:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7257999999999997e-05 2024-09-18T04:57:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:01.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252634+0000", 2024-09-18T04:57:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2144999999999995e-05 2024-09-18T04:57:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54076 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3622912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222121+0000", 2024-09-18T04:57:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.73379106, 2024-09-18T04:57:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030699046000000001, 2024-09-18T04:57:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:01.019 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54076 2024-09-18T04:57:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222121+0000", 2024-09-18T04:57:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222121+0000", 2024-09-18T04:57:01.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222120+0000", 2024-09-18T04:57:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222123+0000", 2024-09-18T04:57:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2199999999999999e-06 2024-09-18T04:57:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222129+0000", 2024-09-18T04:57:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5250000000000002e-06 2024-09-18T04:57:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222134+0000", 2024-09-18T04:57:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8180000000000003e-06 2024-09-18T04:57:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252677+0000", 2024-09-18T04:57:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030542969999999999 2024-09-18T04:57:01.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252750+0000", 2024-09-18T04:57:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2601000000000005e-05 2024-09-18T04:57:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252820+0000", 2024-09-18T04:57:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0409999999999998e-05 2024-09-18T04:57:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54077 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3638272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222138+0000", 2024-09-18T04:57:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.733773559999999, 2024-09-18T04:57:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031008865, 2024-09-18T04:57:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54077 2024-09-18T04:57:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222138+0000", 2024-09-18T04:57:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222138+0000", 2024-09-18T04:57:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222138+0000", 2024-09-18T04:57:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222140+0000", 2024-09-18T04:57:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9700000000000002e-06 2024-09-18T04:57:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222146+0000", 2024-09-18T04:57:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1460000000000001e-06 2024-09-18T04:57:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222151+0000", 2024-09-18T04:57:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.763e-06 2024-09-18T04:57:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252989+0000", 2024-09-18T04:57:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030838580000000001 2024-09-18T04:57:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253072+0000", 2024-09-18T04:57:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2204000000000001e-05 2024-09-18T04:57:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253147+0000", 2024-09-18T04:57:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5661000000000006e-05 2024-09-18T04:57:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54078 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222155+0000", 2024-09-18T04:57:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.733756974, 2024-09-18T04:57:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031175118000000002, 2024-09-18T04:57:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:01.028 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:01.029 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:01.029 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.029 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54078 2024-09-18T04:57:01.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.032 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:01.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:01.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222155+0000", 2024-09-18T04:57:01.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:01.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222155+0000", 2024-09-18T04:57:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222155+0000", 2024-09-18T04:57:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222157+0000", 2024-09-18T04:57:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0660000000000002e-06 2024-09-18T04:57:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222163+0000", 2024-09-18T04:57:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2820000000000002e-06 2024-09-18T04:57:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222168+0000", 2024-09-18T04:57:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7979999999999996e-06 2024-09-18T04:57:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253174+0000", 2024-09-18T04:57:01.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031005721999999999 2024-09-18T04:57:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253251+0000", 2024-09-18T04:57:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7717000000000001e-05 2024-09-18T04:57:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253330+0000", 2024-09-18T04:57:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8858999999999998e-05 2024-09-18T04:57:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54079 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3668992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222172+0000", 2024-09-18T04:57:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.733739824000001, 2024-09-18T04:57:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031327661, 2024-09-18T04:57:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54079 2024-09-18T04:57:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.036 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222172+0000", 2024-09-18T04:57:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222172+0000", 2024-09-18T04:57:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222172+0000", 2024-09-18T04:57:01.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222174+0000", 2024-09-18T04:57:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0159999999999998e-06 2024-09-18T04:57:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222180+0000", 2024-09-18T04:57:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9950000000000002e-06 2024-09-18T04:57:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222185+0000", 2024-09-18T04:57:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7679999999999999e-06 2024-09-18T04:57:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253360+0000", 2024-09-18T04:57:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031175423000000001 2024-09-18T04:57:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253430+0000", 2024-09-18T04:57:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9929000000000001e-05 2024-09-18T04:57:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:01.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253500+0000", 2024-09-18T04:57:01.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9896999999999998e-05 2024-09-18T04:57:01.040 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.040 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:01.040 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.041 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54332 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3623936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:01.041 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.229403+0000", 2024-09-18T04:57:01.041 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.726509140999999, 2024-09-18T04:57:01.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029249332999999999, 2024-09-18T04:57:01.041 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:01.041 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:01.041 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:01.041 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:01.041 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.041 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54332 2024-09-18T04:57:01.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.041 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:01.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229403+0000", 2024-09-18T04:57:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229403+0000", 2024-09-18T04:57:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229402+0000", 2024-09-18T04:57:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229405+0000", 2024-09-18T04:57:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0150000000000002e-06 2024-09-18T04:57:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229412+0000", 2024-09-18T04:57:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6260000000000003e-06 2024-09-18T04:57:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229417+0000", 2024-09-18T04:57:01.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6199999999999998e-06 2024-09-18T04:57:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258523+0000", 2024-09-18T04:57:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029106396999999999 2024-09-18T04:57:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258589+0000", 2024-09-18T04:57:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6054000000000001e-05 2024-09-18T04:57:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258652+0000", 2024-09-18T04:57:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2917999999999996e-05 2024-09-18T04:57:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54333 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3639296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.229446+0000", 2024-09-18T04:57:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.726466143, 2024-09-18T04:57:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029358565, 2024-09-18T04:57:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:01.045 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54333 2024-09-18T04:57:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229446+0000", 2024-09-18T04:57:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229446+0000", 2024-09-18T04:57:01.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:01.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:01.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229445+0000", 2024-09-18T04:57:01.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:01.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229448+0000", 2024-09-18T04:57:01.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8200000000000001e-06 2024-09-18T04:57:01.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229456+0000", 2024-09-18T04:57:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.371e-06 2024-09-18T04:57:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229473+0000", 2024-09-18T04:57:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6595e-05 2024-09-18T04:57:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258677+0000", 2024-09-18T04:57:01.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029203863 2024-09-18T04:57:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258742+0000", 2024-09-18T04:57:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5273999999999999e-05 2024-09-18T04:57:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258804+0000", 2024-09-18T04:57:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2176999999999998e-05 2024-09-18T04:57:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54334 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3654656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.229481+0000", 2024-09-18T04:57:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 10.726430862000001, 2024-09-18T04:57:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029474845999999999, 2024-09-18T04:57:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54334 2024-09-18T04:57:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:01.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229481+0000", 2024-09-18T04:57:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229481+0000", 2024-09-18T04:57:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229481+0000", 2024-09-18T04:57:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229483+0000", 2024-09-18T04:57:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7580000000000001e-06 2024-09-18T04:57:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229490+0000", 2024-09-18T04:57:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7020000000000004e-06 2024-09-18T04:57:01.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229495+0000", 2024-09-18T04:57:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7770000000000002e-06 2024-09-18T04:57:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258830+0000", 2024-09-18T04:57:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029334828 2024-09-18T04:57:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258896+0000", 2024-09-18T04:57:01.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6384999999999995e-05 2024-09-18T04:57:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258956+0000", 2024-09-18T04:57:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9889999999999997e-05 2024-09-18T04:57:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:62520 3.13 3.84cdb893 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T04:57:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:01.055 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.6584757509999992, 2024-09-18T04:57:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94452134799999998, 2024-09-18T04:57:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:57:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 62520 2024-09-18T04:57:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:01.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297441+0000", 2024-09-18T04:57:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9840000000000001e-06 2024-09-18T04:57:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297449+0000", 2024-09-18T04:57:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3999999999999992e-06 2024-09-18T04:57:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297455+0000", 2024-09-18T04:57:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3350000000000003e-06 2024-09-18T04:57:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:01.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297552+0000", 2024-09-18T04:57:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6602999999999996e-05 2024-09-18T04:57:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:57:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297559+0000", 2024-09-18T04:57:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.52e-06 2024-09-18T04:57:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241943+0000", 2024-09-18T04:57:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94438357100000003 2024-09-18T04:57:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:01.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:01.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241958+0000", 2024-09-18T04:57:01.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4598e-05 2024-09-18T04:57:01.060 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.060 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:01.060 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.060 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:01.060 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:01.060 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:01.061 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:01.059+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:01.062 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:57:01.162 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:01.159+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:01.172 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:01.173 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:01.173 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:01.173 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:01.181 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:57:01.263 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:01.261+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:01.289 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:01.289 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:01.289 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:01.289 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:01.290 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:01.294 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T04:57:01.298 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:57:01.339 INFO:teuthology.orchestra.run.smithi002.stdout:leaving image image_client.0-clone23 intact 2024-09-18T04:57:01.364 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:01.362+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:01.455 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:01.456 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:01.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:01.456 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:01.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.456 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9316 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:01.456 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:01.456 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 25.685833110000001, 2024-09-18T04:57:01.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044027208999999998, 2024-09-18T04:57:01.456 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:01.456 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:01.456 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:01.456 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:01.457 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.457 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9316 2024-09-18T04:57:01.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.457 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:01.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:01.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:01.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:01.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:01.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:01.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:01.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:01.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767144+0000", 2024-09-18T04:57:01.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3070000000000001e-06 2024-09-18T04:57:01.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:01.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767150+0000", 2024-09-18T04:57:01.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1700000000000002e-06 2024-09-18T04:57:01.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767163+0000", 2024-09-18T04:57:01.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2670999999999999e-05 2024-09-18T04:57:01.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:01.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810995+0000", 2024-09-18T04:57:01.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043832310999999999 2024-09-18T04:57:01.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:01.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811087+0000", 2024-09-18T04:57:01.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1345e-05 2024-09-18T04:57:01.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:01.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811169+0000", 2024-09-18T04:57:01.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2848000000000004e-05 2024-09-18T04:57:01.466 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.466 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:01.467 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.467 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9317 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1728512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:01.468 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:57:01.468 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 25.685807475000001, 2024-09-18T04:57:01.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044190170000000001, 2024-09-18T04:57:01.468 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:01.469 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:01.469 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:01.469 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:01.469 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.469 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9317 2024-09-18T04:57:01.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.470 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:01.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:01.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:57:01.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:01.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:57:01.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:01.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767167+0000", 2024-09-18T04:57:01.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:01.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767170+0000", 2024-09-18T04:57:01.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7259999999999998e-06 2024-09-18T04:57:01.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:01.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767176+0000", 2024-09-18T04:57:01.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1460000000000001e-06 2024-09-18T04:57:01.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767182+0000", 2024-09-18T04:57:01.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1020000000000002e-06 2024-09-18T04:57:01.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:01.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811204+0000", 2024-09-18T04:57:01.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044022039999999998 2024-09-18T04:57:01.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:01.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811286+0000", 2024-09-18T04:57:01.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1623000000000001e-05 2024-09-18T04:57:01.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:01.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811358+0000", 2024-09-18T04:57:01.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2125000000000005e-05 2024-09-18T04:57:01.480 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.480 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:01.480 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.480 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9318 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1777664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:01.481 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:01.481 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 25.685788930000001, 2024-09-18T04:57:01.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044380181999999997, 2024-09-18T04:57:01.481 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:01.481 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:01.481 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:01.482 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:01.482 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.482 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9318 2024-09-18T04:57:01.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.482 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:01.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:01.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:01.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:01.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:01.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:01.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:01.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:01.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767189+0000", 2024-09-18T04:57:01.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5909999999999998e-06 2024-09-18T04:57:01.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:01.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767194+0000", 2024-09-18T04:57:01.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7690000000000002e-06 2024-09-18T04:57:01.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767199+0000", 2024-09-18T04:57:01.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189999999999999e-06 2024-09-18T04:57:01.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:01.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811394+0000", 2024-09-18T04:57:01.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044195350000000001 2024-09-18T04:57:01.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:01.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811483+0000", 2024-09-18T04:57:01.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8282999999999995e-05 2024-09-18T04:57:01.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:01.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811567+0000", 2024-09-18T04:57:01.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3855000000000003e-05 2024-09-18T04:57:01.489 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.489 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:01.490 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.490 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9325 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:01.490 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:01.490 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 25.684058392000001, 2024-09-18T04:57:01.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044063989999999997, 2024-09-18T04:57:01.491 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:01.491 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:01.491 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:01.491 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:01.491 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.491 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9325 2024-09-18T04:57:01.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.492 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:01.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:01.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:01.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:01.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:01.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:01.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:01.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:01.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768919+0000", 2024-09-18T04:57:01.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:57:01.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:01.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768925+0000", 2024-09-18T04:57:01.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2310000000000001e-06 2024-09-18T04:57:01.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768929+0000", 2024-09-18T04:57:01.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4469999999999999e-06 2024-09-18T04:57:01.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:01.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812823+0000", 2024-09-18T04:57:01.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043893975000000002 2024-09-18T04:57:01.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:01.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812908+0000", 2024-09-18T04:57:01.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4862000000000002e-05 2024-09-18T04:57:01.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:01.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812981+0000", 2024-09-18T04:57:01.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2892000000000006e-05 2024-09-18T04:57:01.498 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.499 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:01.499 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.499 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9334 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:01.499 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:57:01.500 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 25.682039606, 2024-09-18T04:57:01.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044042853999999999, 2024-09-18T04:57:01.500 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:01.500 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:01.500 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:01.501 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:01.501 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.501 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9334 2024-09-18T04:57:01.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.501 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:01.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:01.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:57:01.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:01.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:57:01.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:01.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770935+0000", 2024-09-18T04:57:01.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:01.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770937+0000", 2024-09-18T04:57:01.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1489999999999999e-06 2024-09-18T04:57:01.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:01.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770940+0000", 2024-09-18T04:57:01.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9569999999999998e-06 2024-09-18T04:57:01.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770945+0000", 2024-09-18T04:57:01.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:57:01.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:01.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814838+0000", 2024-09-18T04:57:01.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043892635999999999 2024-09-18T04:57:01.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:01.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814920+0000", 2024-09-18T04:57:01.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2175000000000005e-05 2024-09-18T04:57:01.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:01.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814979+0000", 2024-09-18T04:57:01.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8372000000000003e-05 2024-09-18T04:57:01.508 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.509 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:01.509 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.509 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9352 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1159168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:01.509 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:57:01.509 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 25.679818429000001, 2024-09-18T04:57:01.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044011269999999998, 2024-09-18T04:57:01.509 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:01.509 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:01.510 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:01.510 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:01.510 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.510 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9352 2024-09-18T04:57:01.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.510 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:01.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:01.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:57:01.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:01.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:57:01.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:01.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773156+0000", 2024-09-18T04:57:01.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:01.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773159+0000", 2024-09-18T04:57:01.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2919999999999998e-06 2024-09-18T04:57:01.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:01.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773165+0000", 2024-09-18T04:57:01.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0580000000000003e-06 2024-09-18T04:57:01.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773177+0000", 2024-09-18T04:57:01.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.204e-05 2024-09-18T04:57:01.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:01.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817082+0000", 2024-09-18T04:57:01.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043905528999999999 2024-09-18T04:57:01.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:01.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817122+0000", 2024-09-18T04:57:01.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9258999999999998e-05 2024-09-18T04:57:01.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:01.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817168+0000", 2024-09-18T04:57:01.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6545000000000002e-05 2024-09-18T04:57:01.517 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.517 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:01.517 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.517 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9353 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1208320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:01.517 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:01.518 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 25.679794131000001, 2024-09-18T04:57:01.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044105537, 2024-09-18T04:57:01.518 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:01.518 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:01.518 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:01.518 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:01.518 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.518 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9353 2024-09-18T04:57:01.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.519 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:01.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:01.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:01.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:01.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:01.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:01.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:01.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:01.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773183+0000", 2024-09-18T04:57:01.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2929999999999999e-06 2024-09-18T04:57:01.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:01.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773190+0000", 2024-09-18T04:57:01.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4330000000000001e-06 2024-09-18T04:57:01.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773194+0000", 2024-09-18T04:57:01.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.865e-06 2024-09-18T04:57:01.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:01.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817196+0000", 2024-09-18T04:57:01.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044001708 2024-09-18T04:57:01.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:01.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817236+0000", 2024-09-18T04:57:01.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0043999999999998e-05 2024-09-18T04:57:01.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:01.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817287+0000", 2024-09-18T04:57:01.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0605000000000001e-05 2024-09-18T04:57:01.526 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.526 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:01.527 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.527 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9354 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:01.527 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:57:01.527 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 25.679776619999998, 2024-09-18T04:57:01.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044199796, 2024-09-18T04:57:01.527 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:01.527 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:01.527 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:01.527 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:01.528 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.528 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9354 2024-09-18T04:57:01.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.528 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:01.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:01.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:57:01.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:01.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:57:01.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:01.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773198+0000", 2024-09-18T04:57:01.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:01.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773200+0000", 2024-09-18T04:57:01.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4159999999999998e-06 2024-09-18T04:57:01.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:01.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773206+0000", 2024-09-18T04:57:01.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.789e-06 2024-09-18T04:57:01.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773210+0000", 2024-09-18T04:57:01.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2679999999999996e-06 2024-09-18T04:57:01.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:01.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817314+0000", 2024-09-18T04:57:01.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044103187000000002 2024-09-18T04:57:01.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:01.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817353+0000", 2024-09-18T04:57:01.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8973000000000001e-05 2024-09-18T04:57:01.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:01.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817399+0000", 2024-09-18T04:57:01.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5877999999999997e-05 2024-09-18T04:57:01.535 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.535 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:01.535 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.535 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46591 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2097152~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:01.536 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:01.536 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.754920202999999, 2024-09-18T04:57:01.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043987310000000002, 2024-09-18T04:57:01.536 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:01.536 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:01.536 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:01.536 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:01.537 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.537 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46591 2024-09-18T04:57:01.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.537 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:01.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:01.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:01.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:01.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:01.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:01.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698057+0000", 2024-09-18T04:57:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.531e-06 2024-09-18T04:57:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698064+0000", 2024-09-18T04:57:01.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-18T04:57:01.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698069+0000", 2024-09-18T04:57:01.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.091e-06 2024-09-18T04:57:01.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:01.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741843+0000", 2024-09-18T04:57:01.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043773470000000002 2024-09-18T04:57:01.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:01.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741928+0000", 2024-09-18T04:57:01.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5113999999999999e-05 2024-09-18T04:57:01.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:01.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742042+0000", 2024-09-18T04:57:01.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114527 2024-09-18T04:57:01.542 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.542 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:01.542 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.542 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46592 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2121728~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:01.542 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:57:01.542 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.754901776000001, 2024-09-18T04:57:01.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044077948999999998, 2024-09-18T04:57:01.542 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:01.542 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:01.543 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:01.543 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:01.543 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.543 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46592 2024-09-18T04:57:01.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.543 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:01.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:01.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:57:01.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:01.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:57:01.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:01.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698073+0000", 2024-09-18T04:57:01.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:01.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698076+0000", 2024-09-18T04:57:01.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.481e-06 2024-09-18T04:57:01.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:01.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698082+0000", 2024-09-18T04:57:01.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8730000000000004e-06 2024-09-18T04:57:01.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698086+0000", 2024-09-18T04:57:01.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:57:01.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:01.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742055+0000", 2024-09-18T04:57:01.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043969170000000002 2024-09-18T04:57:01.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:01.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742093+0000", 2024-09-18T04:57:01.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7707000000000003e-05 2024-09-18T04:57:01.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:01.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742152+0000", 2024-09-18T04:57:01.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8323999999999999e-05 2024-09-18T04:57:01.547 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.547 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:01.547 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.547 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46593 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2211840~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:01.547 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:01.547 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.754884873, 2024-09-18T04:57:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044259917000000003, 2024-09-18T04:57:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46593 2024-09-18T04:57:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:01.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:01.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698092+0000", 2024-09-18T04:57:01.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2249999999999999e-06 2024-09-18T04:57:01.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:01.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698098+0000", 2024-09-18T04:57:01.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:57:01.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698103+0000", 2024-09-18T04:57:01.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:57:01.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:01.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742182+0000", 2024-09-18T04:57:01.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044079508000000003 2024-09-18T04:57:01.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:01.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742257+0000", 2024-09-18T04:57:01.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4678000000000006e-05 2024-09-18T04:57:01.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:01.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742350+0000", 2024-09-18T04:57:01.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3531999999999998e-05 2024-09-18T04:57:01.553 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.553 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:01.553 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.554 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46594 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2236416~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:01.554 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:01.554 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.754869176, 2024-09-18T04:57:01.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044378984000000003, 2024-09-18T04:57:01.554 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:01.554 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:01.554 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:01.555 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:01.555 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.555 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46594 2024-09-18T04:57:01.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.555 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:01.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:01.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:01.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:01.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:01.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:01.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:01.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:01.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698108+0000", 2024-09-18T04:57:01.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0880000000000002e-06 2024-09-18T04:57:01.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:01.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698114+0000", 2024-09-18T04:57:01.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6269999999999997e-06 2024-09-18T04:57:01.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698118+0000", 2024-09-18T04:57:01.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4050000000000002e-06 2024-09-18T04:57:01.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:01.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742365+0000", 2024-09-18T04:57:01.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044246799000000003 2024-09-18T04:57:01.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:01.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742407+0000", 2024-09-18T04:57:01.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2008000000000002e-05 2024-09-18T04:57:01.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:01.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742485+0000", 2024-09-18T04:57:01.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8405999999999994e-05 2024-09-18T04:57:01.560 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.560 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:01.560 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.560 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46595 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:01.560 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:01.560 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.754853103, 2024-09-18T04:57:01.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.04440463, 2024-09-18T04:57:01.561 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:01.561 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:01.561 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:01.561 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:01.561 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.561 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46595 2024-09-18T04:57:01.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.561 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:01.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:01.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:01.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:01.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:01.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:01.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:01.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:01.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698124+0000", 2024-09-18T04:57:01.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1840000000000002e-06 2024-09-18T04:57:01.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:01.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698130+0000", 2024-09-18T04:57:01.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.947e-06 2024-09-18T04:57:01.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698136+0000", 2024-09-18T04:57:01.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6520000000000003e-06 2024-09-18T04:57:01.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:01.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742521+0000", 2024-09-18T04:57:01.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044385446000000002 2024-09-18T04:57:01.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:01.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742527+0000", 2024-09-18T04:57:01.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:57:01.565 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.565 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:01.565 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.566 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46596 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:01.566 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:01.566 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.754834859000001, 2024-09-18T04:57:01.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044418066999999999, 2024-09-18T04:57:01.566 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:01.566 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:01.566 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:01.566 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:01.566 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.567 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46596 2024-09-18T04:57:01.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.567 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:01.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:01.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:01.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:01.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:01.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:01.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:01.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:01.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698146+0000", 2024-09-18T04:57:01.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2779999999999999e-06 2024-09-18T04:57:01.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:01.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698153+0000", 2024-09-18T04:57:01.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3849999999999999e-06 2024-09-18T04:57:01.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698164+0000", 2024-09-18T04:57:01.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1375999999999999e-05 2024-09-18T04:57:01.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:01.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742554+0000", 2024-09-18T04:57:01.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044389482000000001 2024-09-18T04:57:01.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:01.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742559+0000", 2024-09-18T04:57:01.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9540000000000003e-06 2024-09-18T04:57:01.572 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.573 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:01.573 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.573 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46597 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:01.574 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:57:01.574 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.754806494, 2024-09-18T04:57:01.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044416513999999997, 2024-09-18T04:57:01.574 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:01.574 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:01.574 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:01.574 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:01.575 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.575 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46597 2024-09-18T04:57:01.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.575 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:01.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:01.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:57:01.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:01.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:57:01.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:01.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698168+0000", 2024-09-18T04:57:01.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:01.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698175+0000", 2024-09-18T04:57:01.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4520000000000002e-06 2024-09-18T04:57:01.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:01.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698181+0000", 2024-09-18T04:57:01.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4150000000000004e-06 2024-09-18T04:57:01.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698186+0000", 2024-09-18T04:57:01.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7489999999999998e-06 2024-09-18T04:57:01.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:01.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742580+0000", 2024-09-18T04:57:01.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044394325999999998 2024-09-18T04:57:01.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:01.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742585+0000", 2024-09-18T04:57:01.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9849999999999997e-06 2024-09-18T04:57:01.582 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.582 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:01.583 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.583 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46598 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:01.583 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:57:01.583 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 11.754699018, 2024-09-18T04:57:01.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044335316999999999, 2024-09-18T04:57:01.583 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:01.583 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:01.584 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:01.584 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:01.584 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.584 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46598 2024-09-18T04:57:01.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.584 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:01.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:01.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:57:01.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:01.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:57:01.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:01.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698190+0000", 2024-09-18T04:57:01.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999137 2024-09-18T04:57:01.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:01.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698280+0000", 2024-09-18T04:57:01.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0409999999999997e-05 2024-09-18T04:57:01.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:01.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698289+0000", 2024-09-18T04:57:01.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9490000000000002e-06 2024-09-18T04:57:01.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698296+0000", 2024-09-18T04:57:01.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2670000000000003e-06 2024-09-18T04:57:01.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:01.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742607+0000", 2024-09-18T04:57:01.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044311072999999999 2024-09-18T04:57:01.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:01.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742612+0000", 2024-09-18T04:57:01.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9400000000000001e-06 2024-09-18T04:57:01.588 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.588 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:01.588 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.589 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57986 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3086336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:01.589 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:01.589 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.767853295, 2024-09-18T04:57:01.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044047731999999999, 2024-09-18T04:57:01.589 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:01.589 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:01.589 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:01.589 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:01.589 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.590 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57986 2024-09-18T04:57:01.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.590 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:01.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:01.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:01.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685124+0000", 2024-09-18T04:57:01.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2859999999999998e-06 2024-09-18T04:57:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685130+0000", 2024-09-18T04:57:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5400000000000001e-06 2024-09-18T04:57:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685136+0000", 2024-09-18T04:57:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.152e-06 2024-09-18T04:57:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:01.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.728976+0000", 2024-09-18T04:57:01.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043840037999999998 2024-09-18T04:57:01.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:01.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729064+0000", 2024-09-18T04:57:01.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8709999999999996e-05 2024-09-18T04:57:01.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:01.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729170+0000", 2024-09-18T04:57:01.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000105386 2024-09-18T04:57:01.594 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.594 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:01.594 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.594 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57987 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3101696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:01.594 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:57:01.594 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.76780432, 2024-09-18T04:57:01.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044238123999999997, 2024-09-18T04:57:01.595 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:01.595 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:01.595 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:01.595 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:01.595 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.595 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57987 2024-09-18T04:57:01.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.595 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:01.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:01.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:57:01.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:01.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:57:01.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:01.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685170+0000", 2024-09-18T04:57:01.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:01.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:01.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685175+0000", 2024-09-18T04:57:01.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9069999999999998e-06 2024-09-18T04:57:01.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:01.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685187+0000", 2024-09-18T04:57:01.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.189e-05 2024-09-18T04:57:01.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685195+0000", 2024-09-18T04:57:01.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9459999999999998e-06 2024-09-18T04:57:01.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:01.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729212+0000", 2024-09-18T04:57:01.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044017239 2024-09-18T04:57:01.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:01.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729301+0000", 2024-09-18T04:57:01.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8887999999999993e-05 2024-09-18T04:57:01.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:01.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729409+0000", 2024-09-18T04:57:01.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108119 2024-09-18T04:57:01.602 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.602 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:01.602 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.602 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57988 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3117056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:01.603 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:57:01.603 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.767762201, 2024-09-18T04:57:01.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044429835000000001, 2024-09-18T04:57:01.603 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:01.603 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:01.603 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:01.603 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:01.604 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.604 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57988 2024-09-18T04:57:01.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.604 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:01.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:01.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:57:01.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:01.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:57:01.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:01.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685212+0000", 2024-09-18T04:57:01.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.606 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:01.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685216+0000", 2024-09-18T04:57:01.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7390000000000001e-06 2024-09-18T04:57:01.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:01.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685224+0000", 2024-09-18T04:57:01.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2260000000000005e-06 2024-09-18T04:57:01.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685230+0000", 2024-09-18T04:57:01.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0299999999999999e-06 2024-09-18T04:57:01.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:01.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729454+0000", 2024-09-18T04:57:01.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044223248999999999 2024-09-18T04:57:01.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:01.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729539+0000", 2024-09-18T04:57:01.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5029000000000002e-05 2024-09-18T04:57:01.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:01.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729643+0000", 2024-09-18T04:57:01.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104267 2024-09-18T04:57:01.610 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.610 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:01.611 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.611 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57989 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3132416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:01.611 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:57:01.611 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 10.767739449, 2024-09-18T04:57:01.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044642881000000002, 2024-09-18T04:57:01.612 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:01.612 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:01.612 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:01.612 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:01.612 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.613 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57989 2024-09-18T04:57:01.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.613 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:01.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:01.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:57:01.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:01.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:57:01.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:01.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685235+0000", 2024-09-18T04:57:01.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:01.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685238+0000", 2024-09-18T04:57:01.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.667e-06 2024-09-18T04:57:01.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:01.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685245+0000", 2024-09-18T04:57:01.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7909999999999999e-06 2024-09-18T04:57:01.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685250+0000", 2024-09-18T04:57:01.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3730000000000001e-06 2024-09-18T04:57:01.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:01.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729685+0000", 2024-09-18T04:57:01.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044434552000000002 2024-09-18T04:57:01.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:01.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729775+0000", 2024-09-18T04:57:01.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0395000000000004e-05 2024-09-18T04:57:01.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:01.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729879+0000", 2024-09-18T04:57:01.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010361099999999999 2024-09-18T04:57:01.621 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.621 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:01.621 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.622 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.622 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:01.622 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:01.623 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:57:01.624 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-09-18T04:57:01.625 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:01.521+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:01.627 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:01.625+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:01.726 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:01.724+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:01.729 INFO:teuthology.orchestra.run.smithi002.stdout:2856 writesame 0x17aa1e thru 0x185362 data_size 0x1(0xa945 bytes) 2024-09-18T04:57:01.739 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:57:01.741 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:01.741 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:01.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.741 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:91375 3.4 3:265c2e99:::rbd_data.11ac679235d7.0000000000000004:1b [read 319488~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-18T04:57:01.742 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:01.736960+0000", 2024-09-18T04:57:01.742 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0024105849999999998, 2024-09-18T04:57:01.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00244451, 2024-09-18T04:57:01.742 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:01.742 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:01.742 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:01.742 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:01.742 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.742 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 91375 2024-09-18T04:57:01.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.742 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:01.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:01.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.736960+0000", 2024-09-18T04:57:01.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:01.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.736960+0000", 2024-09-18T04:57:01.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:01.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.736960+0000", 2024-09-18T04:57:01.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:01.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.736962+0000", 2024-09-18T04:57:01.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3999999999999999e-06 2024-09-18T04:57:01.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:01.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.736969+0000", 2024-09-18T04:57:01.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7089999999999996e-06 2024-09-18T04:57:01.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.736974+0000", 2024-09-18T04:57:01.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2429999999999996e-06 2024-09-18T04:57:01.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:01.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.739367+0000", 2024-09-18T04:57:01.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0023925169999999998 2024-09-18T04:57:01.745 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.745 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:01.746 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.746 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:91376 3.4 3.99743a64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-18T04:57:01.746 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:01.736980+0000", 2024-09-18T04:57:01.746 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0023912629999999998, 2024-09-18T04:57:01.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025064089999999998, 2024-09-18T04:57:01.746 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:01.746 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:57:01.746 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:01.746 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:01.746 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.747 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 91376 2024-09-18T04:57:01.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.747 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:01.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:01.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.736980+0000", 2024-09-18T04:57:01.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:01.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.736980+0000", 2024-09-18T04:57:01.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:01.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.736979+0000", 2024-09-18T04:57:01.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:01.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.736987+0000", 2024-09-18T04:57:01.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0630000000000006e-06 2024-09-18T04:57:01.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:01.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.736995+0000", 2024-09-18T04:57:01.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4959999999999999e-06 2024-09-18T04:57:01.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.737000+0000", 2024-09-18T04:57:01.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2730000000000002e-06 2024-09-18T04:57:01.749 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.750 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:01.750 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.750 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:91377 3.4 3.99743a64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-18T04:57:01.750 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:01.737005+0000", 2024-09-18T04:57:01.751 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.002365827, 2024-09-18T04:57:01.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025437569999999998, 2024-09-18T04:57:01.751 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:01.751 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:57:01.751 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:01.751 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:01.751 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.751 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 91377 2024-09-18T04:57:01.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.751 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:01.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:01.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.737005+0000", 2024-09-18T04:57:01.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:01.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.737005+0000", 2024-09-18T04:57:01.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:01.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.737005+0000", 2024-09-18T04:57:01.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:01.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.737007+0000", 2024-09-18T04:57:01.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1909999999999999e-06 2024-09-18T04:57:01.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:01.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.737013+0000", 2024-09-18T04:57:01.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5599999999999999e-06 2024-09-18T04:57:01.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.737018+0000", 2024-09-18T04:57:01.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8810000000000004e-06 2024-09-18T04:57:01.755 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.755 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:01.755 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.756 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:91378 3.4 3.99743a64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-18T04:57:01.756 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:01.737023+0000", 2024-09-18T04:57:01.756 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0023479260000000002, 2024-09-18T04:57:01.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025859250000000002, 2024-09-18T04:57:01.756 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:01.756 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:57:01.756 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:01.757 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:01.757 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.757 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 91378 2024-09-18T04:57:01.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.757 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:01.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:01.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.737023+0000", 2024-09-18T04:57:01.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:01.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.737023+0000", 2024-09-18T04:57:01.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:01.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.737023+0000", 2024-09-18T04:57:01.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:01.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.737025+0000", 2024-09-18T04:57:01.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3159999999999999e-06 2024-09-18T04:57:01.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:01.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.737031+0000", 2024-09-18T04:57:01.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.297e-06 2024-09-18T04:57:01.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.737036+0000", 2024-09-18T04:57:01.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9819999999999999e-06 2024-09-18T04:57:01.760 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.760 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:01.760 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.761 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:91379 3.4 3.99743a64 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e55)", 2024-09-18T04:57:01.761 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:01.737041+0000", 2024-09-18T04:57:01.761 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0023303849999999999, 2024-09-18T04:57:01.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026249839999999999, 2024-09-18T04:57:01.761 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:01.761 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "queued for pg", 2024-09-18T04:57:01.761 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:01.761 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:01.762 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:01.762 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 91379 2024-09-18T04:57:01.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.762 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:01.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:01.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.737041+0000", 2024-09-18T04:57:01.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:01.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.737041+0000", 2024-09-18T04:57:01.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:01.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:01.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.737040+0000", 2024-09-18T04:57:01.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:01.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:01.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.737042+0000", 2024-09-18T04:57:01.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1940000000000001e-06 2024-09-18T04:57:01.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:01.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.737049+0000", 2024-09-18T04:57:01.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4659999999999996e-06 2024-09-18T04:57:01.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:01.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:01.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:01.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.737054+0000", 2024-09-18T04:57:01.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0329999999999999e-06 2024-09-18T04:57:01.765 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.765 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:01.765 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.765 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:01.766 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:57:01.766 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 5 2024-09-18T04:57:01.766 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:01.768 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:57:01.845 INFO:teuthology.orchestra.run.smithi002.stdout:2857 trunc from 0xc8a910c to 0xb13c9b3 2024-09-18T04:57:01.848 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:01.843+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:01.927 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:01.925+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:01.934 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:57:01.934 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:57:01.956 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:57:01.957 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:57:01.966 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:01.967 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:01.967 INFO:teuthology.orchestra.run.smithi002.stdout:2858 trunc from 0xb13c9b3 to 0x6163459 2024-09-18T04:57:01.980 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:57:01.981 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:57:01.982 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:57:02.019 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:57:02.019 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:57:02.028 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:02.026+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:02.047 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:57:02.047 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:57:02.049 INFO:teuthology.orchestra.run.smithi002.stdout:2860 punch from 0x35daed to 0x35fa95, (0x1fa8 bytes) 2024-09-18T04:57:02.049 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:02.050 INFO:teuthology.orchestra.run.smithi002.stdout:2862 write 0x2a76582 thru 0x2a84aba (0xe539 bytes) 2024-09-18T04:57:02.079 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:57:02.079 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:57:02.107 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:57:02.108 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:57:02.127 INFO:teuthology.orchestra.run.smithi002.stdout:2863 punch from 0x522e3e to 0x52fe56, (0xd018 bytes) 2024-09-18T04:57:02.127 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:02.127 INFO:teuthology.orchestra.run.smithi002.stdout:2865 read 0x38d476c thru 0x38e2016 (0xd8ab bytes) 2024-09-18T04:57:02.129 INFO:teuthology.orchestra.run.smithi002.stdout:2867 write 0x22726c thru 0x22e8c1 (0x7656 bytes) 2024-09-18T04:57:02.129 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:02.127+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:02.135 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:57:02.135 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:57:02.143 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:02.143 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:02.147 INFO:teuthology.orchestra.run.smithi002.stdout:2868 write 0xe7a36d5 thru 0xe7a47f5 (0x1121 bytes) 2024-09-18T04:57:02.150 INFO:teuthology.orchestra.run.smithi002.stdout:2871 read 0x1fd8638 thru 0x1fe587f (0xd248 bytes) 2024-09-18T04:57:02.152 INFO:teuthology.orchestra.run.smithi002.stdout:2872 writesame 0x5ff004 thru 0x608232 data_size 0x1(0x922f bytes) 2024-09-18T04:57:02.156 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:57:02.240 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:02.238+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:02.242 INFO:teuthology.orchestra.run.smithi002.stdout:2873 write 0x5e9771 thru 0x5f1d79 (0x8609 bytes) 2024-09-18T04:57:02.245 INFO:teuthology.orchestra.run.smithi002.stdout:2874 write 0x4f823bd thru 0x4f8c426 (0xa06a bytes) 2024-09-18T04:57:02.293 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-18T04:57:02.331 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:02.329+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:02.351 INFO:teuthology.orchestra.run.smithi002.stdout:2876 read 0x1f080ce thru 0x1f17cd5 (0xfc08 bytes) 2024-09-18T04:57:02.353 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:02.353 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:02.353 INFO:teuthology.orchestra.run.smithi002.stdout:2883 writesame 0x992b9ff thru 0x9935589 data_size 0xd(0x9b8b bytes) 2024-09-18T04:57:02.354 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:02.354 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:02.365 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:57:02.366 INFO:teuthology.orchestra.run.smithi002.stdout:2884 write 0x5941fea thru 0x59463ca (0x43e1 bytes) 2024-09-18T04:57:02.408 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:02.408 INFO:teuthology.orchestra.run.smithi002.stdout:2886 writesame 0x7dc92fd thru 0x7dd280b data_size 0x1(0x950f bytes) 2024-09-18T04:57:02.422 INFO:teuthology.orchestra.run.smithi002.stdout:2887 writesame 0x6c8eab thru 0x6cb620 data_size 0x1(0x2776 bytes) 2024-09-18T04:57:02.425 INFO:teuthology.orchestra.run.smithi002.stdout:2889 write 0x80eefc2 thru 0x80fa5a9 (0xb5e8 bytes) 2024-09-18T04:57:02.428 INFO:teuthology.orchestra.run.smithi002.stdout:2890 write 0x893e01a thru 0x89410be (0x30a5 bytes) 2024-09-18T04:57:02.431 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:02.429+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:02.434 INFO:teuthology.orchestra.run.smithi002.stdout:2892 write 0x409bcf8 thru 0x40a090b (0x4c14 bytes) 2024-09-18T04:57:02.492 INFO:teuthology.orchestra.run.smithi002.stdout:2893 write 0x122e0f1 thru 0x1233cff (0x5c0f bytes) 2024-09-18T04:57:02.532 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:02.530+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:02.541 INFO:teuthology.orchestra.run.smithi002.stdout:2894 read 0x59a6fad thru 0x59b21ba (0xb20e bytes) 2024-09-18T04:57:02.542 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:02.542 INFO:teuthology.orchestra.run.smithi002.stdout:2896 writesame 0x7231e87 thru 0x7236555 data_size 0x1(0x46cf bytes) 2024-09-18T04:57:02.555 INFO:teuthology.orchestra.run.smithi002.stdout:2897 write 0xb275e46 thru 0xb27f23b (0x93f6 bytes) 2024-09-18T04:57:02.556 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:02.556 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:02.565 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:02.565 INFO:teuthology.orchestra.run.smithi002.stdout:2903 write 0xc7805f9 thru 0xc7856d7 (0x50df bytes) 2024-09-18T04:57:02.567 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:57:02.587 INFO:teuthology.orchestra.run.smithi002.stdout:2904 trunc from 0xe7a47f6 to 0x1688a03 2024-09-18T04:57:02.634 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:02.631+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:02.700 INFO:teuthology.orchestra.run.smithi002.stdout:2907 punch from 0x1035c6d to 0x103b7a0, (0x5b33 bytes) 2024-09-18T04:57:02.719 INFO:teuthology.orchestra.run.smithi002.stdout:2908 write 0xd3be002 thru 0xd3c4f9c (0x6f9b bytes) 2024-09-18T04:57:02.727 INFO:teuthology.orchestra.run.smithi002.stdout:2909 read 0xce58231 thru 0xce5ad22 (0x2af2 bytes) 2024-09-18T04:57:02.728 INFO:teuthology.orchestra.run.smithi002.stdout:2910 write 0xb50457b thru 0xb508d21 (0x47a7 bytes) 2024-09-18T04:57:02.734 INFO:teuthology.orchestra.run.smithi002.stdout:2911 write 0x1a85ba8 thru 0x1a8ed02 (0x915b bytes) 2024-09-18T04:57:02.734 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:02.732+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:02.736 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:02.736 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:02.745 INFO:teuthology.orchestra.run.smithi002.stdout:2912 punch from 0x317f93e to 0x318da9c, (0xe15e bytes) 2024-09-18T04:57:02.748 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:57:02.751 INFO:teuthology.orchestra.run.smithi002.stdout:2913 write 0xdd77b90 thru 0xdd879c7 (0xfe38 bytes) 2024-09-18T04:57:02.761 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:02.762 INFO:teuthology.orchestra.run.smithi002.stdout:2915 write 0x22672d0 thru 0x22718bf (0xa5f0 bytes) 2024-09-18T04:57:02.773 INFO:teuthology.orchestra.run.smithi002.stdout:2916 read 0x7f6e2b8 thru 0x7f73a63 (0x57ac bytes) 2024-09-18T04:57:02.773 INFO:teuthology.orchestra.run.smithi002.stdout:2917 punch from 0x63b5cd5 to 0x63bc408, (0x6733 bytes) 2024-09-18T04:57:02.773 INFO:teuthology.orchestra.run.smithi002.stdout:2918 read 0x90a72c5 thru 0x90b131e (0xa05a bytes) 2024-09-18T04:57:02.774 INFO:teuthology.orchestra.run.smithi002.stdout:2920 write 0x463bb80 thru 0x4642f3d (0x73be bytes) 2024-09-18T04:57:02.786 INFO:teuthology.orchestra.run.smithi002.stdout:2921 write 0xe077e43 thru 0xe07a0e5 (0x22a3 bytes) 2024-09-18T04:57:02.788 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:02.788 INFO:teuthology.orchestra.run.smithi002.stdout:2923 read 0x23077cd thru 0x230a46f (0x2ca3 bytes) 2024-09-18T04:57:02.788 INFO:teuthology.orchestra.run.smithi002.stdout:2924 write 0x5500583 thru 0x55087d4 (0x8252 bytes) 2024-09-18T04:57:02.798 INFO:teuthology.orchestra.run.smithi002.stdout:2925 punch from 0x847372a to 0x84773f1, (0x3cc7 bytes) 2024-09-18T04:57:02.799 INFO:teuthology.orchestra.run.smithi002.stdout:2926 read 0x17165bf thru 0x17173a7 (0xde9 bytes) 2024-09-18T04:57:02.799 INFO:teuthology.orchestra.run.smithi002.stdout:2927 read 0xa9c613c thru 0xa9cf64a (0x950f bytes) 2024-09-18T04:57:02.799 INFO:teuthology.orchestra.run.smithi002.stdout:2928 punch from 0x3bf046a to 0x3bf1625, (0x11bb bytes) 2024-09-18T04:57:02.799 INFO:teuthology.orchestra.run.smithi002.stdout:2932 writesame 0xb8262f3 thru 0xb82c936 data_size 0x77(0x6644 bytes) 2024-09-18T04:57:02.805 INFO:teuthology.orchestra.run.smithi002.stdout:2933 read 0x3f39cf8 thru 0x3f3daf3 (0x3dfc bytes) 2024-09-18T04:57:02.805 INFO:teuthology.orchestra.run.smithi002.stdout:2934 write 0x84ed869 thru 0x84fd250 (0xf9e8 bytes) 2024-09-18T04:57:02.815 INFO:teuthology.orchestra.run.smithi002.stdout:2935 write 0xb68b124 thru 0xb68e2fe (0x31db bytes) 2024-09-18T04:57:02.817 INFO:teuthology.orchestra.run.smithi002.stdout:2937 write 0x4daed96 thru 0x4db973e (0xa9a9 bytes) 2024-09-18T04:57:02.831 INFO:teuthology.orchestra.run.smithi002.stdout:2938 read 0x6c02840 thru 0x6c0afcb (0x878c bytes) 2024-09-18T04:57:02.831 INFO:teuthology.orchestra.run.smithi002.stdout:2939 read 0xa7224d8 thru 0xa723ef1 (0x1a1a bytes) 2024-09-18T04:57:02.831 INFO:teuthology.orchestra.run.smithi002.stdout:2940 writesame 0x2d27355 thru 0x2d2ce64 data_size 0x5e(0x5b10 bytes) 2024-09-18T04:57:02.835 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:02.833+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:02.840 INFO:teuthology.orchestra.run.smithi002.stdout:2941 read 0xb070a11 thru 0xb075082 (0x4672 bytes) 2024-09-18T04:57:02.840 INFO:teuthology.orchestra.run.smithi002.stdout:2942 read 0x3504a64 thru 0x350c662 (0x7bff bytes) 2024-09-18T04:57:02.840 INFO:teuthology.orchestra.run.smithi002.stdout:2946 read 0x684d247 thru 0x6853c0b (0x69c5 bytes) 2024-09-18T04:57:02.840 INFO:teuthology.orchestra.run.smithi002.stdout:2947 read 0x7bb5957 thru 0x7bb61b7 (0x861 bytes) 2024-09-18T04:57:02.841 INFO:teuthology.orchestra.run.smithi002.stdout:2948 punch from 0xb5741da to 0xb584079, (0xfe9f bytes) 2024-09-18T04:57:02.855 INFO:teuthology.orchestra.run.smithi002.stdout:2949 trunc from 0xe07a0e6 to 0x6ba7cf 2024-09-18T04:57:02.887 INFO:teuthology.orchestra.run.smithi002.stdout:2950 punch from 0x2b160c to 0x2bb4a1, (0x9e95 bytes) 2024-09-18T04:57:02.890 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:02.891 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:02.896 INFO:teuthology.orchestra.run.smithi002.stdout:2953 trunc from 0x6ba7cf to 0x8069196 2024-09-18T04:57:02.901 INFO:teuthology.orchestra.run.smithi002.stdout:2955 read 0x7fa8f2c thru 0x7fa9af8 (0xbcd bytes) 2024-09-18T04:57:02.901 INFO:teuthology.orchestra.run.smithi002.stdout:2956 writesame 0x50e0927 thru 0x50ea531 data_size 0x1(0x9c0b bytes) 2024-09-18T04:57:02.903 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:57:02.916 INFO:teuthology.orchestra.run.smithi002.stdout:2958 read 0x4e0fc59 thru 0x4e15339 (0x56e1 bytes) 2024-09-18T04:57:02.923 INFO:teuthology.orchestra.run.smithi002.stdout:2959 writesame 0x9fcaf97 thru 0x9fcf097 data_size 0x9(0x4101 bytes) 2024-09-18T04:57:02.923 INFO:teuthology.orchestra.run.smithi002.stdout:2961 write 0x798288b thru 0x7987869 (0x4fdf bytes) 2024-09-18T04:57:02.931 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:02.931 INFO:teuthology.orchestra.run.smithi002.stdout:2963 punch from 0x7d73906 to 0x7d7f618, (0xbd12 bytes) 2024-09-18T04:57:02.931 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:02.931 INFO:teuthology.orchestra.run.smithi002.stdout:2969 clone 25 order 25 su 2097152 sc 2 2024-09-18T04:57:02.936 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:02.934+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:03.024 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:03.024 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:03.032 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:57:03.038 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:03.035+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:03.139 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:03.136+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:03.178 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:03.179 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:03.187 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:57:03.240 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:03.237+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:03.290 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T04:57:03.291 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T04:57:03.291 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T04:57:03.291 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-18T04:57:03.299 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:03.299 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:03.308 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:57:03.341 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:03.339+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:03.355 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone24 from 0x9fcf098 (overlap 0x6ba7cf) to 0x2b0fec 2024-09-18T04:57:03.433 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-09-18T04:57:03.442 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:03.439+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:03.458 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:03.458 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:03.467 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:57:03.551 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:03.549+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:03.609 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:03.609 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:03.611 INFO:tasks.ceph.mon.a.smithi002.stderr:2024-09-18T04:57:03.609+0000 7f7d3b87f640 -1 mon.a@0(leader).osd e57 definitely_dead 0 2024-09-18T04:57:03.617 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:57:03.644 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:03.642+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:03.651 INFO:teuthology.orchestra.run.smithi002.stdout:2971 read 0x709a08d thru 0x70a549d (0xb411 bytes) 2024-09-18T04:57:03.661 INFO:teuthology.orchestra.run.smithi002.stdout:2972 writesame 0xb20f061 thru 0xb21eccf data_size 0xd(0xfc6f bytes) 2024-09-18T04:57:03.665 INFO:teuthology.orchestra.run.smithi002.stdout:2973 write 0x6193275 thru 0x6194969 (0x16f5 bytes) 2024-09-18T04:57:03.673 INFO:teuthology.orchestra.run.smithi002.stdout:2974 trunc from 0xb21ecd0 to 0xcef5b77 2024-09-18T04:57:03.681 INFO:teuthology.orchestra.run.smithi002.stdout:2975 trunc from 0xcef5b77 to 0xad444ba 2024-09-18T04:57:03.691 INFO:teuthology.orchestra.run.smithi002.stdout:2976 read 0x342bd09 thru 0x342e89a (0x2b92 bytes) 2024-09-18T04:57:03.696 INFO:teuthology.orchestra.run.smithi002.stdout:2977 trunc from 0xad444ba to 0x6389dbe 2024-09-18T04:57:03.714 INFO:teuthology.orchestra.run.smithi002.stdout:2978 trunc from 0x6389dbe to 0x7fd0b65 2024-09-18T04:57:03.720 INFO:teuthology.orchestra.run.smithi002.stdout:2979 write 0x15ece40 thru 0x15f7e9c (0xb05d bytes) 2024-09-18T04:57:03.744 INFO:teuthology.orchestra.run.smithi002.stdout:2980 read 0x271e3ee thru 0x2722c5b (0x486e bytes) 2024-09-18T04:57:03.745 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:03.742+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:03.748 INFO:teuthology.orchestra.run.smithi002.stdout:2984 trunc from 0x7fd0b65 to 0x4893e6b 2024-09-18T04:57:03.765 INFO:teuthology.orchestra.run.smithi002.stdout:2985 writesame 0x7e1d47b thru 0x7e247fc data_size 0x5(0x7382 bytes) 2024-09-18T04:57:03.768 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:03.768 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:03.774 INFO:teuthology.orchestra.run.smithi002.stdout:2986 trunc from 0x7e247fd to 0x70f931 2024-09-18T04:57:03.777 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:57:03.804 INFO:teuthology.orchestra.run.smithi002.stdout:2987 trunc from 0x70f931 to 0x8187422 2024-09-18T04:57:03.822 INFO:teuthology.orchestra.run.smithi002.stdout:2990 writesame 0xde48ef1 thru 0xde55cf5 data_size 0x1(0xce05 bytes) 2024-09-18T04:57:03.822 INFO:teuthology.orchestra.run.smithi002.stdout:2992 writesame 0x6c8646b thru 0x6c8707d data_size 0x1(0xc13 bytes) 2024-09-18T04:57:03.825 INFO:teuthology.orchestra.run.smithi002.stdout:2994 punch from 0x8a776ef to 0x8a77a23, (0x334 bytes) 2024-09-18T04:57:03.825 INFO:teuthology.orchestra.run.smithi002.stdout:2995 punch from 0x7e6af29 to 0x7e7516d, (0xa244 bytes) 2024-09-18T04:57:03.825 INFO:teuthology.orchestra.run.smithi002.stdout:2996 read 0xa2f69d5 thru 0xa2f6e06 (0x432 bytes) 2024-09-18T04:57:03.825 INFO:teuthology.orchestra.run.smithi002.stdout:2997 write 0xa51e848 thru 0xa529cf3 (0xb4ac bytes) 2024-09-18T04:57:03.828 INFO:teuthology.orchestra.run.smithi002.stdout:2998 writesame 0x307240 thru 0x311540 data_size 0x1(0xa301 bytes) 2024-09-18T04:57:03.847 INFO:teuthology.orchestra.run.smithi002.stdout:2999 read 0x4f27e9 thru 0x500099 (0xd8b1 bytes) 2024-09-18T04:57:03.848 INFO:teuthology.orchestra.run.smithi002.stdout:3000 writesame 0x2f6f82f thru 0x2f70413 data_size 0x3(0xbe5 bytes) 2024-09-18T04:57:03.849 INFO:teuthology.orchestra.run.smithi002.stdout:3001 write 0xc1ae9c thru 0xc207b7 (0x591c bytes) 2024-09-18T04:57:03.850 INFO:teuthology.orchestra.run.smithi002.stdout:3002 punch from 0x408c0e2 to 0x408ea3f, (0x295d bytes) 2024-09-18T04:57:03.862 INFO:teuthology.orchestra.run.smithi002.stdout:3003 trunc from 0xde55cf6 to 0x3868b15 2024-09-18T04:57:03.879 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:03.843+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:03.883 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:03.883 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:03.885 INFO:teuthology.orchestra.run.smithi002.stdout:3004 read 0x35b83e6 thru 0x35c554d (0xd168 bytes) 2024-09-18T04:57:03.885 INFO:teuthology.orchestra.run.smithi002.stdout:3005 read 0x2aa6820 thru 0x2aa867e (0x1e5f bytes) 2024-09-18T04:57:03.885 INFO:teuthology.orchestra.run.smithi002.stdout:3006 read 0x362e901 thru 0x36316d2 (0x2dd2 bytes) 2024-09-18T04:57:03.886 INFO:teuthology.orchestra.run.smithi002.stdout:3007 read 0x351f93c thru 0x352dd20 (0xe3e5 bytes) 2024-09-18T04:57:03.893 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:57:03.898 INFO:teuthology.orchestra.run.smithi002.stdout:3010 write 0x820c06c thru 0x821bc9e (0xfc33 bytes) 2024-09-18T04:57:03.901 INFO:teuthology.orchestra.run.smithi002.stdout:3011 write 0x1a60945 thru 0x1a67cbe (0x737a bytes) 2024-09-18T04:57:03.902 INFO:teuthology.orchestra.run.smithi002.stdout:3012 read 0x3658f4f thru 0x365b190 (0x2242 bytes) 2024-09-18T04:57:03.903 INFO:teuthology.orchestra.run.smithi002.stdout:3014 read 0x3b10f7b thru 0x3b1df32 (0xcfb8 bytes) 2024-09-18T04:57:03.912 INFO:teuthology.orchestra.run.smithi002.stdout:3016 writesame 0xbaf6a1e thru 0xbb061ff data_size 0x1(0xf7e2 bytes) 2024-09-18T04:57:03.913 INFO:teuthology.orchestra.run.smithi002.stdout:3017 write 0x1e1c4b5 thru 0x1e28757 (0xc2a3 bytes) 2024-09-18T04:57:03.914 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:03.915 INFO:teuthology.orchestra.run.smithi002.stdout:3020 read 0x3281367 thru 0x328f94e (0xe5e8 bytes) 2024-09-18T04:57:03.915 INFO:teuthology.orchestra.run.smithi002.stdout:3021 clone 26 order 20 su 131072 sc 10 2024-09-18T04:57:03.946 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:03.944+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:04.034 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:04.034 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:04.042 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:57:04.085 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:04.083+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:04.144 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:04.144 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:04.148 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:04.146+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:04.152 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:57:04.153 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:57:04.249 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:04.246+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:04.277 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:57:04.277 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:57:04.293 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:57:04.294 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:57:04.300 INFO:teuthology.orchestra.run.smithi002.stderr:marked out osd.1. 2024-09-18T04:57:04.300 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:04.300 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:04.310 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:57:04.312 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T04:57:04.315 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:57:04.315 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:57:04.331 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:57:04.331 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:57:04.348 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:57:04.348 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:57:04.349 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:04.347+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:04.364 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:57:04.365 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:57:04.381 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:57:04.382 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:57:04.399 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:57:04.399 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:57:04.415 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:04.415 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:04.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:04.416 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:04.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.416 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26189 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 237568~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:04.416 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:04.416 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.772713954, 2024-09-18T04:57:04.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022263431, 2024-09-18T04:57:04.416 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:04.416 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:04.416 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:04.416 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:04.416 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:04.417 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26189 2024-09-18T04:57:04.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.417 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:04.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:04.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:04.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:04.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:04.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:04.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:04.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:04.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:04.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639787+0000", 2024-09-18T04:57:04.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9759999999999998e-06 2024-09-18T04:57:04.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:04.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639792+0000", 2024-09-18T04:57:04.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5829999999999997e-06 2024-09-18T04:57:04.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:04.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639796+0000", 2024-09-18T04:57:04.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0690000000000003e-06 2024-09-18T04:57:04.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:04.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.661944+0000", 2024-09-18T04:57:04.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022147463999999999 2024-09-18T04:57:04.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:04.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.661985+0000", 2024-09-18T04:57:04.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1360999999999999e-05 2024-09-18T04:57:04.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:04.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662048+0000", 2024-09-18T04:57:04.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3298000000000003e-05 2024-09-18T04:57:04.420 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.420 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:04.420 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.420 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26199 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 4096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:04.420 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:04.421 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.772698668, 2024-09-18T04:57:04.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022348776000000001, 2024-09-18T04:57:04.421 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:04.421 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:04.421 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:04.421 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:04.421 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:04.421 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26199 2024-09-18T04:57:04.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.421 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:04.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:04.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:04.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:04.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:04.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:04.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:04.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:04.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:04.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639806+0000", 2024-09-18T04:57:04.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1539999999999999e-06 2024-09-18T04:57:04.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:04.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639812+0000", 2024-09-18T04:57:04.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5520000000000004e-06 2024-09-18T04:57:04.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:04.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639816+0000", 2024-09-18T04:57:04.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1840000000000001e-06 2024-09-18T04:57:04.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:04.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662063+0000", 2024-09-18T04:57:04.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022247125999999999 2024-09-18T04:57:04.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:04.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662098+0000", 2024-09-18T04:57:04.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4968000000000001e-05 2024-09-18T04:57:04.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:04.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662149+0000", 2024-09-18T04:57:04.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1122000000000003e-05 2024-09-18T04:57:04.425 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.425 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:04.425 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.425 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26200 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 32768~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:04.425 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:04.425 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.772679143000001, 2024-09-18T04:57:04.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022501118000000001, 2024-09-18T04:57:04.425 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:04.426 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:04.426 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:04.426 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:04.426 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:04.426 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26200 2024-09-18T04:57:04.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.426 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:04.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:04.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:04.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:04.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:04.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:04.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:04.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:04.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:04.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639822+0000", 2024-09-18T04:57:04.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.007e-06 2024-09-18T04:57:04.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:04.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639827+0000", 2024-09-18T04:57:04.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5709999999999997e-06 2024-09-18T04:57:04.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:04.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639831+0000", 2024-09-18T04:57:04.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2819999999999998e-06 2024-09-18T04:57:04.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:04.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662181+0000", 2024-09-18T04:57:04.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022349878 2024-09-18T04:57:04.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:04.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662252+0000", 2024-09-18T04:57:04.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0542000000000003e-05 2024-09-18T04:57:04.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:04.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662321+0000", 2024-09-18T04:57:04.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9167999999999999e-05 2024-09-18T04:57:04.430 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.430 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:04.430 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.430 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26201 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 61440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:04.430 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:04.430 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.772663852000001, 2024-09-18T04:57:04.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022569783, 2024-09-18T04:57:04.430 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:04.430 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:04.430 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:04.431 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:04.431 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:04.431 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26201 2024-09-18T04:57:04.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.431 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:04.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:04.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:04.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:04.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:04.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:04.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:04.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:04.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:04.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639837+0000", 2024-09-18T04:57:04.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.928e-06 2024-09-18T04:57:04.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:04.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639843+0000", 2024-09-18T04:57:04.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8130000000000001e-06 2024-09-18T04:57:04.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:04.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639847+0000", 2024-09-18T04:57:04.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0999999999999997e-06 2024-09-18T04:57:04.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:04.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662335+0000", 2024-09-18T04:57:04.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022488207999999999 2024-09-18T04:57:04.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:04.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662370+0000", 2024-09-18T04:57:04.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5292e-05 2024-09-18T04:57:04.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:04.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662405+0000", 2024-09-18T04:57:04.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4765999999999999e-05 2024-09-18T04:57:04.434 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.434 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:04.434 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.434 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26202 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 90112~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:04.434 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:57:04.435 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.772648435000001, 2024-09-18T04:57:04.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022655937000000001, 2024-09-18T04:57:04.435 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:04.435 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:04.435 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:04.435 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:04.435 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:04.435 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26202 2024-09-18T04:57:04.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.435 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:04.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:04.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:57:04.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:04.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:57:04.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:04.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639850+0000", 2024-09-18T04:57:04.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:04.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:04.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639852+0000", 2024-09-18T04:57:04.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9700000000000002e-06 2024-09-18T04:57:04.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:04.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639858+0000", 2024-09-18T04:57:04.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.434e-06 2024-09-18T04:57:04.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:04.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639862+0000", 2024-09-18T04:57:04.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3499999999999999e-06 2024-09-18T04:57:04.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:04.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662429+0000", 2024-09-18T04:57:04.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022567409 2024-09-18T04:57:04.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:04.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662469+0000", 2024-09-18T04:57:04.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9801000000000001e-05 2024-09-18T04:57:04.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:04.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662507+0000", 2024-09-18T04:57:04.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7311000000000002e-05 2024-09-18T04:57:04.439 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.439 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:04.439 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.439 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26203 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 118784~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:04.439 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:57:04.439 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.772633280000001, 2024-09-18T04:57:04.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022730358999999999, 2024-09-18T04:57:04.439 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:04.439 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:04.440 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:04.440 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:04.440 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:04.440 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26203 2024-09-18T04:57:04.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.440 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:04.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:04.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:57:04.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:04.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:57:04.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:04.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639865+0000", 2024-09-18T04:57:04.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:04.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:04.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639867+0000", 2024-09-18T04:57:04.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0889999999999998e-06 2024-09-18T04:57:04.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:04.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639873+0000", 2024-09-18T04:57:04.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4979999999999997e-06 2024-09-18T04:57:04.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:04.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639877+0000", 2024-09-18T04:57:04.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4510000000000002e-06 2024-09-18T04:57:04.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:04.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662528+0000", 2024-09-18T04:57:04.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022650791999999999 2024-09-18T04:57:04.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:04.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662563+0000", 2024-09-18T04:57:04.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4530000000000003e-05 2024-09-18T04:57:04.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:04.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662596+0000", 2024-09-18T04:57:04.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3463000000000001e-05 2024-09-18T04:57:04.444 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.444 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:04.444 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.444 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26204 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 147456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:04.444 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:04.444 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.772617887999999, 2024-09-18T04:57:04.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022803233999999999, 2024-09-18T04:57:04.444 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:04.444 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:04.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:04.445 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:04.445 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:04.445 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26204 2024-09-18T04:57:04.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.445 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:04.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:04.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:04.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:04.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:04.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:04.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:04.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:04.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:04.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639883+0000", 2024-09-18T04:57:04.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6520000000000002e-06 2024-09-18T04:57:04.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:04.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639888+0000", 2024-09-18T04:57:04.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2870000000000004e-06 2024-09-18T04:57:04.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:04.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639893+0000", 2024-09-18T04:57:04.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3839999999999999e-06 2024-09-18T04:57:04.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:04.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662617+0000", 2024-09-18T04:57:04.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022723876 2024-09-18T04:57:04.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:04.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662651+0000", 2024-09-18T04:57:04.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3797000000000002e-05 2024-09-18T04:57:04.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:04.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662684+0000", 2024-09-18T04:57:04.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3828999999999997e-05 2024-09-18T04:57:04.449 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.449 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:04.449 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.449 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26205 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 176128~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:04.449 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:04.450 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.772594927, 2024-09-18T04:57:04.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022919108000000001, 2024-09-18T04:57:04.450 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:04.450 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:04.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:04.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:04.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:04.450 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26205 2024-09-18T04:57:04.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.450 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:04.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:04.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:04.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:04.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:04.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:04.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:04.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:04.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:04.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639906+0000", 2024-09-18T04:57:04.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.531e-06 2024-09-18T04:57:04.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:04.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639913+0000", 2024-09-18T04:57:04.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.331e-06 2024-09-18T04:57:04.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:04.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639920+0000", 2024-09-18T04:57:04.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5399999999999998e-06 2024-09-18T04:57:04.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:04.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662706+0000", 2024-09-18T04:57:04.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.02278587 2024-09-18T04:57:04.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:04.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662764+0000", 2024-09-18T04:57:04.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7630000000000002e-05 2024-09-18T04:57:04.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:04.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662823+0000", 2024-09-18T04:57:04.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9648000000000002e-05 2024-09-18T04:57:04.454 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.454 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:04.454 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.454 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26206 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 204800~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:04.454 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:04.454 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.772574528, 2024-09-18T04:57:04.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023047517, 2024-09-18T04:57:04.454 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:04.454 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:04.454 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:04.455 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:04.455 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:04.455 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26206 2024-09-18T04:57:04.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.455 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:04.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:04.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:04.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:04.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:04.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:04.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:04.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:04.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:04.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639926+0000", 2024-09-18T04:57:04.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3120000000000001e-06 2024-09-18T04:57:04.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:04.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639932+0000", 2024-09-18T04:57:04.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4890000000000003e-06 2024-09-18T04:57:04.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:04.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639936+0000", 2024-09-18T04:57:04.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.07e-06 2024-09-18T04:57:04.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:04.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662850+0000", 2024-09-18T04:57:04.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022913579999999999 2024-09-18T04:57:04.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:04.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662903+0000", 2024-09-18T04:57:04.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3176e-05 2024-09-18T04:57:04.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:04.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662972+0000", 2024-09-18T04:57:04.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9270999999999995e-05 2024-09-18T04:57:04.458 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.459 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:04.459 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.459 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26207 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 233472~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:04.459 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:57:04.459 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.772559384000001, 2024-09-18T04:57:04.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023139541999999999, 2024-09-18T04:57:04.459 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:04.459 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:04.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:04.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:04.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:04.460 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26207 2024-09-18T04:57:04.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.460 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:04.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:04.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:57:04.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:04.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:57:04.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:04.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639939+0000", 2024-09-18T04:57:04.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:04.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:04.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639941+0000", 2024-09-18T04:57:04.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0080000000000001e-06 2024-09-18T04:57:04.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:04.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639947+0000", 2024-09-18T04:57:04.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.485e-06 2024-09-18T04:57:04.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:04.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639951+0000", 2024-09-18T04:57:04.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4549999999999997e-06 2024-09-18T04:57:04.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:04.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663005+0000", 2024-09-18T04:57:04.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023053569999999999 2024-09-18T04:57:04.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:04.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663042+0000", 2024-09-18T04:57:04.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7245e-05 2024-09-18T04:57:04.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:04.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663079+0000", 2024-09-18T04:57:04.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7119000000000002e-05 2024-09-18T04:57:04.463 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.463 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:04.463 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.463 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26208 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 524288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:04.464 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:04.464 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.772542636000001, 2024-09-18T04:57:04.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023283564999999999, 2024-09-18T04:57:04.464 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:04.464 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:04.464 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:04.464 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:04.464 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:04.464 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26208 2024-09-18T04:57:04.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.464 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:04.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:04.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:04.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:04.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:04.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:04.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:04.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:04.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:04.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639958+0000", 2024-09-18T04:57:04.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0899999999999999e-06 2024-09-18T04:57:04.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:04.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639963+0000", 2024-09-18T04:57:04.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2209999999999996e-06 2024-09-18T04:57:04.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:04.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639967+0000", 2024-09-18T04:57:04.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1169999999999997e-06 2024-09-18T04:57:04.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:04.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663114+0000", 2024-09-18T04:57:04.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023146102000000002 2024-09-18T04:57:04.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:04.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663183+0000", 2024-09-18T04:57:04.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9686999999999999e-05 2024-09-18T04:57:04.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:04.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663240+0000", 2024-09-18T04:57:04.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6728000000000003e-05 2024-09-18T04:57:04.468 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.468 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:04.468 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.468 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26209 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 552960~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:04.468 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:04.468 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.772524860000001, 2024-09-18T04:57:04.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023440966000000001, 2024-09-18T04:57:04.468 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:04.468 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:04.468 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:04.468 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:04.469 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:04.469 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26209 2024-09-18T04:57:04.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.469 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:04.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:04.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:04.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:04.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:04.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:04.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:04.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:04.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:04.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639976+0000", 2024-09-18T04:57:04.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4940000000000002e-06 2024-09-18T04:57:04.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:04.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639982+0000", 2024-09-18T04:57:04.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0560000000000001e-06 2024-09-18T04:57:04.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:04.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639987+0000", 2024-09-18T04:57:04.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-18T04:57:04.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:04.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663278+0000", 2024-09-18T04:57:04.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023290789999999999 2024-09-18T04:57:04.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:04.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663348+0000", 2024-09-18T04:57:04.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0837e-05 2024-09-18T04:57:04.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:04.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663415+0000", 2024-09-18T04:57:04.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6736000000000006e-05 2024-09-18T04:57:04.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.472 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:04.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.473 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26210 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 581632~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:04.473 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:04.473 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.772507852, 2024-09-18T04:57:04.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023616437000000001, 2024-09-18T04:57:04.473 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:04.473 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:04.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:04.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:04.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:04.473 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26210 2024-09-18T04:57:04.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.474 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:04.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:04.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:04.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:04.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:04.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:04.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:04.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:04.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:04.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639993+0000", 2024-09-18T04:57:04.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.419e-06 2024-09-18T04:57:04.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:04.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640006+0000", 2024-09-18T04:57:04.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2918000000000001e-05 2024-09-18T04:57:04.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:04.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640011+0000", 2024-09-18T04:57:04.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9320000000000004e-06 2024-09-18T04:57:04.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:04.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663454+0000", 2024-09-18T04:57:04.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023443035000000001 2024-09-18T04:57:04.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:04.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663532+0000", 2024-09-18T04:57:04.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7443999999999999e-05 2024-09-18T04:57:04.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:04.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663608+0000", 2024-09-18T04:57:04.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6093000000000005e-05 2024-09-18T04:57:04.478 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.478 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:04.478 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.478 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26211 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 610304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:04.478 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:57:04.478 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.772483300000001, 2024-09-18T04:57:04.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023774337999999999, 2024-09-18T04:57:04.479 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:04.479 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:04.479 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:04.479 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:04.479 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:04.479 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26211 2024-09-18T04:57:04.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.480 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:04.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:04.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:57:04.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:04.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:57:04.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:04.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640015+0000", 2024-09-18T04:57:04.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:04.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:04.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640023+0000", 2024-09-18T04:57:04.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3900000000000004e-06 2024-09-18T04:57:04.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:04.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640036+0000", 2024-09-18T04:57:04.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3091999999999999e-05 2024-09-18T04:57:04.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:04.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640041+0000", 2024-09-18T04:57:04.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6219999999999997e-06 2024-09-18T04:57:04.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:04.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663644+0000", 2024-09-18T04:57:04.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023602643999999999 2024-09-18T04:57:04.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:04.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663718+0000", 2024-09-18T04:57:04.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4481999999999996e-05 2024-09-18T04:57:04.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:04.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663790+0000", 2024-09-18T04:57:04.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1589999999999997e-05 2024-09-18T04:57:04.484 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.484 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:04.484 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.485 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26212 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 638976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:04.485 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:57:04.485 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.772450259999999, 2024-09-18T04:57:04.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023924748999999999, 2024-09-18T04:57:04.485 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:04.485 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:04.485 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:04.485 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:04.485 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:04.485 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26212 2024-09-18T04:57:04.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.486 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:04.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:04.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:57:04.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:04.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:57:04.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:04.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640048+0000", 2024-09-18T04:57:04.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:04.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:04.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640051+0000", 2024-09-18T04:57:04.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.095e-06 2024-09-18T04:57:04.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:04.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640060+0000", 2024-09-18T04:57:04.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8929999999999994e-06 2024-09-18T04:57:04.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:04.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640065+0000", 2024-09-18T04:57:04.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0270000000000003e-06 2024-09-18T04:57:04.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:04.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663824+0000", 2024-09-18T04:57:04.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023758873999999999 2024-09-18T04:57:04.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:04.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663904+0000", 2024-09-18T04:57:04.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9856000000000003e-05 2024-09-18T04:57:04.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:04.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663974+0000", 2024-09-18T04:57:04.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9450999999999996e-05 2024-09-18T04:57:04.490 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.490 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:04.490 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.490 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26213 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 667648~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:04.490 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:04.490 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.772429088999999, 2024-09-18T04:57:04.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024059812999999999, 2024-09-18T04:57:04.491 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:04.491 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:04.491 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:04.491 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:04.491 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:04.491 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26213 2024-09-18T04:57:04.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.491 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:04.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:04.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:04.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:04.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:04.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:04.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:04.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:04.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:04.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640072+0000", 2024-09-18T04:57:04.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4200000000000001e-06 2024-09-18T04:57:04.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:04.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640078+0000", 2024-09-18T04:57:04.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0179999999999998e-06 2024-09-18T04:57:04.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:04.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640083+0000", 2024-09-18T04:57:04.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:57:04.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:04.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664008+0000", 2024-09-18T04:57:04.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023925287 2024-09-18T04:57:04.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:04.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664059+0000", 2024-09-18T04:57:04.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0730999999999999e-05 2024-09-18T04:57:04.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:04.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664130+0000", 2024-09-18T04:57:04.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0801000000000002e-05 2024-09-18T04:57:04.494 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.494 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:04.494 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.495 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26214 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 696320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:04.495 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:04.495 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.772412118999998, 2024-09-18T04:57:04.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024223738000000002, 2024-09-18T04:57:04.495 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:04.495 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:04.495 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:04.495 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:04.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:04.496 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26214 2024-09-18T04:57:04.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.496 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:04.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:04.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:04.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:04.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:04.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:04.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:04.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:04.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:04.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640089+0000", 2024-09-18T04:57:04.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0849999999999999e-06 2024-09-18T04:57:04.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:04.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640095+0000", 2024-09-18T04:57:04.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9279999999999998e-06 2024-09-18T04:57:04.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:04.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640099+0000", 2024-09-18T04:57:04.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:57:04.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:04.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664166+0000", 2024-09-18T04:57:04.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024066513000000001 2024-09-18T04:57:04.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:04.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664239+0000", 2024-09-18T04:57:04.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3201999999999995e-05 2024-09-18T04:57:04.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:04.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664311+0000", 2024-09-18T04:57:04.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1434999999999996e-05 2024-09-18T04:57:04.499 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.499 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:04.499 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.500 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26215 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 724992~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:04.500 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:57:04.500 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.772395358000001, 2024-09-18T04:57:04.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024302284, 2024-09-18T04:57:04.500 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:04.500 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:04.500 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:04.500 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:04.500 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:04.501 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26215 2024-09-18T04:57:04.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.501 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:04.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:04.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:57:04.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:04.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:57:04.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:04.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640103+0000", 2024-09-18T04:57:04.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:04.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:04.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640106+0000", 2024-09-18T04:57:04.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2120000000000002e-06 2024-09-18T04:57:04.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:04.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640111+0000", 2024-09-18T04:57:04.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7880000000000003e-06 2024-09-18T04:57:04.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:04.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640116+0000", 2024-09-18T04:57:04.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6099999999999999e-06 2024-09-18T04:57:04.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:04.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664327+0000", 2024-09-18T04:57:04.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024210941999999999 2024-09-18T04:57:04.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:04.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664367+0000", 2024-09-18T04:57:04.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9666000000000003e-05 2024-09-18T04:57:04.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:04.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664406+0000", 2024-09-18T04:57:04.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9402e-05 2024-09-18T04:57:04.505 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.505 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:04.505 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.505 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26216 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 753664~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:04.505 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:04.505 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.772378864, 2024-09-18T04:57:04.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024452854999999999, 2024-09-18T04:57:04.505 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:04.506 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:04.506 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:04.506 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:04.506 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:04.506 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26216 2024-09-18T04:57:04.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.506 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:04.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:04.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:04.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:04.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:04.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:04.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:04.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:04.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:04.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640122+0000", 2024-09-18T04:57:04.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3599999999999999e-06 2024-09-18T04:57:04.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:04.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640128+0000", 2024-09-18T04:57:04.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6339999999999998e-06 2024-09-18T04:57:04.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640132+0000", 2024-09-18T04:57:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:57:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664436+0000", 2024-09-18T04:57:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024303585999999999 2024-09-18T04:57:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:04.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664507+0000", 2024-09-18T04:57:04.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1046999999999999e-05 2024-09-18T04:57:04.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:04.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664573+0000", 2024-09-18T04:57:04.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6217000000000007e-05 2024-09-18T04:57:04.510 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.510 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:04.510 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.511 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26217 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 782336~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:04.511 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:04.511 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.77236263, 2024-09-18T04:57:04.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024526287000000001, 2024-09-18T04:57:04.511 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:04.511 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:04.511 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:04.511 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:04.511 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:04.511 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26217 2024-09-18T04:57:04.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.512 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:04.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:04.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:04.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:04.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:04.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:04.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:04.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:04.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:04.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:04.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640140+0000", 2024-09-18T04:57:04.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2989999999999998e-06 2024-09-18T04:57:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640146+0000", 2024-09-18T04:57:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9569999999999999e-06 2024-09-18T04:57:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640151+0000", 2024-09-18T04:57:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:57:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:04.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664590+0000", 2024-09-18T04:57:04.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0244389 2024-09-18T04:57:04.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:04.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664628+0000", 2024-09-18T04:57:04.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7910999999999997e-05 2024-09-18T04:57:04.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:04.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:04.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:04.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664663+0000", 2024-09-18T04:57:04.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5024999999999998e-05 2024-09-18T04:57:04.516 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.516 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:04.516 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.516 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:04.516 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:04.516 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:04.518 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:57:04.518 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:04.515+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:04.619 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:04.616+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:04.661 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:04.662 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:04.662 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:57:04.662 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:04.670 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:57:04.719 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:04.718+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:04.816 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:05.144 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:05.144 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:57:05.144 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:57:05.144 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:05.146 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:57:05.146 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:05.143+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:05.246 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:05.244+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:05.302 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-18T04:57:05.305 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:05.305 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:05.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:05.305 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:05.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.305 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:57:05.305 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:57:05.305 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 53.907731830000003, 2024-09-18T04:57:05.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037718511000000003, 2024-09-18T04:57:05.305 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:05.305 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:57:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:57:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-18T04:57:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:57:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:05.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:57:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393125+0000", 2024-09-18T04:57:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:57:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393133+0000", 2024-09-18T04:57:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.225e-06 2024-09-18T04:57:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393144+0000", 2024-09-18T04:57:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1279e-05 2024-09-18T04:57:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393153+0000", 2024-09-18T04:57:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4540000000000007e-06 2024-09-18T04:57:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393203+0000", 2024-09-18T04:57:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9963000000000003e-05 2024-09-18T04:57:05.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:57:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393210+0000", 2024-09-18T04:57:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5689999999999999e-06 2024-09-18T04:57:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427399+0000", 2024-09-18T04:57:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034188784999999999 2024-09-18T04:57:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:05.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427532+0000", 2024-09-18T04:57:05.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013264700000000001 2024-09-18T04:57:05.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:57:05.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427685+0000", 2024-09-18T04:57:05.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015300599999999999 2024-09-18T04:57:05.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:05.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.428520+0000", 2024-09-18T04:57:05.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00083502499999999998 2024-09-18T04:57:05.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:57:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430772+0000", 2024-09-18T04:57:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022518880000000001 2024-09-18T04:57:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:57:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430817+0000", 2024-09-18T04:57:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5077000000000002e-05 2024-09-18T04:57:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430845+0000", 2024-09-18T04:57:05.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8036999999999999e-05 2024-09-18T04:57:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54065 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3453952~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221852+0000", 2024-09-18T04:57:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.079006473, 2024-09-18T04:57:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029151789000000001, 2024-09-18T04:57:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:05.312 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54065 2024-09-18T04:57:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221852+0000", 2024-09-18T04:57:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221852+0000", 2024-09-18T04:57:05.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221851+0000", 2024-09-18T04:57:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221853+0000", 2024-09-18T04:57:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9570000000000001e-06 2024-09-18T04:57:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:05.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221859+0000", 2024-09-18T04:57:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0410000000000003e-06 2024-09-18T04:57:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221870+0000", 2024-09-18T04:57:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0801e-05 2024-09-18T04:57:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.250810+0000", 2024-09-18T04:57:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.028940006000000001 2024-09-18T04:57:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.250882+0000", 2024-09-18T04:57:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1737999999999997e-05 2024-09-18T04:57:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251003+0000", 2024-09-18T04:57:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000121605 2024-09-18T04:57:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54066 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3469312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221875+0000", 2024-09-18T04:57:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.078983294, 2024-09-18T04:57:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029331864999999999, 2024-09-18T04:57:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54066 2024-09-18T04:57:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:05.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221875+0000", 2024-09-18T04:57:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221875+0000", 2024-09-18T04:57:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221874+0000", 2024-09-18T04:57:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221877+0000", 2024-09-18T04:57:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1280000000000002e-06 2024-09-18T04:57:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221883+0000", 2024-09-18T04:57:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7109999999999998e-06 2024-09-18T04:57:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221888+0000", 2024-09-18T04:57:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.668e-06 2024-09-18T04:57:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251030+0000", 2024-09-18T04:57:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029142208999999999 2024-09-18T04:57:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251138+0000", 2024-09-18T04:57:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107518 2024-09-18T04:57:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251207+0000", 2024-09-18T04:57:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8978000000000003e-05 2024-09-18T04:57:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54067 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3484672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221893+0000", 2024-09-18T04:57:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.078965149, 2024-09-18T04:57:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029508817999999999, 2024-09-18T04:57:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:05.321 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54067 2024-09-18T04:57:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221893+0000", 2024-09-18T04:57:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221893+0000", 2024-09-18T04:57:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221893+0000", 2024-09-18T04:57:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221895+0000", 2024-09-18T04:57:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9709999999999998e-06 2024-09-18T04:57:05.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221902+0000", 2024-09-18T04:57:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4379999999999998e-06 2024-09-18T04:57:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221907+0000", 2024-09-18T04:57:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0019999999999997e-06 2024-09-18T04:57:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:05.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251260+0000", 2024-09-18T04:57:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029353338999999999 2024-09-18T04:57:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251336+0000", 2024-09-18T04:57:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5591000000000002e-05 2024-09-18T04:57:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251402+0000", 2024-09-18T04:57:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5752000000000003e-05 2024-09-18T04:57:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.325 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54068 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3500032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221912+0000", 2024-09-18T04:57:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.078946459999999, 2024-09-18T04:57:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029654553, 2024-09-18T04:57:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.326 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54068 2024-09-18T04:57:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221912+0000", 2024-09-18T04:57:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221912+0000", 2024-09-18T04:57:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221911+0000", 2024-09-18T04:57:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221916+0000", 2024-09-18T04:57:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0130000000000001e-06 2024-09-18T04:57:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221922+0000", 2024-09-18T04:57:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.105e-06 2024-09-18T04:57:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221928+0000", 2024-09-18T04:57:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0430000000000004e-06 2024-09-18T04:57:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251427+0000", 2024-09-18T04:57:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029498586 2024-09-18T04:57:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251495+0000", 2024-09-18T04:57:05.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8125000000000003e-05 2024-09-18T04:57:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251566+0000", 2024-09-18T04:57:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1145999999999999e-05 2024-09-18T04:57:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54069 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3515392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221945+0000", 2024-09-18T04:57:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.078912851, 2024-09-18T04:57:05.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029786185999999999, 2024-09-18T04:57:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54069 2024-09-18T04:57:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221945+0000", 2024-09-18T04:57:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221945+0000", 2024-09-18T04:57:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221945+0000", 2024-09-18T04:57:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:05.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221947+0000", 2024-09-18T04:57:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6189999999999998e-06 2024-09-18T04:57:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221955+0000", 2024-09-18T04:57:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6249999999999998e-06 2024-09-18T04:57:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221968+0000", 2024-09-18T04:57:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2846e-05 2024-09-18T04:57:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251595+0000", 2024-09-18T04:57:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029627158000000001 2024-09-18T04:57:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251666+0000", 2024-09-18T04:57:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1437000000000001e-05 2024-09-18T04:57:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251731+0000", 2024-09-18T04:57:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5071000000000001e-05 2024-09-18T04:57:05.334 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54070 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221973+0000", 2024-09-18T04:57:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.078884867999999, 2024-09-18T04:57:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029891219, 2024-09-18T04:57:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:05.335 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54070 2024-09-18T04:57:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221973+0000", 2024-09-18T04:57:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221973+0000", 2024-09-18T04:57:05.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221973+0000", 2024-09-18T04:57:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221985+0000", 2024-09-18T04:57:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1888999999999999e-05 2024-09-18T04:57:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221992+0000", 2024-09-18T04:57:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2830000000000004e-06 2024-09-18T04:57:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222008+0000", 2024-09-18T04:57:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5608999999999999e-05 2024-09-18T04:57:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251757+0000", 2024-09-18T04:57:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029749652000000001 2024-09-18T04:57:05.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251826+0000", 2024-09-18T04:57:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9103000000000006e-05 2024-09-18T04:57:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251864+0000", 2024-09-18T04:57:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8179000000000002e-05 2024-09-18T04:57:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.340 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54071 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3546112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:05.340 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222014+0000", 2024-09-18T04:57:05.340 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.078844437000001, 2024-09-18T04:57:05.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03002749, 2024-09-18T04:57:05.340 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:05.340 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:05.340 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:05.340 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:05.340 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.340 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54071 2024-09-18T04:57:05.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.340 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222014+0000", 2024-09-18T04:57:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222014+0000", 2024-09-18T04:57:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222013+0000", 2024-09-18T04:57:05.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222016+0000", 2024-09-18T04:57:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6639999999999998e-06 2024-09-18T04:57:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222023+0000", 2024-09-18T04:57:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.849e-06 2024-09-18T04:57:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222028+0000", 2024-09-18T04:57:05.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0030000000000002e-06 2024-09-18T04:57:05.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:05.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251895+0000", 2024-09-18T04:57:05.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029867482000000001 2024-09-18T04:57:05.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251963+0000", 2024-09-18T04:57:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8038000000000001e-05 2024-09-18T04:57:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252041+0000", 2024-09-18T04:57:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8028999999999994e-05 2024-09-18T04:57:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54072 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3561472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222034+0000", 2024-09-18T04:57:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.078824366999999, 2024-09-18T04:57:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030190214, 2024-09-18T04:57:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54072 2024-09-18T04:57:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:05.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222034+0000", 2024-09-18T04:57:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222034+0000", 2024-09-18T04:57:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222033+0000", 2024-09-18T04:57:05.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222035+0000", 2024-09-18T04:57:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0490000000000002e-06 2024-09-18T04:57:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222051+0000", 2024-09-18T04:57:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5914e-05 2024-09-18T04:57:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222057+0000", 2024-09-18T04:57:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4269999999999999e-06 2024-09-18T04:57:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252079+0000", 2024-09-18T04:57:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030022338999999999 2024-09-18T04:57:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252157+0000", 2024-09-18T04:57:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7459999999999994e-05 2024-09-18T04:57:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252224+0000", 2024-09-18T04:57:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7415000000000004e-05 2024-09-18T04:57:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.349 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54073 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3576832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222062+0000", 2024-09-18T04:57:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.07879659, 2024-09-18T04:57:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030280892, 2024-09-18T04:57:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54073 2024-09-18T04:57:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222062+0000", 2024-09-18T04:57:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222062+0000", 2024-09-18T04:57:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222061+0000", 2024-09-18T04:57:05.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222063+0000", 2024-09-18T04:57:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0729999999999999e-06 2024-09-18T04:57:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222070+0000", 2024-09-18T04:57:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3199999999999996e-06 2024-09-18T04:57:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222074+0000", 2024-09-18T04:57:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8659999999999997e-06 2024-09-18T04:57:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252255+0000", 2024-09-18T04:57:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030180350000000002 2024-09-18T04:57:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252305+0000", 2024-09-18T04:57:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0086000000000001e-05 2024-09-18T04:57:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252342+0000", 2024-09-18T04:57:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7594e-05 2024-09-18T04:57:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54074 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3592192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222079+0000", 2024-09-18T04:57:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.078779199, 2024-09-18T04:57:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030370844000000001, 2024-09-18T04:57:05.354 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54074 2024-09-18T04:57:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222079+0000", 2024-09-18T04:57:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222079+0000", 2024-09-18T04:57:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222079+0000", 2024-09-18T04:57:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:05.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222081+0000", 2024-09-18T04:57:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0820000000000001e-06 2024-09-18T04:57:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222087+0000", 2024-09-18T04:57:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0789999999999997e-06 2024-09-18T04:57:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222097+0000", 2024-09-18T04:57:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0446e-05 2024-09-18T04:57:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252365+0000", 2024-09-18T04:57:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030267544 2024-09-18T04:57:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252402+0000", 2024-09-18T04:57:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7117999999999999e-05 2024-09-18T04:57:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252450+0000", 2024-09-18T04:57:05.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7930999999999999e-05 2024-09-18T04:57:05.359 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.359 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:05.359 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.359 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54075 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3607552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:05.359 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222103+0000", 2024-09-18T04:57:05.359 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.078755147000001, 2024-09-18T04:57:05.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030531175000000001, 2024-09-18T04:57:05.359 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:05.359 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:05.359 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:05.359 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:05.360 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.360 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54075 2024-09-18T04:57:05.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.360 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:05.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:05.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222103+0000", 2024-09-18T04:57:05.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:05.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222103+0000", 2024-09-18T04:57:05.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:05.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222101+0000", 2024-09-18T04:57:05.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:57:05.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:05.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222105+0000", 2024-09-18T04:57:05.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4089999999999999e-06 2024-09-18T04:57:05.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:05.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222111+0000", 2024-09-18T04:57:05.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3720000000000001e-06 2024-09-18T04:57:05.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222116+0000", 2024-09-18T04:57:05.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.544e-06 2024-09-18T04:57:05.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:05.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252475+0000", 2024-09-18T04:57:05.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030358975999999999 2024-09-18T04:57:05.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:05.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252562+0000", 2024-09-18T04:57:05.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7257999999999997e-05 2024-09-18T04:57:05.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:05.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252634+0000", 2024-09-18T04:57:05.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2144999999999995e-05 2024-09-18T04:57:05.363 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.363 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:05.363 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.363 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54076 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3622912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:05.364 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222121+0000", 2024-09-18T04:57:05.364 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.078737256, 2024-09-18T04:57:05.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030699046000000001, 2024-09-18T04:57:05.364 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:05.364 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:05.364 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:05.364 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:05.364 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.364 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54076 2024-09-18T04:57:05.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.364 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:05.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:05.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222121+0000", 2024-09-18T04:57:05.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:05.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222121+0000", 2024-09-18T04:57:05.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:05.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222120+0000", 2024-09-18T04:57:05.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:05.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222123+0000", 2024-09-18T04:57:05.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2199999999999999e-06 2024-09-18T04:57:05.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:05.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222129+0000", 2024-09-18T04:57:05.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5250000000000002e-06 2024-09-18T04:57:05.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222134+0000", 2024-09-18T04:57:05.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8180000000000003e-06 2024-09-18T04:57:05.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:05.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252677+0000", 2024-09-18T04:57:05.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030542969999999999 2024-09-18T04:57:05.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:05.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252750+0000", 2024-09-18T04:57:05.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2601000000000005e-05 2024-09-18T04:57:05.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:05.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252820+0000", 2024-09-18T04:57:05.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0409999999999998e-05 2024-09-18T04:57:05.368 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.368 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:05.368 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.368 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54077 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3638272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:05.368 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222138+0000", 2024-09-18T04:57:05.369 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.078719756, 2024-09-18T04:57:05.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031008865, 2024-09-18T04:57:05.369 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:05.369 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:05.369 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:05.369 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:05.369 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.369 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54077 2024-09-18T04:57:05.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.369 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:05.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:05.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222138+0000", 2024-09-18T04:57:05.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:05.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222138+0000", 2024-09-18T04:57:05.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:05.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222138+0000", 2024-09-18T04:57:05.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:05.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222140+0000", 2024-09-18T04:57:05.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9700000000000002e-06 2024-09-18T04:57:05.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:05.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222146+0000", 2024-09-18T04:57:05.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1460000000000001e-06 2024-09-18T04:57:05.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222151+0000", 2024-09-18T04:57:05.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.763e-06 2024-09-18T04:57:05.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:05.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252989+0000", 2024-09-18T04:57:05.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030838580000000001 2024-09-18T04:57:05.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:05.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253072+0000", 2024-09-18T04:57:05.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2204000000000001e-05 2024-09-18T04:57:05.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:05.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253147+0000", 2024-09-18T04:57:05.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5661000000000006e-05 2024-09-18T04:57:05.373 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.373 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:05.373 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.373 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54078 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:05.373 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222155+0000", 2024-09-18T04:57:05.373 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.078703170000001, 2024-09-18T04:57:05.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031175118000000002, 2024-09-18T04:57:05.373 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:05.373 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:05.373 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:05.373 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:05.374 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.374 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54078 2024-09-18T04:57:05.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.374 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:05.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:05.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222155+0000", 2024-09-18T04:57:05.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:05.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222155+0000", 2024-09-18T04:57:05.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:05.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222155+0000", 2024-09-18T04:57:05.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:05.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222157+0000", 2024-09-18T04:57:05.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0660000000000002e-06 2024-09-18T04:57:05.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:05.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222163+0000", 2024-09-18T04:57:05.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2820000000000002e-06 2024-09-18T04:57:05.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222168+0000", 2024-09-18T04:57:05.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7979999999999996e-06 2024-09-18T04:57:05.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:05.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253174+0000", 2024-09-18T04:57:05.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031005721999999999 2024-09-18T04:57:05.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:05.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253251+0000", 2024-09-18T04:57:05.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7717000000000001e-05 2024-09-18T04:57:05.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:05.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253330+0000", 2024-09-18T04:57:05.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8858999999999998e-05 2024-09-18T04:57:05.377 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.377 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:05.377 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.378 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54079 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3668992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:05.378 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222172+0000", 2024-09-18T04:57:05.378 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.078686019999999, 2024-09-18T04:57:05.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031327661, 2024-09-18T04:57:05.378 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:05.378 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:05.378 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:05.378 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:05.378 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.378 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54079 2024-09-18T04:57:05.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.378 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:05.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:05.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222172+0000", 2024-09-18T04:57:05.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:05.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222172+0000", 2024-09-18T04:57:05.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:05.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222172+0000", 2024-09-18T04:57:05.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:05.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222174+0000", 2024-09-18T04:57:05.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0159999999999998e-06 2024-09-18T04:57:05.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:05.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222180+0000", 2024-09-18T04:57:05.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9950000000000002e-06 2024-09-18T04:57:05.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222185+0000", 2024-09-18T04:57:05.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7679999999999999e-06 2024-09-18T04:57:05.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:05.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253360+0000", 2024-09-18T04:57:05.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031175423000000001 2024-09-18T04:57:05.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:05.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253430+0000", 2024-09-18T04:57:05.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9929000000000001e-05 2024-09-18T04:57:05.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:05.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253500+0000", 2024-09-18T04:57:05.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9896999999999998e-05 2024-09-18T04:57:05.382 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.382 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:05.382 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.382 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54332 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3623936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:05.382 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.229403+0000", 2024-09-18T04:57:05.383 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.071455337, 2024-09-18T04:57:05.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029249332999999999, 2024-09-18T04:57:05.383 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:05.383 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:05.383 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:05.383 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:05.383 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.383 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54332 2024-09-18T04:57:05.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.383 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:05.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:05.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229403+0000", 2024-09-18T04:57:05.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:05.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229403+0000", 2024-09-18T04:57:05.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:05.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229402+0000", 2024-09-18T04:57:05.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:05.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229405+0000", 2024-09-18T04:57:05.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0150000000000002e-06 2024-09-18T04:57:05.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:05.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229412+0000", 2024-09-18T04:57:05.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6260000000000003e-06 2024-09-18T04:57:05.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229417+0000", 2024-09-18T04:57:05.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6199999999999998e-06 2024-09-18T04:57:05.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:05.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258523+0000", 2024-09-18T04:57:05.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029106396999999999 2024-09-18T04:57:05.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:05.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258589+0000", 2024-09-18T04:57:05.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6054000000000001e-05 2024-09-18T04:57:05.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:05.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258652+0000", 2024-09-18T04:57:05.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2917999999999996e-05 2024-09-18T04:57:05.387 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.387 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:05.387 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.387 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54333 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3639296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:05.388 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.229446+0000", 2024-09-18T04:57:05.388 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.071412339, 2024-09-18T04:57:05.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029358565, 2024-09-18T04:57:05.388 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:05.388 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:05.388 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:05.388 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:05.388 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.388 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54333 2024-09-18T04:57:05.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.388 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:05.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:05.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229446+0000", 2024-09-18T04:57:05.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:05.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229446+0000", 2024-09-18T04:57:05.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:05.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229445+0000", 2024-09-18T04:57:05.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:05.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229448+0000", 2024-09-18T04:57:05.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8200000000000001e-06 2024-09-18T04:57:05.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:05.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229456+0000", 2024-09-18T04:57:05.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.371e-06 2024-09-18T04:57:05.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229473+0000", 2024-09-18T04:57:05.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6595e-05 2024-09-18T04:57:05.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:05.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258677+0000", 2024-09-18T04:57:05.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029203863 2024-09-18T04:57:05.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:05.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258742+0000", 2024-09-18T04:57:05.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5273999999999999e-05 2024-09-18T04:57:05.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:05.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258804+0000", 2024-09-18T04:57:05.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2176999999999998e-05 2024-09-18T04:57:05.392 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.392 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:05.392 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.393 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54334 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3654656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:05.393 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.229481+0000", 2024-09-18T04:57:05.393 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.071377057999999, 2024-09-18T04:57:05.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029474845999999999, 2024-09-18T04:57:05.393 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:05.393 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:05.393 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:05.393 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:05.393 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.393 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54334 2024-09-18T04:57:05.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.393 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:05.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:05.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229481+0000", 2024-09-18T04:57:05.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:05.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229481+0000", 2024-09-18T04:57:05.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:05.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229481+0000", 2024-09-18T04:57:05.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:05.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229483+0000", 2024-09-18T04:57:05.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7580000000000001e-06 2024-09-18T04:57:05.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:05.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229490+0000", 2024-09-18T04:57:05.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7020000000000004e-06 2024-09-18T04:57:05.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229495+0000", 2024-09-18T04:57:05.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7770000000000002e-06 2024-09-18T04:57:05.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:05.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258830+0000", 2024-09-18T04:57:05.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029334828 2024-09-18T04:57:05.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:05.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258896+0000", 2024-09-18T04:57:05.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6384999999999995e-05 2024-09-18T04:57:05.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:05.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258956+0000", 2024-09-18T04:57:05.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9889999999999997e-05 2024-09-18T04:57:05.397 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.397 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:05.397 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.397 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:62520 3.13 3.84cdb893 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T04:57:05.398 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:05.398 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.003421947, 2024-09-18T04:57:05.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94452134799999998, 2024-09-18T04:57:05.398 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:05.398 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:57:05.398 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:05.398 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:05.398 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.398 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 62520 2024-09-18T04:57:05.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.398 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:05.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:05.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:05.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:05.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:05.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:05.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:05.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:05.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:05.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297441+0000", 2024-09-18T04:57:05.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9840000000000001e-06 2024-09-18T04:57:05.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:05.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297449+0000", 2024-09-18T04:57:05.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3999999999999992e-06 2024-09-18T04:57:05.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297455+0000", 2024-09-18T04:57:05.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3350000000000003e-06 2024-09-18T04:57:05.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:05.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297552+0000", 2024-09-18T04:57:05.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6602999999999996e-05 2024-09-18T04:57:05.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:57:05.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297559+0000", 2024-09-18T04:57:05.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.52e-06 2024-09-18T04:57:05.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:05.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241943+0000", 2024-09-18T04:57:05.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94438357100000003 2024-09-18T04:57:05.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:05.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:05.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:05.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241958+0000", 2024-09-18T04:57:05.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4598e-05 2024-09-18T04:57:05.402 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.402 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:05.402 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.402 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:05.402 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:05.402 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:05.404 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:05.402+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:05.405 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:57:05.504 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:05.502+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:05.557 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:05.557 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:05.557 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:05.557 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:05.566 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:57:05.605 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:05.602+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:05.672 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:05.672 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:05.672 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:05.672 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:05.672 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:05.680 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:57:05.724 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:05.722+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:05.788 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:05.788 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:05.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:05.788 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:05.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.788 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9316 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:05.788 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:05.789 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.017888963000001, 2024-09-18T04:57:05.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044027208999999998, 2024-09-18T04:57:05.789 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:05.789 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:05.789 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:05.789 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:05.789 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.789 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9316 2024-09-18T04:57:05.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.789 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:05.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:05.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:05.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:05.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:05.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:05.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:05.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:05.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767144+0000", 2024-09-18T04:57:05.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3070000000000001e-06 2024-09-18T04:57:05.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:05.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767150+0000", 2024-09-18T04:57:05.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1700000000000002e-06 2024-09-18T04:57:05.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767163+0000", 2024-09-18T04:57:05.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2670999999999999e-05 2024-09-18T04:57:05.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:05.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810995+0000", 2024-09-18T04:57:05.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043832310999999999 2024-09-18T04:57:05.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:05.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811087+0000", 2024-09-18T04:57:05.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1345e-05 2024-09-18T04:57:05.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:05.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811169+0000", 2024-09-18T04:57:05.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2848000000000004e-05 2024-09-18T04:57:05.793 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.793 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:05.793 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.793 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9317 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1728512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:05.793 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:57:05.793 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.017863328000001, 2024-09-18T04:57:05.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044190170000000001, 2024-09-18T04:57:05.794 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:05.794 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:05.794 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:05.794 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:05.794 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.794 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9317 2024-09-18T04:57:05.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.794 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:05.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:05.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:57:05.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:05.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:57:05.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:05.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767167+0000", 2024-09-18T04:57:05.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:05.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767170+0000", 2024-09-18T04:57:05.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7259999999999998e-06 2024-09-18T04:57:05.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:05.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767176+0000", 2024-09-18T04:57:05.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1460000000000001e-06 2024-09-18T04:57:05.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767182+0000", 2024-09-18T04:57:05.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1020000000000002e-06 2024-09-18T04:57:05.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:05.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811204+0000", 2024-09-18T04:57:05.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044022039999999998 2024-09-18T04:57:05.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:05.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811286+0000", 2024-09-18T04:57:05.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1623000000000001e-05 2024-09-18T04:57:05.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:05.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811358+0000", 2024-09-18T04:57:05.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2125000000000005e-05 2024-09-18T04:57:05.798 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.798 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:05.798 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.798 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9318 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1777664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:05.798 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:05.798 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.017844783000001, 2024-09-18T04:57:05.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044380181999999997, 2024-09-18T04:57:05.798 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:05.798 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:05.799 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:05.799 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:05.799 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.799 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9318 2024-09-18T04:57:05.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.799 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:05.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:05.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:05.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:05.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:05.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:05.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:05.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:05.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767189+0000", 2024-09-18T04:57:05.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5909999999999998e-06 2024-09-18T04:57:05.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:05.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767194+0000", 2024-09-18T04:57:05.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7690000000000002e-06 2024-09-18T04:57:05.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767199+0000", 2024-09-18T04:57:05.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189999999999999e-06 2024-09-18T04:57:05.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:05.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811394+0000", 2024-09-18T04:57:05.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044195350000000001 2024-09-18T04:57:05.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:05.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811483+0000", 2024-09-18T04:57:05.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8282999999999995e-05 2024-09-18T04:57:05.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:05.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811567+0000", 2024-09-18T04:57:05.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3855000000000003e-05 2024-09-18T04:57:05.803 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.803 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:05.803 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.803 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9325 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:05.803 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:05.803 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.016114245000001, 2024-09-18T04:57:05.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044063989999999997, 2024-09-18T04:57:05.803 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:05.803 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:05.804 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:05.804 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:05.804 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.804 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9325 2024-09-18T04:57:05.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.804 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:05.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:05.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:05.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:05.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:05.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:05.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:05.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:05.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768919+0000", 2024-09-18T04:57:05.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:57:05.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:05.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768925+0000", 2024-09-18T04:57:05.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2310000000000001e-06 2024-09-18T04:57:05.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768929+0000", 2024-09-18T04:57:05.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4469999999999999e-06 2024-09-18T04:57:05.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:05.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812823+0000", 2024-09-18T04:57:05.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043893975000000002 2024-09-18T04:57:05.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:05.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812908+0000", 2024-09-18T04:57:05.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4862000000000002e-05 2024-09-18T04:57:05.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:05.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812981+0000", 2024-09-18T04:57:05.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2892000000000006e-05 2024-09-18T04:57:05.807 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.807 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:05.807 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.808 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9334 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:05.808 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:57:05.808 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.014095459, 2024-09-18T04:57:05.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044042853999999999, 2024-09-18T04:57:05.808 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:05.808 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:05.808 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:05.808 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:05.808 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.808 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9334 2024-09-18T04:57:05.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.808 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:05.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:05.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:57:05.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:05.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:57:05.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:05.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770935+0000", 2024-09-18T04:57:05.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:05.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770937+0000", 2024-09-18T04:57:05.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1489999999999999e-06 2024-09-18T04:57:05.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:05.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770940+0000", 2024-09-18T04:57:05.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9569999999999998e-06 2024-09-18T04:57:05.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770945+0000", 2024-09-18T04:57:05.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:57:05.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:05.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814838+0000", 2024-09-18T04:57:05.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043892635999999999 2024-09-18T04:57:05.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:05.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814920+0000", 2024-09-18T04:57:05.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2175000000000005e-05 2024-09-18T04:57:05.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:05.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814979+0000", 2024-09-18T04:57:05.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8372000000000003e-05 2024-09-18T04:57:05.812 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.812 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:05.812 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.812 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9352 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1159168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:05.812 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:57:05.812 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.011874282000001, 2024-09-18T04:57:05.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044011269999999998, 2024-09-18T04:57:05.812 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:05.813 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:05.813 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:05.813 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:05.813 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.813 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9352 2024-09-18T04:57:05.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.813 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:05.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:05.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:57:05.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:05.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:57:05.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:05.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773156+0000", 2024-09-18T04:57:05.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:05.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773159+0000", 2024-09-18T04:57:05.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2919999999999998e-06 2024-09-18T04:57:05.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:05.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773165+0000", 2024-09-18T04:57:05.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0580000000000003e-06 2024-09-18T04:57:05.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773177+0000", 2024-09-18T04:57:05.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.204e-05 2024-09-18T04:57:05.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:05.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817082+0000", 2024-09-18T04:57:05.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043905528999999999 2024-09-18T04:57:05.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:05.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817122+0000", 2024-09-18T04:57:05.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9258999999999998e-05 2024-09-18T04:57:05.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:05.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817168+0000", 2024-09-18T04:57:05.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6545000000000002e-05 2024-09-18T04:57:05.816 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.816 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:05.816 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.817 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9353 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1208320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:05.817 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:05.817 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.011849984000001, 2024-09-18T04:57:05.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044105537, 2024-09-18T04:57:05.817 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:05.817 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:05.817 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:05.817 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:05.817 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.817 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9353 2024-09-18T04:57:05.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.818 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:05.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:05.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:05.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:05.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:05.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:05.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:05.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:05.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773183+0000", 2024-09-18T04:57:05.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2929999999999999e-06 2024-09-18T04:57:05.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:05.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773190+0000", 2024-09-18T04:57:05.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4330000000000001e-06 2024-09-18T04:57:05.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773194+0000", 2024-09-18T04:57:05.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.865e-06 2024-09-18T04:57:05.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:05.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817196+0000", 2024-09-18T04:57:05.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044001708 2024-09-18T04:57:05.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:05.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817236+0000", 2024-09-18T04:57:05.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0043999999999998e-05 2024-09-18T04:57:05.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:05.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817287+0000", 2024-09-18T04:57:05.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0605000000000001e-05 2024-09-18T04:57:05.821 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.821 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:05.821 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.821 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9354 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:05.822 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:57:05.822 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.011832472999998, 2024-09-18T04:57:05.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044199796, 2024-09-18T04:57:05.822 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:05.822 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:05.822 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:05.822 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:05.822 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.822 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9354 2024-09-18T04:57:05.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.822 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:05.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:05.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:57:05.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:05.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:57:05.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:05.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773198+0000", 2024-09-18T04:57:05.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:05.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773200+0000", 2024-09-18T04:57:05.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4159999999999998e-06 2024-09-18T04:57:05.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:05.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773206+0000", 2024-09-18T04:57:05.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.789e-06 2024-09-18T04:57:05.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773210+0000", 2024-09-18T04:57:05.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2679999999999996e-06 2024-09-18T04:57:05.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:05.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817314+0000", 2024-09-18T04:57:05.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044103187000000002 2024-09-18T04:57:05.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:05.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817353+0000", 2024-09-18T04:57:05.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8973000000000001e-05 2024-09-18T04:57:05.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:05.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817399+0000", 2024-09-18T04:57:05.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5877999999999997e-05 2024-09-18T04:57:05.826 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.826 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:05.826 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.826 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46591 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2097152~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:05.826 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:05.826 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.086976056000001, 2024-09-18T04:57:05.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043987310000000002, 2024-09-18T04:57:05.827 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:05.827 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:05.827 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:05.827 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:05.827 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.827 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46591 2024-09-18T04:57:05.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.827 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:05.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:05.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:05.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:05.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:05.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:05.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:05.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:05.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698057+0000", 2024-09-18T04:57:05.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.531e-06 2024-09-18T04:57:05.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:05.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698064+0000", 2024-09-18T04:57:05.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-18T04:57:05.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698069+0000", 2024-09-18T04:57:05.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.091e-06 2024-09-18T04:57:05.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:05.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741843+0000", 2024-09-18T04:57:05.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043773470000000002 2024-09-18T04:57:05.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:05.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741928+0000", 2024-09-18T04:57:05.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5113999999999999e-05 2024-09-18T04:57:05.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:05.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742042+0000", 2024-09-18T04:57:05.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114527 2024-09-18T04:57:05.831 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.831 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:05.831 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.831 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46592 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2121728~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:05.831 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:57:05.831 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.086957629, 2024-09-18T04:57:05.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044077948999999998, 2024-09-18T04:57:05.831 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:05.831 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:05.832 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:05.832 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:05.832 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.832 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46592 2024-09-18T04:57:05.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.832 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:05.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:05.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:57:05.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:05.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:57:05.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:05.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698073+0000", 2024-09-18T04:57:05.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:05.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698076+0000", 2024-09-18T04:57:05.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.481e-06 2024-09-18T04:57:05.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:05.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698082+0000", 2024-09-18T04:57:05.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8730000000000004e-06 2024-09-18T04:57:05.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698086+0000", 2024-09-18T04:57:05.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:57:05.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:05.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742055+0000", 2024-09-18T04:57:05.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043969170000000002 2024-09-18T04:57:05.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:05.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742093+0000", 2024-09-18T04:57:05.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7707000000000003e-05 2024-09-18T04:57:05.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:05.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742152+0000", 2024-09-18T04:57:05.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8323999999999999e-05 2024-09-18T04:57:05.835 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.835 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:05.836 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.836 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46593 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2211840~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:05.836 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:05.836 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.086940726000002, 2024-09-18T04:57:05.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044259917000000003, 2024-09-18T04:57:05.836 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:05.836 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:05.836 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:05.836 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:05.836 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.836 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46593 2024-09-18T04:57:05.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.837 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:05.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:05.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:05.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:05.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:05.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:05.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:05.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:05.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698092+0000", 2024-09-18T04:57:05.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2249999999999999e-06 2024-09-18T04:57:05.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:05.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698098+0000", 2024-09-18T04:57:05.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:57:05.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698103+0000", 2024-09-18T04:57:05.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:57:05.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:05.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742182+0000", 2024-09-18T04:57:05.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044079508000000003 2024-09-18T04:57:05.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:05.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742257+0000", 2024-09-18T04:57:05.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4678000000000006e-05 2024-09-18T04:57:05.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:05.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742350+0000", 2024-09-18T04:57:05.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3531999999999998e-05 2024-09-18T04:57:05.840 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.840 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:05.840 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.840 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46594 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2236416~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:05.841 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:05.841 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.086925029, 2024-09-18T04:57:05.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044378984000000003, 2024-09-18T04:57:05.841 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:05.841 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:05.841 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:05.841 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:05.841 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.841 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46594 2024-09-18T04:57:05.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.841 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:05.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:05.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:05.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:05.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:05.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:05.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:05.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:05.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698108+0000", 2024-09-18T04:57:05.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0880000000000002e-06 2024-09-18T04:57:05.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:05.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698114+0000", 2024-09-18T04:57:05.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6269999999999997e-06 2024-09-18T04:57:05.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698118+0000", 2024-09-18T04:57:05.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4050000000000002e-06 2024-09-18T04:57:05.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:05.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742365+0000", 2024-09-18T04:57:05.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044246799000000003 2024-09-18T04:57:05.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:05.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742407+0000", 2024-09-18T04:57:05.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2008000000000002e-05 2024-09-18T04:57:05.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:05.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742485+0000", 2024-09-18T04:57:05.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8405999999999994e-05 2024-09-18T04:57:05.845 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.845 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:05.845 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.845 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46595 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:05.845 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:05.845 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.086908955999998, 2024-09-18T04:57:05.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.04440463, 2024-09-18T04:57:05.845 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:05.846 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:05.846 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:05.846 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:05.846 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.846 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46595 2024-09-18T04:57:05.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.846 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:05.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:05.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:05.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:05.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:05.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:05.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:05.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:05.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698124+0000", 2024-09-18T04:57:05.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1840000000000002e-06 2024-09-18T04:57:05.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:05.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698130+0000", 2024-09-18T04:57:05.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.947e-06 2024-09-18T04:57:05.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698136+0000", 2024-09-18T04:57:05.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6520000000000003e-06 2024-09-18T04:57:05.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:05.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742521+0000", 2024-09-18T04:57:05.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044385446000000002 2024-09-18T04:57:05.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:05.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742527+0000", 2024-09-18T04:57:05.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:57:05.849 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.849 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:05.849 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.849 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46596 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:05.849 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:05.850 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.086890711999999, 2024-09-18T04:57:05.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044418066999999999, 2024-09-18T04:57:05.850 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:05.850 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:05.850 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:05.850 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:05.850 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.850 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46596 2024-09-18T04:57:05.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.850 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:05.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:05.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:05.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:05.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:05.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:05.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:05.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:05.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698146+0000", 2024-09-18T04:57:05.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2779999999999999e-06 2024-09-18T04:57:05.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:05.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698153+0000", 2024-09-18T04:57:05.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3849999999999999e-06 2024-09-18T04:57:05.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698164+0000", 2024-09-18T04:57:05.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1375999999999999e-05 2024-09-18T04:57:05.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:05.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742554+0000", 2024-09-18T04:57:05.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044389482000000001 2024-09-18T04:57:05.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:05.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742559+0000", 2024-09-18T04:57:05.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9540000000000003e-06 2024-09-18T04:57:05.853 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.853 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:05.853 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.854 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46597 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:05.854 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:57:05.854 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.086862347, 2024-09-18T04:57:05.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044416513999999997, 2024-09-18T04:57:05.854 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:05.854 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:05.854 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:05.854 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:05.854 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.854 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46597 2024-09-18T04:57:05.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.854 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:05.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:05.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:57:05.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:05.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:57:05.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:05.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698168+0000", 2024-09-18T04:57:05.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:05.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698175+0000", 2024-09-18T04:57:05.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4520000000000002e-06 2024-09-18T04:57:05.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:05.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698181+0000", 2024-09-18T04:57:05.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4150000000000004e-06 2024-09-18T04:57:05.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698186+0000", 2024-09-18T04:57:05.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7489999999999998e-06 2024-09-18T04:57:05.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:05.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742580+0000", 2024-09-18T04:57:05.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044394325999999998 2024-09-18T04:57:05.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:05.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742585+0000", 2024-09-18T04:57:05.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9849999999999997e-06 2024-09-18T04:57:05.857 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.858 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:05.858 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.858 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46598 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:05.858 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:57:05.858 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.086754871, 2024-09-18T04:57:05.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044335316999999999, 2024-09-18T04:57:05.858 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:05.858 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:05.858 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:05.858 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:05.858 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.859 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46598 2024-09-18T04:57:05.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.859 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:05.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:05.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:57:05.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:05.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:57:05.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:05.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698190+0000", 2024-09-18T04:57:05.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999137 2024-09-18T04:57:05.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:05.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698280+0000", 2024-09-18T04:57:05.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0409999999999997e-05 2024-09-18T04:57:05.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:05.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698289+0000", 2024-09-18T04:57:05.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9490000000000002e-06 2024-09-18T04:57:05.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698296+0000", 2024-09-18T04:57:05.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2670000000000003e-06 2024-09-18T04:57:05.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:05.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742607+0000", 2024-09-18T04:57:05.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044311072999999999 2024-09-18T04:57:05.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:05.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742612+0000", 2024-09-18T04:57:05.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9400000000000001e-06 2024-09-18T04:57:05.862 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.862 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:05.862 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.862 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57986 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3086336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:05.863 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:05.863 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.099909148, 2024-09-18T04:57:05.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044047731999999999, 2024-09-18T04:57:05.863 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:05.863 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:05.863 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:05.863 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:05.863 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.863 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57986 2024-09-18T04:57:05.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.863 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:05.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:05.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:05.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:05.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:05.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:05.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:05.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:05.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685124+0000", 2024-09-18T04:57:05.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2859999999999998e-06 2024-09-18T04:57:05.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:05.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685130+0000", 2024-09-18T04:57:05.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5400000000000001e-06 2024-09-18T04:57:05.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685136+0000", 2024-09-18T04:57:05.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.152e-06 2024-09-18T04:57:05.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:05.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.728976+0000", 2024-09-18T04:57:05.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043840037999999998 2024-09-18T04:57:05.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:05.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729064+0000", 2024-09-18T04:57:05.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8709999999999996e-05 2024-09-18T04:57:05.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:05.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729170+0000", 2024-09-18T04:57:05.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000105386 2024-09-18T04:57:05.867 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.867 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:05.867 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.867 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57987 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3101696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:05.867 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:57:05.867 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.099860173, 2024-09-18T04:57:05.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044238123999999997, 2024-09-18T04:57:05.867 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:05.868 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:05.868 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:05.868 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:05.868 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.868 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57987 2024-09-18T04:57:05.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.868 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:05.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:05.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:57:05.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:05.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:57:05.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:05.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685170+0000", 2024-09-18T04:57:05.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:05.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:05.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685175+0000", 2024-09-18T04:57:05.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9069999999999998e-06 2024-09-18T04:57:05.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:05.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685187+0000", 2024-09-18T04:57:05.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.189e-05 2024-09-18T04:57:05.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685195+0000", 2024-09-18T04:57:05.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9459999999999998e-06 2024-09-18T04:57:05.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:05.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729212+0000", 2024-09-18T04:57:05.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044017239 2024-09-18T04:57:05.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:05.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729301+0000", 2024-09-18T04:57:05.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8887999999999993e-05 2024-09-18T04:57:05.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:05.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729409+0000", 2024-09-18T04:57:05.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108119 2024-09-18T04:57:05.871 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.871 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:05.872 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.872 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57988 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3117056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:05.872 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:57:05.872 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.099818054, 2024-09-18T04:57:05.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044429835000000001, 2024-09-18T04:57:05.872 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:05.872 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:05.872 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:05.872 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:05.872 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.872 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57988 2024-09-18T04:57:05.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.873 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:05.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:05.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:57:05.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:05.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:57:05.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:05.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685212+0000", 2024-09-18T04:57:05.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:05.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685216+0000", 2024-09-18T04:57:05.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7390000000000001e-06 2024-09-18T04:57:05.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:05.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685224+0000", 2024-09-18T04:57:05.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2260000000000005e-06 2024-09-18T04:57:05.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685230+0000", 2024-09-18T04:57:05.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0299999999999999e-06 2024-09-18T04:57:05.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:05.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729454+0000", 2024-09-18T04:57:05.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044223248999999999 2024-09-18T04:57:05.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:05.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729539+0000", 2024-09-18T04:57:05.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5029000000000002e-05 2024-09-18T04:57:05.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:05.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729643+0000", 2024-09-18T04:57:05.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104267 2024-09-18T04:57:05.876 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.876 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:05.876 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.876 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57989 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3132416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:05.877 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:57:05.877 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 15.099795302, 2024-09-18T04:57:05.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044642881000000002, 2024-09-18T04:57:05.877 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:05.877 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:05.877 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:05.877 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:05.877 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:05.877 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57989 2024-09-18T04:57:05.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.877 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:05.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:05.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:57:05.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:05.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:57:05.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:05.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:05.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685235+0000", 2024-09-18T04:57:05.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:05.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:05.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685238+0000", 2024-09-18T04:57:05.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.667e-06 2024-09-18T04:57:05.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:05.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685245+0000", 2024-09-18T04:57:05.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7909999999999999e-06 2024-09-18T04:57:05.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:05.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685250+0000", 2024-09-18T04:57:05.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3730000000000001e-06 2024-09-18T04:57:05.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:05.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729685+0000", 2024-09-18T04:57:05.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044434552000000002 2024-09-18T04:57:05.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:05.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729775+0000", 2024-09-18T04:57:05.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0395000000000004e-05 2024-09-18T04:57:05.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:05.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:05.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:05.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729879+0000", 2024-09-18T04:57:05.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010361099999999999 2024-09-18T04:57:05.881 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.881 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:05.881 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.881 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:05.881 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:05.881 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:05.882 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:57:05.883 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:05.880+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:05.983 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:05.981+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:06.029 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:06.029 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:06.029 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:06.029 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:06.037 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:57:06.084 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:06.082+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:06.183 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:06.183 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:06.183 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:06.183 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:06.183 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:06.184 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:06.182+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:06.192 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:57:06.285 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:06.283+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:06.361 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:06.361 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:06.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:06.361 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:06.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.362 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14125 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:06.362 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:57:06.362 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.230643651000001, 2024-09-18T04:57:06.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052956470999999998, 2024-09-18T04:57:06.362 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:06.362 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:06.362 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:06.362 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:06.362 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:06.362 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14125 2024-09-18T04:57:06.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.362 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:06.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:06.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:57:06.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:06.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:57:06.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:06.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126849+0000", 2024-09-18T04:57:06.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:06.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:06.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126852+0000", 2024-09-18T04:57:06.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3659999999999999e-06 2024-09-18T04:57:06.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:06.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126858+0000", 2024-09-18T04:57:06.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4540000000000004e-06 2024-09-18T04:57:06.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:06.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126863+0000", 2024-09-18T04:57:06.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1780000000000002e-06 2024-09-18T04:57:06.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:06.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179608+0000", 2024-09-18T04:57:06.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052745159999999999 2024-09-18T04:57:06.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:06.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179703+0000", 2024-09-18T04:57:06.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4773999999999998e-05 2024-09-18T04:57:06.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:06.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179806+0000", 2024-09-18T04:57:06.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010296799999999999 2024-09-18T04:57:06.366 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.366 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:06.366 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.366 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14136 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1732608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:06.367 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:06.367 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.228483099999998, 2024-09-18T04:57:06.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052973412999999997, 2024-09-18T04:57:06.367 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:06.367 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:06.367 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:06.367 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:06.367 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:06.367 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14136 2024-09-18T04:57:06.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.368 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:06.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:06.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:06.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:06.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:06.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:06.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:06.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:06.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:06.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129013+0000", 2024-09-18T04:57:06.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.96e-06 2024-09-18T04:57:06.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:06.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129020+0000", 2024-09-18T04:57:06.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6809999999999998e-06 2024-09-18T04:57:06.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:06.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129026+0000", 2024-09-18T04:57:06.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:57:06.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:06.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181873+0000", 2024-09-18T04:57:06.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052847238999999997 2024-09-18T04:57:06.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:06.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181932+0000", 2024-09-18T04:57:06.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9339000000000002e-05 2024-09-18T04:57:06.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:06.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181984+0000", 2024-09-18T04:57:06.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1348999999999999e-05 2024-09-18T04:57:06.371 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.371 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:06.371 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.372 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14137 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1781760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:06.372 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:57:06.372 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.228462695000001, 2024-09-18T04:57:06.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053128914999999999, 2024-09-18T04:57:06.372 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:06.373 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:06.373 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:06.373 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:06.373 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:06.373 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14137 2024-09-18T04:57:06.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.374 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:06.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:06.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:57:06.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:06.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:57:06.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:06.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129030+0000", 2024-09-18T04:57:06.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:06.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:06.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129033+0000", 2024-09-18T04:57:06.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7439999999999999e-06 2024-09-18T04:57:06.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:06.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129039+0000", 2024-09-18T04:57:06.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.314e-06 2024-09-18T04:57:06.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:06.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129044+0000", 2024-09-18T04:57:06.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9919999999999998e-06 2024-09-18T04:57:06.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:06.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182015+0000", 2024-09-18T04:57:06.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052970875000000001 2024-09-18T04:57:06.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:06.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182058+0000", 2024-09-18T04:57:06.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2997000000000002e-05 2024-09-18T04:57:06.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:06.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182160+0000", 2024-09-18T04:57:06.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101437 2024-09-18T04:57:06.379 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.379 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:06.379 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.379 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14138 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1830912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:06.379 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:57:06.380 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.228444661000001, 2024-09-18T04:57:06.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053256435999999997, 2024-09-18T04:57:06.380 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:06.380 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:06.380 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:06.380 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:06.380 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:06.381 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14138 2024-09-18T04:57:06.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.381 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:06.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:06.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:57:06.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:06.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:57:06.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:06.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129048+0000", 2024-09-18T04:57:06.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:06.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:06.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129056+0000", 2024-09-18T04:57:06.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.413e-06 2024-09-18T04:57:06.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:06.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129062+0000", 2024-09-18T04:57:06.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6810000000000001e-06 2024-09-18T04:57:06.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:06.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129068+0000", 2024-09-18T04:57:06.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0459999999999996e-06 2024-09-18T04:57:06.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:06.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182191+0000", 2024-09-18T04:57:06.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053123210999999997 2024-09-18T04:57:06.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:06.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182238+0000", 2024-09-18T04:57:06.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7404999999999998e-05 2024-09-18T04:57:06.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:06.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182305+0000", 2024-09-18T04:57:06.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7028999999999998e-05 2024-09-18T04:57:06.385 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.385 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:06.385 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.386 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15910 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2236416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:06.386 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:06.386 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.129357128999999, 2024-09-18T04:57:06.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053241054000000003, 2024-09-18T04:57:06.386 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:06.386 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:06.386 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:06.386 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:06.386 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:06.386 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15910 2024-09-18T04:57:06.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.387 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:06.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:06.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:06.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:06.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:06.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:06.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:06.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:06.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:06.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228138+0000", 2024-09-18T04:57:06.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:57:06.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:06.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228144+0000", 2024-09-18T04:57:06.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5369999999999997e-06 2024-09-18T04:57:06.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:06.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228149+0000", 2024-09-18T04:57:06.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1209999999999998e-06 2024-09-18T04:57:06.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:06.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280822+0000", 2024-09-18T04:57:06.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052673194999999999 2024-09-18T04:57:06.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:06.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280903+0000", 2024-09-18T04:57:06.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1179000000000003e-05 2024-09-18T04:57:06.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:06.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281377+0000", 2024-09-18T04:57:06.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047417899999999997 2024-09-18T04:57:06.392 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.392 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:06.392 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.393 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15911 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2285568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:06.393 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:06.393 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.129340397, 2024-09-18T04:57:06.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053415151000000001, 2024-09-18T04:57:06.393 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:06.393 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:06.393 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:06.393 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:06.393 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:06.394 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15911 2024-09-18T04:57:06.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.394 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:06.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:06.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:06.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:06.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:06.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:06.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:06.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:06.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:06.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228155+0000", 2024-09-18T04:57:06.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:57:06.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:06.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228161+0000", 2024-09-18T04:57:06.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:57:06.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:06.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228166+0000", 2024-09-18T04:57:06.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2320000000000001e-06 2024-09-18T04:57:06.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:06.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281405+0000", 2024-09-18T04:57:06.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053239336999999998 2024-09-18T04:57:06.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:06.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281482+0000", 2024-09-18T04:57:06.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6569999999999994e-05 2024-09-18T04:57:06.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:06.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281568+0000", 2024-09-18T04:57:06.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6328000000000004e-05 2024-09-18T04:57:06.398 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.398 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:06.398 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.398 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15918 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2662400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:06.398 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:06.398 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.127571415999999, 2024-09-18T04:57:06.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053091065, 2024-09-18T04:57:06.399 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:06.399 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:06.399 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:06.399 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:06.399 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:06.399 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15918 2024-09-18T04:57:06.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.399 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:06.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:06.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:06.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:06.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:06.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:06.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:06.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:06.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:06.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229924+0000", 2024-09-18T04:57:06.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:57:06.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:06.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229930+0000", 2024-09-18T04:57:06.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.066e-06 2024-09-18T04:57:06.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:06.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229935+0000", 2024-09-18T04:57:06.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9119999999999997e-06 2024-09-18T04:57:06.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:06.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282840+0000", 2024-09-18T04:57:06.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052905119 2024-09-18T04:57:06.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:06.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282922+0000", 2024-09-18T04:57:06.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2081000000000002e-05 2024-09-18T04:57:06.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:06.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283013+0000", 2024-09-18T04:57:06.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0828000000000005e-05 2024-09-18T04:57:06.403 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.403 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:06.403 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.404 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15919 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2711552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:06.404 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:06.404 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.127554138000001, 2024-09-18T04:57:06.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053293021000000003, 2024-09-18T04:57:06.405 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:06.405 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:06.405 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:06.405 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:06.405 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:06.405 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15919 2024-09-18T04:57:06.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.405 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:06.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:06.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:06.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:06.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:06.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:06.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:06.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:06.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:06.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229941+0000", 2024-09-18T04:57:06.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2639999999999998e-06 2024-09-18T04:57:06.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:06.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229947+0000", 2024-09-18T04:57:06.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9750000000000004e-06 2024-09-18T04:57:06.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:06.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229952+0000", 2024-09-18T04:57:06.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7929999999999997e-06 2024-09-18T04:57:06.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:06.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283062+0000", 2024-09-18T04:57:06.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05310952 2024-09-18T04:57:06.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:06.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283162+0000", 2024-09-18T04:57:06.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100098 2024-09-18T04:57:06.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:06.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283232+0000", 2024-09-18T04:57:06.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0693999999999998e-05 2024-09-18T04:57:06.410 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.410 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:06.410 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.411 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15920 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2760704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:06.411 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:06.411 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.127537376999999, 2024-09-18T04:57:06.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053502367000000002, 2024-09-18T04:57:06.411 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:06.411 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:06.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:06.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:06.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:06.411 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15920 2024-09-18T04:57:06.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.412 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:06.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:06.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:06.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:06.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:06.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:06.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:06.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:06.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:06.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229958+0000", 2024-09-18T04:57:06.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1770000000000001e-06 2024-09-18T04:57:06.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:06.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229964+0000", 2024-09-18T04:57:06.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8470000000000001e-06 2024-09-18T04:57:06.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:06.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229968+0000", 2024-09-18T04:57:06.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5680000000000001e-06 2024-09-18T04:57:06.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:06.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283269+0000", 2024-09-18T04:57:06.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053300620999999999 2024-09-18T04:57:06.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:06.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283359+0000", 2024-09-18T04:57:06.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9971999999999998e-05 2024-09-18T04:57:06.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:06.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283458+0000", 2024-09-18T04:57:06.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9483999999999998e-05 2024-09-18T04:57:06.418 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.418 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:06.418 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.418 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15921 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2809856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:06.419 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:57:06.419 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.127520827000001, 2024-09-18T04:57:06.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053800977, 2024-09-18T04:57:06.419 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:06.419 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:06.419 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:06.419 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:06.419 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:06.419 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15921 2024-09-18T04:57:06.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.419 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:06.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:06.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:57:06.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:06.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:57:06.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:06.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229972+0000", 2024-09-18T04:57:06.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:06.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:06.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229974+0000", 2024-09-18T04:57:06.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2910000000000002e-06 2024-09-18T04:57:06.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:06.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229982+0000", 2024-09-18T04:57:06.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7249999999999997e-06 2024-09-18T04:57:06.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:06.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229989+0000", 2024-09-18T04:57:06.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6429999999999998e-06 2024-09-18T04:57:06.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:06.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283601+0000", 2024-09-18T04:57:06.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053612211 2024-09-18T04:57:06.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:06.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283681+0000", 2024-09-18T04:57:06.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0125999999999998e-05 2024-09-18T04:57:06.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:06.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283774+0000", 2024-09-18T04:57:06.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2362999999999996e-05 2024-09-18T04:57:06.423 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.423 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:06.423 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.424 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15922 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2859008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:06.424 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:57:06.424 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.127475887999999, 2024-09-18T04:57:06.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053882270000000003, 2024-09-18T04:57:06.424 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:06.424 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:06.424 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:06.424 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:06.425 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:06.425 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15922 2024-09-18T04:57:06.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.425 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:06.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:06.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:57:06.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:06.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:57:06.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:06.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230017+0000", 2024-09-18T04:57:06.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:06.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:06.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230019+0000", 2024-09-18T04:57:06.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4210000000000002e-06 2024-09-18T04:57:06.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:06.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230027+0000", 2024-09-18T04:57:06.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000002e-06 2024-09-18T04:57:06.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:06.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230032+0000", 2024-09-18T04:57:06.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0089999999999998e-06 2024-09-18T04:57:06.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:06.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283800+0000", 2024-09-18T04:57:06.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053768418999999998 2024-09-18T04:57:06.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:06.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283849+0000", 2024-09-18T04:57:06.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9339000000000003e-05 2024-09-18T04:57:06.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:06.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283900+0000", 2024-09-18T04:57:06.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0421999999999999e-05 2024-09-18T04:57:06.429 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.429 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:06.429 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.429 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15923 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2924544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:06.429 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:06.429 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.127457019000001, 2024-09-18T04:57:06.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054073016000000002, 2024-09-18T04:57:06.429 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:06.430 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:06.430 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:06.430 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:06.430 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:06.430 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15923 2024-09-18T04:57:06.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.430 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:06.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:06.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:06.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:06.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:06.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:06.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:06.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:06.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:06.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230038+0000", 2024-09-18T04:57:06.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.463e-06 2024-09-18T04:57:06.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:06.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230045+0000", 2024-09-18T04:57:06.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3570000000000003e-06 2024-09-18T04:57:06.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:06.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230050+0000", 2024-09-18T04:57:06.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7299999999999996e-06 2024-09-18T04:57:06.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:06.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283928+0000", 2024-09-18T04:57:06.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053878218999999998 2024-09-18T04:57:06.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:06.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284010+0000", 2024-09-18T04:57:06.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2577000000000006e-05 2024-09-18T04:57:06.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:06.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284109+0000", 2024-09-18T04:57:06.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9102000000000001e-05 2024-09-18T04:57:06.434 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.434 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:06.434 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.434 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15924 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:06.435 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:06.435 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.127438549000001, 2024-09-18T04:57:06.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054108470999999998, 2024-09-18T04:57:06.435 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:06.435 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:06.435 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:06.435 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:06.435 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:06.435 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15924 2024-09-18T04:57:06.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.435 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:06.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:06.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:06.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:06.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:06.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:06.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:06.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:06.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:06.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230061+0000", 2024-09-18T04:57:06.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8240000000000002e-06 2024-09-18T04:57:06.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:06.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230068+0000", 2024-09-18T04:57:06.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4320000000000004e-06 2024-09-18T04:57:06.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:06.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230073+0000", 2024-09-18T04:57:06.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.921e-06 2024-09-18T04:57:06.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:06.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284153+0000", 2024-09-18T04:57:06.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054080793000000002 2024-09-18T04:57:06.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:06.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284163+0000", 2024-09-18T04:57:06.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.859e-06 2024-09-18T04:57:06.439 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.439 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:06.439 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.439 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15925 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:06.440 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:06.440 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.127414433999999, 2024-09-18T04:57:06.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054134632000000002, 2024-09-18T04:57:06.440 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:06.440 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:06.440 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:06.440 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:06.440 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:06.440 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15925 2024-09-18T04:57:06.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.441 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:06.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:06.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:06.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:06.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:06.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:06.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:06.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:06.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:06.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230081+0000", 2024-09-18T04:57:06.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4109999999999999e-06 2024-09-18T04:57:06.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:06.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230087+0000", 2024-09-18T04:57:06.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.161e-06 2024-09-18T04:57:06.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:06.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230092+0000", 2024-09-18T04:57:06.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9259999999999999e-06 2024-09-18T04:57:06.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:06.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284204+0000", 2024-09-18T04:57:06.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054111546000000003 2024-09-18T04:57:06.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:06.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284214+0000", 2024-09-18T04:57:06.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9639999999999998e-06 2024-09-18T04:57:06.445 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.445 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:06.445 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.445 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15926 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:06.445 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:06.445 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.12739711, 2024-09-18T04:57:06.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054143193999999999, 2024-09-18T04:57:06.445 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:06.445 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:06.445 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:06.446 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:06.446 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:06.446 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15926 2024-09-18T04:57:06.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.446 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:06.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:06.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:06.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:06.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:06.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:06.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:06.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:06.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:06.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230098+0000", 2024-09-18T04:57:06.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1399999999999998e-06 2024-09-18T04:57:06.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:06.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230104+0000", 2024-09-18T04:57:06.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-18T04:57:06.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:06.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230109+0000", 2024-09-18T04:57:06.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9250000000000003e-06 2024-09-18T04:57:06.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:06.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284231+0000", 2024-09-18T04:57:06.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054122297999999999 2024-09-18T04:57:06.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:06.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284239+0000", 2024-09-18T04:57:06.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1689999999999992e-06 2024-09-18T04:57:06.450 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.451 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:06.451 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.451 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15927 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:06.451 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:06.451 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.127380063, 2024-09-18T04:57:06.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054168435000000001, 2024-09-18T04:57:06.452 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:06.452 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:06.452 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:06.452 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:06.452 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:06.452 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15927 2024-09-18T04:57:06.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.452 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:06.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:06.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:06.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:06.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:06.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:06.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:06.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:06.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:06.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230116+0000", 2024-09-18T04:57:06.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5150000000000001e-06 2024-09-18T04:57:06.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:06.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230122+0000", 2024-09-18T04:57:06.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1190000000000002e-06 2024-09-18T04:57:06.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:06.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230126+0000", 2024-09-18T04:57:06.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.792e-06 2024-09-18T04:57:06.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:06.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284273+0000", 2024-09-18T04:57:06.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054146086000000003 2024-09-18T04:57:06.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:06.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284282+0000", 2024-09-18T04:57:06.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2590000000000006e-06 2024-09-18T04:57:06.457 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.457 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:06.457 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.458 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15928 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:06.458 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:57:06.458 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.127362680000001, 2024-09-18T04:57:06.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054189739000000001, 2024-09-18T04:57:06.458 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:06.458 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:06.458 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:06.458 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:06.458 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:06.459 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15928 2024-09-18T04:57:06.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.459 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:06.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:06.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:57:06.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:06.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:57:06.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:06.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230130+0000", 2024-09-18T04:57:06.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:06.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:06.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230133+0000", 2024-09-18T04:57:06.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.244e-06 2024-09-18T04:57:06.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:06.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230139+0000", 2024-09-18T04:57:06.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9830000000000001e-06 2024-09-18T04:57:06.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:06.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230143+0000", 2024-09-18T04:57:06.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7530000000000001e-06 2024-09-18T04:57:06.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:06.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284313+0000", 2024-09-18T04:57:06.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054169332000000001 2024-09-18T04:57:06.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:06.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284320+0000", 2024-09-18T04:57:06.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8699999999999992e-06 2024-09-18T04:57:06.462 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.463 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:06.463 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.463 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15929 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:06.463 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:06.463 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.127346240000001, 2024-09-18T04:57:06.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054218771999999998, 2024-09-18T04:57:06.463 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:06.463 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:06.463 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:06.464 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:06.464 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:06.464 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15929 2024-09-18T04:57:06.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.464 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:06.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:06.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:06.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:06.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:06.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:06.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:06.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:06.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:06.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230149+0000", 2024-09-18T04:57:06.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-18T04:57:06.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:06.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230155+0000", 2024-09-18T04:57:06.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2469999999999997e-06 2024-09-18T04:57:06.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:06.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230160+0000", 2024-09-18T04:57:06.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7129999999999996e-06 2024-09-18T04:57:06.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284355+0000", 2024-09-18T04:57:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054194900999999997 2024-09-18T04:57:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284366+0000", 2024-09-18T04:57:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0993e-05 2024-09-18T04:57:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:06.468 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.468 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15930 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:06.468 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:06.468 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.127329240000002, 2024-09-18T04:57:06.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054244713999999999, 2024-09-18T04:57:06.468 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:06.468 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:06.468 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:06.468 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:06.468 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:06.469 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15930 2024-09-18T04:57:06.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.469 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:06.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:06.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:06.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:06.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:06.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:06.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:06.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:06.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:06.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230166+0000", 2024-09-18T04:57:06.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1009999999999999e-06 2024-09-18T04:57:06.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:06.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230172+0000", 2024-09-18T04:57:06.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.981e-06 2024-09-18T04:57:06.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:06.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230184+0000", 2024-09-18T04:57:06.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2449000000000001e-05 2024-09-18T04:57:06.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:06.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284400+0000", 2024-09-18T04:57:06.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054215452999999997 2024-09-18T04:57:06.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:06.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284409+0000", 2024-09-18T04:57:06.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0839999999999998e-06 2024-09-18T04:57:06.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.472 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:06.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.473 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15931 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:06.473 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:57:06.473 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 30.127303451, 2024-09-18T04:57:06.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054261928000000001, 2024-09-18T04:57:06.473 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:06.473 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:06.473 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:06.473 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:06.473 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:06.474 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15931 2024-09-18T04:57:06.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.474 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:06.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:06.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:57:06.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:06.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:57:06.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:06.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:06.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230189+0000", 2024-09-18T04:57:06.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:06.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:06.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230192+0000", 2024-09-18T04:57:06.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6599999999999999e-06 2024-09-18T04:57:06.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:06.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230199+0000", 2024-09-18T04:57:06.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5570000000000001e-06 2024-09-18T04:57:06.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:06.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230204+0000", 2024-09-18T04:57:06.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2270000000000001e-06 2024-09-18T04:57:06.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:06.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284442+0000", 2024-09-18T04:57:06.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054238383000000001 2024-09-18T04:57:06.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:06.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:06.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:06.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284452+0000", 2024-09-18T04:57:06.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6069999999999997e-06 2024-09-18T04:57:06.477 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.477 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:06.477 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.477 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:06.477 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:06.478 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:06.478 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:57:06.479 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:06.476+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:06.480 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:57:06.481 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone25 from 0xbb06200 (overlap 0x70f931) to 0x0 2024-09-18T04:57:06.491 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-09-18T04:57:06.500 INFO:teuthology.orchestra.run.smithi002.stdout:3022 trunc from 0xbb06200 to 0xb90195a 2024-09-18T04:57:06.536 INFO:teuthology.orchestra.run.smithi002.stdout:3023 write 0xcd4a8b1 thru 0xcd52e59 (0x85a9 bytes) 2024-09-18T04:57:06.540 INFO:teuthology.orchestra.run.smithi002.stdout:3025 write 0xb267525 thru 0xb2698aa (0x2386 bytes) 2024-09-18T04:57:06.544 INFO:teuthology.orchestra.run.smithi002.stdout:3026 read 0x6616c26 thru 0x661cb9c (0x5f77 bytes) 2024-09-18T04:57:06.550 INFO:teuthology.orchestra.run.smithi002.stdout:3027 write 0xda8b04e thru 0xda8ea54 (0x3a07 bytes) 2024-09-18T04:57:06.553 INFO:teuthology.orchestra.run.smithi002.stdout:3028 write 0x17bfe0c thru 0x17ca8c8 (0xaabd bytes) 2024-09-18T04:57:06.559 INFO:teuthology.orchestra.run.smithi002.stdout:3031 read 0xb7e7efa thru 0xb7f2762 (0xa869 bytes) 2024-09-18T04:57:06.559 INFO:teuthology.orchestra.run.smithi002.stdout:3032 punch from 0x890c7e6 to 0x8916e95, (0xa6af bytes) 2024-09-18T04:57:06.559 INFO:teuthology.orchestra.run.smithi002.stdout:3034 write 0x785b1e3 thru 0x7861739 (0x6557 bytes) 2024-09-18T04:57:06.563 INFO:teuthology.orchestra.run.smithi002.stdout:3037 write 0x270ebd5 thru 0x271997b (0xada7 bytes) 2024-09-18T04:57:06.567 INFO:teuthology.orchestra.run.smithi002.stdout:truncating to largest ever: 0xee25ec8 2024-09-18T04:57:06.569 INFO:teuthology.orchestra.run.smithi002.stdout:3038 trunc from 0xda8ea55 to 0xee25ec8 2024-09-18T04:57:06.575 INFO:teuthology.orchestra.run.smithi002.stdout:3039 write 0x49f9a8e thru 0x49fdd8f (0x4302 bytes) 2024-09-18T04:57:06.578 INFO:teuthology.orchestra.run.smithi002.stdout:3040 read 0x20dbb4f thru 0x20dd5e1 (0x1a93 bytes) 2024-09-18T04:57:06.578 INFO:teuthology.orchestra.run.smithi002.stdout:3041 read 0x31439ad thru 0x31462fa (0x294e bytes) 2024-09-18T04:57:06.579 INFO:teuthology.orchestra.run.smithi002.stdout:3042 write 0xab648b1 thru 0xab6c2c1 (0x7a11 bytes) 2024-09-18T04:57:06.579 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:06.577+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:06.584 INFO:teuthology.orchestra.run.smithi002.stdout:3043 write 0x43f7f00 thru 0x43fa5c8 (0x26c9 bytes) 2024-09-18T04:57:06.587 INFO:teuthology.orchestra.run.smithi002.stdout:3050 write 0x9e129f1 thru 0x9e16066 (0x3676 bytes) 2024-09-18T04:57:06.592 INFO:teuthology.orchestra.run.smithi002.stdout:3051 writesame 0x6f2275f thru 0x6f233ec data_size 0x1(0xc8e bytes) 2024-09-18T04:57:06.597 INFO:teuthology.orchestra.run.smithi002.stdout:3052 punch from 0xe2c583e to 0xe2c6c15, (0x13d7 bytes) 2024-09-18T04:57:06.597 INFO:teuthology.orchestra.run.smithi002.stdout:3055 writesame 0xce4e4f4 thru 0xce52788 data_size 0x7(0x4295 bytes) 2024-09-18T04:57:06.606 INFO:teuthology.orchestra.run.smithi002.stdout:3056 trunc from 0xee25ec8 to 0x908edf5 2024-09-18T04:57:06.629 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:06.629 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:06.630 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:06.630 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:06.635 INFO:teuthology.orchestra.run.smithi002.stdout:3057 trunc from 0x908edf5 to 0x828e63e 2024-09-18T04:57:06.641 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:57:06.647 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:57:06.647 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:57:06.655 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:06.655 INFO:teuthology.orchestra.run.smithi002.stdout:3059 flatten 2024-09-18T04:57:06.673 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:57:06.673 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:57:06.679 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:06.677+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:06.695 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:57:06.695 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:57:06.713 INFO:teuthology.orchestra.run.smithi002.stdout:3060 punch from 0x8ee287 to 0x8f504a, (0x6dc3 bytes) 2024-09-18T04:57:06.713 INFO:teuthology.orchestra.run.smithi002.stdout:3061 clone 27 order 19 su 262144 sc 11 2024-09-18T04:57:06.714 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:57:06.714 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:57:06.730 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:57:06.731 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:57:06.746 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:57:06.747 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:57:06.762 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:57:06.763 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:57:06.767 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:06.767 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:06.776 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:57:06.779 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:57:06.780 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:57:06.780 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:06.778+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:06.881 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:06.879+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:06.885 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:06.885 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:06.894 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:57:06.982 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:06.980+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:07.041 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:07.042 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:07.050 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:57:07.084 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:07.081+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:07.156 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:07.156 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:07.164 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:57:07.184 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:07.182+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:07.271 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:07.272 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:07.280 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:57:07.328 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:07.325+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:07.386 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:07.384+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:07.392 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:07.392 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:07.399 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:57:07.405 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T04:57:07.487 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:07.484+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:07.552 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:07.552 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:07.559 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:57:07.607 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:07.604+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:07.669 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:07.670 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:07.672 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:57:07.716 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:07.714+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:07.790 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:07.787+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:07.818 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:07.818 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:07.827 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:57:07.891 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:07.889+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:07.929 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:07.929 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:07.938 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:57:08.040 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:08.038+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:08.090 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:08.090 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:08.098 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:08.095+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:08.099 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:57:08.199 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:08.197+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:08.254 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:08.254 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:08.263 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:57:08.296 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:08.293+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:08.322 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-18T04:57:08.396 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:08.394+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:08.411 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:08.412 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:08.420 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:57:08.471 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-09-18T04:57:08.472 INFO:teuthology.orchestra.run.smithi002.stdout:3063 write 0x41840f8 thru 0x4185766 (0x166f bytes) 2024-09-18T04:57:08.479 INFO:teuthology.orchestra.run.smithi002.stdout:3065 read 0x583f512 thru 0x584e59e (0xf08d bytes) 2024-09-18T04:57:08.479 INFO:teuthology.orchestra.run.smithi002.stdout:3067 read 0x1aaf7ce thru 0x1ab5b18 (0x634b bytes) 2024-09-18T04:57:08.487 INFO:teuthology.orchestra.run.smithi002.stdout:3068 write 0xa6e14e3 thru 0xa6eff8a (0xeaa8 bytes) 2024-09-18T04:57:08.490 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:08.490 INFO:teuthology.orchestra.run.smithi002.stdout:3071 read 0x2d05966 thru 0x2d0a4c3 (0x4b5e bytes) 2024-09-18T04:57:08.490 INFO:teuthology.orchestra.run.smithi002.stdout:3073 punch from 0x6f6b294 to 0x6f78b5b, (0xd8c7 bytes) 2024-09-18T04:57:08.491 INFO:teuthology.orchestra.run.smithi002.stdout:3074 read 0x584cb2d thru 0x5852226 (0x56fa bytes) 2024-09-18T04:57:08.491 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:08.502 INFO:teuthology.orchestra.run.smithi002.stdout:3076 trunc from 0xa6eff8b to 0x5403af 2024-09-18T04:57:08.504 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:08.502+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:08.522 INFO:teuthology.orchestra.run.smithi002.stdout:3077 write 0x2beae3b thru 0x2bf5c87 (0xae4d bytes) 2024-09-18T04:57:08.525 INFO:teuthology.orchestra.run.smithi002.stdout:3078 writesame 0xa7c350 thru 0xa7f090 data_size 0x7(0x2d41 bytes) 2024-09-18T04:57:08.528 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:08.528 INFO:teuthology.orchestra.run.smithi002.stdout:3080 punch from 0x441d8f to 0x442461, (0x6d2 bytes) 2024-09-18T04:57:08.532 INFO:teuthology.orchestra.run.smithi002.stdout:3082 trunc from 0x2bf5c88 to 0x6ada876 2024-09-18T04:57:08.538 INFO:teuthology.orchestra.run.smithi002.stdout:3084 punch from 0x10c5e48 to 0x10d47e6, (0xe99e bytes) 2024-09-18T04:57:08.538 INFO:teuthology.orchestra.run.smithi002.stdout:3086 punch from 0x8ffbe5 to 0x90e446, (0xe861 bytes) 2024-09-18T04:57:08.539 INFO:teuthology.orchestra.run.smithi002.stdout:3089 read 0x3498fb9 thru 0x349e75b (0x57a3 bytes) 2024-09-18T04:57:08.539 INFO:teuthology.orchestra.run.smithi002.stdout:3090 writesame 0x10b5983 thru 0x10bf1d1 data_size 0x3(0x984f bytes) 2024-09-18T04:57:08.546 INFO:teuthology.orchestra.run.smithi002.stdout:3091 write 0xa4b771 thru 0xa4caf2 (0x1382 bytes) 2024-09-18T04:57:08.548 INFO:teuthology.orchestra.run.smithi002.stdout:3092 flatten 2024-09-18T04:57:08.569 INFO:teuthology.orchestra.run.smithi002.stdout:3093 write 0x4a3c46c thru 0x4a45d29 (0x98be bytes) 2024-09-18T04:57:08.573 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:08.573 INFO:teuthology.orchestra.run.smithi002.stdout:3096 punch from 0x3b7e15d to 0x3b88aa8, (0xa94b bytes) 2024-09-18T04:57:08.573 INFO:teuthology.orchestra.run.smithi002.stdout:3097 read 0x3639502 thru 0x363a6e4 (0x11e3 bytes) 2024-09-18T04:57:08.575 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:08.575 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:08.583 INFO:teuthology.orchestra.run.smithi002.stdout:3099 writesame 0xa5ed7a3 thru 0xa5f9cb2 data_size 0x18(0xc510 bytes) 2024-09-18T04:57:08.583 INFO:teuthology.orchestra.run.smithi002.stdout:3100 write 0x80c20a9 thru 0x80c7567 (0x54bf bytes) 2024-09-18T04:57:08.584 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:57:08.587 INFO:teuthology.orchestra.run.smithi002.stdout:3101 write 0x96ca177 thru 0x96d9b5b (0xf9e5 bytes) 2024-09-18T04:57:08.590 INFO:teuthology.orchestra.run.smithi002.stdout:3102 punch from 0x71747b5 to 0x7175f2d, (0x1778 bytes) 2024-09-18T04:57:08.590 INFO:teuthology.orchestra.run.smithi002.stdout:3104 read 0x2cdddd9 thru 0x2ce28f1 (0x4b19 bytes) 2024-09-18T04:57:08.591 INFO:teuthology.orchestra.run.smithi002.stdout:3105 read 0x70e6482 thru 0x70e99d7 (0x3556 bytes) 2024-09-18T04:57:08.591 INFO:teuthology.orchestra.run.smithi002.stdout:3106 read 0x3e4be0f thru 0x3e50a72 (0x4c64 bytes) 2024-09-18T04:57:08.591 INFO:teuthology.orchestra.run.smithi002.stdout:3108 read 0x2b69f9b thru 0x2b709ac (0x6a12 bytes) 2024-09-18T04:57:08.591 INFO:teuthology.orchestra.run.smithi002.stdout:3109 read 0x95fd658 thru 0x960cd85 (0xf72e bytes) 2024-09-18T04:57:08.598 INFO:teuthology.orchestra.run.smithi002.stdout:3112 writesame 0xbce1da9 thru 0xbceccc9 data_size 0x1(0xaf21 bytes) 2024-09-18T04:57:08.598 INFO:teuthology.orchestra.run.smithi002.stdout:3113 punch from 0x3ab7668 to 0x3ac40f6, (0xca8e bytes) 2024-09-18T04:57:08.598 INFO:teuthology.orchestra.run.smithi002.stdout:3115 read 0x447c03 thru 0x452908 (0xad06 bytes) 2024-09-18T04:57:08.599 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:08.595+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:08.599 INFO:teuthology.orchestra.run.smithi002.stdout:3116 write 0xf2f6dc thru 0xf3c6d4 (0xcff9 bytes) 2024-09-18T04:57:08.601 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:08.601 INFO:teuthology.orchestra.run.smithi002.stdout:3118 read 0x6292acb thru 0x6294477 (0x19ad bytes) 2024-09-18T04:57:08.604 INFO:teuthology.orchestra.run.smithi002.stdout:3119 trunc from 0xbceccca to 0xdd476c3 2024-09-18T04:57:08.609 INFO:teuthology.orchestra.run.smithi002.stdout:3120 write 0x5cb2623 thru 0x5cc14da (0xeeb8 bytes) 2024-09-18T04:57:08.614 INFO:teuthology.orchestra.run.smithi002.stdout:3121 read 0x25dcc3e thru 0x25ea37e (0xd741 bytes) 2024-09-18T04:57:08.614 INFO:teuthology.orchestra.run.smithi002.stdout:3122 write 0x3e07c6b thru 0x3e0cc0b (0x4fa1 bytes) 2024-09-18T04:57:08.618 INFO:teuthology.orchestra.run.smithi002.stdout:3123 write 0x8c1582d thru 0x8c202da (0xaaae bytes) 2024-09-18T04:57:08.621 INFO:teuthology.orchestra.run.smithi002.stdout:3124 read 0x3297606 thru 0x32a190e (0xa309 bytes) 2024-09-18T04:57:08.621 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:08.621 INFO:teuthology.orchestra.run.smithi002.stdout:3126 write 0x4eca513 thru 0x4ecabbb (0x6a9 bytes) 2024-09-18T04:57:08.629 INFO:teuthology.orchestra.run.smithi002.stdout:3127 write 0xde3088d thru 0xde405e0 (0xfd54 bytes) 2024-09-18T04:57:08.632 INFO:teuthology.orchestra.run.smithi002.stdout:3129 read 0xd583718 thru 0xd58de25 (0xa70e bytes) 2024-09-18T04:57:08.632 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:08.632 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:08.632 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:08.632 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:08.632 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:08.632 INFO:teuthology.orchestra.run.smithi002.stdout:3135 punch from 0x27c18fb to 0x27ce849, (0xcf4e bytes) 2024-09-18T04:57:08.638 INFO:teuthology.orchestra.run.smithi002.stdout:3136 writesame 0xe41f073 thru 0xe423876 data_size 0x16(0x4804 bytes) 2024-09-18T04:57:08.638 INFO:teuthology.orchestra.run.smithi002.stdout:3138 write 0x7e43eac thru 0x7e4ab10 (0x6c65 bytes) 2024-09-18T04:57:08.641 INFO:teuthology.orchestra.run.smithi002.stdout:3139 write 0x603aed8 thru 0x603ff0f (0x5038 bytes) 2024-09-18T04:57:08.643 INFO:teuthology.orchestra.run.smithi002.stdout:3140 writesame 0x3e53c9c thru 0x3e5f567 data_size 0x2(0xb8cc bytes) 2024-09-18T04:57:08.655 INFO:teuthology.orchestra.run.smithi002.stdout:3142 writesame 0x714e438 thru 0x7154d08 data_size 0x1(0x68d1 bytes) 2024-09-18T04:57:08.677 INFO:teuthology.orchestra.run.smithi002.stdout:3143 trunc from 0xe423877 to 0x8600807 2024-09-18T04:57:08.686 INFO:teuthology.orchestra.run.smithi002.stdout:3144 write 0x1cf34c7 thru 0x1cf4dad (0x18e7 bytes) 2024-09-18T04:57:08.695 INFO:teuthology.orchestra.run.smithi002.stdout:3145 write 0x8ed45d0 thru 0x8edc3e0 (0x7e11 bytes) 2024-09-18T04:57:08.697 INFO:teuthology.orchestra.run.smithi002.stdout:3146 write 0x8772cb0 thru 0x877664d (0x399e bytes) 2024-09-18T04:57:08.699 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:08.696+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:08.700 INFO:teuthology.orchestra.run.smithi002.stdout:3148 clone 28 order 24 su 1048576 sc 9 2024-09-18T04:57:08.732 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:08.733 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:08.741 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:57:08.791 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:57:08.835 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:08.832+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:08.883 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:08.884 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:08.893 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:57:08.916 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:57:08.917 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:57:08.933 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:57:08.933 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:57:08.937 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:08.934+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:08.948 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:57:08.949 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:57:08.964 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:57:08.965 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:57:08.981 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:57:08.981 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:57:08.995 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:08.995 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:08.997 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:57:08.998 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:57:09.002 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:09.000+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:09.004 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:57:09.015 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:57:09.015 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:57:09.031 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:57:09.032 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:57:09.103 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:09.101+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:09.113 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:09.113 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:09.113 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:09.113 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:09.121 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:57:09.204 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:09.201+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:09.229 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:09.229 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:09.229 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:09.229 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:09.229 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:09.238 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:57:09.306 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:09.304+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:09.315 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T04:57:09.315 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T04:57:09.315 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-09-18T04:57:09.315 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T04:57:09.315 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T04:57:09.316 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-09-18T04:57:09.394 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:09.395 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:09.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:09.395 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:09.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.395 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9316 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.395 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:09.395 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.624918149999999, 2024-09-18T04:57:09.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044027208999999998, 2024-09-18T04:57:09.395 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.395 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.395 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.395 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.396 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.396 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9316 2024-09-18T04:57:09.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.396 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:09.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:09.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:09.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767144+0000", 2024-09-18T04:57:09.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3070000000000001e-06 2024-09-18T04:57:09.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767150+0000", 2024-09-18T04:57:09.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1700000000000002e-06 2024-09-18T04:57:09.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767163+0000", 2024-09-18T04:57:09.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2670999999999999e-05 2024-09-18T04:57:09.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810995+0000", 2024-09-18T04:57:09.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043832310999999999 2024-09-18T04:57:09.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811087+0000", 2024-09-18T04:57:09.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1345e-05 2024-09-18T04:57:09.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811169+0000", 2024-09-18T04:57:09.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2848000000000004e-05 2024-09-18T04:57:09.399 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.399 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.399 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.400 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9317 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1728512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.400 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:57:09.400 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.624892514999999, 2024-09-18T04:57:09.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044190170000000001, 2024-09-18T04:57:09.400 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.400 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.400 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.400 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.400 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.400 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9317 2024-09-18T04:57:09.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.400 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:57:09.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:57:09.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767167+0000", 2024-09-18T04:57:09.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767170+0000", 2024-09-18T04:57:09.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7259999999999998e-06 2024-09-18T04:57:09.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767176+0000", 2024-09-18T04:57:09.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1460000000000001e-06 2024-09-18T04:57:09.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767182+0000", 2024-09-18T04:57:09.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1020000000000002e-06 2024-09-18T04:57:09.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811204+0000", 2024-09-18T04:57:09.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044022039999999998 2024-09-18T04:57:09.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811286+0000", 2024-09-18T04:57:09.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1623000000000001e-05 2024-09-18T04:57:09.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811358+0000", 2024-09-18T04:57:09.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2125000000000005e-05 2024-09-18T04:57:09.404 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.404 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.404 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.405 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9318 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1777664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.405 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:09.405 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.624873970000003, 2024-09-18T04:57:09.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044380181999999997, 2024-09-18T04:57:09.405 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.405 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.405 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.405 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.405 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.405 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9318 2024-09-18T04:57:09.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.405 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:09.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:09.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:09.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767189+0000", 2024-09-18T04:57:09.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5909999999999998e-06 2024-09-18T04:57:09.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767194+0000", 2024-09-18T04:57:09.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7690000000000002e-06 2024-09-18T04:57:09.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767199+0000", 2024-09-18T04:57:09.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189999999999999e-06 2024-09-18T04:57:09.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811394+0000", 2024-09-18T04:57:09.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044195350000000001 2024-09-18T04:57:09.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811483+0000", 2024-09-18T04:57:09.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8282999999999995e-05 2024-09-18T04:57:09.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811567+0000", 2024-09-18T04:57:09.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3855000000000003e-05 2024-09-18T04:57:09.409 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.409 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.409 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.409 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9325 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.409 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:09.410 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.623143431999999, 2024-09-18T04:57:09.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044063989999999997, 2024-09-18T04:57:09.410 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.410 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.410 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.410 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.410 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.410 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9325 2024-09-18T04:57:09.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.410 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:09.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:09.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:09.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768919+0000", 2024-09-18T04:57:09.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:57:09.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768925+0000", 2024-09-18T04:57:09.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2310000000000001e-06 2024-09-18T04:57:09.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768929+0000", 2024-09-18T04:57:09.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4469999999999999e-06 2024-09-18T04:57:09.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812823+0000", 2024-09-18T04:57:09.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043893975000000002 2024-09-18T04:57:09.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812908+0000", 2024-09-18T04:57:09.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4862000000000002e-05 2024-09-18T04:57:09.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812981+0000", 2024-09-18T04:57:09.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2892000000000006e-05 2024-09-18T04:57:09.414 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.414 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.414 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.414 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9334 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.415 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:57:09.415 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.621124645999998, 2024-09-18T04:57:09.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044042853999999999, 2024-09-18T04:57:09.415 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.415 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.415 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.415 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.415 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.415 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9334 2024-09-18T04:57:09.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.415 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:57:09.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:57:09.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770935+0000", 2024-09-18T04:57:09.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770937+0000", 2024-09-18T04:57:09.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1489999999999999e-06 2024-09-18T04:57:09.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770940+0000", 2024-09-18T04:57:09.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9569999999999998e-06 2024-09-18T04:57:09.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770945+0000", 2024-09-18T04:57:09.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:57:09.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814838+0000", 2024-09-18T04:57:09.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043892635999999999 2024-09-18T04:57:09.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814920+0000", 2024-09-18T04:57:09.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2175000000000005e-05 2024-09-18T04:57:09.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814979+0000", 2024-09-18T04:57:09.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8372000000000003e-05 2024-09-18T04:57:09.419 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.419 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.419 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.420 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9352 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1159168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.420 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:57:09.420 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.618903469000003, 2024-09-18T04:57:09.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044011269999999998, 2024-09-18T04:57:09.420 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.420 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.420 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.420 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.420 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.420 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9352 2024-09-18T04:57:09.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.421 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:57:09.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:57:09.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773156+0000", 2024-09-18T04:57:09.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773159+0000", 2024-09-18T04:57:09.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2919999999999998e-06 2024-09-18T04:57:09.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.422 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773165+0000", 2024-09-18T04:57:09.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0580000000000003e-06 2024-09-18T04:57:09.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773177+0000", 2024-09-18T04:57:09.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.204e-05 2024-09-18T04:57:09.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817082+0000", 2024-09-18T04:57:09.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043905528999999999 2024-09-18T04:57:09.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817122+0000", 2024-09-18T04:57:09.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9258999999999998e-05 2024-09-18T04:57:09.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817168+0000", 2024-09-18T04:57:09.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6545000000000002e-05 2024-09-18T04:57:09.425 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.425 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.425 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.425 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9353 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1208320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.425 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:09.425 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.618879171000003, 2024-09-18T04:57:09.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044105537, 2024-09-18T04:57:09.425 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.425 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.425 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.425 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.426 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.426 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9353 2024-09-18T04:57:09.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.426 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:09.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.427 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:09.428 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-18T04:57:09.428 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:09.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:09.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:09.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773183+0000", 2024-09-18T04:57:09.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2929999999999999e-06 2024-09-18T04:57:09.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773190+0000", 2024-09-18T04:57:09.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4330000000000001e-06 2024-09-18T04:57:09.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773194+0000", 2024-09-18T04:57:09.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.865e-06 2024-09-18T04:57:09.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817196+0000", 2024-09-18T04:57:09.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044001708 2024-09-18T04:57:09.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817236+0000", 2024-09-18T04:57:09.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0043999999999998e-05 2024-09-18T04:57:09.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817287+0000", 2024-09-18T04:57:09.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0605000000000001e-05 2024-09-18T04:57:09.431 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.432 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.432 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.432 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9354 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.432 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:57:09.432 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.61886166, 2024-09-18T04:57:09.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044199796, 2024-09-18T04:57:09.432 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.432 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.432 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.432 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.432 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.432 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9354 2024-09-18T04:57:09.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.433 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:57:09.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:57:09.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773198+0000", 2024-09-18T04:57:09.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773200+0000", 2024-09-18T04:57:09.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4159999999999998e-06 2024-09-18T04:57:09.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773206+0000", 2024-09-18T04:57:09.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.789e-06 2024-09-18T04:57:09.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773210+0000", 2024-09-18T04:57:09.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2679999999999996e-06 2024-09-18T04:57:09.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817314+0000", 2024-09-18T04:57:09.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044103187000000002 2024-09-18T04:57:09.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817353+0000", 2024-09-18T04:57:09.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8973000000000001e-05 2024-09-18T04:57:09.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817399+0000", 2024-09-18T04:57:09.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5877999999999997e-05 2024-09-18T04:57:09.436 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.436 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.436 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.436 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46591 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2097152~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:09.437 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:09.437 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.694005242999999, 2024-09-18T04:57:09.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043987310000000002, 2024-09-18T04:57:09.437 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.437 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.437 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.437 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.437 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.437 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46591 2024-09-18T04:57:09.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.437 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:09.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:09.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:09.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698057+0000", 2024-09-18T04:57:09.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.531e-06 2024-09-18T04:57:09.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698064+0000", 2024-09-18T04:57:09.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-18T04:57:09.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698069+0000", 2024-09-18T04:57:09.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.091e-06 2024-09-18T04:57:09.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741843+0000", 2024-09-18T04:57:09.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043773470000000002 2024-09-18T04:57:09.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741928+0000", 2024-09-18T04:57:09.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5113999999999999e-05 2024-09-18T04:57:09.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742042+0000", 2024-09-18T04:57:09.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114527 2024-09-18T04:57:09.441 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.441 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.441 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.441 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46592 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2121728~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:09.441 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:57:09.441 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.693986815999999, 2024-09-18T04:57:09.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044077948999999998, 2024-09-18T04:57:09.441 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.442 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.442 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.442 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.442 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.442 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46592 2024-09-18T04:57:09.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.442 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:57:09.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:57:09.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698073+0000", 2024-09-18T04:57:09.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698076+0000", 2024-09-18T04:57:09.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.481e-06 2024-09-18T04:57:09.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698082+0000", 2024-09-18T04:57:09.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8730000000000004e-06 2024-09-18T04:57:09.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698086+0000", 2024-09-18T04:57:09.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:57:09.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742055+0000", 2024-09-18T04:57:09.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043969170000000002 2024-09-18T04:57:09.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742093+0000", 2024-09-18T04:57:09.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7707000000000003e-05 2024-09-18T04:57:09.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742152+0000", 2024-09-18T04:57:09.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8323999999999999e-05 2024-09-18T04:57:09.446 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.446 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.446 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.447 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46593 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2211840~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:09.447 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:09.447 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.693969913, 2024-09-18T04:57:09.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044259917000000003, 2024-09-18T04:57:09.447 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.447 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.447 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.447 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.447 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.447 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46593 2024-09-18T04:57:09.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.447 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:09.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:09.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:09.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698092+0000", 2024-09-18T04:57:09.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2249999999999999e-06 2024-09-18T04:57:09.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698098+0000", 2024-09-18T04:57:09.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:57:09.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698103+0000", 2024-09-18T04:57:09.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:57:09.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742182+0000", 2024-09-18T04:57:09.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044079508000000003 2024-09-18T04:57:09.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742257+0000", 2024-09-18T04:57:09.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4678000000000006e-05 2024-09-18T04:57:09.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742350+0000", 2024-09-18T04:57:09.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3531999999999998e-05 2024-09-18T04:57:09.451 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.451 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.451 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.452 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46594 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2236416~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:09.452 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:09.452 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.693954216000002, 2024-09-18T04:57:09.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044378984000000003, 2024-09-18T04:57:09.452 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.452 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.452 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.452 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.452 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.452 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46594 2024-09-18T04:57:09.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.452 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:09.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:09.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:09.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698108+0000", 2024-09-18T04:57:09.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0880000000000002e-06 2024-09-18T04:57:09.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698114+0000", 2024-09-18T04:57:09.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6269999999999997e-06 2024-09-18T04:57:09.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698118+0000", 2024-09-18T04:57:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4050000000000002e-06 2024-09-18T04:57:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742365+0000", 2024-09-18T04:57:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044246799000000003 2024-09-18T04:57:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742407+0000", 2024-09-18T04:57:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2008000000000002e-05 2024-09-18T04:57:09.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742485+0000", 2024-09-18T04:57:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8405999999999994e-05 2024-09-18T04:57:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46595 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.693938143, 2024-09-18T04:57:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.04440463, 2024-09-18T04:57:09.456 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.457 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:09.457 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.457 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.457 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.457 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46595 2024-09-18T04:57:09.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.457 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:09.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698124+0000", 2024-09-18T04:57:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1840000000000002e-06 2024-09-18T04:57:09.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698130+0000", 2024-09-18T04:57:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.947e-06 2024-09-18T04:57:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698136+0000", 2024-09-18T04:57:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6520000000000003e-06 2024-09-18T04:57:09.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742521+0000", 2024-09-18T04:57:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044385446000000002 2024-09-18T04:57:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742527+0000", 2024-09-18T04:57:09.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:57:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46596 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.693919899000001, 2024-09-18T04:57:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044418066999999999, 2024-09-18T04:57:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.461 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46596 2024-09-18T04:57:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698146+0000", 2024-09-18T04:57:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2779999999999999e-06 2024-09-18T04:57:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698153+0000", 2024-09-18T04:57:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3849999999999999e-06 2024-09-18T04:57:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698164+0000", 2024-09-18T04:57:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1375999999999999e-05 2024-09-18T04:57:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742554+0000", 2024-09-18T04:57:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044389482000000001 2024-09-18T04:57:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742559+0000", 2024-09-18T04:57:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9540000000000003e-06 2024-09-18T04:57:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.466 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46597 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:57:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.693891533999999, 2024-09-18T04:57:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044416513999999997, 2024-09-18T04:57:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46597 2024-09-18T04:57:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:57:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:57:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698168+0000", 2024-09-18T04:57:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698175+0000", 2024-09-18T04:57:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4520000000000002e-06 2024-09-18T04:57:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698181+0000", 2024-09-18T04:57:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4150000000000004e-06 2024-09-18T04:57:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698186+0000", 2024-09-18T04:57:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7489999999999998e-06 2024-09-18T04:57:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742580+0000", 2024-09-18T04:57:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044394325999999998 2024-09-18T04:57:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742585+0000", 2024-09-18T04:57:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9849999999999997e-06 2024-09-18T04:57:09.470 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46598 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:57:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.693784057999999, 2024-09-18T04:57:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044335316999999999, 2024-09-18T04:57:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.471 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46598 2024-09-18T04:57:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:57:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:57:09.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698190+0000", 2024-09-18T04:57:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999137 2024-09-18T04:57:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698280+0000", 2024-09-18T04:57:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0409999999999997e-05 2024-09-18T04:57:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698289+0000", 2024-09-18T04:57:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9490000000000002e-06 2024-09-18T04:57:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698296+0000", 2024-09-18T04:57:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2670000000000003e-06 2024-09-18T04:57:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742607+0000", 2024-09-18T04:57:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044311072999999999 2024-09-18T04:57:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742612+0000", 2024-09-18T04:57:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9400000000000001e-06 2024-09-18T04:57:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57986 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3086336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.706938335, 2024-09-18T04:57:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044047731999999999, 2024-09-18T04:57:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.475 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.476 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.476 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.476 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.476 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57986 2024-09-18T04:57:09.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.476 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:09.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:09.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:09.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685124+0000", 2024-09-18T04:57:09.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2859999999999998e-06 2024-09-18T04:57:09.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685130+0000", 2024-09-18T04:57:09.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5400000000000001e-06 2024-09-18T04:57:09.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685136+0000", 2024-09-18T04:57:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.152e-06 2024-09-18T04:57:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.728976+0000", 2024-09-18T04:57:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043840037999999998 2024-09-18T04:57:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729064+0000", 2024-09-18T04:57:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8709999999999996e-05 2024-09-18T04:57:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729170+0000", 2024-09-18T04:57:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000105386 2024-09-18T04:57:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57987 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3101696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:57:09.480 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.706889360000002, 2024-09-18T04:57:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044238123999999997, 2024-09-18T04:57:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57987 2024-09-18T04:57:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:57:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:57:09.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685170+0000", 2024-09-18T04:57:09.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:09.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685175+0000", 2024-09-18T04:57:09.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9069999999999998e-06 2024-09-18T04:57:09.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685187+0000", 2024-09-18T04:57:09.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.189e-05 2024-09-18T04:57:09.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685195+0000", 2024-09-18T04:57:09.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9459999999999998e-06 2024-09-18T04:57:09.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729212+0000", 2024-09-18T04:57:09.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044017239 2024-09-18T04:57:09.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729301+0000", 2024-09-18T04:57:09.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8887999999999993e-05 2024-09-18T04:57:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729409+0000", 2024-09-18T04:57:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108119 2024-09-18T04:57:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57988 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3117056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:57:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.706847240999998, 2024-09-18T04:57:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044429835000000001, 2024-09-18T04:57:09.486 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.487 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.487 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.487 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.487 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.487 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57988 2024-09-18T04:57:09.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.487 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:57:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:57:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685212+0000", 2024-09-18T04:57:09.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685216+0000", 2024-09-18T04:57:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7390000000000001e-06 2024-09-18T04:57:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685224+0000", 2024-09-18T04:57:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2260000000000005e-06 2024-09-18T04:57:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685230+0000", 2024-09-18T04:57:09.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0299999999999999e-06 2024-09-18T04:57:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729454+0000", 2024-09-18T04:57:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044223248999999999 2024-09-18T04:57:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729539+0000", 2024-09-18T04:57:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5029000000000002e-05 2024-09-18T04:57:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729643+0000", 2024-09-18T04:57:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104267 2024-09-18T04:57:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57989 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3132416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:57:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.706824488999999, 2024-09-18T04:57:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044642881000000002, 2024-09-18T04:57:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.491 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57989 2024-09-18T04:57:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:57:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:57:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685235+0000", 2024-09-18T04:57:09.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685238+0000", 2024-09-18T04:57:09.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.667e-06 2024-09-18T04:57:09.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685245+0000", 2024-09-18T04:57:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7909999999999999e-06 2024-09-18T04:57:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685250+0000", 2024-09-18T04:57:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3730000000000001e-06 2024-09-18T04:57:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729685+0000", 2024-09-18T04:57:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044434552000000002 2024-09-18T04:57:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729775+0000", 2024-09-18T04:57:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0395000000000004e-05 2024-09-18T04:57:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729879+0000", 2024-09-18T04:57:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010361099999999999 2024-09-18T04:57:09.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.496 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.496 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.496 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.496 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.496 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:09.496 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:57:09.497 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:09.424+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:09.497 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-09-18T04:57:09.542 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:09.539+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:09.612 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:09.613 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:09.613 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:09.613 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:09.622 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:57:09.630 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:09.627+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:09.631 INFO:teuthology.orchestra.run.smithi002.stdout:3149 write 0xbbb1e0a thru 0xbbc08b5 (0xeaac bytes) 2024-09-18T04:57:09.642 INFO:teuthology.orchestra.run.smithi002.stdout:3150 write 0xe70af7b thru 0xe71a3c8 (0xf44e bytes) 2024-09-18T04:57:09.645 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:09.657 INFO:teuthology.orchestra.run.smithi002.stdout:3153 trunc from 0xe71a3c9 to 0x3f5d5c8 2024-09-18T04:57:09.693 INFO:teuthology.orchestra.run.smithi002.stdout:3154 write 0x67ecf70 thru 0x67f8f7d (0xc00e bytes) 2024-09-18T04:57:09.694 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:09.694 INFO:teuthology.orchestra.run.smithi002.stdout:3156 read 0x5b72507 thru 0x5b732b3 (0xdad bytes) 2024-09-18T04:57:09.694 INFO:teuthology.orchestra.run.smithi002.stdout:3158 read 0x580f5dc thru 0x5810351 (0xd76 bytes) 2024-09-18T04:57:09.699 INFO:teuthology.orchestra.run.smithi002.stdout:3159 trunc from 0x67f8f7e to 0x2731319 2024-09-18T04:57:09.718 INFO:teuthology.orchestra.run.smithi002.stdout:3160 trunc from 0x2731319 to 0xce6cd19 2024-09-18T04:57:09.723 INFO:teuthology.orchestra.run.smithi002.stdout:3161 write 0x5ec628e thru 0x5ecbe6b (0x5bde bytes) 2024-09-18T04:57:09.725 INFO:teuthology.orchestra.run.smithi002.stdout:3162 writesame 0x47dcc90 thru 0x47e3946 data_size 0x1(0x6cb7 bytes) 2024-09-18T04:57:09.730 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:09.727+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:09.739 INFO:teuthology.orchestra.run.smithi002.stdout:3163 read 0x7204847 thru 0x720f643 (0xadfd bytes) 2024-09-18T04:57:09.739 INFO:teuthology.orchestra.run.smithi002.stdout:3165 punch from 0x88da909 to 0x88de6e4, (0x3ddb bytes) 2024-09-18T04:57:09.744 INFO:teuthology.orchestra.run.smithi002.stdout:3166 trunc from 0xce6cd19 to 0x8f797c9 2024-09-18T04:57:09.751 INFO:teuthology.orchestra.run.smithi002.stdout:3167 write 0x1d5de12 thru 0x1d65bba (0x7da9 bytes) 2024-09-18T04:57:09.752 INFO:teuthology.orchestra.run.smithi002.stdout:3168 write 0x39993e thru 0x3a15b2 (0x7c75 bytes) 2024-09-18T04:57:09.756 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:09.756 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:09.757 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:09.757 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:09.757 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:09.759 INFO:teuthology.orchestra.run.smithi002.stdout:3169 trunc from 0x8f797c9 to 0xd87ce76 2024-09-18T04:57:09.766 INFO:teuthology.orchestra.run.smithi002.stdout:3170 punch from 0x8f624f8 to 0x8f6ddc8, (0xb8d0 bytes) 2024-09-18T04:57:09.766 INFO:teuthology.orchestra.run.smithi002.stdout:3172 read 0x1abc746 thru 0x1ac46a2 (0x7f5d bytes) 2024-09-18T04:57:09.768 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:57:09.772 INFO:teuthology.orchestra.run.smithi002.stdout:3173 trunc from 0xd87ce76 to 0x7d8a224 2024-09-18T04:57:09.786 INFO:teuthology.orchestra.run.smithi002.stdout:3174 trunc from 0x7d8a224 to 0xd552ca6 2024-09-18T04:57:09.791 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:09.803 INFO:teuthology.orchestra.run.smithi002.stdout:3176 trunc from 0xd552ca6 to 0x2e34848 2024-09-18T04:57:09.811 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:09.824 INFO:teuthology.orchestra.run.smithi002.stdout:3179 write 0x9eb7d10 thru 0x9ec10b4 (0x93a5 bytes) 2024-09-18T04:57:09.826 INFO:teuthology.orchestra.run.smithi002.stdout:3180 read 0x780d735 thru 0x781ccf6 (0xf5c2 bytes) 2024-09-18T04:57:09.829 INFO:teuthology.orchestra.run.smithi002.stdout:3184 trunc from 0x9ec10b5 to 0x83025d5 2024-09-18T04:57:09.831 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:09.829+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:09.840 INFO:teuthology.orchestra.run.smithi002.stdout:3186 read 0xc95a05 thru 0xc97cec (0x22e8 bytes) 2024-09-18T04:57:09.840 INFO:teuthology.orchestra.run.smithi002.stdout:3187 read 0x5a8d65e thru 0x5a95143 (0x7ae6 bytes) 2024-09-18T04:57:09.840 INFO:teuthology.orchestra.run.smithi002.stdout:3188 writesame 0x78c022f thru 0x78cef97 data_size 0x9(0xed69 bytes) 2024-09-18T04:57:09.857 INFO:teuthology.orchestra.run.smithi002.stdout:3189 write 0xdf68f29 thru 0xdf6c131 (0x3209 bytes) 2024-09-18T04:57:09.871 INFO:teuthology.orchestra.run.smithi002.stdout:3190 trunc from 0xdf6c132 to 0x3c8e732 2024-09-18T04:57:09.882 INFO:teuthology.orchestra.run.smithi002.stdout:3192 trunc from 0x3c8e732 to 0x44fcc18 2024-09-18T04:57:09.887 INFO:teuthology.orchestra.run.smithi002.stdout:3193 read 0x14b078e thru 0x14bb3ac (0xac1f bytes) 2024-09-18T04:57:09.901 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:09.901 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:09.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:09.901 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:09.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.901 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14125 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.901 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:57:09.901 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.77057954, 2024-09-18T04:57:09.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052956470999999998, 2024-09-18T04:57:09.901 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.902 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.902 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.902 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.902 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.902 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14125 2024-09-18T04:57:09.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.902 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:57:09.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:57:09.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126849+0000", 2024-09-18T04:57:09.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126852+0000", 2024-09-18T04:57:09.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3659999999999999e-06 2024-09-18T04:57:09.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126858+0000", 2024-09-18T04:57:09.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4540000000000004e-06 2024-09-18T04:57:09.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126863+0000", 2024-09-18T04:57:09.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1780000000000002e-06 2024-09-18T04:57:09.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179608+0000", 2024-09-18T04:57:09.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052745159999999999 2024-09-18T04:57:09.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179703+0000", 2024-09-18T04:57:09.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4773999999999998e-05 2024-09-18T04:57:09.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179806+0000", 2024-09-18T04:57:09.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010296799999999999 2024-09-18T04:57:09.905 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.905 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.905 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.906 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14136 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1732608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.906 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:09.906 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.768418988999997, 2024-09-18T04:57:09.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052973412999999997, 2024-09-18T04:57:09.906 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.906 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.906 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.906 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.906 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.906 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14136 2024-09-18T04:57:09.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.907 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:09.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:09.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:09.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129013+0000", 2024-09-18T04:57:09.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.96e-06 2024-09-18T04:57:09.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129020+0000", 2024-09-18T04:57:09.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6809999999999998e-06 2024-09-18T04:57:09.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129026+0000", 2024-09-18T04:57:09.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:57:09.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181873+0000", 2024-09-18T04:57:09.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052847238999999997 2024-09-18T04:57:09.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181932+0000", 2024-09-18T04:57:09.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9339000000000002e-05 2024-09-18T04:57:09.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181984+0000", 2024-09-18T04:57:09.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1348999999999999e-05 2024-09-18T04:57:09.910 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.910 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.910 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.910 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14137 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1781760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.910 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:57:09.910 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.768398584000003, 2024-09-18T04:57:09.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053128914999999999, 2024-09-18T04:57:09.911 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.911 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.911 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.911 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.911 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.911 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14137 2024-09-18T04:57:09.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.911 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:57:09.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:57:09.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129030+0000", 2024-09-18T04:57:09.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129033+0000", 2024-09-18T04:57:09.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7439999999999999e-06 2024-09-18T04:57:09.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129039+0000", 2024-09-18T04:57:09.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.314e-06 2024-09-18T04:57:09.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129044+0000", 2024-09-18T04:57:09.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9919999999999998e-06 2024-09-18T04:57:09.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182015+0000", 2024-09-18T04:57:09.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052970875000000001 2024-09-18T04:57:09.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182058+0000", 2024-09-18T04:57:09.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2997000000000002e-05 2024-09-18T04:57:09.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182160+0000", 2024-09-18T04:57:09.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101437 2024-09-18T04:57:09.915 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.915 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.915 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.915 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14138 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1830912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.915 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:57:09.915 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.768380550000003, 2024-09-18T04:57:09.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053256435999999997, 2024-09-18T04:57:09.916 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.916 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.916 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.916 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.916 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.916 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14138 2024-09-18T04:57:09.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.916 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:57:09.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:57:09.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129048+0000", 2024-09-18T04:57:09.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129056+0000", 2024-09-18T04:57:09.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.413e-06 2024-09-18T04:57:09.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129062+0000", 2024-09-18T04:57:09.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6810000000000001e-06 2024-09-18T04:57:09.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129068+0000", 2024-09-18T04:57:09.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0459999999999996e-06 2024-09-18T04:57:09.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182191+0000", 2024-09-18T04:57:09.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053123210999999997 2024-09-18T04:57:09.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182238+0000", 2024-09-18T04:57:09.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7404999999999998e-05 2024-09-18T04:57:09.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182305+0000", 2024-09-18T04:57:09.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7028999999999998e-05 2024-09-18T04:57:09.919 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.920 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.920 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.920 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15910 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2236416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.920 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:09.920 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.669293017999998, 2024-09-18T04:57:09.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053241054000000003, 2024-09-18T04:57:09.920 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.920 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.920 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.920 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.920 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.921 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15910 2024-09-18T04:57:09.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.921 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:09.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.921 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.921 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:09.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:09.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228138+0000", 2024-09-18T04:57:09.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:57:09.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228144+0000", 2024-09-18T04:57:09.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5369999999999997e-06 2024-09-18T04:57:09.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228149+0000", 2024-09-18T04:57:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1209999999999998e-06 2024-09-18T04:57:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280822+0000", 2024-09-18T04:57:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052673194999999999 2024-09-18T04:57:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280903+0000", 2024-09-18T04:57:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1179000000000003e-05 2024-09-18T04:57:09.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281377+0000", 2024-09-18T04:57:09.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047417899999999997 2024-09-18T04:57:09.924 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.924 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.924 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.924 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15911 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2285568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.924 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:09.924 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.669276285999999, 2024-09-18T04:57:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053415151000000001, 2024-09-18T04:57:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15911 2024-09-18T04:57:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:09.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:09.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228155+0000", 2024-09-18T04:57:09.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:57:09.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228161+0000", 2024-09-18T04:57:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:57:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228166+0000", 2024-09-18T04:57:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2320000000000001e-06 2024-09-18T04:57:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281405+0000", 2024-09-18T04:57:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053239336999999998 2024-09-18T04:57:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281482+0000", 2024-09-18T04:57:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6569999999999994e-05 2024-09-18T04:57:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281568+0000", 2024-09-18T04:57:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6328000000000004e-05 2024-09-18T04:57:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15918 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2662400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.667507305000001, 2024-09-18T04:57:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053091065, 2024-09-18T04:57:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15918 2024-09-18T04:57:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:09.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:09.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:09.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229924+0000", 2024-09-18T04:57:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:57:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229930+0000", 2024-09-18T04:57:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.066e-06 2024-09-18T04:57:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229935+0000", 2024-09-18T04:57:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9119999999999997e-06 2024-09-18T04:57:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282840+0000", 2024-09-18T04:57:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052905119 2024-09-18T04:57:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282922+0000", 2024-09-18T04:57:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2081000000000002e-05 2024-09-18T04:57:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283013+0000", 2024-09-18T04:57:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0828000000000005e-05 2024-09-18T04:57:09.934 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.934 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.934 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.934 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15919 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2711552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.934 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:09.934 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.667490026999999, 2024-09-18T04:57:09.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053293021000000003, 2024-09-18T04:57:09.934 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.934 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.934 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.934 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15919 2024-09-18T04:57:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:09.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229941+0000", 2024-09-18T04:57:09.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2639999999999998e-06 2024-09-18T04:57:09.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229947+0000", 2024-09-18T04:57:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9750000000000004e-06 2024-09-18T04:57:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229952+0000", 2024-09-18T04:57:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7929999999999997e-06 2024-09-18T04:57:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283062+0000", 2024-09-18T04:57:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05310952 2024-09-18T04:57:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283162+0000", 2024-09-18T04:57:09.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100098 2024-09-18T04:57:09.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283232+0000", 2024-09-18T04:57:09.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0693999999999998e-05 2024-09-18T04:57:09.938 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.938 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.938 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.938 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15920 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2760704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.938 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.667473266000002, 2024-09-18T04:57:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053502367000000002, 2024-09-18T04:57:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15920 2024-09-18T04:57:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229958+0000", 2024-09-18T04:57:09.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1770000000000001e-06 2024-09-18T04:57:09.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229964+0000", 2024-09-18T04:57:09.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8470000000000001e-06 2024-09-18T04:57:09.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229968+0000", 2024-09-18T04:57:09.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5680000000000001e-06 2024-09-18T04:57:09.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283269+0000", 2024-09-18T04:57:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053300620999999999 2024-09-18T04:57:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283359+0000", 2024-09-18T04:57:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9971999999999998e-05 2024-09-18T04:57:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283458+0000", 2024-09-18T04:57:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9483999999999998e-05 2024-09-18T04:57:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15921 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2809856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:57:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.667456715999997, 2024-09-18T04:57:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053800977, 2024-09-18T04:57:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15921 2024-09-18T04:57:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:57:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:57:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229972+0000", 2024-09-18T04:57:09.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229974+0000", 2024-09-18T04:57:09.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2910000000000002e-06 2024-09-18T04:57:09.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229982+0000", 2024-09-18T04:57:09.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7249999999999997e-06 2024-09-18T04:57:09.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229989+0000", 2024-09-18T04:57:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6429999999999998e-06 2024-09-18T04:57:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283601+0000", 2024-09-18T04:57:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053612211 2024-09-18T04:57:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283681+0000", 2024-09-18T04:57:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0125999999999998e-05 2024-09-18T04:57:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283774+0000", 2024-09-18T04:57:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2362999999999996e-05 2024-09-18T04:57:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15922 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2859008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:57:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.667411776999998, 2024-09-18T04:57:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053882270000000003, 2024-09-18T04:57:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15922 2024-09-18T04:57:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:57:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:57:09.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230017+0000", 2024-09-18T04:57:09.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230019+0000", 2024-09-18T04:57:09.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4210000000000002e-06 2024-09-18T04:57:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230027+0000", 2024-09-18T04:57:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000002e-06 2024-09-18T04:57:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230032+0000", 2024-09-18T04:57:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0089999999999998e-06 2024-09-18T04:57:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283800+0000", 2024-09-18T04:57:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053768418999999998 2024-09-18T04:57:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283849+0000", 2024-09-18T04:57:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9339000000000003e-05 2024-09-18T04:57:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283900+0000", 2024-09-18T04:57:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0421999999999999e-05 2024-09-18T04:57:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.952 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15923 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2924544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.952 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:09.952 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.667392907999997, 2024-09-18T04:57:09.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054073016000000002, 2024-09-18T04:57:09.952 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.952 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:09.952 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.952 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.952 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.952 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15923 2024-09-18T04:57:09.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230038+0000", 2024-09-18T04:57:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.463e-06 2024-09-18T04:57:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230045+0000", 2024-09-18T04:57:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3570000000000003e-06 2024-09-18T04:57:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230050+0000", 2024-09-18T04:57:09.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7299999999999996e-06 2024-09-18T04:57:09.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283928+0000", 2024-09-18T04:57:09.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053878218999999998 2024-09-18T04:57:09.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:09.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284010+0000", 2024-09-18T04:57:09.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2577000000000006e-05 2024-09-18T04:57:09.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284109+0000", 2024-09-18T04:57:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9102000000000001e-05 2024-09-18T04:57:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15924 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.667374438000003, 2024-09-18T04:57:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054108470999999998, 2024-09-18T04:57:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15924 2024-09-18T04:57:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:09.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:09.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230061+0000", 2024-09-18T04:57:09.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8240000000000002e-06 2024-09-18T04:57:09.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230068+0000", 2024-09-18T04:57:09.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4320000000000004e-06 2024-09-18T04:57:09.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230073+0000", 2024-09-18T04:57:09.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.921e-06 2024-09-18T04:57:09.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284153+0000", 2024-09-18T04:57:09.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054080793000000002 2024-09-18T04:57:09.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284163+0000", 2024-09-18T04:57:09.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.859e-06 2024-09-18T04:57:09.960 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.960 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.960 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.960 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15925 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.960 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:09.960 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.667350323000001, 2024-09-18T04:57:09.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054134632000000002, 2024-09-18T04:57:09.960 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.961 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:09.961 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.961 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.961 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.961 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15925 2024-09-18T04:57:09.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.961 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:09.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230081+0000", 2024-09-18T04:57:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4109999999999999e-06 2024-09-18T04:57:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230087+0000", 2024-09-18T04:57:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.161e-06 2024-09-18T04:57:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230092+0000", 2024-09-18T04:57:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9259999999999999e-06 2024-09-18T04:57:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284204+0000", 2024-09-18T04:57:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054111546000000003 2024-09-18T04:57:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284214+0000", 2024-09-18T04:57:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9639999999999998e-06 2024-09-18T04:57:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15926 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.667332999000003, 2024-09-18T04:57:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054143193999999999, 2024-09-18T04:57:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15926 2024-09-18T04:57:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:09.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:09.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230098+0000", 2024-09-18T04:57:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1399999999999998e-06 2024-09-18T04:57:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230104+0000", 2024-09-18T04:57:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-18T04:57:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230109+0000", 2024-09-18T04:57:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9250000000000003e-06 2024-09-18T04:57:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284231+0000", 2024-09-18T04:57:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054122297999999999 2024-09-18T04:57:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284239+0000", 2024-09-18T04:57:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1689999999999992e-06 2024-09-18T04:57:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15927 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.667315952000003, 2024-09-18T04:57:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054168435000000001, 2024-09-18T04:57:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15927 2024-09-18T04:57:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230116+0000", 2024-09-18T04:57:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5150000000000001e-06 2024-09-18T04:57:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230122+0000", 2024-09-18T04:57:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1190000000000002e-06 2024-09-18T04:57:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230126+0000", 2024-09-18T04:57:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.792e-06 2024-09-18T04:57:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284273+0000", 2024-09-18T04:57:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054146086000000003 2024-09-18T04:57:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284282+0000", 2024-09-18T04:57:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2590000000000006e-06 2024-09-18T04:57:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15928 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:57:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.667298569000003, 2024-09-18T04:57:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054189739000000001, 2024-09-18T04:57:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15928 2024-09-18T04:57:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:57:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:57:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230130+0000", 2024-09-18T04:57:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230133+0000", 2024-09-18T04:57:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.244e-06 2024-09-18T04:57:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230139+0000", 2024-09-18T04:57:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9830000000000001e-06 2024-09-18T04:57:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230143+0000", 2024-09-18T04:57:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7530000000000001e-06 2024-09-18T04:57:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284313+0000", 2024-09-18T04:57:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054169332000000001 2024-09-18T04:57:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284320+0000", 2024-09-18T04:57:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8699999999999992e-06 2024-09-18T04:57:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15929 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.667282129, 2024-09-18T04:57:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054218771999999998, 2024-09-18T04:57:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15929 2024-09-18T04:57:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230149+0000", 2024-09-18T04:57:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-18T04:57:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230155+0000", 2024-09-18T04:57:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2469999999999997e-06 2024-09-18T04:57:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230160+0000", 2024-09-18T04:57:09.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7129999999999996e-06 2024-09-18T04:57:09.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284355+0000", 2024-09-18T04:57:09.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054194900999999997 2024-09-18T04:57:09.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284366+0000", 2024-09-18T04:57:09.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0993e-05 2024-09-18T04:57:09.980 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15930 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.667265129, 2024-09-18T04:57:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054244713999999999, 2024-09-18T04:57:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15930 2024-09-18T04:57:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230166+0000", 2024-09-18T04:57:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1009999999999999e-06 2024-09-18T04:57:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230172+0000", 2024-09-18T04:57:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.981e-06 2024-09-18T04:57:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230184+0000", 2024-09-18T04:57:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2449000000000001e-05 2024-09-18T04:57:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284400+0000", 2024-09-18T04:57:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054215452999999997 2024-09-18T04:57:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284409+0000", 2024-09-18T04:57:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0839999999999998e-06 2024-09-18T04:57:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15931 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:57:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.667239340000002, 2024-09-18T04:57:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054261928000000001, 2024-09-18T04:57:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15931 2024-09-18T04:57:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:57:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:57:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230189+0000", 2024-09-18T04:57:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230192+0000", 2024-09-18T04:57:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6599999999999999e-06 2024-09-18T04:57:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230199+0000", 2024-09-18T04:57:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5570000000000001e-06 2024-09-18T04:57:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230204+0000", 2024-09-18T04:57:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2270000000000001e-06 2024-09-18T04:57:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284442+0000", 2024-09-18T04:57:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054238383000000001 2024-09-18T04:57:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284452+0000", 2024-09-18T04:57:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6069999999999997e-06 2024-09-18T04:57:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.989 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.989 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.989 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:09.989 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:09.989 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:09.990 INFO:teuthology.orchestra.run.smithi002.stdout:3194 write 0xbcef550 thru 0xbcff161 (0xfc12 bytes) 2024-09-18T04:57:09.990 INFO:teuthology.orchestra.run.smithi002.stdout:3196 punch from 0x3dbdacb to 0x3dc8bf0, (0xb125 bytes) 2024-09-18T04:57:09.990 INFO:teuthology.orchestra.run.smithi002.stdout:3197 write 0x5a7d02e thru 0x5a870fa (0xa0cd bytes) 2024-09-18T04:57:09.990 INFO:teuthology.orchestra.run.smithi002.stdout:3198 trunc from 0xbcff162 to 0xa219c5f 2024-09-18T04:57:09.991 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:57:09.994 INFO:teuthology.orchestra.run.smithi002.stdout:3200 read 0x739d078 thru 0x73ab319 (0xe2a2 bytes) 2024-09-18T04:57:09.994 INFO:teuthology.orchestra.run.smithi002.stdout:3201 write 0xd9fd3e3 thru 0xda01258 (0x3e76 bytes) 2024-09-18T04:57:09.994 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:09.994 INFO:teuthology.orchestra.run.smithi002.stdout:3203 read 0xc7ce4f6 thru 0xc7d9cf2 (0xb7fd bytes) 2024-09-18T04:57:09.994 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:09.994 INFO:teuthology.orchestra.run.smithi002.stdout:3206 write 0x82d07fb thru 0x82dedca (0xe5d0 bytes) 2024-09-18T04:57:09.994 INFO:teuthology.orchestra.run.smithi002.stdout:3214 read 0x22bfb3f thru 0x22c5211 (0x56d3 bytes) 2024-09-18T04:57:09.994 INFO:teuthology.orchestra.run.smithi002.stdout:3216 write 0x445ebec thru 0x44630b2 (0x44c7 bytes) 2024-09-18T04:57:09.994 INFO:teuthology.orchestra.run.smithi002.stdout:3217 write 0xadf29f5 thru 0xadfaf22 (0x852e bytes) 2024-09-18T04:57:09.995 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:09.995 INFO:teuthology.orchestra.run.smithi002.stdout:3219 punch from 0x37c794d to 0x37c97f8, (0x1eab bytes) 2024-09-18T04:57:09.995 INFO:teuthology.orchestra.run.smithi002.stdout:3221 writesame 0xbf8d4e thru 0xc0736e data_size 0x1(0xe621 bytes) 2024-09-18T04:57:09.995 INFO:teuthology.orchestra.run.smithi002.stdout:3222 write 0x8cbeaba thru 0x8cbf79b (0xce2 bytes) 2024-09-18T04:57:09.995 INFO:teuthology.orchestra.run.smithi002.stdout:3226 trunc from 0xda01259 to 0xdb202ea 2024-09-18T04:57:09.995 INFO:teuthology.orchestra.run.smithi002.stdout:3227 write 0x4df0300 thru 0x4e00084 (0xfd85 bytes) 2024-09-18T04:57:09.995 INFO:teuthology.orchestra.run.smithi002.stdout:3229 write 0x35b7c76 thru 0x35c5a24 (0xddaf bytes) 2024-09-18T04:57:09.995 INFO:teuthology.orchestra.run.smithi002.stdout:3230 read 0xa53ed26 thru 0xa54ab34 (0xbe0f bytes) 2024-09-18T04:57:09.995 INFO:teuthology.orchestra.run.smithi002.stdout:3231 write 0x9418595 thru 0x941a713 (0x217f bytes) 2024-09-18T04:57:09.995 INFO:teuthology.orchestra.run.smithi002.stdout:3232 read 0xc140735 thru 0xc148339 (0x7c05 bytes) 2024-09-18T04:57:09.995 INFO:teuthology.orchestra.run.smithi002.stdout:3233 read 0xb730788 thru 0xb73d180 (0xc9f9 bytes) 2024-09-18T04:57:09.995 INFO:teuthology.orchestra.run.smithi002.stdout:3235 read 0x45ad8b4 thru 0x45af6b0 (0x1dfd bytes) 2024-09-18T04:57:09.995 INFO:teuthology.orchestra.run.smithi002.stdout:3236 read 0xdb1cf3b thru 0xdb202e9 (0x33af bytes) 2024-09-18T04:57:09.996 INFO:teuthology.orchestra.run.smithi002.stdout:3237 read 0x751193f thru 0x75160d2 (0x4794 bytes) 2024-09-18T04:57:09.996 INFO:teuthology.orchestra.run.smithi002.stdout:3240 write 0x51b7abb thru 0x51c59d5 (0xdf1b bytes) 2024-09-18T04:57:09.996 INFO:teuthology.orchestra.run.smithi002.stdout:3242 trunc from 0xdb202ea to 0x8826e7f 2024-09-18T04:57:09.996 INFO:teuthology.orchestra.run.smithi002.stdout:3246 punch from 0x3693f94 to 0x369db92, (0x9bfe bytes) 2024-09-18T04:57:09.996 INFO:teuthology.orchestra.run.smithi002.stdout:3247 writesame 0x2e4bcda thru 0x2e4edb1 data_size 0xc(0x30d8 bytes) 2024-09-18T04:57:09.996 INFO:teuthology.orchestra.run.smithi002.stdout:3248 write 0x596c95a thru 0x59719bc (0x5063 bytes) 2024-09-18T04:57:09.996 INFO:teuthology.orchestra.run.smithi002.stdout:3249 read 0x42c419d thru 0x42c6d94 (0x2bf8 bytes) 2024-09-18T04:57:09.996 INFO:teuthology.orchestra.run.smithi002.stdout:3250 writesame 0x521cc01 thru 0x521d907 data_size 0x17(0xd07 bytes) 2024-09-18T04:57:09.996 INFO:teuthology.orchestra.run.smithi002.stdout:3252 punch from 0x1e4f9f0 to 0x1e5bb92, (0xc1a2 bytes) 2024-09-18T04:57:09.996 INFO:teuthology.orchestra.run.smithi002.stdout:3253 trunc from 0x8826e7f to 0x8238db7 2024-09-18T04:57:09.996 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:09.987+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:09.997 INFO:teuthology.orchestra.run.smithi002.stdout:3254 writesame 0x44a2cc1 thru 0x44a302f data_size 0x1(0x36f bytes) 2024-09-18T04:57:10.002 INFO:teuthology.orchestra.run.smithi002.stdout:3255 write 0xc5b388e thru 0xc5be17f (0xa8f2 bytes) 2024-09-18T04:57:10.005 INFO:teuthology.orchestra.run.smithi002.stdout:3256 trunc from 0xc5be180 to 0xc1659eb 2024-09-18T04:57:10.011 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:10.012 INFO:teuthology.orchestra.run.smithi002.stdout:3261 write 0x2a049a6 thru 0x2a0a0d1 (0x572c bytes) 2024-09-18T04:57:10.013 INFO:teuthology.orchestra.run.smithi002.stdout:3262 read 0x99e0d82 thru 0x99ebf15 (0xb194 bytes) 2024-09-18T04:57:10.022 INFO:teuthology.orchestra.run.smithi002.stdout:3264 trunc from 0xc1659eb to 0x3cba5d8 2024-09-18T04:57:10.036 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:10.036 INFO:teuthology.orchestra.run.smithi002.stdout:3267 read 0x16a7b7b thru 0x16b1374 (0x97fa bytes) 2024-09-18T04:57:10.051 INFO:teuthology.orchestra.run.smithi002.stdout:3269 trunc from 0x3cba5d8 to 0xe9f9385 2024-09-18T04:57:10.056 INFO:teuthology.orchestra.run.smithi002.stdout:3270 write 0x3dc9e81 thru 0x3dd7fde (0xe15e bytes) 2024-09-18T04:57:10.057 INFO:teuthology.orchestra.run.smithi002.stdout:3271 read 0xa9371e7 thru 0xa9402dd (0x90f7 bytes) 2024-09-18T04:57:10.058 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:10.058 INFO:teuthology.orchestra.run.smithi002.stdout:3273 writesame 0x640314 thru 0x649299 data_size 0x1(0x8f86 bytes) 2024-09-18T04:57:10.080 INFO:teuthology.orchestra.run.smithi002.stdout:3274 trunc from 0xe9f9385 to 0xadf0cc5 2024-09-18T04:57:10.091 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:10.089+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:10.092 INFO:teuthology.orchestra.run.smithi002.stdout:3275 writesame 0xc301210 thru 0xc30b739 data_size 0x57(0xa52a bytes) 2024-09-18T04:57:10.092 INFO:teuthology.orchestra.run.smithi002.stdout:3276 writesame 0x6367b26 thru 0x6372d0d data_size 0x4(0xb1e8 bytes) 2024-09-18T04:57:10.098 INFO:teuthology.orchestra.run.smithi002.stdout:3277 write 0x14fbb5 thru 0x15c7ee (0xcc3a bytes) 2024-09-18T04:57:10.099 INFO:teuthology.orchestra.run.smithi002.stdout:3278 punch from 0xd8bb0d to 0xd8e104, (0x25f7 bytes) 2024-09-18T04:57:10.099 INFO:teuthology.orchestra.run.smithi002.stdout:3279 read 0xcaccf7 thru 0xcb9a66 (0xcd70 bytes) 2024-09-18T04:57:10.107 INFO:teuthology.orchestra.run.smithi002.stdout:3280 writesame 0xe23b69a thru 0xe248a7c data_size 0x93(0xd3e3 bytes) 2024-09-18T04:57:10.107 INFO:teuthology.orchestra.run.smithi002.stdout:3281 read 0xb112aad thru 0xb11e877 (0xbdcb bytes) 2024-09-18T04:57:10.107 INFO:teuthology.orchestra.run.smithi002.stdout:3282 read 0x25baca2 thru 0x25bcb1f (0x1e7e bytes) 2024-09-18T04:57:10.108 INFO:teuthology.orchestra.run.smithi002.stdout:3283 read 0x582ea07 thru 0x582f9e5 (0xfdf bytes) 2024-09-18T04:57:10.108 INFO:teuthology.orchestra.run.smithi002.stdout:3284 read 0xc30c271 thru 0xc3119a0 (0x5730 bytes) 2024-09-18T04:57:10.108 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:10.108 INFO:teuthology.orchestra.run.smithi002.stdout:3287 read 0x6ee38bf thru 0x6ee58cb (0x200d bytes) 2024-09-18T04:57:10.108 INFO:teuthology.orchestra.run.smithi002.stdout:3289 punch from 0x2174f1f to 0x21803ed, (0xb4ce bytes) 2024-09-18T04:57:10.114 INFO:teuthology.orchestra.run.smithi002.stdout:3290 writesame 0xe4b781d thru 0xe4b9943 data_size 0x9(0x2127 bytes) 2024-09-18T04:57:10.114 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:10.114 INFO:teuthology.orchestra.run.smithi002.stdout:3293 punch from 0xa3f2e5d to 0xa400c04, (0xdda7 bytes) 2024-09-18T04:57:10.114 INFO:teuthology.orchestra.run.smithi002.stdout:3294 punch from 0xb10d514 to 0xb119cb5, (0xc7a1 bytes) 2024-09-18T04:57:10.118 INFO:teuthology.orchestra.run.smithi002.stdout:3295 write 0xe77df44 thru 0xe784367 (0x6424 bytes) 2024-09-18T04:57:10.121 INFO:teuthology.orchestra.run.smithi002.stdout:3296 writesame 0x1ad8bd1 thru 0x1ade1f5 data_size 0x1(0x5625 bytes) 2024-09-18T04:57:10.126 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:10.126 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:10.126 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:10.126 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:10.131 INFO:teuthology.orchestra.run.smithi002.stdout:3297 write 0x937c736 thru 0x9388e50 (0xc71b bytes) 2024-09-18T04:57:10.134 INFO:teuthology.orchestra.run.smithi002.stdout:3298 writesame 0x361ddeb thru 0x3620be1 data_size 0x7(0x2df7 bytes) 2024-09-18T04:57:10.135 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:57:10.137 INFO:teuthology.orchestra.run.smithi002.stdout:3301 write 0xc67a22e thru 0xc684634 (0xa407 bytes) 2024-09-18T04:57:10.140 INFO:teuthology.orchestra.run.smithi002.stdout:3302 write 0xc9a8290 thru 0xc9adf21 (0x5c92 bytes) 2024-09-18T04:57:10.141 INFO:teuthology.orchestra.run.smithi002.stdout:3303 write 0x9ff8835 thru 0xa006b16 (0xe2e2 bytes) 2024-09-18T04:57:10.145 INFO:teuthology.orchestra.run.smithi002.stdout:3304 punch from 0x7215189 to 0x7220201, (0xb078 bytes) 2024-09-18T04:57:10.145 INFO:teuthology.orchestra.run.smithi002.stdout:3305 write 0x3dce948 thru 0x3dd9029 (0xa6e2 bytes) 2024-09-18T04:57:10.146 INFO:teuthology.orchestra.run.smithi002.stdout:3306 read 0x71535de thru 0x7160b02 (0xd525 bytes) 2024-09-18T04:57:10.147 INFO:teuthology.orchestra.run.smithi002.stdout:3307 write 0x5b19fe8 thru 0x5b266b6 (0xc6cf bytes) 2024-09-18T04:57:10.158 INFO:teuthology.orchestra.run.smithi002.stdout:3308 trunc from 0xe784368 to 0x67f1400 2024-09-18T04:57:10.170 INFO:teuthology.orchestra.run.smithi002.stdout:3310 clone 29 order 19 su 262144 sc 13 2024-09-18T04:57:10.192 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:10.190+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:10.247 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:10.247 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:10.248 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:10.248 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:10.248 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:10.257 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:57:10.294 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:10.292+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:10.342 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T04:57:10.388 INFO:teuthology.orchestra.run.smithi002.stdout:leaving image image_client.0-clone28 intact 2024-09-18T04:57:10.395 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:10.392+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:10.414 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:10.415 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:10.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:10.415 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:10.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.415 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:15989 3.4 e32/19 3:238b8fe4:::rbd_data.11ac7a27a554.0000000000000005:head v 32'150, mlcod=32'147)", 2024-09-18T04:57:10.415 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.245449+0000", 2024-09-18T04:57:10.415 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.164717809000003, 2024-09-18T04:57:10.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028002180000000002, 2024-09-18T04:57:10.415 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:10.416 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:10.416 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:10.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:10.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.245449+0000", 2024-09-18T04:57:10.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:10.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.245449+0000", 2024-09-18T04:57:10.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:10.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.245448+0000", 2024-09-18T04:57:10.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:10.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:10.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.246149+0000", 2024-09-18T04:57:10.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00070047499999999999 2024-09-18T04:57:10.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:10.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.246164+0000", 2024-09-18T04:57:10.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5532999999999998e-05 2024-09-18T04:57:10.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:10.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.246171+0000", 2024-09-18T04:57:10.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8990000000000004e-06 2024-09-18T04:57:10.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:10.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.246218+0000", 2024-09-18T04:57:10.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6789000000000002e-05 2024-09-18T04:57:10.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:10.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.246242+0000", 2024-09-18T04:57:10.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3706999999999999e-05 2024-09-18T04:57:10.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:10.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.248217+0000", 2024-09-18T04:57:10.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019752060000000002 2024-09-18T04:57:10.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:10.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.248249+0000", 2024-09-18T04:57:10.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2252000000000002e-05 2024-09-18T04:57:10.421 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.421 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:10.421 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.422 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:15991 3.4 e32/19 3:21a074f1:::rbd_data.11ac7a27a554.0000000000000006:head v 32'152, mlcod=32'148)", 2024-09-18T04:57:10.422 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.249009+0000", 2024-09-18T04:57:10.422 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.161157256000003, 2024-09-18T04:57:10.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021236060000000001, 2024-09-18T04:57:10.422 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:10.422 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:10.422 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:10.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:10.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249009+0000", 2024-09-18T04:57:10.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:10.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249009+0000", 2024-09-18T04:57:10.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:10.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249009+0000", 2024-09-18T04:57:10.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:10.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:10.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249228+0000", 2024-09-18T04:57:10.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021908999999999999 2024-09-18T04:57:10.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:10.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249243+0000", 2024-09-18T04:57:10.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5031999999999999e-05 2024-09-18T04:57:10.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:10.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249250+0000", 2024-09-18T04:57:10.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2919999999999998e-06 2024-09-18T04:57:10.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:10.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249294+0000", 2024-09-18T04:57:10.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3479000000000002e-05 2024-09-18T04:57:10.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:10.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249314+0000", 2024-09-18T04:57:10.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0596000000000001e-05 2024-09-18T04:57:10.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:10.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.251093+0000", 2024-09-18T04:57:10.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001778342 2024-09-18T04:57:10.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:10.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.251133+0000", 2024-09-18T04:57:10.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0503999999999997e-05 2024-09-18T04:57:10.427 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.427 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:10.427 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.427 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:18826 3.4 e32/19 3:23a67d65:::rbd_data.11ac7a27a554.0000000000000010:head v 32'162, mlcod=32'159)", 2024-09-18T04:57:10.428 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.666171+0000", 2024-09-18T04:57:10.428 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.743995583, 2024-09-18T04:57:10.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022865569999999998, 2024-09-18T04:57:10.428 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:10.428 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:10.428 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:10.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:10.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666171+0000", 2024-09-18T04:57:10.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:10.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666171+0000", 2024-09-18T04:57:10.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:10.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666170+0000", 2024-09-18T04:57:10.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:10.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:10.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666654+0000", 2024-09-18T04:57:10.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00048368400000000001 2024-09-18T04:57:10.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:10.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666670+0000", 2024-09-18T04:57:10.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5818e-05 2024-09-18T04:57:10.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:10.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666677+0000", 2024-09-18T04:57:10.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5750000000000003e-06 2024-09-18T04:57:10.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:10.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666734+0000", 2024-09-18T04:57:10.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6663999999999998e-05 2024-09-18T04:57:10.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:10.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666757+0000", 2024-09-18T04:57:10.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2691e-05 2024-09-18T04:57:10.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:10.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.668417+0000", 2024-09-18T04:57:10.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001660501 2024-09-18T04:57:10.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:10.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.668458+0000", 2024-09-18T04:57:10.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0364999999999997e-05 2024-09-18T04:57:10.432 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.432 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:10.432 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.432 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:22065 3.4 e32/19 3:267b21b8:::rbd_object_map.11ac7a27a554:head v 32'217, mlcod=32'213)", 2024-09-18T04:57:10.432 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:37.044092+0000", 2024-09-18T04:57:10.432 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 33.366074711000003, 2024-09-18T04:57:10.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001661636, 2024-09-18T04:57:10.432 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:10.432 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:10.433 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:10.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:10.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044092+0000", 2024-09-18T04:57:10.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:10.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044092+0000", 2024-09-18T04:57:10.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:10.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044091+0000", 2024-09-18T04:57:10.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:10.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:10.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044096+0000", 2024-09-18T04:57:10.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9230000000000001e-06 2024-09-18T04:57:10.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:10.436 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044106+0000", 2024-09-18T04:57:10.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7569999999999999e-06 2024-09-18T04:57:10.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:10.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044111+0000", 2024-09-18T04:57:10.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4839999999999998e-06 2024-09-18T04:57:10.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:10.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044147+0000", 2024-09-18T04:57:10.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6736999999999997e-05 2024-09-18T04:57:10.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:10.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.044158+0000", 2024-09-18T04:57:10.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1178999999999999e-05 2024-09-18T04:57:10.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:10.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.045717+0000", 2024-09-18T04:57:10.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015584850000000001 2024-09-18T04:57:10.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:10.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:37.045754+0000", 2024-09-18T04:57:10.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6671000000000002e-05 2024-09-18T04:57:10.438 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.439 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:10.439 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.439 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:36051 3.4 e42/19 3:25d849c5:::rbd_data.11ac728e6e9a.0000000000000002:head v 42'294, mlcod=40'292)", 2024-09-18T04:57:10.439 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:46.443141+0000", 2024-09-18T04:57:10.439 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.967025829000001, 2024-09-18T04:57:10.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037584760000000002, 2024-09-18T04:57:10.439 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:10.439 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:10.439 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:10.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:10.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443141+0000", 2024-09-18T04:57:10.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:10.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443141+0000", 2024-09-18T04:57:10.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:10.440 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443141+0000", 2024-09-18T04:57:10.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:10.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.440 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:10.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443812+0000", 2024-09-18T04:57:10.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067104899999999995 2024-09-18T04:57:10.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:10.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443831+0000", 2024-09-18T04:57:10.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9074999999999999e-05 2024-09-18T04:57:10.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:10.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443841+0000", 2024-09-18T04:57:10.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8919999999999995e-06 2024-09-18T04:57:10.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:10.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443933+0000", 2024-09-18T04:57:10.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2348000000000004e-05 2024-09-18T04:57:10.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:10.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443957+0000", 2024-09-18T04:57:10.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3567e-05 2024-09-18T04:57:10.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:10.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.446862+0000", 2024-09-18T04:57:10.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029052919999999999 2024-09-18T04:57:10.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:10.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.446899+0000", 2024-09-18T04:57:10.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7515999999999998e-05 2024-09-18T04:57:10.443 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.443 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:10.443 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.443 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:67383 3.4 e51/19 3:2583a267:::rbd_data.11ac8b4c71f8.00000000000001a3:head v 51'413, mlcod=50'410)", 2024-09-18T04:57:10.443 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.349429+0000", 2024-09-18T04:57:10.444 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.060737956000001, 2024-09-18T04:57:10.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018230950000000001, 2024-09-18T04:57:10.444 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:10.444 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:10.444 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:10.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:10.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.349429+0000", 2024-09-18T04:57:10.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:10.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.349429+0000", 2024-09-18T04:57:10.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:10.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.349428+0000", 2024-09-18T04:57:10.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:10.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:10.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.349434+0000", 2024-09-18T04:57:10.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.648e-06 2024-09-18T04:57:10.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:10.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.349443+0000", 2024-09-18T04:57:10.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5410000000000006e-06 2024-09-18T04:57:10.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:10.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.349450+0000", 2024-09-18T04:57:10.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5490000000000003e-06 2024-09-18T04:57:10.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:10.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.349494+0000", 2024-09-18T04:57:10.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4212999999999998e-05 2024-09-18T04:57:10.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:10.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.349514+0000", 2024-09-18T04:57:10.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.965e-05 2024-09-18T04:57:10.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:10.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.351215+0000", 2024-09-18T04:57:10.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017009690000000001 2024-09-18T04:57:10.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:10.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.351252+0000", 2024-09-18T04:57:10.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7023000000000001e-05 2024-09-18T04:57:10.447 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.448 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:10.448 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.448 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:90054 3.4 e54/19 3:265c2e99:::rbd_data.11ac679235d7.0000000000000004:head v 54'424, mlcod=53'422)", 2024-09-18T04:57:10.448 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:00.486527+0000", 2024-09-18T04:57:10.448 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 9.9236393029999999, 2024-09-18T04:57:10.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00244005, 2024-09-18T04:57:10.448 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:10.448 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:10.449 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:10.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:10.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:00.486527+0000", 2024-09-18T04:57:10.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:10.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:00.486527+0000", 2024-09-18T04:57:10.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:10.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:00.486527+0000", 2024-09-18T04:57:10.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:10.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:10.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:00.486998+0000", 2024-09-18T04:57:10.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047111199999999998 2024-09-18T04:57:10.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:10.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:00.487010+0000", 2024-09-18T04:57:10.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2007999999999999e-05 2024-09-18T04:57:10.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:10.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:00.487016+0000", 2024-09-18T04:57:10.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1090000000000003e-06 2024-09-18T04:57:10.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:10.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:00.487053+0000", 2024-09-18T04:57:10.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7125000000000001e-05 2024-09-18T04:57:10.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:10.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:00.487071+0000", 2024-09-18T04:57:10.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7391e-05 2024-09-18T04:57:10.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:10.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:00.488926+0000", 2024-09-18T04:57:10.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00185516 2024-09-18T04:57:10.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:10.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:00.488967+0000", 2024-09-18T04:57:10.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1798000000000002e-05 2024-09-18T04:57:10.453 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.453 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:10.453 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.453 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:92355 3.4 e55/19 3:202900f5:::rbd_data.11ac51791e54.0000000000000007:head v 55'427, mlcod=54'425)", 2024-09-18T04:57:10.453 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:01.828032+0000", 2024-09-18T04:57:10.453 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 8.5821347110000001, 2024-09-18T04:57:10.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019153250000000001, 2024-09-18T04:57:10.453 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:10.453 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:10.453 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:10.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:10.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.828032+0000", 2024-09-18T04:57:10.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:10.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.828032+0000", 2024-09-18T04:57:10.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:10.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.828030+0000", 2024-09-18T04:57:10.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999976 2024-09-18T04:57:10.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:10.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.828332+0000", 2024-09-18T04:57:10.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00030202799999999998 2024-09-18T04:57:10.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:10.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.828352+0000", 2024-09-18T04:57:10.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0475e-05 2024-09-18T04:57:10.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:10.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.828362+0000", 2024-09-18T04:57:10.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.55e-06 2024-09-18T04:57:10.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:10.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.828411+0000", 2024-09-18T04:57:10.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8899000000000001e-05 2024-09-18T04:57:10.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:10.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.828435+0000", 2024-09-18T04:57:10.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4697000000000001e-05 2024-09-18T04:57:10.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:10.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.829907+0000", 2024-09-18T04:57:10.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001471665 2024-09-18T04:57:10.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:10.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.829947+0000", 2024-09-18T04:57:10.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0219000000000002e-05 2024-09-18T04:57:10.457 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.457 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:10.458 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.458 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPush(3.1a 60/59 [PushOp(3:5d34c0ec:::rbd_data.11ac14fc1d07.0000000000000000:head, version: 29'34, data_included: [0~262144,514400~30572,786432~524288,1572864~231572], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:5d34c0ec:::rbd_data.11ac14fc1d07.0000000000000000:head@29'34, size: 2097152, copy_subset: [0~2097152], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1804436, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-18T04:57:10.458 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:06.327000+0000", 2024-09-18T04:57:10.458 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.0831664410000004, 2024-09-18T04:57:10.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002127688, 2024-09-18T04:57:10.458 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:10.458 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:10.458 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:10.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:10.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.327000+0000", 2024-09-18T04:57:10.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:10.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.327000+0000", 2024-09-18T04:57:10.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:10.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.327000+0000", 2024-09-18T04:57:10.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:10.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:10.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.328145+0000", 2024-09-18T04:57:10.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001145418 2024-09-18T04:57:10.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:10.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.328164+0000", 2024-09-18T04:57:10.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.931e-05 2024-09-18T04:57:10.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:10.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.328190+0000", 2024-09-18T04:57:10.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6078e-05 2024-09-18T04:57:10.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:10.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.328243+0000", 2024-09-18T04:57:10.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2781999999999997e-05 2024-09-18T04:57:10.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:10.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.328250+0000", 2024-09-18T04:57:10.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7549999999999997e-06 2024-09-18T04:57:10.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:10.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.329128+0000", 2024-09-18T04:57:10.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00087802699999999997 2024-09-18T04:57:10.462 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.462 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:10.462 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.463 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPush(3.1a 60/59 [PushOp(3:5f22f35c:::rbd_data.11ac7a27a554.0000000000000018:head, version: 32'57, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:5f22f35c:::rbd_data.11ac7a27a554.0000000000000018:head@32'57, size: 1048576, copy_subset: [0~1048576], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-18T04:57:10.463 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:06.339467+0000", 2024-09-18T04:57:10.463 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.0706995690000003, 2024-09-18T04:57:10.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021495609999999999, 2024-09-18T04:57:10.463 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:10.463 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:10.463 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:10.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:10.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.339467+0000", 2024-09-18T04:57:10.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:10.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.339467+0000", 2024-09-18T04:57:10.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:10.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.339466+0000", 2024-09-18T04:57:10.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:10.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:10.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.340457+0000", 2024-09-18T04:57:10.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00099054000000000004 2024-09-18T04:57:10.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:10.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.340477+0000", 2024-09-18T04:57:10.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9500999999999998e-05 2024-09-18T04:57:10.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:10.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.340491+0000", 2024-09-18T04:57:10.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4968e-05 2024-09-18T04:57:10.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:10.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.340555+0000", 2024-09-18T04:57:10.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3110999999999999e-05 2024-09-18T04:57:10.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:10.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.340562+0000", 2024-09-18T04:57:10.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.61e-06 2024-09-18T04:57:10.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:10.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.341617+0000", 2024-09-18T04:57:10.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010545140000000001 2024-09-18T04:57:10.467 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.467 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:10.467 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.468 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:104253 3.0 3:014cd031:::rbd_data.11acf685aece.0000000000000091:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T04:57:10.468 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:06.635820+0000", 2024-09-18T04:57:10.468 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.774346311, 2024-09-18T04:57:10.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021358190000000002, 2024-09-18T04:57:10.468 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:10.468 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:10.468 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:10.468 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:10.468 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:10.468 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104253 2024-09-18T04:57:10.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.469 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:10.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:10.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635820+0000", 2024-09-18T04:57:10.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:10.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635820+0000", 2024-09-18T04:57:10.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:10.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635819+0000", 2024-09-18T04:57:10.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999981 2024-09-18T04:57:10.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:10.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635829+0000", 2024-09-18T04:57:10.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.066e-05 2024-09-18T04:57:10.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:10.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635846+0000", 2024-09-18T04:57:10.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7058000000000002e-05 2024-09-18T04:57:10.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:10.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635855+0000", 2024-09-18T04:57:10.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6649999999999992e-06 2024-09-18T04:57:10.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:10.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635909+0000", 2024-09-18T04:57:10.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4156999999999997e-05 2024-09-18T04:57:10.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635993+0000", 2024-09-18T04:57:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3363999999999997e-05 2024-09-18T04:57:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-18T04:57:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.636106+0000", 2024-09-18T04:57:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000113682 2024-09-18T04:57:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:57:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.636749+0000", 2024-09-18T04:57:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00064277099999999999 2024-09-18T04:57:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:10.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.637800+0000", 2024-09-18T04:57:10.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010504900000000001 2024-09-18T04:57:10.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:10.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.637854+0000", 2024-09-18T04:57:10.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4676999999999998e-05 2024-09-18T04:57:10.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:10.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.637956+0000", 2024-09-18T04:57:10.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010197800000000001 2024-09-18T04:57:10.474 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.474 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:10.474 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.474 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:104256 3.4 3:22227238:::rbd_data.11acf685aece.0000000000000094:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T04:57:10.474 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:06.635869+0000", 2024-09-18T04:57:10.474 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.7742979920000002, 2024-09-18T04:57:10.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019687379999999998, 2024-09-18T04:57:10.474 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:10.474 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:10.474 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:10.475 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:10.475 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:10.475 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104256 2024-09-18T04:57:10.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.475 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:10.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:10.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635869+0000", 2024-09-18T04:57:10.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:10.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635869+0000", 2024-09-18T04:57:10.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:10.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635868+0000", 2024-09-18T04:57:10.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:10.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:10.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635874+0000", 2024-09-18T04:57:10.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6180000000000003e-06 2024-09-18T04:57:10.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:10.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635881+0000", 2024-09-18T04:57:10.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5839999999999997e-06 2024-09-18T04:57:10.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:10.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635887+0000", 2024-09-18T04:57:10.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9320000000000001e-06 2024-09-18T04:57:10.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:10.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635940+0000", 2024-09-18T04:57:10.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2954000000000002e-05 2024-09-18T04:57:10.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:10.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.636028+0000", 2024-09-18T04:57:10.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7862999999999995e-05 2024-09-18T04:57:10.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-18T04:57:10.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.636143+0000", 2024-09-18T04:57:10.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011521100000000001 2024-09-18T04:57:10.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:57:10.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.636970+0000", 2024-09-18T04:57:10.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00082689899999999997 2024-09-18T04:57:10.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:10.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.637690+0000", 2024-09-18T04:57:10.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00071944100000000003 2024-09-18T04:57:10.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:10.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.637776+0000", 2024-09-18T04:57:10.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5902000000000005e-05 2024-09-18T04:57:10.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:10.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.637837+0000", 2024-09-18T04:57:10.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1797999999999993e-05 2024-09-18T04:57:10.480 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.480 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:10.480 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.481 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:104394 3.1a e60/59 3:5d3c2665:::rbd_data.11acf685aece.0000000000000002:head v 60'167, mlcod=60'165)", 2024-09-18T04:57:10.481 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:06.663312+0000", 2024-09-18T04:57:10.481 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.7468544260000001, 2024-09-18T04:57:10.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022438900000000001, 2024-09-18T04:57:10.481 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:10.481 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:10.481 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:10.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:10.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.663312+0000", 2024-09-18T04:57:10.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.663312+0000", 2024-09-18T04:57:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.663311+0000", 2024-09-18T04:57:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:57:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.664199+0000", 2024-09-18T04:57:10.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00088800900000000004 2024-09-18T04:57:10.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:10.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.664222+0000", 2024-09-18T04:57:10.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3734999999999999e-05 2024-09-18T04:57:10.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:10.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.664235+0000", 2024-09-18T04:57:10.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2617e-05 2024-09-18T04:57:10.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:10.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.664308+0000", 2024-09-18T04:57:10.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2632000000000005e-05 2024-09-18T04:57:10.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:10.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.664332+0000", 2024-09-18T04:57:10.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4341999999999998e-05 2024-09-18T04:57:10.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:10.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.665489+0000", 2024-09-18T04:57:10.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011568640000000001 2024-09-18T04:57:10.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:10.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.665556+0000", 2024-09-18T04:57:10.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7286999999999995e-05 2024-09-18T04:57:10.485 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.485 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:10.485 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.486 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:104673 3.0 3:014f1eff:::rbd_data.11acf685aece.000000000000001f:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write 0~0] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T04:57:10.486 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:06.688140+0000", 2024-09-18T04:57:10.486 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.7220264649999999, 2024-09-18T04:57:10.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0016468139999999999, 2024-09-18T04:57:10.486 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:10.486 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:10.486 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:10.486 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:10.486 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:10.486 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104673 2024-09-18T04:57:10.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.486 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:10.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:10.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.688140+0000", 2024-09-18T04:57:10.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:10.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.688140+0000", 2024-09-18T04:57:10.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:10.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.688138+0000", 2024-09-18T04:57:10.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999981 2024-09-18T04:57:10.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:10.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.688171+0000", 2024-09-18T04:57:10.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.3098000000000001e-05 2024-09-18T04:57:10.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:10.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.688189+0000", 2024-09-18T04:57:10.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7555999999999999e-05 2024-09-18T04:57:10.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:10.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.688198+0000", 2024-09-18T04:57:10.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5559999999999996e-06 2024-09-18T04:57:10.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:10.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.688255+0000", 2024-09-18T04:57:10.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6657999999999999e-05 2024-09-18T04:57:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.688376+0000", 2024-09-18T04:57:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012102 2024-09-18T04:57:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-18T04:57:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.688525+0000", 2024-09-18T04:57:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000149276 2024-09-18T04:57:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:57:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.689338+0000", 2024-09-18T04:57:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00081219300000000001 2024-09-18T04:57:10.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:10.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.689615+0000", 2024-09-18T04:57:10.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027728099999999999 2024-09-18T04:57:10.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:10.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.689678+0000", 2024-09-18T04:57:10.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3467999999999996e-05 2024-09-18T04:57:10.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:10.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.689787+0000", 2024-09-18T04:57:10.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108693 2024-09-18T04:57:10.492 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.492 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:10.492 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.492 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:105545 3.1a e63/61 3:5f592997:::rbd_data.11ac491a2851.0000000000000000:head v 63'172, mlcod=62'170)", 2024-09-18T04:57:10.492 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:09.668693+0000", 2024-09-18T04:57:10.492 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.74147365899999995, 2024-09-18T04:57:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003007455, 2024-09-18T04:57:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.668693+0000", 2024-09-18T04:57:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.668693+0000", 2024-09-18T04:57:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:10.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.668691+0000", 2024-09-18T04:57:10.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999981 2024-09-18T04:57:10.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:10.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.670050+0000", 2024-09-18T04:57:10.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001358529 2024-09-18T04:57:10.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:10.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.670067+0000", 2024-09-18T04:57:10.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7050000000000001e-05 2024-09-18T04:57:10.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:10.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.670075+0000", 2024-09-18T04:57:10.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1030000000000002e-06 2024-09-18T04:57:10.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:10.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.670124+0000", 2024-09-18T04:57:10.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9228999999999999e-05 2024-09-18T04:57:10.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:10.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.670143+0000", 2024-09-18T04:57:10.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9097999999999999e-05 2024-09-18T04:57:10.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:10.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671662+0000", 2024-09-18T04:57:10.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015189929999999999 2024-09-18T04:57:10.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:10.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671701+0000", 2024-09-18T04:57:10.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8262000000000001e-05 2024-09-18T04:57:10.497 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.497 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:10.497 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.497 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:105546 3.9 e63/19 3:94add86d:::rbd_data.11ac491a2851.0000000000000003:head v 63'270, mlcod=57'268)", 2024-09-18T04:57:10.497 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:09.671085+0000", 2024-09-18T04:57:10.498 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.73908177100000005, 2024-09-18T04:57:10.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025510419999999999, 2024-09-18T04:57:10.498 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:10.498 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:10.498 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:10.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:10.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671085+0000", 2024-09-18T04:57:10.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:10.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671085+0000", 2024-09-18T04:57:10.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:10.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671084+0000", 2024-09-18T04:57:10.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:10.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:10.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671945+0000", 2024-09-18T04:57:10.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00086074600000000002 2024-09-18T04:57:10.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:10.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671959+0000", 2024-09-18T04:57:10.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3708e-05 2024-09-18T04:57:10.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:10.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671965+0000", 2024-09-18T04:57:10.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5590000000000002e-06 2024-09-18T04:57:10.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:10.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.672010+0000", 2024-09-18T04:57:10.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4591e-05 2024-09-18T04:57:10.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:10.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.672028+0000", 2024-09-18T04:57:10.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7731e-05 2024-09-18T04:57:10.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.673599+0000", 2024-09-18T04:57:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015717940000000001 2024-09-18T04:57:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.673636+0000", 2024-09-18T04:57:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6647000000000003e-05 2024-09-18T04:57:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:105549 3.0 3:041c4dc5:::rbd_data.11ac491a2851.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 7340032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T04:57:10.504 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:09.671361+0000", 2024-09-18T04:57:10.504 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.73880597000000003, 2024-09-18T04:57:10.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026345079999999998, 2024-09-18T04:57:10.504 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:10.504 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:10.504 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:10.504 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:10.504 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:10.504 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 105549 2024-09-18T04:57:10.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.504 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:10.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:10.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671361+0000", 2024-09-18T04:57:10.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:10.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671361+0000", 2024-09-18T04:57:10.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:10.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671359+0000", 2024-09-18T04:57:10.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999981 2024-09-18T04:57:10.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671411+0000", 2024-09-18T04:57:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1700999999999998e-05 2024-09-18T04:57:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671437+0000", 2024-09-18T04:57:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6795999999999999e-05 2024-09-18T04:57:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671448+0000", 2024-09-18T04:57:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0124e-05 2024-09-18T04:57:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:10.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671502+0000", 2024-09-18T04:57:10.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4768999999999997e-05 2024-09-18T04:57:10.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:10.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671682+0000", 2024-09-18T04:57:10.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018011000000000001 2024-09-18T04:57:10.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-18T04:57:10.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671897+0000", 2024-09-18T04:57:10.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000214112 2024-09-18T04:57:10.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:57:10.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.672805+0000", 2024-09-18T04:57:10.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00090863399999999996 2024-09-18T04:57:10.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:10.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.673922+0000", 2024-09-18T04:57:10.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001117067 2024-09-18T04:57:10.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.673971+0000", 2024-09-18T04:57:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8814999999999999e-05 2024-09-18T04:57:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.673995+0000", 2024-09-18T04:57:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4207999999999999e-05 2024-09-18T04:57:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:105660 3.0 3:041c4dc5:::rbd_data.11ac491a2851.0000000000000006:head [stat,truncate 5242880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T04:57:10.510 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:09.802588+0000", 2024-09-18T04:57:10.510 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.60757843, 2024-09-18T04:57:10.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001891818, 2024-09-18T04:57:10.510 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:10.510 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:10.510 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:10.510 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:10.510 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:10.510 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 105660 2024-09-18T04:57:10.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.510 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:10.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.802588+0000", 2024-09-18T04:57:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.802588+0000", 2024-09-18T04:57:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.802586+0000", 2024-09-18T04:57:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999976 2024-09-18T04:57:10.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:10.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.802596+0000", 2024-09-18T04:57:10.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0019e-05 2024-09-18T04:57:10.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:10.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.802619+0000", 2024-09-18T04:57:10.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2734999999999998e-05 2024-09-18T04:57:10.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:10.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.802631+0000", 2024-09-18T04:57:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2218999999999999e-05 2024-09-18T04:57:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.802692+0000", 2024-09-18T04:57:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1080000000000005e-05 2024-09-18T04:57:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.802806+0000", 2024-09-18T04:57:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000113688 2024-09-18T04:57:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-18T04:57:10.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.802941+0000", 2024-09-18T04:57:10.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000135196 2024-09-18T04:57:10.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:57:10.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.803610+0000", 2024-09-18T04:57:10.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00066944400000000005 2024-09-18T04:57:10.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:10.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.804151+0000", 2024-09-18T04:57:10.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00054033100000000001 2024-09-18T04:57:10.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:10.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.804442+0000", 2024-09-18T04:57:10.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00029086100000000001 2024-09-18T04:57:10.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:10.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.804480+0000", 2024-09-18T04:57:10.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8577e-05 2024-09-18T04:57:10.515 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.515 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:10.516 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.516 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:105911 3.0 3:041c4dc5:::rbd_data.11ac491a2851.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T04:57:10.516 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:10.056416+0000", 2024-09-18T04:57:10.516 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.35375064699999997, 2024-09-18T04:57:10.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017911119999999999, 2024-09-18T04:57:10.516 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:10.516 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:10.516 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:10.516 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:10.516 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:10.516 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 105911 2024-09-18T04:57:10.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.517 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:10.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:10.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.056416+0000", 2024-09-18T04:57:10.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:10.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.056416+0000", 2024-09-18T04:57:10.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:10.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.056414+0000", 2024-09-18T04:57:10.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999981 2024-09-18T04:57:10.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:10.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.056426+0000", 2024-09-18T04:57:10.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1584e-05 2024-09-18T04:57:10.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:10.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.056442+0000", 2024-09-18T04:57:10.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6297e-05 2024-09-18T04:57:10.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:10.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.056449+0000", 2024-09-18T04:57:10.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9070000000000001e-06 2024-09-18T04:57:10.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:10.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.056491+0000", 2024-09-18T04:57:10.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1885000000000003e-05 2024-09-18T04:57:10.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:10.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.056546+0000", 2024-09-18T04:57:10.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4614999999999998e-05 2024-09-18T04:57:10.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-18T04:57:10.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.059571+0000", 2024-09-18T04:57:10.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030249389999999999 2024-09-18T04:57:10.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:10.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.074240+0000", 2024-09-18T04:57:10.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014668976 2024-09-18T04:57:10.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:57:10.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.074250+0000", 2024-09-18T04:57:10.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0461e-05 2024-09-18T04:57:10.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:10.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.074285+0000", 2024-09-18T04:57:10.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4541e-05 2024-09-18T04:57:10.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:10.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.074327+0000", 2024-09-18T04:57:10.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2626000000000002e-05 2024-09-18T04:57:10.522 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.522 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:10.522 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.522 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(client.4524.0:105911 3.0 e63/61 ondisk, result = 0)", 2024-09-18T04:57:10.522 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:10.069182+0000", 2024-09-18T04:57:10.522 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.34098498599999999, 2024-09-18T04:57:10.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0050614400000000004, 2024-09-18T04:57:10.522 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:10.522 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:10.522 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:10.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:10.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.069182+0000", 2024-09-18T04:57:10.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:10.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.069182+0000", 2024-09-18T04:57:10.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:10.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.069180+0000", 2024-09-18T04:57:10.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:57:10.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:10.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.069188+0000", 2024-09-18T04:57:10.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6569999999999997e-06 2024-09-18T04:57:10.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:10.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.069208+0000", 2024-09-18T04:57:10.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9775e-05 2024-09-18T04:57:10.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:10.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.069220+0000", 2024-09-18T04:57:10.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.242e-05 2024-09-18T04:57:10.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:10.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.074226+0000", 2024-09-18T04:57:10.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0050062099999999997 2024-09-18T04:57:10.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:10.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.074235+0000", 2024-09-18T04:57:10.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6959999999999994e-06 2024-09-18T04:57:10.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:10.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.074243+0000", 2024-09-18T04:57:10.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0709999999999995e-06 2024-09-18T04:57:10.526 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.526 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:10.526 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.526 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:10.526 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:10.526 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:10.528 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:10.525+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:10.528 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:57:10.529 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-09-18T04:57:10.629 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:10.626+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:10.646 INFO:teuthology.orchestra.run.smithi002.stdout:3311 write 0xa20a95 thru 0xa2a750 (0x9cbc bytes) 2024-09-18T04:57:10.660 INFO:teuthology.orchestra.run.smithi002.stdout:3312 write 0x7baf4fe thru 0x7baf52b (0x2e bytes) 2024-09-18T04:57:10.663 INFO:teuthology.orchestra.run.smithi002.stdout:3314 punch from 0x5306047 to 0x530af03, (0x4ebc bytes) 2024-09-18T04:57:10.663 INFO:teuthology.orchestra.run.smithi002.stdout:3315 punch from 0x2ed433e to 0x2edc521, (0x81e3 bytes) 2024-09-18T04:57:10.664 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:10.666 INFO:teuthology.orchestra.run.smithi002.stdout:3318 trunc from 0x7baf52c to 0x50234c0 2024-09-18T04:57:10.686 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:10.686 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:10.686 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:10.686 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:10.698 INFO:teuthology.orchestra.run.smithi002.stdout:3319 trunc from 0x50234c0 to 0xd993300 2024-09-18T04:57:10.699 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:57:10.703 INFO:teuthology.orchestra.run.smithi002.stdout:3322 write 0x21f800f thru 0x2206032 (0xe024 bytes) 2024-09-18T04:57:10.721 INFO:teuthology.orchestra.run.smithi002.stdout:3323 trunc from 0xd993300 to 0xfbe558 2024-09-18T04:57:10.730 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:10.728+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:10.748 INFO:teuthology.orchestra.run.smithi002.stdout:3324 punch from 0xef6a08 to 0xf0582e, (0xee26 bytes) 2024-09-18T04:57:10.749 INFO:teuthology.orchestra.run.smithi002.stdout:3326 read 0x6df642 thru 0x6ed301 (0xdcc0 bytes) 2024-09-18T04:57:10.749 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:10.756 INFO:teuthology.orchestra.run.smithi002.stdout:3328 write 0x241a3b2 thru 0x2425a50 (0xb69f bytes) 2024-09-18T04:57:10.758 INFO:teuthology.orchestra.run.smithi002.stdout:3330 punch from 0x21cc853 to 0x21d7620, (0xadcd bytes) 2024-09-18T04:57:10.766 INFO:teuthology.orchestra.run.smithi002.stdout:3331 trunc from 0x2425a51 to 0x7a8a8fc 2024-09-18T04:57:10.786 INFO:teuthology.orchestra.run.smithi002.stdout:3332 write 0xea5856d thru 0xea5c812 (0x42a6 bytes) 2024-09-18T04:57:10.788 INFO:teuthology.orchestra.run.smithi002.stdout:3333 write 0xa26506a thru 0xa2692e1 (0x4278 bytes) 2024-09-18T04:57:10.791 INFO:teuthology.orchestra.run.smithi002.stdout:3334 write 0x559c6af thru 0x559d909 (0x125b bytes) 2024-09-18T04:57:10.794 INFO:teuthology.orchestra.run.smithi002.stdout:3335 write 0xc72325 thru 0xc770cd (0x4da9 bytes) 2024-09-18T04:57:10.797 INFO:teuthology.orchestra.run.smithi002.stdout:3339 read 0x4bf6c2d thru 0x4bf7400 (0x7d4 bytes) 2024-09-18T04:57:10.797 INFO:teuthology.orchestra.run.smithi002.stdout:3342 read 0x82f2d0 thru 0x83b4a4 (0xc1d5 bytes) 2024-09-18T04:57:10.798 INFO:teuthology.orchestra.run.smithi002.stdout:3344 trunc from 0xea5c813 to 0xea4102d 2024-09-18T04:57:10.823 INFO:teuthology.orchestra.run.smithi002.stdout:3345 trunc from 0xea4102d to 0x2a0b4b 2024-09-18T04:57:10.831 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:10.828+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:10.849 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:10.849 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:10.849 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:10.849 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:10.850 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:10.851 INFO:teuthology.orchestra.run.smithi002.stdout:3346 read 0x30bf1 thru 0x3fbc2 (0xefd2 bytes) 2024-09-18T04:57:10.859 INFO:teuthology.orchestra.run.smithi002.stdout:3348 write 0x1650955 thru 0x165d5ea (0xcc96 bytes) 2024-09-18T04:57:10.860 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:57:10.876 INFO:teuthology.orchestra.run.smithi002.stdout:3349 write 0x86cb48f thru 0x86d2869 (0x73db bytes) 2024-09-18T04:57:10.880 INFO:teuthology.orchestra.run.smithi002.stdout:3350 trunc from 0x86d286a to 0x737e737 2024-09-18T04:57:10.890 INFO:teuthology.orchestra.run.smithi002.stdout:3351 punch from 0x237f71b to 0x238dad0, (0xe3b5 bytes) 2024-09-18T04:57:10.890 INFO:teuthology.orchestra.run.smithi002.stdout:3352 read 0x66c5943 thru 0x66c9157 (0x3815 bytes) 2024-09-18T04:57:10.890 INFO:teuthology.orchestra.run.smithi002.stdout:3353 trunc from 0x737e737 to 0x745725f 2024-09-18T04:57:10.894 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:10.894 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:10.895 INFO:teuthology.orchestra.run.smithi002.stdout:3359 write 0x128da1a thru 0x128ec9c (0x1283 bytes) 2024-09-18T04:57:10.908 INFO:teuthology.orchestra.run.smithi002.stdout:3360 write 0xc64de62 thru 0xc653387 (0x5526 bytes) 2024-09-18T04:57:10.912 INFO:teuthology.orchestra.run.smithi002.stdout:3361 read 0xb75973a thru 0xb75d7b1 (0x4078 bytes) 2024-09-18T04:57:10.912 INFO:teuthology.orchestra.run.smithi002.stdout:3362 read 0x2cbf7cc thru 0x2cc1ce3 (0x2518 bytes) 2024-09-18T04:57:10.912 INFO:teuthology.orchestra.run.smithi002.stdout:3364 write 0x63e588f thru 0x63ef2fa (0x9a6c bytes) 2024-09-18T04:57:10.914 INFO:teuthology.orchestra.run.smithi002.stdout:3365 write 0x1519944 thru 0x151d490 (0x3b4d bytes) 2024-09-18T04:57:10.919 INFO:teuthology.orchestra.run.smithi002.stdout:3366 trunc from 0xc653388 to 0xe1e5485 2024-09-18T04:57:10.925 INFO:teuthology.orchestra.run.smithi002.stdout:3367 read 0x46fb110 thru 0x4707819 (0xc70a bytes) 2024-09-18T04:57:10.925 INFO:teuthology.orchestra.run.smithi002.stdout:3368 read 0x4c81e2e thru 0x4c8f711 (0xd8e4 bytes) 2024-09-18T04:57:10.925 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:10.925 INFO:teuthology.orchestra.run.smithi002.stdout:3371 writesame 0xc75273e thru 0xc761081 data_size 0x2(0xe944 bytes) 2024-09-18T04:57:10.932 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:10.929+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:10.946 INFO:teuthology.orchestra.run.smithi002.stdout:3372 trunc from 0xe1e5485 to 0xa82c320 2024-09-18T04:57:10.955 INFO:teuthology.orchestra.run.smithi002.stdout:3373 punch from 0x23e82c2 to 0x23f698e, (0xe6cc bytes) 2024-09-18T04:57:10.955 INFO:teuthology.orchestra.run.smithi002.stdout:3375 read 0x64baa6d thru 0x64c321b (0x87af bytes) 2024-09-18T04:57:10.956 INFO:teuthology.orchestra.run.smithi002.stdout:3376 read 0x1dd8c01 thru 0x1de30a3 (0xa4a3 bytes) 2024-09-18T04:57:10.966 INFO:teuthology.orchestra.run.smithi002.stdout:3378 trunc from 0xa82c320 to 0xf13d9d 2024-09-18T04:57:10.982 INFO:teuthology.orchestra.run.smithi002.stdout:3381 trunc from 0xf13d9d to 0x41adc24 2024-09-18T04:57:10.988 INFO:teuthology.orchestra.run.smithi002.stdout:3382 read 0x1fb0274 thru 0x1fb6784 (0x6511 bytes) 2024-09-18T04:57:10.988 INFO:teuthology.orchestra.run.smithi002.stdout:3383 read 0x30519a thru 0x306375 (0x11dc bytes) 2024-09-18T04:57:10.996 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:10.996 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:10.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:10.996 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:10.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.996 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11151 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1802240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:10.997 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889328+0000", 2024-09-18T04:57:10.997 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.102913913999998, 2024-09-18T04:57:10.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040037528000000003, 2024-09-18T04:57:10.997 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:10.997 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:10.997 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:10.998 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:10.998 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:10.998 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11151 2024-09-18T04:57:10.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.998 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:10.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:10.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889328+0000", 2024-09-18T04:57:10.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:10.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:10.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:10.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889328+0000", 2024-09-18T04:57:10.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:10.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:11.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889328+0000", 2024-09-18T04:57:11.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:11.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:11.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889330+0000", 2024-09-18T04:57:11.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.148e-06 2024-09-18T04:57:11.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:11.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889336+0000", 2024-09-18T04:57:11.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8220000000000003e-06 2024-09-18T04:57:11.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:11.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889340+0000", 2024-09-18T04:57:11.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.318e-06 2024-09-18T04:57:11.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:11.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929171+0000", 2024-09-18T04:57:11.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039830466000000002 2024-09-18T04:57:11.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:11.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929268+0000", 2024-09-18T04:57:11.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7569e-05 2024-09-18T04:57:11.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:11.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929366+0000", 2024-09-18T04:57:11.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7573999999999998e-05 2024-09-18T04:57:11.003 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.003 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:11.004 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.004 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11152 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1867776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:11.004 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889344+0000", 2024-09-18T04:57:11.004 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.102898328999999, 2024-09-18T04:57:11.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040227974, 2024-09-18T04:57:11.004 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:11.004 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:11.004 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:11.004 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:11.004 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:11.005 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11152 2024-09-18T04:57:11.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.005 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:11.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:11.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889344+0000", 2024-09-18T04:57:11.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:11.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889344+0000", 2024-09-18T04:57:11.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:11.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889344+0000", 2024-09-18T04:57:11.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:11.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:11.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889346+0000", 2024-09-18T04:57:11.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.99e-06 2024-09-18T04:57:11.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:11.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889351+0000", 2024-09-18T04:57:11.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4260000000000002e-06 2024-09-18T04:57:11.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:11.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889370+0000", 2024-09-18T04:57:11.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9262999999999998e-05 2024-09-18T04:57:11.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:11.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929397+0000", 2024-09-18T04:57:11.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040026679000000003 2024-09-18T04:57:11.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929479+0000", 2024-09-18T04:57:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2467999999999997e-05 2024-09-18T04:57:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929572+0000", 2024-09-18T04:57:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2519e-05 2024-09-18T04:57:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.009 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11153 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1916928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:11.009 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889377+0000", 2024-09-18T04:57:11.009 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.102865354999999, 2024-09-18T04:57:11.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040407884999999998, 2024-09-18T04:57:11.009 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:11.009 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:11.009 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:11.009 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:11.009 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:11.009 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11153 2024-09-18T04:57:11.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.009 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:11.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:11.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889377+0000", 2024-09-18T04:57:11.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:11.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889377+0000", 2024-09-18T04:57:11.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:11.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889376+0000", 2024-09-18T04:57:11.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:11.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889385+0000", 2024-09-18T04:57:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1030000000000002e-06 2024-09-18T04:57:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889392+0000", 2024-09-18T04:57:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1799999999999999e-06 2024-09-18T04:57:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889396+0000", 2024-09-18T04:57:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6890000000000003e-06 2024-09-18T04:57:11.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:11.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929603+0000", 2024-09-18T04:57:11.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040206652000000002 2024-09-18T04:57:11.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:11.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929688+0000", 2024-09-18T04:57:11.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5327000000000005e-05 2024-09-18T04:57:11.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:11.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929785+0000", 2024-09-18T04:57:11.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6362999999999999e-05 2024-09-18T04:57:11.014 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.014 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:11.014 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.014 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11154 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:11.014 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:57:11.014 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.102841525000002, 2024-09-18T04:57:11.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040591919999999997, 2024-09-18T04:57:11.014 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:11.014 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:11.014 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:11.014 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:11.015 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:11.015 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11154 2024-09-18T04:57:11.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.015 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:11.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:11.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:57:11.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:11.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:57:11.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:11.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889400+0000", 2024-09-18T04:57:11.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:11.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:11.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889403+0000", 2024-09-18T04:57:11.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3999999999999999e-06 2024-09-18T04:57:11.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:11.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889409+0000", 2024-09-18T04:57:11.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3380000000000001e-06 2024-09-18T04:57:11.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:11.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889414+0000", 2024-09-18T04:57:11.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5789999999999997e-06 2024-09-18T04:57:11.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:11.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929811+0000", 2024-09-18T04:57:11.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040397486000000003 2024-09-18T04:57:11.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:11.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929898+0000", 2024-09-18T04:57:11.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6370000000000001e-05 2024-09-18T04:57:11.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:11.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929993+0000", 2024-09-18T04:57:11.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5136000000000005e-05 2024-09-18T04:57:11.018 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.018 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:11.018 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.019 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11155 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2015232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:11.019 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:57:11.019 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.102824347000002, 2024-09-18T04:57:11.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040769156000000001, 2024-09-18T04:57:11.019 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:11.019 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:11.019 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:11.019 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:11.019 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:11.019 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11155 2024-09-18T04:57:11.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.019 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:11.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:11.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:57:11.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:11.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:57:11.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:11.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:57:11.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:11.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889420+0000", 2024-09-18T04:57:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.181e-06 2024-09-18T04:57:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889426+0000", 2024-09-18T04:57:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8599999999999998e-06 2024-09-18T04:57:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889430+0000", 2024-09-18T04:57:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4399999999999998e-06 2024-09-18T04:57:11.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:11.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930020+0000", 2024-09-18T04:57:11.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040589698 2024-09-18T04:57:11.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:11.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930105+0000", 2024-09-18T04:57:11.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4812e-05 2024-09-18T04:57:11.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:11.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930187+0000", 2024-09-18T04:57:11.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2540999999999995e-05 2024-09-18T04:57:11.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.023 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:11.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.023 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11156 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2064384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:11.023 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:57:11.023 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.102784053000001, 2024-09-18T04:57:11.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040934419, 2024-09-18T04:57:11.023 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:11.023 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:11.023 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11156 2024-09-18T04:57:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:57:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:57:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889457+0000", 2024-09-18T04:57:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889462+0000", 2024-09-18T04:57:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8350000000000003e-06 2024-09-18T04:57:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889481+0000", 2024-09-18T04:57:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8845999999999999e-05 2024-09-18T04:57:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889495+0000", 2024-09-18T04:57:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4229e-05 2024-09-18T04:57:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930216+0000", 2024-09-18T04:57:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040721081999999999 2024-09-18T04:57:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:11.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930299+0000", 2024-09-18T04:57:11.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2995000000000001e-05 2024-09-18T04:57:11.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:11.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930393+0000", 2024-09-18T04:57:11.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3185999999999999e-05 2024-09-18T04:57:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11402 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1495040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896220+0000", 2024-09-18T04:57:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.096022083000001, 2024-09-18T04:57:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040050388999999999, 2024-09-18T04:57:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:11.029 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11402 2024-09-18T04:57:11.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.029 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:11.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:11.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896220+0000", 2024-09-18T04:57:11.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:11.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896220+0000", 2024-09-18T04:57:11.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896220+0000", 2024-09-18T04:57:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896222+0000", 2024-09-18T04:57:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0049999999999999e-06 2024-09-18T04:57:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896227+0000", 2024-09-18T04:57:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6130000000000003e-06 2024-09-18T04:57:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896232+0000", 2024-09-18T04:57:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4100000000000001e-06 2024-09-18T04:57:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936072+0000", 2024-09-18T04:57:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039840253999999999 2024-09-18T04:57:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936175+0000", 2024-09-18T04:57:11.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103037 2024-09-18T04:57:11.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:11.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936271+0000", 2024-09-18T04:57:11.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5383999999999993e-05 2024-09-18T04:57:11.032 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.032 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:11.032 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.032 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11403 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1544192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:11.032 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896250+0000", 2024-09-18T04:57:11.032 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.095992703999997, 2024-09-18T04:57:11.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040244017999999999, 2024-09-18T04:57:11.033 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:11.033 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:11.033 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:11.033 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:11.033 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:11.033 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11403 2024-09-18T04:57:11.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.033 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:11.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:11.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896250+0000", 2024-09-18T04:57:11.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896250+0000", 2024-09-18T04:57:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896249+0000", 2024-09-18T04:57:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896251+0000", 2024-09-18T04:57:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.136e-06 2024-09-18T04:57:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896256+0000", 2024-09-18T04:57:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9350000000000002e-06 2024-09-18T04:57:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896259+0000", 2024-09-18T04:57:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8839999999999998e-06 2024-09-18T04:57:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936320+0000", 2024-09-18T04:57:11.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040061173999999998 2024-09-18T04:57:11.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:11.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936401+0000", 2024-09-18T04:57:11.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0987000000000002e-05 2024-09-18T04:57:11.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:11.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936494+0000", 2024-09-18T04:57:11.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2281999999999995e-05 2024-09-18T04:57:11.036 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.036 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:11.036 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.037 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11404 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1609728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:11.037 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:57:11.037 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.095979890999999, 2024-09-18T04:57:11.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040669012999999997, 2024-09-18T04:57:11.037 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:11.037 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:11.037 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:11.037 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:11.037 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:11.037 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11404 2024-09-18T04:57:11.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.037 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:57:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:57:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:57:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:57:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:11.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896263+0000", 2024-09-18T04:57:11.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2729999999999999e-06 2024-09-18T04:57:11.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:11.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896266+0000", 2024-09-18T04:57:11.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8200000000000001e-06 2024-09-18T04:57:11.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:11.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896269+0000", 2024-09-18T04:57:11.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3199999999999998e-06 2024-09-18T04:57:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936538+0000", 2024-09-18T04:57:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040269394999999999 2024-09-18T04:57:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936657+0000", 2024-09-18T04:57:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119498 2024-09-18T04:57:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936931+0000", 2024-09-18T04:57:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027391900000000002 2024-09-18T04:57:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11405 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1658880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:57:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.095971632999998, 2024-09-18T04:57:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040891034999999999, 2024-09-18T04:57:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11405 2024-09-18T04:57:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:57:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:57:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:11.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896270+0000", 2024-09-18T04:57:11.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:57:11.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:11.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896272+0000", 2024-09-18T04:57:11.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.124e-06 2024-09-18T04:57:11.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:11.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896274+0000", 2024-09-18T04:57:11.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8660000000000002e-06 2024-09-18T04:57:11.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:11.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896285+0000", 2024-09-18T04:57:11.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0101e-05 2024-09-18T04:57:11.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:11.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936986+0000", 2024-09-18T04:57:11.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040701213999999999 2024-09-18T04:57:11.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937070+0000", 2024-09-18T04:57:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4211999999999999e-05 2024-09-18T04:57:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937162+0000", 2024-09-18T04:57:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1728e-05 2024-09-18T04:57:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.046 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11406 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1708032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:11.046 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:57:11.046 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.095955463999999, 2024-09-18T04:57:11.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041085251000000003, 2024-09-18T04:57:11.046 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:11.046 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:11.046 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:11.046 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:11.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:11.048 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11406 2024-09-18T04:57:11.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.048 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:11.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:11.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:57:11.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:11.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:57:11.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:57:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:57:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896288+0000", 2024-09-18T04:57:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2440000000000001e-06 2024-09-18T04:57:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896291+0000", 2024-09-18T04:57:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8289999999999999e-06 2024-09-18T04:57:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896293+0000", 2024-09-18T04:57:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5399999999999998e-06 2024-09-18T04:57:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937201+0000", 2024-09-18T04:57:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040907418000000001 2024-09-18T04:57:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:11.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937283+0000", 2024-09-18T04:57:11.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2460999999999995e-05 2024-09-18T04:57:11.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:11.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937372+0000", 2024-09-18T04:57:11.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8987999999999996e-05 2024-09-18T04:57:11.051 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.051 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:11.051 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.051 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11407 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:11.051 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:57:11.052 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.095945643999997, 2024-09-18T04:57:11.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041438580000000003, 2024-09-18T04:57:11.052 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:11.052 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:11.052 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:11.052 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:11.052 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:11.052 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11407 2024-09-18T04:57:11.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.052 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:11.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:11.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:57:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:57:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896296+0000", 2024-09-18T04:57:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896298+0000", 2024-09-18T04:57:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:57:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896304+0000", 2024-09-18T04:57:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6570000000000002e-06 2024-09-18T04:57:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896308+0000", 2024-09-18T04:57:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0940000000000001e-06 2024-09-18T04:57:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937540+0000", 2024-09-18T04:57:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041231469 2024-09-18T04:57:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937625+0000", 2024-09-18T04:57:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5767999999999996e-05 2024-09-18T04:57:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937735+0000", 2024-09-18T04:57:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010974 2024-09-18T04:57:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11408 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:57:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.095930005, 2024-09-18T04:57:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041653055000000001, 2024-09-18T04:57:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11408 2024-09-18T04:57:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:57:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:57:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:57:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896314+0000", 2024-09-18T04:57:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0140000000000001e-06 2024-09-18T04:57:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896320+0000", 2024-09-18T04:57:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-18T04:57:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896324+0000", 2024-09-18T04:57:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4410000000000003e-06 2024-09-18T04:57:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937769+0000", 2024-09-18T04:57:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041445243 2024-09-18T04:57:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937861+0000", 2024-09-18T04:57:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1471000000000006e-05 2024-09-18T04:57:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:11.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937965+0000", 2024-09-18T04:57:11.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104516 2024-09-18T04:57:11.060 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.060 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:11.060 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.060 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11409 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:11.060 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:57:11.060 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.095913949, 2024-09-18T04:57:11.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041840467999999999, 2024-09-18T04:57:11.061 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:11.061 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:11.061 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:11.061 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:11.061 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:11.061 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11409 2024-09-18T04:57:11.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.061 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:11.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:11.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:57:11.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:57:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:57:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896331+0000", 2024-09-18T04:57:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5950000000000001e-06 2024-09-18T04:57:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896336+0000", 2024-09-18T04:57:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7259999999999999e-06 2024-09-18T04:57:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896341+0000", 2024-09-18T04:57:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5619999999999997e-06 2024-09-18T04:57:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937995+0000", 2024-09-18T04:57:11.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041654060999999999 2024-09-18T04:57:11.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:11.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938078+0000", 2024-09-18T04:57:11.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2869999999999998e-05 2024-09-18T04:57:11.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:11.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938169+0000", 2024-09-18T04:57:11.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1040999999999998e-05 2024-09-18T04:57:11.064 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.064 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:11.064 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.065 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11410 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1921024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:11.065 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:57:11.065 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.095897465999997, 2024-09-18T04:57:11.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042060342000000001, 2024-09-18T04:57:11.065 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:11.065 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:11.065 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:11.065 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:11.065 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:11.065 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11410 2024-09-18T04:57:11.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.066 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:11.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:11.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:57:11.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:11.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:57:11.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896344+0000", 2024-09-18T04:57:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896360+0000", 2024-09-18T04:57:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5068e-05 2024-09-18T04:57:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896366+0000", 2024-09-18T04:57:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5119999999999997e-06 2024-09-18T04:57:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896371+0000", 2024-09-18T04:57:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:57:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938198+0000", 2024-09-18T04:57:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041826579000000003 2024-09-18T04:57:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938297+0000", 2024-09-18T04:57:11.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9067000000000005e-05 2024-09-18T04:57:11.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:11.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938405+0000", 2024-09-18T04:57:11.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108492 2024-09-18T04:57:11.069 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.069 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:11.069 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.069 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11411 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1970176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:11.069 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:57:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.095866624000003, 2024-09-18T04:57:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042259191000000002, 2024-09-18T04:57:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11411 2024-09-18T04:57:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:57:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:57:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896375+0000", 2024-09-18T04:57:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:11.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896377+0000", 2024-09-18T04:57:11.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2699999999999999e-06 2024-09-18T04:57:11.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:11.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896383+0000", 2024-09-18T04:57:11.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6740000000000002e-06 2024-09-18T04:57:11.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:11.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896388+0000", 2024-09-18T04:57:11.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:57:11.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938435+0000", 2024-09-18T04:57:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042047169000000002 2024-09-18T04:57:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938534+0000", 2024-09-18T04:57:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8939999999999998e-05 2024-09-18T04:57:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938635+0000", 2024-09-18T04:57:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101208 2024-09-18T04:57:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11412 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:57:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.095850642000002, 2024-09-18T04:57:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042828079999999998, 2024-09-18T04:57:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11412 2024-09-18T04:57:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:57:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:57:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:11.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896391+0000", 2024-09-18T04:57:11.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:11.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:11.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896393+0000", 2024-09-18T04:57:11.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0810000000000001e-06 2024-09-18T04:57:11.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:11.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896399+0000", 2024-09-18T04:57:11.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7230000000000001e-06 2024-09-18T04:57:11.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:11.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896403+0000", 2024-09-18T04:57:11.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3470000000000001e-06 2024-09-18T04:57:11.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:11.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939043+0000", 2024-09-18T04:57:11.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042639829999999997 2024-09-18T04:57:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939124+0000", 2024-09-18T04:57:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0809000000000005e-05 2024-09-18T04:57:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939220+0000", 2024-09-18T04:57:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5659e-05 2024-09-18T04:57:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11413 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2068480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:57:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.095834947, 2024-09-18T04:57:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043042108000000003, 2024-09-18T04:57:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11413 2024-09-18T04:57:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:57:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:57:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:57:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896409+0000", 2024-09-18T04:57:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.165e-06 2024-09-18T04:57:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896415+0000", 2024-09-18T04:57:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6149999999999996e-06 2024-09-18T04:57:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896419+0000", 2024-09-18T04:57:11.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2649999999999998e-06 2024-09-18T04:57:11.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:11.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939262+0000", 2024-09-18T04:57:11.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042843444000000001 2024-09-18T04:57:11.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:11.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939351+0000", 2024-09-18T04:57:11.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8944999999999996e-05 2024-09-18T04:57:11.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:11.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939449+0000", 2024-09-18T04:57:11.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8029000000000006e-05 2024-09-18T04:57:11.083 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.083 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:11.083 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.083 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11669 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2023424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:11.083 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.907990+0000", 2024-09-18T04:57:11.083 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.084252053, 2024-09-18T04:57:11.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040045942000000001, 2024-09-18T04:57:11.083 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:11.083 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:11.084 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:11.084 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:11.084 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:11.084 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11669 2024-09-18T04:57:11.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.084 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:11.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:11.084 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.907990+0000", 2024-09-18T04:57:11.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:11.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.907990+0000", 2024-09-18T04:57:11.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:11.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.907989+0000", 2024-09-18T04:57:11.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:11.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:11.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.907993+0000", 2024-09-18T04:57:11.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6109999999999998e-06 2024-09-18T04:57:11.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:11.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908003+0000", 2024-09-18T04:57:11.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7610000000000002e-06 2024-09-18T04:57:11.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:11.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908009+0000", 2024-09-18T04:57:11.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7660000000000004e-06 2024-09-18T04:57:11.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:11.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.947929+0000", 2024-09-18T04:57:11.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039920297 2024-09-18T04:57:11.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:11.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.947982+0000", 2024-09-18T04:57:11.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3176e-05 2024-09-18T04:57:11.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:11.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.948036+0000", 2024-09-18T04:57:11.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4052999999999999e-05 2024-09-18T04:57:11.087 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.087 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:11.087 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.088 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11670 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2072576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:11.088 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.908014+0000", 2024-09-18T04:57:11.088 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.084228172000003, 2024-09-18T04:57:11.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040179074000000002, 2024-09-18T04:57:11.088 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:11.088 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:11.088 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:11.088 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:11.088 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:11.088 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11670 2024-09-18T04:57:11.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.088 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:11.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:11.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908014+0000", 2024-09-18T04:57:11.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:11.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908014+0000", 2024-09-18T04:57:11.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:11.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:11.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908014+0000", 2024-09-18T04:57:11.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:11.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:11.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908016+0000", 2024-09-18T04:57:11.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3889999999999999e-06 2024-09-18T04:57:11.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:11.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908023+0000", 2024-09-18T04:57:11.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5599999999999999e-06 2024-09-18T04:57:11.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:11.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908028+0000", 2024-09-18T04:57:11.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.242e-06 2024-09-18T04:57:11.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:11.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.948075+0000", 2024-09-18T04:57:11.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040047117 2024-09-18T04:57:11.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:11.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.948128+0000", 2024-09-18T04:57:11.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3127999999999997e-05 2024-09-18T04:57:11.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:11.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:11.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:11.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.948193+0000", 2024-09-18T04:57:11.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5042000000000005e-05 2024-09-18T04:57:11.092 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.092 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:11.092 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.092 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:11.092 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:11.092 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:11.093 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:57:11.094 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:11.091+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:11.094 INFO:teuthology.orchestra.run.smithi002.stdout:3384 writesame 0xb8bd701 thru 0xb8c343b data_size 0x1(0x5d3b bytes) 2024-09-18T04:57:11.094 INFO:teuthology.orchestra.run.smithi002.stdout:3385 write 0x1f126e2 thru 0x1f1bb00 (0x941f bytes) 2024-09-18T04:57:11.094 INFO:teuthology.orchestra.run.smithi002.stdout:3386 punch from 0xa8ec9dc to 0xa8fa593, (0xdbb7 bytes) 2024-09-18T04:57:11.095 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:57:11.098 INFO:teuthology.orchestra.run.smithi002.stdout:3387 trunc from 0xb8c343c to 0xb143a05 2024-09-18T04:57:11.098 INFO:teuthology.orchestra.run.smithi002.stdout:3388 read 0x8a31f71 thru 0x8a3703f (0x50cf bytes) 2024-09-18T04:57:11.098 INFO:teuthology.orchestra.run.smithi002.stdout:3389 trunc from 0xb143a05 to 0x2873472 2024-09-18T04:57:11.098 INFO:teuthology.orchestra.run.smithi002.stdout:3390 write 0x40a3a7 thru 0x40eb63 (0x47bd bytes) 2024-09-18T04:57:11.098 INFO:teuthology.orchestra.run.smithi002.stdout:3392 punch from 0x3e510 to 0x4043d, (0x1f2d bytes) 2024-09-18T04:57:11.098 INFO:teuthology.orchestra.run.smithi002.stdout:3393 trunc from 0x2873472 to 0x1fef41b 2024-09-18T04:57:11.098 INFO:teuthology.orchestra.run.smithi002.stdout:3395 write 0x8fc2dbf thru 0x8fc2fb8 (0x1fa bytes) 2024-09-18T04:57:11.098 INFO:teuthology.orchestra.run.smithi002.stdout:3396 trunc from 0x8fc2fb9 to 0xa72098b 2024-09-18T04:57:11.098 INFO:teuthology.orchestra.run.smithi002.stdout:3397 read 0x92d15d3 thru 0x92dde31 (0xc85f bytes) 2024-09-18T04:57:11.098 INFO:teuthology.orchestra.run.smithi002.stdout:3398 write 0x96a80b5 thru 0x96b1a6c (0x99b8 bytes) 2024-09-18T04:57:11.098 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:11.098 INFO:teuthology.orchestra.run.smithi002.stdout:3401 write 0x7cd38ee thru 0x7cdcd9f (0x94b2 bytes) 2024-09-18T04:57:11.099 INFO:teuthology.orchestra.run.smithi002.stdout:3402 write 0xab8259c thru 0xab8c39e (0x9e03 bytes) 2024-09-18T04:57:11.099 INFO:teuthology.orchestra.run.smithi002.stdout:3403 read 0x55a31fa thru 0x55b2110 (0xef17 bytes) 2024-09-18T04:57:11.099 INFO:teuthology.orchestra.run.smithi002.stdout:3404 read 0x7e41f86 thru 0x7e463c6 (0x4441 bytes) 2024-09-18T04:57:11.099 INFO:teuthology.orchestra.run.smithi002.stdout:3408 write 0x9b0c636 thru 0x9b1bdd9 (0xf7a4 bytes) 2024-09-18T04:57:11.099 INFO:teuthology.orchestra.run.smithi002.stdout:3410 write 0xae0ab24 thru 0xae17c29 (0xd106 bytes) 2024-09-18T04:57:11.099 INFO:teuthology.orchestra.run.smithi002.stdout:3411 write 0xf2d2aa thru 0xf32a6d (0x57c4 bytes) 2024-09-18T04:57:11.099 INFO:teuthology.orchestra.run.smithi002.stdout:3412 punch from 0x3f3b712 to 0x3f484c1, (0xcdaf bytes) 2024-09-18T04:57:11.099 INFO:teuthology.orchestra.run.smithi002.stdout:3413 writesame 0x1fdecdc thru 0x1fecba1 data_size 0xf(0xdec6 bytes) 2024-09-18T04:57:11.099 INFO:teuthology.orchestra.run.smithi002.stdout:3415 write 0x88a18e thru 0x88c80e (0x2681 bytes) 2024-09-18T04:57:11.099 INFO:teuthology.orchestra.run.smithi002.stdout:3417 write 0x75dc703 thru 0x75e8d19 (0xc617 bytes) 2024-09-18T04:57:11.100 INFO:teuthology.orchestra.run.smithi002.stdout:3418 write 0xa92346 thru 0xa949b6 (0x2671 bytes) 2024-09-18T04:57:11.100 INFO:teuthology.orchestra.run.smithi002.stdout:3419 writesame 0x2eddf56 thru 0x2ede093 data_size 0x1(0x13e bytes) 2024-09-18T04:57:11.100 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:11.107 INFO:teuthology.orchestra.run.smithi002.stdout:3423 write 0xed96582 thru 0xed9f5b1 (0x9030 bytes) 2024-09-18T04:57:11.110 INFO:teuthology.orchestra.run.smithi002.stdout:3424 read 0x4d2b761 thru 0x4d322c5 (0x6b65 bytes) 2024-09-18T04:57:11.110 INFO:teuthology.orchestra.run.smithi002.stdout:3425 read 0x8c14182 thru 0x8c1d9b8 (0x9837 bytes) 2024-09-18T04:57:11.110 INFO:teuthology.orchestra.run.smithi002.stdout:3426 read 0x5bdb08d thru 0x5bdd052 (0x1fc6 bytes) 2024-09-18T04:57:11.110 INFO:teuthology.orchestra.run.smithi002.stdout:3428 read 0x582b4d7 thru 0x5837cc7 (0xc7f1 bytes) 2024-09-18T04:57:11.110 INFO:teuthology.orchestra.run.smithi002.stdout:3430 read 0x28ea138 thru 0x28ed558 (0x3421 bytes) 2024-09-18T04:57:11.111 INFO:teuthology.orchestra.run.smithi002.stdout:3431 punch from 0x1831f44 to 0x183e2a3, (0xc35f bytes) 2024-09-18T04:57:11.122 INFO:teuthology.orchestra.run.smithi002.stdout:3432 trunc from 0xed9f5b2 to 0x3dd78d3 2024-09-18T04:57:11.146 INFO:teuthology.orchestra.run.smithi002.stdout:3433 write 0xa8f5bc4 thru 0xa8fd0e7 (0x7524 bytes) 2024-09-18T04:57:11.149 INFO:teuthology.orchestra.run.smithi002.stdout:3434 punch from 0x4af2595 to 0x4b01fa0, (0xfa0b bytes) 2024-09-18T04:57:11.149 INFO:teuthology.orchestra.run.smithi002.stdout:3436 punch from 0x719e85c to 0x71a7041, (0x87e5 bytes) 2024-09-18T04:57:11.150 INFO:teuthology.orchestra.run.smithi002.stdout:3437 trunc from 0xa8fd0e8 to 0x90a4e28 2024-09-18T04:57:11.160 INFO:teuthology.orchestra.run.smithi002.stdout:3442 read 0x74cd8b1 thru 0x74cdcba (0x40a bytes) 2024-09-18T04:57:11.160 INFO:teuthology.orchestra.run.smithi002.stdout:3445 read 0x4beb660 thru 0x4bf36c8 (0x8069 bytes) 2024-09-18T04:57:11.160 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:11.160 INFO:teuthology.orchestra.run.smithi002.stdout:3447 write 0x83d283 thru 0x8475a4 (0xa322 bytes) 2024-09-18T04:57:11.173 INFO:teuthology.orchestra.run.smithi002.stdout:3449 write 0xd206cca thru 0xd20f511 (0x8848 bytes) 2024-09-18T04:57:11.176 INFO:teuthology.orchestra.run.smithi002.stdout:3451 write 0x2e7af49 thru 0x2e84845 (0x98fd bytes) 2024-09-18T04:57:11.180 INFO:teuthology.orchestra.run.smithi002.stdout:3452 read 0x74a523e thru 0x74a8997 (0x375a bytes) 2024-09-18T04:57:11.180 INFO:teuthology.orchestra.run.smithi002.stdout:3453 write 0x2ad0622 thru 0x2ad99a4 (0x9383 bytes) 2024-09-18T04:57:11.182 INFO:teuthology.orchestra.run.smithi002.stdout:3454 write 0x625018 thru 0x627f50 (0x2f39 bytes) 2024-09-18T04:57:11.185 INFO:teuthology.orchestra.run.smithi002.stdout:3455 read 0xcac35fa thru 0xcacf6e1 (0xc0e8 bytes) 2024-09-18T04:57:11.185 INFO:teuthology.orchestra.run.smithi002.stdout:3456 writesame 0x7ec7338 thru 0x7ec77c8 data_size 0x1(0x491 bytes) 2024-09-18T04:57:11.188 INFO:teuthology.orchestra.run.smithi002.stdout:3457 write 0x6db2939 thru 0x6dbe00b (0xb6d3 bytes) 2024-09-18T04:57:11.194 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:11.192+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:11.197 INFO:teuthology.orchestra.run.smithi002.stdout:3458 write 0xdbb3882 thru 0xdbbbe86 (0x8605 bytes) 2024-09-18T04:57:11.201 INFO:teuthology.orchestra.run.smithi002.stdout:3459 punch from 0x47c7ea5 to 0x47cd394, (0x54ef bytes) 2024-09-18T04:57:11.201 INFO:teuthology.orchestra.run.smithi002.stdout:3460 write 0x5ef27cf thru 0x5ef7352 (0x4b84 bytes) 2024-09-18T04:57:11.203 INFO:teuthology.orchestra.run.smithi002.stdout:3465 read 0xa30cf46 thru 0xa31ce68 (0xff23 bytes) 2024-09-18T04:57:11.206 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:11.206 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:11.207 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:57:11.207 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:11.214 INFO:teuthology.orchestra.run.smithi002.stdout:3466 trunc from 0xdbbbe87 to 0x4936758 2024-09-18T04:57:11.216 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:57:11.227 INFO:teuthology.orchestra.run.smithi002.stdout:3467 punch from 0xd89b08 to 0xd8dbb9, (0x40b1 bytes) 2024-09-18T04:57:11.234 INFO:teuthology.orchestra.run.smithi002.stdout:3468 trunc from 0x4936758 to 0xafac875 2024-09-18T04:57:11.235 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:57:11.235 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:57:11.239 INFO:teuthology.orchestra.run.smithi002.stdout:3469 clone 30 order 24 su 2097152 sc 3 2024-09-18T04:57:11.253 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:57:11.253 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:57:11.268 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:57:11.268 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:57:11.284 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:57:11.284 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:57:11.301 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:57:11.302 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:57:11.302 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:11.299+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:11.317 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:57:11.317 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:57:11.333 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-18T04:57:11.335 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:57:11.335 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:57:11.353 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:57:11.354 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:57:11.365 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:11.365 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:11.374 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:57:11.398 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone29 from 0xafac875 (overlap 0x2a0b4b) to 0x43454 2024-09-18T04:57:11.399 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:11.395+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:11.479 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-09-18T04:57:11.497 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:11.495+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:11.522 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:11.522 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:11.531 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:57:11.595 INFO:teuthology.orchestra.run.smithi002.stdout:3472 read 0x3e1855f thru 0x3e1867d (0x11f bytes) 2024-09-18T04:57:11.595 INFO:teuthology.orchestra.run.smithi002.stdout:3473 read 0x7411799 thru 0x7418c1f (0x7487 bytes) 2024-09-18T04:57:11.595 INFO:teuthology.orchestra.run.smithi002.stdout:3474 punch from 0x464c3f1 to 0x4656047, (0x9c56 bytes) 2024-09-18T04:57:11.598 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:11.595+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:11.599 INFO:teuthology.orchestra.run.smithi002.stdout:3475 punch from 0x61ceb5d to 0x61d5da1, (0x7244 bytes) 2024-09-18T04:57:11.599 INFO:teuthology.orchestra.run.smithi002.stdout:3477 read 0x1f4972f thru 0x1f4b87e (0x2150 bytes) 2024-09-18T04:57:11.609 INFO:teuthology.orchestra.run.smithi002.stdout:3478 writesame 0xede7c2e thru 0xedf76cb data_size 0x6f(0xfa9e bytes) 2024-09-18T04:57:11.614 INFO:teuthology.orchestra.run.smithi002.stdout:3479 write 0xee19054 thru 0xee1b51b (0x24c8 bytes) 2024-09-18T04:57:11.618 INFO:teuthology.orchestra.run.smithi002.stdout:3480 write 0x934ce55 thru 0x93547df (0x798b bytes) 2024-09-18T04:57:11.623 INFO:teuthology.orchestra.run.smithi002.stdout:3481 write 0x5469090 thru 0x54740c7 (0xb038 bytes) 2024-09-18T04:57:11.626 INFO:teuthology.orchestra.run.smithi002.stdout:3483 write 0x673176f thru 0x673d723 (0xbfb5 bytes) 2024-09-18T04:57:11.629 INFO:teuthology.orchestra.run.smithi002.stdout:3484 writesame 0x448e130 thru 0x44986fa data_size 0x1(0xa5cb bytes) 2024-09-18T04:57:11.651 INFO:teuthology.orchestra.run.smithi002.stdout:3485 write 0xd241a64 thru 0xd24914a (0x76e7 bytes) 2024-09-18T04:57:11.654 INFO:teuthology.orchestra.run.smithi002.stdout:3486 writesame 0xb5a0e09 thru 0xb5ad496 data_size 0x82(0xc68e bytes) 2024-09-18T04:57:11.659 INFO:teuthology.orchestra.run.smithi002.stdout:3487 trunc from 0xee1b51c to 0xd1d7b93 2024-09-18T04:57:11.665 INFO:teuthology.orchestra.run.smithi002.stdout:3490 write 0x1203d8 thru 0x123676 (0x329f bytes) 2024-09-18T04:57:11.680 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:11.680 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:11.680 INFO:teuthology.orchestra.run.smithi002.stdout:3492 trunc from 0xd1d7b93 to 0xed8474d 2024-09-18T04:57:11.688 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:57:11.692 INFO:teuthology.orchestra.run.smithi002.stdout:3495 trunc from 0xed8474d to 0x7b6a465 2024-09-18T04:57:11.706 INFO:teuthology.orchestra.run.smithi002.stdout:3496 punch from 0x2d6b19e to 0x2d6bb85, (0x9e7 bytes) 2024-09-18T04:57:11.707 INFO:teuthology.orchestra.run.smithi002.stdout:3498 read 0x6ea4076 thru 0x6ea436b (0x2f6 bytes) 2024-09-18T04:57:11.707 INFO:teuthology.orchestra.run.smithi002.stdout:3499 writesame 0xd80903 thru 0xd9083f data_size 0x13(0xff3d bytes) 2024-09-18T04:57:11.709 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:11.718 INFO:teuthology.orchestra.run.smithi002.stdout:3504 writesame 0xb0dc461 thru 0xb0e992e data_size 0x1(0xd4ce bytes) 2024-09-18T04:57:11.719 INFO:teuthology.orchestra.run.smithi002.stdout:3506 punch from 0x5e48fcb to 0x5e57fcb, (0xf000 bytes) 2024-09-18T04:57:11.729 INFO:teuthology.orchestra.run.smithi002.stdout:3507 trunc from 0xb0e992f to 0x1d7092e 2024-09-18T04:57:11.732 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:11.730+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:11.746 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:11.746 INFO:teuthology.orchestra.run.smithi002.stdout:3510 punch from 0x11e015a to 0x11ec320, (0xc1c6 bytes) 2024-09-18T04:57:11.746 INFO:teuthology.orchestra.run.smithi002.stdout:3511 read 0x102268 thru 0x104fe8 (0x2d81 bytes) 2024-09-18T04:57:11.762 INFO:teuthology.orchestra.run.smithi002.stdout:3512 writesame 0xb2ef716 thru 0xb2f2fb9 data_size 0x32(0x38a4 bytes) 2024-09-18T04:57:11.768 INFO:teuthology.orchestra.run.smithi002.stdout:3513 trunc from 0xb2f2fba to 0x65d9af5 2024-09-18T04:57:11.773 INFO:teuthology.orchestra.run.smithi002.stdout:3514 punch from 0x35354f3 to 0x3541b44, (0xc651 bytes) 2024-09-18T04:57:11.773 INFO:teuthology.orchestra.run.smithi002.stdout:3516 read 0x52f10a0 thru 0x52ff34e (0xe2af bytes) 2024-09-18T04:57:11.773 INFO:teuthology.orchestra.run.smithi002.stdout:3519 writesame 0x74e5ad thru 0x74f903 data_size 0x1(0x1357 bytes) 2024-09-18T04:57:11.789 INFO:teuthology.orchestra.run.smithi002.stdout:3520 write 0xd3473a6 thru 0xd34d2b8 (0x5f13 bytes) 2024-09-18T04:57:11.793 INFO:teuthology.orchestra.run.smithi002.stdout:3521 read 0x7330173 thru 0x73359ba (0x5848 bytes) 2024-09-18T04:57:11.793 INFO:teuthology.orchestra.run.smithi002.stdout:3523 read 0x92d4fa8 thru 0x92e3a87 (0xeae0 bytes) 2024-09-18T04:57:11.793 INFO:teuthology.orchestra.run.smithi002.stdout:3524 trunc from 0xd34d2b9 to 0xc976cab 2024-09-18T04:57:11.797 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:11.797 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:11.799 INFO:teuthology.orchestra.run.smithi002.stdout:3525 writesame 0x8ded1de thru 0x8deeb44 data_size 0x1(0x1967 bytes) 2024-09-18T04:57:11.799 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:11.797+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:11.803 INFO:teuthology.orchestra.run.smithi002.stdout:3526 writesame 0x9af9365 thru 0x9b02d16 data_size 0x2(0x99b2 bytes) 2024-09-18T04:57:11.807 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:57:11.814 INFO:teuthology.orchestra.run.smithi002.stdout:3527 read 0xb1b779a thru 0xb1bc8fb (0x5162 bytes) 2024-09-18T04:57:11.814 INFO:teuthology.orchestra.run.smithi002.stdout:3528 read 0xb0be548 thru 0xb0be835 (0x2ee bytes) 2024-09-18T04:57:11.816 INFO:teuthology.orchestra.run.smithi002.stdout:3529 trunc from 0xc976cab to 0xe537a9b 2024-09-18T04:57:11.821 INFO:teuthology.orchestra.run.smithi002.stdout:3530 writesame 0xc52516a thru 0xc5257d6 data_size 0x7(0x66d bytes) 2024-09-18T04:57:11.824 INFO:teuthology.orchestra.run.smithi002.stdout:3531 read 0x7790400 thru 0x7792a9a (0x269b bytes) 2024-09-18T04:57:11.825 INFO:teuthology.orchestra.run.smithi002.stdout:3533 write 0x20cbf18 thru 0x20cf624 (0x370d bytes) 2024-09-18T04:57:11.826 INFO:teuthology.orchestra.run.smithi002.stdout:3534 writesame 0x60c4944 thru 0x60cd116 data_size 0xb(0x87d3 bytes) 2024-09-18T04:57:11.830 INFO:teuthology.orchestra.run.smithi002.stdout:3535 write 0x32f2de9 thru 0x32fef53 (0xc16b bytes) 2024-09-18T04:57:11.832 INFO:teuthology.orchestra.run.smithi002.stdout:3536 read 0xc188a1c thru 0xc19349d (0xaa82 bytes) 2024-09-18T04:57:11.834 INFO:teuthology.orchestra.run.smithi002.stdout:3537 trunc from 0xe537a9b to 0xdc91ad1 2024-09-18T04:57:11.839 INFO:teuthology.orchestra.run.smithi002.stdout:3538 writesame 0x30d5565 thru 0x30db281 data_size 0xb(0x5d1d bytes) 2024-09-18T04:57:11.842 INFO:teuthology.orchestra.run.smithi002.stdout:3539 punch from 0x2498c0 to 0x24b9b2, (0x20f2 bytes) 2024-09-18T04:57:11.856 INFO:teuthology.orchestra.run.smithi002.stdout:3540 trunc from 0xdc91ad1 to 0x1a7cfb4 2024-09-18T04:57:11.886 INFO:teuthology.orchestra.run.smithi002.stdout:3541 writesame 0xd971a62 thru 0xd97a4fb data_size 0x71(0x8a9a bytes) 2024-09-18T04:57:11.886 INFO:teuthology.orchestra.run.smithi002.stdout:3542 write 0xb6afab3 thru 0xb6bafcc (0xb51a bytes) 2024-09-18T04:57:11.888 INFO:teuthology.orchestra.run.smithi002.stdout:3544 punch from 0x498fa7a to 0x499ea2f, (0xefb5 bytes) 2024-09-18T04:57:11.889 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:11.889 INFO:teuthology.orchestra.run.smithi002.stdout:3546 write 0x591c0f9 thru 0x5929965 (0xd86d bytes) 2024-09-18T04:57:11.892 INFO:teuthology.orchestra.run.smithi002.stdout:3547 write 0x4b71e37 thru 0x4b7b1c9 (0x9393 bytes) 2024-09-18T04:57:11.895 INFO:teuthology.orchestra.run.smithi002.stdout:3548 punch from 0x441bac8 to 0x4429368, (0xd8a0 bytes) 2024-09-18T04:57:11.895 INFO:teuthology.orchestra.run.smithi002.stdout:3549 write 0xa549100 thru 0xa54edeb (0x5cec bytes) 2024-09-18T04:57:11.896 INFO:teuthology.orchestra.run.smithi002.stdout:3550 punch from 0xa1ac6fe to 0xa1b57df, (0x90e1 bytes) 2024-09-18T04:57:11.900 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:11.897+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:11.903 INFO:teuthology.orchestra.run.smithi002.stdout:3552 writesame 0xed49e7c thru 0xed55753 data_size 0x46(0xb8d8 bytes) 2024-09-18T04:57:11.903 INFO:teuthology.orchestra.run.smithi002.stdout:3553 read 0x4be7ffa thru 0x4bf4abe (0xcac5 bytes) 2024-09-18T04:57:11.903 INFO:teuthology.orchestra.run.smithi002.stdout:3554 read 0xd5342b2 thru 0xd53d481 (0x91d0 bytes) 2024-09-18T04:57:11.904 INFO:teuthology.orchestra.run.smithi002.stdout:3555 write 0x54a7d0a thru 0x54b5203 (0xd4fa bytes) 2024-09-18T04:57:11.906 INFO:teuthology.orchestra.run.smithi002.stdout:3556 write 0xd00cc2a thru 0xd01685a (0x9c31 bytes) 2024-09-18T04:57:11.909 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:11.909 INFO:teuthology.orchestra.run.smithi002.stdout:3559 read 0x40cae2b thru 0x40d3766 (0x893c bytes) 2024-09-18T04:57:11.909 INFO:teuthology.orchestra.run.smithi002.stdout:3560 write 0x478b469 thru 0x4794804 (0x939c bytes) 2024-09-18T04:57:11.911 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:11.923 INFO:teuthology.orchestra.run.smithi002.stdout:3562 trunc from 0xed55754 to 0x445374c 2024-09-18T04:57:11.933 INFO:teuthology.orchestra.run.smithi002.stdout:3563 read 0x43481f7 thru 0x434f70c (0x7516 bytes) 2024-09-18T04:57:11.942 INFO:teuthology.orchestra.run.smithi002.stdout:3565 write 0x6ca5e12 thru 0x6caf25a (0x9449 bytes) 2024-09-18T04:57:11.945 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:11.945 INFO:teuthology.orchestra.run.smithi002.stdout:3567 read 0x5e6c72 thru 0x5f0a4e (0x9ddd bytes) 2024-09-18T04:57:11.945 INFO:teuthology.orchestra.run.smithi002.stdout:3568 writesame 0x14baec5 thru 0x14be76d data_size 0x5(0x38a9 bytes) 2024-09-18T04:57:11.948 INFO:teuthology.orchestra.run.smithi002.stdout:3569 write 0x2d8518c thru 0x2d8e6ed (0x9562 bytes) 2024-09-18T04:57:11.949 INFO:teuthology.orchestra.run.smithi002.stdout:3570 writesame 0x1c59536 thru 0x1c60875 data_size 0x20(0x7340 bytes) 2024-09-18T04:57:11.951 INFO:teuthology.orchestra.run.smithi002.stdout:3571 read 0x2ce76e8 thru 0x2ce871a (0x1033 bytes) 2024-09-18T04:57:11.951 INFO:teuthology.orchestra.run.smithi002.stdout:3572 writesame 0x2ff6843 thru 0x3003904 data_size 0x9(0xd0c2 bytes) 2024-09-18T04:57:11.954 INFO:teuthology.orchestra.run.smithi002.stdout:3573 read 0x368bf60 thru 0x369b603 (0xf6a4 bytes) 2024-09-18T04:57:11.955 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:11.955 INFO:teuthology.orchestra.run.smithi002.stdout:3575 read 0xc14d60 thru 0xc22558 (0xd7f9 bytes) 2024-09-18T04:57:11.955 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:11.956 INFO:teuthology.orchestra.run.smithi002.stdout:3578 read 0x62a037 thru 0x638557 (0xe521 bytes) 2024-09-18T04:57:11.957 INFO:teuthology.orchestra.run.smithi002.stdout:3579 read 0x5428dfe thru 0x5438703 (0xf906 bytes) 2024-09-18T04:57:11.957 INFO:teuthology.orchestra.run.smithi002.stdout:3580 writesame 0x226c9b1 thru 0x226d7f4 data_size 0x16(0xe44 bytes) 2024-09-18T04:57:11.958 INFO:teuthology.orchestra.run.smithi002.stdout:3581 punch from 0x243a7b5 to 0x2447b83, (0xd3ce bytes) 2024-09-18T04:57:11.958 INFO:teuthology.orchestra.run.smithi002.stdout:3582 read 0x61d3118 thru 0x61dda72 (0xa95b bytes) 2024-09-18T04:57:11.958 INFO:teuthology.orchestra.run.smithi002.stdout:3583 punch from 0xa04779 to 0xa0a246, (0x5acd bytes) 2024-09-18T04:57:11.959 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:11.959 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:11.968 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:57:11.969 INFO:teuthology.orchestra.run.smithi002.stdout:3584 write 0xa8c9ce6 thru 0xa8cdb97 (0x3eb2 bytes) 2024-09-18T04:57:11.971 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:11.971 INFO:teuthology.orchestra.run.smithi002.stdout:3587 punch from 0x9de800 to 0x9dead9, (0x2d9 bytes) 2024-09-18T04:57:11.971 INFO:teuthology.orchestra.run.smithi002.stdout:3588 read 0x9b3976d thru 0x9b419fa (0x828e bytes) 2024-09-18T04:57:11.972 INFO:teuthology.orchestra.run.smithi002.stdout:3589 writesame 0xa60c6ac thru 0xa6161ca data_size 0x5d(0x9b1f bytes) 2024-09-18T04:57:11.977 INFO:teuthology.orchestra.run.smithi002.stdout:3590 trunc from 0xa8cdb98 to 0x770f2f0 2024-09-18T04:57:11.988 INFO:teuthology.orchestra.run.smithi002.stdout:3591 write 0x2eb7ae1 thru 0x2ec7a8c (0xffac bytes) 2024-09-18T04:57:11.989 INFO:teuthology.orchestra.run.smithi002.stdout:3592 read 0x2af161d thru 0x2af343f (0x1e23 bytes) 2024-09-18T04:57:12.000 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:11.997+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:12.003 INFO:teuthology.orchestra.run.smithi002.stdout:3593 write 0xdf13597 thru 0xdf1c877 (0x92e1 bytes) 2024-09-18T04:57:12.006 INFO:teuthology.orchestra.run.smithi002.stdout:3596 clone 31 order 23 su 4194304 sc 11 2024-09-18T04:57:12.071 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:12.072 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:12.080 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:57:12.101 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:12.098+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:12.182 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:12.182 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:12.191 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:57:12.234 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:12.232+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:12.294 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:12.294 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:12.302 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:57:12.303 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:12.299+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:12.403 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:12.401+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:12.404 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone30 from 0xdf1c878 (overlap 0x1a7cfb4) to 0x128aafe 2024-09-18T04:57:12.450 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:12.450 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:12.459 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:57:12.494 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-09-18T04:57:12.496 INFO:teuthology.orchestra.run.smithi002.stdout:3597 punch from 0x2d836f4 to 0x2d84082, (0x98e bytes) 2024-09-18T04:57:12.499 INFO:teuthology.orchestra.run.smithi002.stdout:3598 writesame 0x88c8f50 thru 0x88d3027 data_size 0x4(0xa0d8 bytes) 2024-09-18T04:57:12.508 INFO:teuthology.orchestra.run.smithi002.stdout:3599 punch from 0xbade03f to 0xbae225b, (0x421c bytes) 2024-09-18T04:57:12.508 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:12.508 INFO:teuthology.orchestra.run.smithi002.stdout:3601 punch from 0x28b3dbb to 0x28b681b, (0x2a60 bytes) 2024-09-18T04:57:12.508 INFO:teuthology.orchestra.run.smithi002.stdout:3602 read 0x4958564 thru 0x49633b4 (0xae51 bytes) 2024-09-18T04:57:12.508 INFO:teuthology.orchestra.run.smithi002.stdout:3603 read 0xbc1005b thru 0xbc1d1ae (0xd154 bytes) 2024-09-18T04:57:12.508 INFO:teuthology.orchestra.run.smithi002.stdout:3604 read 0x47d791a thru 0x47e4412 (0xcaf9 bytes) 2024-09-18T04:57:12.509 INFO:teuthology.orchestra.run.smithi002.stdout:3605 trunc from 0xdf1c878 to 0xe2fc719 2024-09-18T04:57:12.529 INFO:teuthology.orchestra.run.smithi002.stdout:3606 trunc from 0xe2fc719 to 0x1a8dd6b 2024-09-18T04:57:12.542 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:12.540+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:12.557 INFO:teuthology.orchestra.run.smithi002.stdout:3607 trunc from 0x1a8dd6b to 0x207fc0d 2024-09-18T04:57:12.564 INFO:teuthology.orchestra.run.smithi002.stdout:3608 flatten 2024-09-18T04:57:12.569 INFO:teuthology.orchestra.run.smithi002.stdout:3609 punch from 0x19abb60 to 0x19b56a6, (0x9b46 bytes) 2024-09-18T04:57:12.580 INFO:teuthology.orchestra.run.smithi002.stdout:3610 write 0x617d318 thru 0x6184d9f (0x7a88 bytes) 2024-09-18T04:57:12.583 INFO:teuthology.orchestra.run.smithi002.stdout:3611 write 0x552b11c thru 0x55369f9 (0xb8de bytes) 2024-09-18T04:57:12.587 INFO:teuthology.orchestra.run.smithi002.stdout:3612 punch from 0x2a31f8b to 0x2a3cd18, (0xad8d bytes) 2024-09-18T04:57:12.587 INFO:teuthology.orchestra.run.smithi002.stdout:3614 read 0x5fe6398 thru 0x5ff3796 (0xd3ff bytes) 2024-09-18T04:57:12.597 INFO:teuthology.orchestra.run.smithi002.stdout:3615 write 0x97246d0 thru 0x97319e7 (0xd318 bytes) 2024-09-18T04:57:12.601 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:12.601 INFO:teuthology.orchestra.run.smithi002.stdout:3618 read 0x7d5d4f7 thru 0x7d67d8f (0xa899 bytes) 2024-09-18T04:57:12.601 INFO:teuthology.orchestra.run.smithi002.stdout:3620 write 0x6eb921 thru 0x6f8586 (0xcc66 bytes) 2024-09-18T04:57:12.603 INFO:teuthology.orchestra.run.smithi002.stdout:3622 read 0x3b1464e thru 0x3b1da43 (0x93f6 bytes) 2024-09-18T04:57:12.604 INFO:teuthology.orchestra.run.smithi002.stdout:3623 trunc from 0x97319e8 to 0x8fe109d 2024-09-18T04:57:12.604 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:12.601+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:12.611 INFO:teuthology.orchestra.run.smithi002.stdout:3625 read 0x4840165 thru 0x484088f (0x72b bytes) 2024-09-18T04:57:12.611 INFO:teuthology.orchestra.run.smithi002.stdout:3626 read 0x6f5b35e thru 0x6f6455e (0x9201 bytes) 2024-09-18T04:57:12.612 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:12.612 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:12.621 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:57:12.624 INFO:teuthology.orchestra.run.smithi002.stdout:3627 writesame 0xe576a72 thru 0xe57bfa0 data_size 0x3(0x552f bytes) 2024-09-18T04:57:12.624 INFO:teuthology.orchestra.run.smithi002.stdout:3628 punch from 0x7e9fc05 to 0x7ead5e1, (0xd9dc bytes) 2024-09-18T04:57:12.624 INFO:teuthology.orchestra.run.smithi002.stdout:3629 punch from 0xd73de36 to 0xd7412bd, (0x3487 bytes) 2024-09-18T04:57:12.624 INFO:teuthology.orchestra.run.smithi002.stdout:3630 writesame 0x133acb3 thru 0x13476ab data_size 0x9(0xc9f9 bytes) 2024-09-18T04:57:12.642 INFO:teuthology.orchestra.run.smithi002.stdout:3631 trunc from 0xe57bfa1 to 0x15d95fe 2024-09-18T04:57:12.667 INFO:teuthology.orchestra.run.smithi002.stdout:3633 write 0x616e0c8 thru 0x617a691 (0xc5ca bytes) 2024-09-18T04:57:12.670 INFO:teuthology.orchestra.run.smithi002.stdout:3635 read 0x5f41733 thru 0x5f46f4b (0x5819 bytes) 2024-09-18T04:57:12.681 INFO:teuthology.orchestra.run.smithi002.stdout:3637 write 0x9ada257 thru 0x9ae3803 (0x95ad bytes) 2024-09-18T04:57:12.682 INFO:teuthology.orchestra.run.smithi002.stdout:3638 punch from 0x7369444 to 0x736ebcc, (0x5788 bytes) 2024-09-18T04:57:12.693 INFO:teuthology.orchestra.run.smithi002.stdout:3641 writesame 0xdc86ce0 thru 0xdc93745 data_size 0x7(0xca66 bytes) 2024-09-18T04:57:12.693 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:12.693 INFO:teuthology.orchestra.run.smithi002.stdout:3643 punch from 0x33c3b74 to 0x33d29a7, (0xee33 bytes) 2024-09-18T04:57:12.693 INFO:teuthology.orchestra.run.smithi002.stdout:3645 read 0x38d1089 thru 0x38d8fe5 (0x7f5d bytes) 2024-09-18T04:57:12.701 INFO:teuthology.orchestra.run.smithi002.stdout:3647 write 0xeb822fa thru 0xeb90f8c (0xec93 bytes) 2024-09-18T04:57:12.705 INFO:teuthology.orchestra.run.smithi002.stdout:3649 read 0x817d907 thru 0x8188ed8 (0xb5d2 bytes) 2024-09-18T04:57:12.705 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:12.702+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:12.711 INFO:teuthology.orchestra.run.smithi002.stdout:3650 trunc from 0xeb90f8d to 0x9855b80 2024-09-18T04:57:12.737 INFO:teuthology.orchestra.run.smithi002.stdout:3651 trunc from 0x9855b80 to 0xef5dc1 2024-09-18T04:57:12.771 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:12.771 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:12.774 INFO:teuthology.orchestra.run.smithi002.stdout:3652 write 0xa7ba29f thru 0xa7c1a2d (0x778f bytes) 2024-09-18T04:57:12.777 INFO:teuthology.orchestra.run.smithi002.stdout:3653 read 0x87ebff4 thru 0x87ed8ee (0x18fb bytes) 2024-09-18T04:57:12.777 INFO:teuthology.orchestra.run.smithi002.stdout:3654 read 0x50cbdb6 thru 0x50d371f (0x796a bytes) 2024-09-18T04:57:12.780 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:57:12.785 INFO:teuthology.orchestra.run.smithi002.stdout:3655 trunc from 0xa7c1a2e to 0x2d9ca5c 2024-09-18T04:57:12.806 INFO:teuthology.orchestra.run.smithi002.stdout:3656 punch from 0x1282f27 to 0x12906a9, (0xd782 bytes) 2024-09-18T04:57:12.806 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:12.806 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:12.806 INFO:teuthology.orchestra.run.smithi002.stdout:3659 read 0x743524 thru 0x744379 (0xe56 bytes) 2024-09-18T04:57:12.807 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:12.807 INFO:teuthology.orchestra.run.smithi002.stdout:3662 punch from 0x20866e6 to 0x208fb29, (0x9443 bytes) 2024-09-18T04:57:12.807 INFO:teuthology.orchestra.run.smithi002.stdout:3663 read 0x86d33d thru 0x879dc5 (0xca89 bytes) 2024-09-18T04:57:12.807 INFO:teuthology.orchestra.run.smithi002.stdout:3664 read 0x136d841 thru 0x1379c36 (0xc3f6 bytes) 2024-09-18T04:57:12.807 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:12.808 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:12.805+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:12.819 INFO:teuthology.orchestra.run.smithi002.stdout:3666 writesame 0x9790b34 thru 0x979d212 data_size 0x7(0xc6df bytes) 2024-09-18T04:57:12.819 INFO:teuthology.orchestra.run.smithi002.stdout:3667 punch from 0x1572d09 to 0x1577310, (0x4607 bytes) 2024-09-18T04:57:12.830 INFO:teuthology.orchestra.run.smithi002.stdout:3668 writesame 0xd8d929a thru 0xd8e7237 data_size 0x3(0xdf9e bytes) 2024-09-18T04:57:12.830 INFO:teuthology.orchestra.run.smithi002.stdout:3669 read 0xcee3ac4 thru 0xceec039 (0x8576 bytes) 2024-09-18T04:57:12.831 INFO:teuthology.orchestra.run.smithi002.stdout:3670 write 0x13fe6e3 thru 0x14053d6 (0x6cf4 bytes) 2024-09-18T04:57:12.840 INFO:teuthology.orchestra.run.smithi002.stdout:3671 writesame 0x6f8c321 thru 0x6f903ed data_size 0x1(0x40cd bytes) 2024-09-18T04:57:12.850 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:12.850 INFO:teuthology.orchestra.run.smithi002.stdout:3674 read 0x53e9408 thru 0x53eddeb (0x49e4 bytes) 2024-09-18T04:57:12.851 INFO:teuthology.orchestra.run.smithi002.stdout:3675 writesame 0xad99f58 thru 0xada81e0 data_size 0xd(0xe289 bytes) 2024-09-18T04:57:12.855 INFO:teuthology.orchestra.run.smithi002.stdout:3676 read 0x894d2ff thru 0x895a3e6 (0xd0e8 bytes) 2024-09-18T04:57:12.855 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:12.868 INFO:teuthology.orchestra.run.smithi002.stdout:3678 trunc from 0xd8e7238 to 0x14184a0 2024-09-18T04:57:12.888 INFO:teuthology.orchestra.run.smithi002.stdout:3679 read 0xd5a586 thru 0xd5a673 (0xee bytes) 2024-09-18T04:57:12.898 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:12.898 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:12.908 INFO:teuthology.orchestra.run.smithi002.stdout:3680 write 0xced92af thru 0xcedb7c2 (0x2514 bytes) 2024-09-18T04:57:12.909 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:12.906+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:12.910 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:57:12.913 INFO:teuthology.orchestra.run.smithi002.stdout:3682 trunc from 0xcedb7c3 to 0xd8133cd 2024-09-18T04:57:12.920 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:12.921 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:12.921 INFO:teuthology.orchestra.run.smithi002.stdout:3689 read 0x206bb31 thru 0x2073c58 (0x8128 bytes) 2024-09-18T04:57:12.921 INFO:teuthology.orchestra.run.smithi002.stdout:3692 read 0x62a220b thru 0x62af257 (0xd04d bytes) 2024-09-18T04:57:12.921 INFO:teuthology.orchestra.run.smithi002.stdout:3693 write 0x64bf9e9 thru 0x64c15be (0x1bd6 bytes) 2024-09-18T04:57:12.930 INFO:teuthology.orchestra.run.smithi002.stdout:3694 trunc from 0xd8133cd to 0x8b68e03 2024-09-18T04:57:12.947 INFO:teuthology.orchestra.run.smithi002.stdout:3695 read 0x1dbfa3a thru 0x1dc0898 (0xe5f bytes) 2024-09-18T04:57:12.956 INFO:teuthology.orchestra.run.smithi002.stdout:3697 write 0xad4de8f thru 0xad5c766 (0xe8d8 bytes) 2024-09-18T04:57:12.959 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:12.959 INFO:teuthology.orchestra.run.smithi002.stdout:3699 writesame 0x21cd7d thru 0x2231a8 data_size 0x6(0x642c bytes) 2024-09-18T04:57:12.961 INFO:teuthology.orchestra.run.smithi002.stdout:3702 read 0x6dfce4e thru 0x6e00159 (0x330c bytes) 2024-09-18T04:57:12.962 INFO:teuthology.orchestra.run.smithi002.stdout:3703 write 0x11e4ea0 thru 0x11ed51b (0x867c bytes) 2024-09-18T04:57:12.964 INFO:teuthology.orchestra.run.smithi002.stdout:3704 read 0xa190a16 thru 0xa1974d1 (0x6abc bytes) 2024-09-18T04:57:12.964 INFO:teuthology.orchestra.run.smithi002.stdout:3705 punch from 0x32b3bf to 0x338d35, (0xd976 bytes) 2024-09-18T04:57:12.964 INFO:teuthology.orchestra.run.smithi002.stdout:3707 clone 32 order 19 su 32768 sc 15 2024-09-18T04:57:13.010 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:13.007+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:13.022 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:13.022 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:13.031 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:57:13.115 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:13.112+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:13.178 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:13.178 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:13.186 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:57:13.227 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:13.224+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:13.312 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:13.309+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:13.333 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:13.333 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:13.343 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:57:13.347 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T04:57:13.366 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:57:13.414 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:13.412+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:13.507 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:13.507 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:13.514 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:13.512+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:13.517 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:57:13.522 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:57:13.522 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:57:13.531 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-09-18T04:57:13.542 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:57:13.542 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:57:13.559 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:57:13.559 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:57:13.567 INFO:teuthology.orchestra.run.smithi002.stdout:3708 read 0x5ca9be0 thru 0x5cb1043 (0x7464 bytes) 2024-09-18T04:57:13.568 INFO:teuthology.orchestra.run.smithi002.stdout:3709 read 0x48ff88 thru 0x49ed95 (0xee0e bytes) 2024-09-18T04:57:13.569 INFO:teuthology.orchestra.run.smithi002.stdout:3711 read 0x174e7c3 thru 0x1751e32 (0x3670 bytes) 2024-09-18T04:57:13.570 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:13.570 INFO:teuthology.orchestra.run.smithi002.stdout:3715 read 0x1857664 thru 0x18585c2 (0xf5f bytes) 2024-09-18T04:57:13.570 INFO:teuthology.orchestra.run.smithi002.stdout:3716 writesame 0x3ab1a6 thru 0x3ac572 data_size 0x1(0x13cd bytes) 2024-09-18T04:57:13.579 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:57:13.579 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:57:13.585 INFO:teuthology.orchestra.run.smithi002.stdout:3719 punch from 0x1b0af07 to 0x1b111e7, (0x62e0 bytes) 2024-09-18T04:57:13.593 INFO:teuthology.orchestra.run.smithi002.stdout:3720 write 0xc5e8364 thru 0xc5edf39 (0x5bd6 bytes) 2024-09-18T04:57:13.601 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:57:13.601 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:57:13.605 INFO:teuthology.orchestra.run.smithi002.stdout:3721 writesame 0xe333825 thru 0xe340b1d data_size 0x33(0xd2f9 bytes) 2024-09-18T04:57:13.605 INFO:teuthology.orchestra.run.smithi002.stdout:3722 write 0xc5b1269 thru 0xc5b5aaf (0x4847 bytes) 2024-09-18T04:57:13.608 INFO:teuthology.orchestra.run.smithi002.stdout:3725 writesame 0xa52bcbc thru 0xa52c5d4 data_size 0x1(0x919 bytes) 2024-09-18T04:57:13.613 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:13.613 INFO:teuthology.orchestra.run.smithi002.stdout:3727 write 0x9af3f42 thru 0x9af4f99 (0x1058 bytes) 2024-09-18T04:57:13.616 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:13.613+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:13.616 INFO:teuthology.orchestra.run.smithi002.stdout:3728 read 0xe08074e thru 0xe08fc16 (0xf4c9 bytes) 2024-09-18T04:57:13.617 INFO:teuthology.orchestra.run.smithi002.stdout:3729 writesame 0x510fb6f thru 0x511e4bc data_size 0x1(0xe94e bytes) 2024-09-18T04:57:13.621 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:57:13.621 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:57:13.638 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:57:13.638 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:57:13.641 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:13.641 INFO:teuthology.orchestra.run.smithi002.stdout:3733 read 0x797f781 thru 0x798d513 (0xdd93 bytes) 2024-09-18T04:57:13.641 INFO:teuthology.orchestra.run.smithi002.stdout:3734 write 0x2c4ad59 thru 0x2c59262 (0xe50a bytes) 2024-09-18T04:57:13.642 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:13.642 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:13.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:13.643 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:13.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.643 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14125 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:13.643 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:57:13.643 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.512245558000004, 2024-09-18T04:57:13.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052956470999999998, 2024-09-18T04:57:13.643 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:13.643 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:13.643 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:13.643 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:13.643 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:13.643 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14125 2024-09-18T04:57:13.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:57:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:57:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:13.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126849+0000", 2024-09-18T04:57:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126852+0000", 2024-09-18T04:57:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3659999999999999e-06 2024-09-18T04:57:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126858+0000", 2024-09-18T04:57:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4540000000000004e-06 2024-09-18T04:57:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126863+0000", 2024-09-18T04:57:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1780000000000002e-06 2024-09-18T04:57:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179608+0000", 2024-09-18T04:57:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052745159999999999 2024-09-18T04:57:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179703+0000", 2024-09-18T04:57:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4773999999999998e-05 2024-09-18T04:57:13.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179806+0000", 2024-09-18T04:57:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010296799999999999 2024-09-18T04:57:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14136 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1732608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.510085007000001, 2024-09-18T04:57:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052973412999999997, 2024-09-18T04:57:13.647 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:13.648 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:13.648 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:13.648 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:13.648 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:13.648 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14136 2024-09-18T04:57:13.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.648 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:13.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:13.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:13.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129013+0000", 2024-09-18T04:57:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.96e-06 2024-09-18T04:57:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129020+0000", 2024-09-18T04:57:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6809999999999998e-06 2024-09-18T04:57:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129026+0000", 2024-09-18T04:57:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:57:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181873+0000", 2024-09-18T04:57:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052847238999999997 2024-09-18T04:57:13.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:13.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181932+0000", 2024-09-18T04:57:13.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9339000000000002e-05 2024-09-18T04:57:13.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:13.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181984+0000", 2024-09-18T04:57:13.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1348999999999999e-05 2024-09-18T04:57:13.652 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.652 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:13.652 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.652 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14137 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1781760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:13.652 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:57:13.652 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.510064602, 2024-09-18T04:57:13.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053128914999999999, 2024-09-18T04:57:13.652 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:13.652 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:13.652 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14137 2024-09-18T04:57:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:57:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:57:13.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129030+0000", 2024-09-18T04:57:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129033+0000", 2024-09-18T04:57:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7439999999999999e-06 2024-09-18T04:57:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:13.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129039+0000", 2024-09-18T04:57:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.314e-06 2024-09-18T04:57:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129044+0000", 2024-09-18T04:57:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9919999999999998e-06 2024-09-18T04:57:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182015+0000", 2024-09-18T04:57:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052970875000000001 2024-09-18T04:57:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182058+0000", 2024-09-18T04:57:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2997000000000002e-05 2024-09-18T04:57:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182160+0000", 2024-09-18T04:57:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101437 2024-09-18T04:57:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14138 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1830912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:13.656 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:57:13.657 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.510046568, 2024-09-18T04:57:13.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053256435999999997, 2024-09-18T04:57:13.657 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:13.657 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:13.657 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:13.657 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:13.657 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:13.657 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14138 2024-09-18T04:57:13.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.657 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:13.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:13.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:57:13.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:13.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:57:13.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:13.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129048+0000", 2024-09-18T04:57:13.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:13.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:13.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129056+0000", 2024-09-18T04:57:13.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.413e-06 2024-09-18T04:57:13.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:13.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129062+0000", 2024-09-18T04:57:13.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6810000000000001e-06 2024-09-18T04:57:13.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:13.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129068+0000", 2024-09-18T04:57:13.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0459999999999996e-06 2024-09-18T04:57:13.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:13.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182191+0000", 2024-09-18T04:57:13.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053123210999999997 2024-09-18T04:57:13.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:13.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182238+0000", 2024-09-18T04:57:13.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7404999999999998e-05 2024-09-18T04:57:13.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:13.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182305+0000", 2024-09-18T04:57:13.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7028999999999998e-05 2024-09-18T04:57:13.661 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.661 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:13.661 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.661 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15910 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2236416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:13.661 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:13.661 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.410959036000001, 2024-09-18T04:57:13.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053241054000000003, 2024-09-18T04:57:13.661 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:13.661 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:13.661 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:13.661 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:13.662 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:13.662 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15910 2024-09-18T04:57:13.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.662 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:13.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:13.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:13.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:13.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:13.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:13.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:13.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:13.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:13.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228138+0000", 2024-09-18T04:57:13.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:57:13.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:13.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228144+0000", 2024-09-18T04:57:13.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5369999999999997e-06 2024-09-18T04:57:13.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:13.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228149+0000", 2024-09-18T04:57:13.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1209999999999998e-06 2024-09-18T04:57:13.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:13.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280822+0000", 2024-09-18T04:57:13.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052673194999999999 2024-09-18T04:57:13.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:13.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280903+0000", 2024-09-18T04:57:13.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1179000000000003e-05 2024-09-18T04:57:13.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:13.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281377+0000", 2024-09-18T04:57:13.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047417899999999997 2024-09-18T04:57:13.665 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.665 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:13.665 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.665 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15911 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2285568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:13.665 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:13.666 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.410942304000002, 2024-09-18T04:57:13.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053415151000000001, 2024-09-18T04:57:13.666 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:13.666 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:13.666 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:13.666 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:13.666 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:13.666 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15911 2024-09-18T04:57:13.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.666 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:13.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:13.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:13.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:13.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:13.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:13.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:13.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:13.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:13.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228155+0000", 2024-09-18T04:57:13.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:57:13.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:13.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228161+0000", 2024-09-18T04:57:13.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:57:13.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:13.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228166+0000", 2024-09-18T04:57:13.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2320000000000001e-06 2024-09-18T04:57:13.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:13.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281405+0000", 2024-09-18T04:57:13.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053239336999999998 2024-09-18T04:57:13.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:13.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281482+0000", 2024-09-18T04:57:13.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6569999999999994e-05 2024-09-18T04:57:13.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:13.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281568+0000", 2024-09-18T04:57:13.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6328000000000004e-05 2024-09-18T04:57:13.669 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.669 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:13.669 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.670 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15918 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2662400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:13.670 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:13.670 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.409173322999997, 2024-09-18T04:57:13.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053091065, 2024-09-18T04:57:13.670 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:13.670 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:13.670 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:13.670 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:13.670 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:13.670 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15918 2024-09-18T04:57:13.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.670 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:13.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:13.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:13.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:13.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:13.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:13.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:13.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:13.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:13.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229924+0000", 2024-09-18T04:57:13.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:57:13.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:13.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229930+0000", 2024-09-18T04:57:13.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.066e-06 2024-09-18T04:57:13.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:13.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229935+0000", 2024-09-18T04:57:13.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9119999999999997e-06 2024-09-18T04:57:13.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:13.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282840+0000", 2024-09-18T04:57:13.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052905119 2024-09-18T04:57:13.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:13.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282922+0000", 2024-09-18T04:57:13.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2081000000000002e-05 2024-09-18T04:57:13.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:13.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283013+0000", 2024-09-18T04:57:13.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0828000000000005e-05 2024-09-18T04:57:13.674 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.674 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:13.674 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.674 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15919 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2711552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:13.674 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:13.674 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.409156045000003, 2024-09-18T04:57:13.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053293021000000003, 2024-09-18T04:57:13.674 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:13.674 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:13.675 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:13.675 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:13.675 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:13.675 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15919 2024-09-18T04:57:13.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.675 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:13.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:13.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:13.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:13.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:13.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:13.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:13.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:13.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:13.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229941+0000", 2024-09-18T04:57:13.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2639999999999998e-06 2024-09-18T04:57:13.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:13.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229947+0000", 2024-09-18T04:57:13.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9750000000000004e-06 2024-09-18T04:57:13.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:13.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229952+0000", 2024-09-18T04:57:13.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7929999999999997e-06 2024-09-18T04:57:13.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:13.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283062+0000", 2024-09-18T04:57:13.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05310952 2024-09-18T04:57:13.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:13.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283162+0000", 2024-09-18T04:57:13.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100098 2024-09-18T04:57:13.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:13.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283232+0000", 2024-09-18T04:57:13.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0693999999999998e-05 2024-09-18T04:57:13.678 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.678 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:13.678 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.679 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15920 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2760704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:13.679 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:13.679 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.409139283999998, 2024-09-18T04:57:13.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053502367000000002, 2024-09-18T04:57:13.679 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:13.679 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:13.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:13.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:13.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:13.679 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15920 2024-09-18T04:57:13.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.679 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:13.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:13.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:13.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:13.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:13.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:13.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:13.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:13.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:13.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229958+0000", 2024-09-18T04:57:13.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1770000000000001e-06 2024-09-18T04:57:13.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:13.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229964+0000", 2024-09-18T04:57:13.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8470000000000001e-06 2024-09-18T04:57:13.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:13.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229968+0000", 2024-09-18T04:57:13.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5680000000000001e-06 2024-09-18T04:57:13.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:13.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283269+0000", 2024-09-18T04:57:13.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053300620999999999 2024-09-18T04:57:13.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:13.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283359+0000", 2024-09-18T04:57:13.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9971999999999998e-05 2024-09-18T04:57:13.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:13.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283458+0000", 2024-09-18T04:57:13.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9483999999999998e-05 2024-09-18T04:57:13.682 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.683 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:13.683 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.683 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15921 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2809856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:13.683 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:57:13.683 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.409122734, 2024-09-18T04:57:13.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053800977, 2024-09-18T04:57:13.683 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:13.683 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:13.683 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:13.684 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:13.684 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:13.684 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15921 2024-09-18T04:57:13.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.684 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:13.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:13.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:57:13.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:13.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:57:13.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:13.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229972+0000", 2024-09-18T04:57:13.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:13.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:13.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229974+0000", 2024-09-18T04:57:13.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2910000000000002e-06 2024-09-18T04:57:13.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:13.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229982+0000", 2024-09-18T04:57:13.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7249999999999997e-06 2024-09-18T04:57:13.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:13.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229989+0000", 2024-09-18T04:57:13.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6429999999999998e-06 2024-09-18T04:57:13.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:13.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283601+0000", 2024-09-18T04:57:13.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053612211 2024-09-18T04:57:13.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:13.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283681+0000", 2024-09-18T04:57:13.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0125999999999998e-05 2024-09-18T04:57:13.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:13.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283774+0000", 2024-09-18T04:57:13.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2362999999999996e-05 2024-09-18T04:57:13.687 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.687 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:13.687 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.688 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15922 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2859008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:13.688 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:57:13.688 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.409077795000002, 2024-09-18T04:57:13.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053882270000000003, 2024-09-18T04:57:13.688 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:13.688 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:13.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:13.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:13.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:13.688 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15922 2024-09-18T04:57:13.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.688 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:13.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:13.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:57:13.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:13.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:57:13.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:13.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230017+0000", 2024-09-18T04:57:13.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:13.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:13.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230019+0000", 2024-09-18T04:57:13.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4210000000000002e-06 2024-09-18T04:57:13.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:13.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230027+0000", 2024-09-18T04:57:13.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000002e-06 2024-09-18T04:57:13.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:13.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230032+0000", 2024-09-18T04:57:13.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0089999999999998e-06 2024-09-18T04:57:13.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:13.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283800+0000", 2024-09-18T04:57:13.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053768418999999998 2024-09-18T04:57:13.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:13.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283849+0000", 2024-09-18T04:57:13.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9339000000000003e-05 2024-09-18T04:57:13.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:13.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283900+0000", 2024-09-18T04:57:13.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0421999999999999e-05 2024-09-18T04:57:13.692 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.692 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:13.692 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.692 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15923 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2924544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:13.692 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:13.692 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.409058926, 2024-09-18T04:57:13.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054073016000000002, 2024-09-18T04:57:13.692 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:13.692 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:13.692 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:13.692 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:13.692 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:13.693 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15923 2024-09-18T04:57:13.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.693 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:13.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:13.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:13.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:13.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:13.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:13.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:13.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:13.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:13.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230038+0000", 2024-09-18T04:57:13.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.463e-06 2024-09-18T04:57:13.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:13.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230045+0000", 2024-09-18T04:57:13.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3570000000000003e-06 2024-09-18T04:57:13.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:13.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230050+0000", 2024-09-18T04:57:13.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7299999999999996e-06 2024-09-18T04:57:13.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:13.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283928+0000", 2024-09-18T04:57:13.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053878218999999998 2024-09-18T04:57:13.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:13.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284010+0000", 2024-09-18T04:57:13.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2577000000000006e-05 2024-09-18T04:57:13.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:13.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284109+0000", 2024-09-18T04:57:13.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9102000000000001e-05 2024-09-18T04:57:13.696 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.696 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:13.696 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.696 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15924 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:13.696 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:13.696 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.409040456, 2024-09-18T04:57:13.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054108470999999998, 2024-09-18T04:57:13.697 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:13.697 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:13.697 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:13.697 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:13.697 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:13.697 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15924 2024-09-18T04:57:13.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.697 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:13.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:13.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:13.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:13.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:13.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:13.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:13.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:13.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:13.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230061+0000", 2024-09-18T04:57:13.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8240000000000002e-06 2024-09-18T04:57:13.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:13.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230068+0000", 2024-09-18T04:57:13.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4320000000000004e-06 2024-09-18T04:57:13.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:13.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230073+0000", 2024-09-18T04:57:13.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.921e-06 2024-09-18T04:57:13.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:13.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284153+0000", 2024-09-18T04:57:13.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054080793000000002 2024-09-18T04:57:13.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:13.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284163+0000", 2024-09-18T04:57:13.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.859e-06 2024-09-18T04:57:13.700 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.700 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:13.700 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.700 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15925 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:13.700 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:13.700 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.409016340999997, 2024-09-18T04:57:13.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054134632000000002, 2024-09-18T04:57:13.701 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:13.701 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:13.701 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:13.701 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:13.701 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:13.701 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15925 2024-09-18T04:57:13.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.701 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:13.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:13.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:13.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:13.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:13.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:13.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:13.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:13.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:13.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230081+0000", 2024-09-18T04:57:13.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4109999999999999e-06 2024-09-18T04:57:13.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:13.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230087+0000", 2024-09-18T04:57:13.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.161e-06 2024-09-18T04:57:13.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:13.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230092+0000", 2024-09-18T04:57:13.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9259999999999999e-06 2024-09-18T04:57:13.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:13.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284204+0000", 2024-09-18T04:57:13.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054111546000000003 2024-09-18T04:57:13.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:13.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284214+0000", 2024-09-18T04:57:13.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9639999999999998e-06 2024-09-18T04:57:13.704 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.704 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:13.704 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.704 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15926 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:13.704 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:13.704 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.408999016999999, 2024-09-18T04:57:13.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054143193999999999, 2024-09-18T04:57:13.705 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:13.705 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:13.705 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:13.705 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:13.705 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:13.705 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15926 2024-09-18T04:57:13.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.705 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:13.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:13.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:13.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:13.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:13.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:13.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:13.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:13.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:13.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230098+0000", 2024-09-18T04:57:13.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1399999999999998e-06 2024-09-18T04:57:13.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:13.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230104+0000", 2024-09-18T04:57:13.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-18T04:57:13.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:13.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230109+0000", 2024-09-18T04:57:13.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9250000000000003e-06 2024-09-18T04:57:13.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:13.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284231+0000", 2024-09-18T04:57:13.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054122297999999999 2024-09-18T04:57:13.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:13.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284239+0000", 2024-09-18T04:57:13.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1689999999999992e-06 2024-09-18T04:57:13.708 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.708 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:13.708 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.709 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15927 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:13.709 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:13.709 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.408981969999999, 2024-09-18T04:57:13.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054168435000000001, 2024-09-18T04:57:13.709 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:13.709 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:13.710 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:13.710 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:13.710 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:13.710 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15927 2024-09-18T04:57:13.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.710 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:13.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:13.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:13.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:13.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:13.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:13.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:13.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:13.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:13.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230116+0000", 2024-09-18T04:57:13.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5150000000000001e-06 2024-09-18T04:57:13.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:13.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230122+0000", 2024-09-18T04:57:13.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1190000000000002e-06 2024-09-18T04:57:13.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:13.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230126+0000", 2024-09-18T04:57:13.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.792e-06 2024-09-18T04:57:13.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:13.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284273+0000", 2024-09-18T04:57:13.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054146086000000003 2024-09-18T04:57:13.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:13.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284282+0000", 2024-09-18T04:57:13.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2590000000000006e-06 2024-09-18T04:57:13.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.713 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:13.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.713 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15928 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:13.713 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:57:13.713 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.408964587, 2024-09-18T04:57:13.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054189739000000001, 2024-09-18T04:57:13.714 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:13.714 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:13.714 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:13.714 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:13.714 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:13.714 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15928 2024-09-18T04:57:13.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.714 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:13.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:13.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:57:13.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:13.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:57:13.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:13.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230130+0000", 2024-09-18T04:57:13.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:13.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:13.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230133+0000", 2024-09-18T04:57:13.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.244e-06 2024-09-18T04:57:13.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:13.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230139+0000", 2024-09-18T04:57:13.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9830000000000001e-06 2024-09-18T04:57:13.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:13.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230143+0000", 2024-09-18T04:57:13.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7530000000000001e-06 2024-09-18T04:57:13.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:13.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284313+0000", 2024-09-18T04:57:13.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054169332000000001 2024-09-18T04:57:13.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:13.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284320+0000", 2024-09-18T04:57:13.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8699999999999992e-06 2024-09-18T04:57:13.717 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.718 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:13.718 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.718 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15929 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:13.718 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:13.718 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.408948146999997, 2024-09-18T04:57:13.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054218771999999998, 2024-09-18T04:57:13.718 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:13.718 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:13.718 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:13.719 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:13.719 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:13.719 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15929 2024-09-18T04:57:13.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.719 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:13.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:13.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:13.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:13.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:13.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:13.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:13.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:13.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:13.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230149+0000", 2024-09-18T04:57:13.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-18T04:57:13.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:13.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230155+0000", 2024-09-18T04:57:13.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2469999999999997e-06 2024-09-18T04:57:13.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:13.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230160+0000", 2024-09-18T04:57:13.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7129999999999996e-06 2024-09-18T04:57:13.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:13.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284355+0000", 2024-09-18T04:57:13.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054194900999999997 2024-09-18T04:57:13.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:13.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284366+0000", 2024-09-18T04:57:13.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0993e-05 2024-09-18T04:57:13.723 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.723 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:13.723 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.723 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15930 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:13.723 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:13.723 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.408931146999997, 2024-09-18T04:57:13.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054244713999999999, 2024-09-18T04:57:13.724 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:13.724 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:13.724 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:13.724 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:13.724 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:13.724 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15930 2024-09-18T04:57:13.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.724 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:13.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:13.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:13.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:13.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:13.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:13.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:13.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:13.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:13.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230166+0000", 2024-09-18T04:57:13.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1009999999999999e-06 2024-09-18T04:57:13.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:13.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230172+0000", 2024-09-18T04:57:13.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.981e-06 2024-09-18T04:57:13.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:13.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230184+0000", 2024-09-18T04:57:13.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2449000000000001e-05 2024-09-18T04:57:13.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:13.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284400+0000", 2024-09-18T04:57:13.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054215452999999997 2024-09-18T04:57:13.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:13.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284409+0000", 2024-09-18T04:57:13.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0839999999999998e-06 2024-09-18T04:57:13.728 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.728 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:13.728 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.728 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15931 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:13.729 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:57:13.729 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.408905357999998, 2024-09-18T04:57:13.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054261928000000001, 2024-09-18T04:57:13.729 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:13.729 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:13.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:13.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:13.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:13.729 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15931 2024-09-18T04:57:13.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.729 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:13.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:13.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:57:13.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:13.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:57:13.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:13.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:13.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230189+0000", 2024-09-18T04:57:13.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:13.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:13.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230192+0000", 2024-09-18T04:57:13.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6599999999999999e-06 2024-09-18T04:57:13.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:13.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230199+0000", 2024-09-18T04:57:13.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5570000000000001e-06 2024-09-18T04:57:13.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:13.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230204+0000", 2024-09-18T04:57:13.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2270000000000001e-06 2024-09-18T04:57:13.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:13.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284442+0000", 2024-09-18T04:57:13.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054238383000000001 2024-09-18T04:57:13.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:13.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:13.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:13.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284452+0000", 2024-09-18T04:57:13.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6069999999999997e-06 2024-09-18T04:57:13.733 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.733 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:13.733 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.733 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:13.733 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:13.733 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:13.734 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:13.735 INFO:teuthology.orchestra.run.smithi002.stdout:3737 read 0xccca25e thru 0xccd4157 (0x9efa bytes) 2024-09-18T04:57:13.735 INFO:teuthology.orchestra.run.smithi002.stdout:3741 write 0xa71ad48 thru 0xa71f648 (0x4901 bytes) 2024-09-18T04:57:13.735 INFO:teuthology.orchestra.run.smithi002.stdout:3742 write 0xcd1babf thru 0xcd2420a (0x874c bytes) 2024-09-18T04:57:13.736 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:57:13.736 INFO:teuthology.orchestra.run.smithi002.stdout:3744 punch from 0xa10adf1 to 0xa10e8d5, (0x3ae4 bytes) 2024-09-18T04:57:13.736 INFO:teuthology.orchestra.run.smithi002.stdout:3745 write 0x37a7310 thru 0x37b25bb (0xb2ac bytes) 2024-09-18T04:57:13.737 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:57:13.737 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:57:13.738 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:13.738 INFO:teuthology.orchestra.run.smithi002.stdout:3747 write 0x6dd89e9 thru 0x6de32bc (0xa8d4 bytes) 2024-09-18T04:57:13.739 INFO:teuthology.orchestra.run.smithi002.stdout:3748 trunc from 0xe340b1e to 0x2c11de1 2024-09-18T04:57:13.739 INFO:teuthology.orchestra.run.smithi002.stdout:3749 trunc from 0x2c11de1 to 0x710181a 2024-09-18T04:57:13.739 INFO:teuthology.orchestra.run.smithi002.stdout:3750 punch from 0x231dfbc to 0x2328bd9, (0xac1d bytes) 2024-09-18T04:57:13.740 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:13.740 INFO:teuthology.orchestra.run.smithi002.stdout:3752 write 0xccd7b1f thru 0xcce1f93 (0xa475 bytes) 2024-09-18T04:57:13.740 INFO:teuthology.orchestra.run.smithi002.stdout:3754 read 0x2d9c07a thru 0x2da7254 (0xb1db bytes) 2024-09-18T04:57:13.740 INFO:teuthology.orchestra.run.smithi002.stdout:3755 writesame 0x5374373 thru 0x5376da2 data_size 0x5a(0x2a30 bytes) 2024-09-18T04:57:13.740 INFO:teuthology.orchestra.run.smithi002.stdout:3756 read 0xadf12e6 thru 0xadf2dea (0x1b05 bytes) 2024-09-18T04:57:13.740 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:13.732+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:13.750 INFO:teuthology.orchestra.run.smithi002.stdout:3757 trunc from 0xcce1f94 to 0x2d08bb1 2024-09-18T04:57:13.761 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:13.761 INFO:teuthology.orchestra.run.smithi002.stdout:3762 punch from 0x170b1be to 0x170c67b, (0x14bd bytes) 2024-09-18T04:57:13.761 INFO:teuthology.orchestra.run.smithi002.stdout:3763 punch from 0x178064f to 0x1787bfc, (0x75ad bytes) 2024-09-18T04:57:13.761 INFO:teuthology.orchestra.run.smithi002.stdout:3765 read 0xb6d4c6 thru 0xb71eb9 (0x49f4 bytes) 2024-09-18T04:57:13.762 INFO:teuthology.orchestra.run.smithi002.stdout:3767 read 0x26f4709 thru 0x26f9865 (0x515d bytes) 2024-09-18T04:57:13.762 INFO:teuthology.orchestra.run.smithi002.stdout:3768 punch from 0xbe30e0 to 0xbe57de, (0x26fe bytes) 2024-09-18T04:57:13.764 INFO:teuthology.orchestra.run.smithi002.stdout:3769 trunc from 0x2d08bb1 to 0xb1cd7e 2024-09-18T04:57:13.803 INFO:teuthology.orchestra.run.smithi002.stdout:3770 read 0x3f3523 thru 0x3fb4c7 (0x7fa5 bytes) 2024-09-18T04:57:13.813 INFO:teuthology.orchestra.run.smithi002.stdout:3771 write 0x4180d9f thru 0x41843fc (0x365e bytes) 2024-09-18T04:57:13.817 INFO:teuthology.orchestra.run.smithi002.stdout:3772 read 0x2722e5a thru 0x272b2e8 (0x848f bytes) 2024-09-18T04:57:13.817 INFO:teuthology.orchestra.run.smithi002.stdout:3773 punch from 0x2167771 to 0x2173927, (0xc1b6 bytes) 2024-09-18T04:57:13.827 INFO:teuthology.orchestra.run.smithi002.stdout:3774 writesame 0x8225bb9 thru 0x8231e60 data_size 0x4(0xc2a8 bytes) 2024-09-18T04:57:13.831 INFO:teuthology.orchestra.run.smithi002.stdout:3775 trunc from 0x8231e61 to 0xb7eaa7d 2024-09-18T04:57:13.835 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:13.833+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:13.838 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:13.838 INFO:teuthology.orchestra.run.smithi002.stdout:3778 read 0x9805ec0 thru 0x9807f79 (0x20ba bytes) 2024-09-18T04:57:13.838 INFO:teuthology.orchestra.run.smithi002.stdout:3780 read 0xab51b4c thru 0xab5a710 (0x8bc5 bytes) 2024-09-18T04:57:13.839 INFO:teuthology.orchestra.run.smithi002.stdout:3781 punch from 0x1f54bb to 0x2048a4, (0xf3e9 bytes) 2024-09-18T04:57:13.839 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:13.839 INFO:teuthology.orchestra.run.smithi002.stdout:3783 read 0x41ea5c thru 0x4277f4 (0x8d99 bytes) 2024-09-18T04:57:13.839 INFO:teuthology.orchestra.run.smithi002.stdout:3784 write 0x4a3c598 thru 0x4a45b6c (0x95d5 bytes) 2024-09-18T04:57:13.843 INFO:teuthology.orchestra.run.smithi002.stdout:3787 punch from 0xa25e899 to 0xa26335b, (0x4ac2 bytes) 2024-09-18T04:57:13.843 INFO:teuthology.orchestra.run.smithi002.stdout:3788 punch from 0x180b153 to 0x180c1cc, (0x1079 bytes) 2024-09-18T04:57:13.843 INFO:teuthology.orchestra.run.smithi002.stdout:3789 punch from 0x11fc4ea to 0x11fe13b, (0x1c51 bytes) 2024-09-18T04:57:13.843 INFO:teuthology.orchestra.run.smithi002.stdout:3790 read 0x2ad238a thru 0x2ae0b3a (0xe7b1 bytes) 2024-09-18T04:57:13.843 INFO:teuthology.orchestra.run.smithi002.stdout:3791 write 0x283eef9 thru 0x2840463 (0x156b bytes) 2024-09-18T04:57:13.846 INFO:teuthology.orchestra.run.smithi002.stdout:3792 write 0x93ad578 thru 0x93b7e51 (0xa8da bytes) 2024-09-18T04:57:13.850 INFO:teuthology.orchestra.run.smithi002.stdout:3793 read 0x15220bd thru 0x15265bd (0x4501 bytes) 2024-09-18T04:57:13.850 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:13.850 INFO:teuthology.orchestra.run.smithi002.stdout:3797 read 0x9d6301 thru 0x9d9160 (0x2e60 bytes) 2024-09-18T04:57:13.856 INFO:teuthology.orchestra.run.smithi002.stdout:3799 write 0xbd6efa3 thru 0xbd7a59e (0xb5fc bytes) 2024-09-18T04:57:13.861 INFO:teuthology.orchestra.run.smithi002.stdout:3800 write 0x7d56cdc thru 0x7d59584 (0x28a9 bytes) 2024-09-18T04:57:13.865 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:13.865 INFO:teuthology.orchestra.run.smithi002.stdout:3802 punch from 0x236ec38 to 0x237383a, (0x4c02 bytes) 2024-09-18T04:57:13.866 INFO:teuthology.orchestra.run.smithi002.stdout:3803 read 0xa6109a0 thru 0xa61a531 (0x9b92 bytes) 2024-09-18T04:57:13.866 INFO:teuthology.orchestra.run.smithi002.stdout:3805 read 0xad5590c thru 0xad6022b (0xa920 bytes) 2024-09-18T04:57:13.866 INFO:teuthology.orchestra.run.smithi002.stdout:3806 trunc from 0xbd7a59f to 0xc5b0f06 2024-09-18T04:57:13.871 INFO:teuthology.orchestra.run.smithi002.stdout:3808 read 0x4337611 thru 0x433c006 (0x49f6 bytes) 2024-09-18T04:57:13.873 INFO:teuthology.orchestra.run.smithi002.stdout:3809 trunc from 0xc5b0f06 to 0xb0191b4 2024-09-18T04:57:13.880 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:13.880 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:13.880 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:13.880 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:13.882 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:13.882 INFO:teuthology.orchestra.run.smithi002.stdout:3811 read 0x3186308 thru 0x318647a (0x173 bytes) 2024-09-18T04:57:13.891 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:57:13.892 INFO:teuthology.orchestra.run.smithi002.stdout:3812 write 0xdc3e9e6 thru 0xdc4abb1 (0xc1cc bytes) 2024-09-18T04:57:13.897 INFO:teuthology.orchestra.run.smithi002.stdout:3813 punch from 0x2e598c2 to 0x2e5e999, (0x50d7 bytes) 2024-09-18T04:57:13.897 INFO:teuthology.orchestra.run.smithi002.stdout:3814 writesame 0x2a2247e thru 0x2a2e8c3 data_size 0x25(0xc446 bytes) 2024-09-18T04:57:13.900 INFO:teuthology.orchestra.run.smithi002.stdout:3817 write 0x253ed03 thru 0x254c70f (0xda0d bytes) 2024-09-18T04:57:13.910 INFO:teuthology.orchestra.run.smithi002.stdout:3819 read 0x6261fed thru 0x62676ee (0x5702 bytes) 2024-09-18T04:57:13.910 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:13.910 INFO:teuthology.orchestra.run.smithi002.stdout:3821 punch from 0x6f4befd to 0x6f4e3a3, (0x24a6 bytes) 2024-09-18T04:57:13.910 INFO:teuthology.orchestra.run.smithi002.stdout:3822 write 0x3e6bab7 thru 0x3e76333 (0xa87d bytes) 2024-09-18T04:57:13.910 INFO:teuthology.orchestra.run.smithi002.stdout:3823 write 0xdb44c8a thru 0xdb45394 (0x70b bytes) 2024-09-18T04:57:13.912 INFO:teuthology.orchestra.run.smithi002.stdout:3824 write 0x14470e4 thru 0x1452925 (0xb842 bytes) 2024-09-18T04:57:13.925 INFO:teuthology.orchestra.run.smithi002.stdout:3825 trunc from 0xdc4abb2 to 0x69adcb6 2024-09-18T04:57:13.936 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:13.934+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:13.948 INFO:teuthology.orchestra.run.smithi002.stdout:3826 write 0xbf3bf0e thru 0xbf41f58 (0x604b bytes) 2024-09-18T04:57:13.952 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:13.952 INFO:teuthology.orchestra.run.smithi002.stdout:3831 write 0x77d4dd7 thru 0x77d8f1f (0x4149 bytes) 2024-09-18T04:57:13.956 INFO:teuthology.orchestra.run.smithi002.stdout:3832 read 0x451ab23 thru 0x451c603 (0x1ae1 bytes) 2024-09-18T04:57:13.966 INFO:teuthology.orchestra.run.smithi002.stdout:3833 write 0xe3cf90d thru 0xe3de7f8 (0xeeec bytes) 2024-09-18T04:57:13.970 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:13.971 INFO:teuthology.orchestra.run.smithi002.stdout:3835 write 0xdbbdce7 thru 0xdbc6307 (0x8621 bytes) 2024-09-18T04:57:13.974 INFO:teuthology.orchestra.run.smithi002.stdout:3836 read 0xd72fd67 thru 0xd73e3d8 (0xe672 bytes) 2024-09-18T04:57:13.975 INFO:teuthology.orchestra.run.smithi002.stdout:3837 write 0x1d9e315 thru 0x1daa90e (0xc5fa bytes) 2024-09-18T04:57:13.980 INFO:teuthology.orchestra.run.smithi002.stdout:3839 read 0x32388da thru 0x3238fef (0x716 bytes) 2024-09-18T04:57:13.980 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:13.980 INFO:teuthology.orchestra.run.smithi002.stdout:3841 read 0xaff6f60 thru 0xb002a4e (0xbaef bytes) 2024-09-18T04:57:13.981 INFO:teuthology.orchestra.run.smithi002.stdout:3842 read 0x9b691bd thru 0x9b6975b (0x59f bytes) 2024-09-18T04:57:13.981 INFO:teuthology.orchestra.run.smithi002.stdout:3845 read 0xae6c6e6 thru 0xae7710f (0xaa2a bytes) 2024-09-18T04:57:13.981 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:13.981 INFO:teuthology.orchestra.run.smithi002.stdout:3848 writesame 0x2fce985 thru 0x2fd8778 data_size 0x16(0x9df4 bytes) 2024-09-18T04:57:13.987 INFO:teuthology.orchestra.run.smithi002.stdout:3849 trunc from 0xe3de7f9 to 0xcf86464 2024-09-18T04:57:13.997 INFO:teuthology.orchestra.run.smithi002.stdout:3850 read 0xa610a7e thru 0xa6157fa (0x4d7d bytes) 2024-09-18T04:57:13.997 INFO:teuthology.orchestra.run.smithi002.stdout:3851 write 0xc35da15 thru 0xc364c5a (0x7246 bytes) 2024-09-18T04:57:14.001 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:14.001 INFO:teuthology.orchestra.run.smithi002.stdout:3854 write 0x7d31bb5 thru 0x7d31ee3 (0x32f bytes) 2024-09-18T04:57:14.003 INFO:teuthology.orchestra.run.smithi002.stdout:3855 write 0x858bfd thru 0x859981 (0xd85 bytes) 2024-09-18T04:57:14.005 INFO:teuthology.orchestra.run.smithi002.stdout:3856 writesame 0x2483447 thru 0x248adc8 data_size 0x67(0x7982 bytes) 2024-09-18T04:57:14.009 INFO:teuthology.orchestra.run.smithi002.stdout:3857 punch from 0x625f55 to 0x633016, (0xd0c1 bytes) 2024-09-18T04:57:14.015 INFO:teuthology.orchestra.run.smithi002.stdout:3859 trunc from 0xcf86464 to 0x78254a9 2024-09-18T04:57:14.018 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:14.019 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:14.019 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:14.019 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:14.019 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:14.029 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:57:14.036 INFO:teuthology.orchestra.run.smithi002.stdout:3860 writesame 0xaba0f38 thru 0xabaf2a3 data_size 0x29(0xe36c bytes) 2024-09-18T04:57:14.036 INFO:teuthology.orchestra.run.smithi002.stdout:3862 writesame 0x6147cfc thru 0x614f9a3 data_size 0x4(0x7ca8 bytes) 2024-09-18T04:57:14.039 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:14.035+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:14.048 INFO:teuthology.orchestra.run.smithi002.stdout:3863 trunc from 0xabaf2a4 to 0x5d49c5c 2024-09-18T04:57:14.060 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:14.073 INFO:teuthology.orchestra.run.smithi002.stdout:3868 writesame 0xa7e299f thru 0xa7e50cd data_size 0x1(0x272f bytes) 2024-09-18T04:57:14.073 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:14.073 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:14.073 INFO:teuthology.orchestra.run.smithi002.stdout:3871 punch from 0xa1ccf05 to 0xa1d5bd6, (0x8cd1 bytes) 2024-09-18T04:57:14.073 INFO:teuthology.orchestra.run.smithi002.stdout:3872 punch from 0x91a9499 to 0x91b6352, (0xceb9 bytes) 2024-09-18T04:57:14.073 INFO:teuthology.orchestra.run.smithi002.stdout:3873 read 0x7a8480a thru 0x7a8fa82 (0xb279 bytes) 2024-09-18T04:57:14.073 INFO:teuthology.orchestra.run.smithi002.stdout:3874 write 0x4f04126 thru 0x4f0bc96 (0x7b71 bytes) 2024-09-18T04:57:14.076 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:14.076 INFO:teuthology.orchestra.run.smithi002.stdout:3877 punch from 0x41f446 to 0x422690, (0x324a bytes) 2024-09-18T04:57:14.087 INFO:teuthology.orchestra.run.smithi002.stdout:3879 trunc from 0xa7e50ce to 0xe3c32b 2024-09-18T04:57:14.104 INFO:teuthology.orchestra.run.smithi002.stdout:3882 read 0x3c4c67 thru 0x3ca14a (0x54e4 bytes) 2024-09-18T04:57:14.114 INFO:teuthology.orchestra.run.smithi002.stdout:3883 write 0x4beb408 thru 0x4bef01f (0x3c18 bytes) 2024-09-18T04:57:14.123 INFO:teuthology.orchestra.run.smithi002.stdout:3884 write 0x5237393 thru 0x524157d (0xa1eb bytes) 2024-09-18T04:57:14.128 INFO:teuthology.orchestra.run.smithi002.stdout:3886 read 0x218481 thru 0x2251a8 (0xcd28 bytes) 2024-09-18T04:57:14.129 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:14.129 INFO:teuthology.orchestra.run.smithi002.stdout:3889 punch from 0x2a89ccb to 0x2a98333, (0xe668 bytes) 2024-09-18T04:57:14.129 INFO:teuthology.orchestra.run.smithi002.stdout:3890 read 0x2402658 thru 0x240dc07 (0xb5b0 bytes) 2024-09-18T04:57:14.129 INFO:teuthology.orchestra.run.smithi002.stdout:3891 read 0x13d0f48 thru 0x13dbef7 (0xafb0 bytes) 2024-09-18T04:57:14.131 INFO:teuthology.orchestra.run.smithi002.stdout:3894 trunc from 0x524157e to 0x306fa75 2024-09-18T04:57:14.138 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:14.135+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:14.141 INFO:teuthology.orchestra.run.smithi002.stdout:3895 read 0x2f3a938 thru 0x2f47c59 (0xd322 bytes) 2024-09-18T04:57:14.142 INFO:teuthology.orchestra.run.smithi002.stdout:3896 write 0xac4dc thru 0xbb86a (0xf38f bytes) 2024-09-18T04:57:14.162 INFO:teuthology.orchestra.run.smithi002.stdout:3897 trunc from 0x306fa75 to 0xc5c24f8 2024-09-18T04:57:14.168 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:14.168 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:14.168 INFO:teuthology.orchestra.run.smithi002.stdout:3900 write 0x579b25c thru 0x579ce2a (0x1bcf bytes) 2024-09-18T04:57:14.169 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:14.170 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:14.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:14.170 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:14.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.170 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:15989 3.4 e32/19 3:238b8fe4:::rbd_data.11ac7a27a554.0000000000000005:head v 32'150, mlcod=32'147)", 2024-09-18T04:57:14.170 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.245449+0000", 2024-09-18T04:57:14.170 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.920181452999998, 2024-09-18T04:57:14.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0028002180000000002, 2024-09-18T04:57:14.171 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.171 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:14.171 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.245449+0000", 2024-09-18T04:57:14.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.245449+0000", 2024-09-18T04:57:14.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.245448+0000", 2024-09-18T04:57:14.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:14.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.246149+0000", 2024-09-18T04:57:14.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00070047499999999999 2024-09-18T04:57:14.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.246164+0000", 2024-09-18T04:57:14.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5532999999999998e-05 2024-09-18T04:57:14.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.246171+0000", 2024-09-18T04:57:14.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8990000000000004e-06 2024-09-18T04:57:14.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.246218+0000", 2024-09-18T04:57:14.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6789000000000002e-05 2024-09-18T04:57:14.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.246242+0000", 2024-09-18T04:57:14.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3706999999999999e-05 2024-09-18T04:57:14.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:14.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.248217+0000", 2024-09-18T04:57:14.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019752060000000002 2024-09-18T04:57:14.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.248249+0000", 2024-09-18T04:57:14.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.2252000000000002e-05 2024-09-18T04:57:14.175 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.175 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.175 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.175 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:15991 3.4 e32/19 3:21a074f1:::rbd_data.11ac7a27a554.0000000000000006:head v 32'152, mlcod=32'148)", 2024-09-18T04:57:14.176 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.249009+0000", 2024-09-18T04:57:14.176 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.916620899999998, 2024-09-18T04:57:14.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021236060000000001, 2024-09-18T04:57:14.176 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.176 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:14.176 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249009+0000", 2024-09-18T04:57:14.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249009+0000", 2024-09-18T04:57:14.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249009+0000", 2024-09-18T04:57:14.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:14.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249228+0000", 2024-09-18T04:57:14.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00021908999999999999 2024-09-18T04:57:14.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249243+0000", 2024-09-18T04:57:14.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5031999999999999e-05 2024-09-18T04:57:14.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249250+0000", 2024-09-18T04:57:14.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2919999999999998e-06 2024-09-18T04:57:14.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249294+0000", 2024-09-18T04:57:14.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3479000000000002e-05 2024-09-18T04:57:14.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.249314+0000", 2024-09-18T04:57:14.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0596000000000001e-05 2024-09-18T04:57:14.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:14.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.251093+0000", 2024-09-18T04:57:14.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001778342 2024-09-18T04:57:14.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.251133+0000", 2024-09-18T04:57:14.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0503999999999997e-05 2024-09-18T04:57:14.181 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.182 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.182 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.182 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:18826 3.4 e32/19 3:23a67d65:::rbd_data.11ac7a27a554.0000000000000010:head v 32'162, mlcod=32'159)", 2024-09-18T04:57:14.182 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.666171+0000", 2024-09-18T04:57:14.182 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 37.499459227000003, 2024-09-18T04:57:14.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022865569999999998, 2024-09-18T04:57:14.182 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.182 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:14.183 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666171+0000", 2024-09-18T04:57:14.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666171+0000", 2024-09-18T04:57:14.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666170+0000", 2024-09-18T04:57:14.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:14.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666654+0000", 2024-09-18T04:57:14.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00048368400000000001 2024-09-18T04:57:14.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666670+0000", 2024-09-18T04:57:14.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5818e-05 2024-09-18T04:57:14.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666677+0000", 2024-09-18T04:57:14.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5750000000000003e-06 2024-09-18T04:57:14.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666734+0000", 2024-09-18T04:57:14.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6663999999999998e-05 2024-09-18T04:57:14.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.666757+0000", 2024-09-18T04:57:14.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2691e-05 2024-09-18T04:57:14.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:14.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.668417+0000", 2024-09-18T04:57:14.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001660501 2024-09-18T04:57:14.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.668458+0000", 2024-09-18T04:57:14.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0364999999999997e-05 2024-09-18T04:57:14.187 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.187 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.187 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.187 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:36051 3.4 e42/19 3:25d849c5:::rbd_data.11ac728e6e9a.0000000000000002:head v 42'294, mlcod=40'292)", 2024-09-18T04:57:14.187 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:46.443141+0000", 2024-09-18T04:57:14.187 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 27.722489473, 2024-09-18T04:57:14.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0037584760000000002, 2024-09-18T04:57:14.187 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.187 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:14.187 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443141+0000", 2024-09-18T04:57:14.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443141+0000", 2024-09-18T04:57:14.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443141+0000", 2024-09-18T04:57:14.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:14.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443812+0000", 2024-09-18T04:57:14.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00067104899999999995 2024-09-18T04:57:14.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443831+0000", 2024-09-18T04:57:14.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9074999999999999e-05 2024-09-18T04:57:14.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443841+0000", 2024-09-18T04:57:14.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8919999999999995e-06 2024-09-18T04:57:14.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443933+0000", 2024-09-18T04:57:14.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2348000000000004e-05 2024-09-18T04:57:14.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.443957+0000", 2024-09-18T04:57:14.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3567e-05 2024-09-18T04:57:14.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:14.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.446862+0000", 2024-09-18T04:57:14.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0029052919999999999 2024-09-18T04:57:14.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:46.446899+0000", 2024-09-18T04:57:14.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7515999999999998e-05 2024-09-18T04:57:14.191 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.191 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.191 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.192 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:67383 3.4 e51/19 3:2583a267:::rbd_data.11ac8b4c71f8.00000000000001a3:head v 51'413, mlcod=50'410)", 2024-09-18T04:57:14.192 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:56.349429+0000", 2024-09-18T04:57:14.192 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 17.816201599999999, 2024-09-18T04:57:14.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0018230950000000001, 2024-09-18T04:57:14.192 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.192 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:14.192 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.349429+0000", 2024-09-18T04:57:14.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.349429+0000", 2024-09-18T04:57:14.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.349428+0000", 2024-09-18T04:57:14.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:14.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.349434+0000", 2024-09-18T04:57:14.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.648e-06 2024-09-18T04:57:14.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.349443+0000", 2024-09-18T04:57:14.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5410000000000006e-06 2024-09-18T04:57:14.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.349450+0000", 2024-09-18T04:57:14.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5490000000000003e-06 2024-09-18T04:57:14.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.349494+0000", 2024-09-18T04:57:14.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4212999999999998e-05 2024-09-18T04:57:14.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.349514+0000", 2024-09-18T04:57:14.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.965e-05 2024-09-18T04:57:14.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:14.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.351215+0000", 2024-09-18T04:57:14.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017009690000000001 2024-09-18T04:57:14.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:56.351252+0000", 2024-09-18T04:57:14.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7023000000000001e-05 2024-09-18T04:57:14.196 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.196 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.196 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.196 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:90054 3.4 e54/19 3:265c2e99:::rbd_data.11ac679235d7.0000000000000004:head v 54'424, mlcod=53'422)", 2024-09-18T04:57:14.196 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:00.486527+0000", 2024-09-18T04:57:14.196 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 13.679102947000001, 2024-09-18T04:57:14.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00244005, 2024-09-18T04:57:14.197 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.197 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:14.197 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:00.486527+0000", 2024-09-18T04:57:14.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:00.486527+0000", 2024-09-18T04:57:14.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:00.486527+0000", 2024-09-18T04:57:14.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:14.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:00.486998+0000", 2024-09-18T04:57:14.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047111199999999998 2024-09-18T04:57:14.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:00.487010+0000", 2024-09-18T04:57:14.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2007999999999999e-05 2024-09-18T04:57:14.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:00.487016+0000", 2024-09-18T04:57:14.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1090000000000003e-06 2024-09-18T04:57:14.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:00.487053+0000", 2024-09-18T04:57:14.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7125000000000001e-05 2024-09-18T04:57:14.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:00.487071+0000", 2024-09-18T04:57:14.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7391e-05 2024-09-18T04:57:14.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:14.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:00.488926+0000", 2024-09-18T04:57:14.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00185516 2024-09-18T04:57:14.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:00.488967+0000", 2024-09-18T04:57:14.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1798000000000002e-05 2024-09-18T04:57:14.200 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.200 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.200 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.201 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:92355 3.4 e55/19 3:202900f5:::rbd_data.11ac51791e54.0000000000000007:head v 55'427, mlcod=54'425)", 2024-09-18T04:57:14.201 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:01.828032+0000", 2024-09-18T04:57:14.201 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 12.337598355000001, 2024-09-18T04:57:14.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019153250000000001, 2024-09-18T04:57:14.201 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.201 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:14.201 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.828032+0000", 2024-09-18T04:57:14.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.828032+0000", 2024-09-18T04:57:14.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.828030+0000", 2024-09-18T04:57:14.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999976 2024-09-18T04:57:14.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.828332+0000", 2024-09-18T04:57:14.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00030202799999999998 2024-09-18T04:57:14.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.828352+0000", 2024-09-18T04:57:14.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0475e-05 2024-09-18T04:57:14.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.828362+0000", 2024-09-18T04:57:14.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.55e-06 2024-09-18T04:57:14.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.828411+0000", 2024-09-18T04:57:14.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8899000000000001e-05 2024-09-18T04:57:14.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.828435+0000", 2024-09-18T04:57:14.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4697000000000001e-05 2024-09-18T04:57:14.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:14.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.829907+0000", 2024-09-18T04:57:14.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001471665 2024-09-18T04:57:14.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:01.829947+0000", 2024-09-18T04:57:14.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0219000000000002e-05 2024-09-18T04:57:14.205 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.205 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.205 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.206 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPush(3.1a 60/59 [PushOp(3:5d34c0ec:::rbd_data.11ac14fc1d07.0000000000000000:head, version: 29'34, data_included: [0~262144,514400~30572,786432~524288,1572864~231572], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:5d34c0ec:::rbd_data.11ac14fc1d07.0000000000000000:head@29'34, size: 2097152, copy_subset: [0~2097152], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1804436, data_complete:false, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-18T04:57:14.206 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:06.327000+0000", 2024-09-18T04:57:14.206 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.8386300850000001, 2024-09-18T04:57:14.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.002127688, 2024-09-18T04:57:14.206 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.206 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:14.206 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.327000+0000", 2024-09-18T04:57:14.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.327000+0000", 2024-09-18T04:57:14.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.327000+0000", 2024-09-18T04:57:14.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:14.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.328145+0000", 2024-09-18T04:57:14.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001145418 2024-09-18T04:57:14.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.328164+0000", 2024-09-18T04:57:14.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.931e-05 2024-09-18T04:57:14.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.328190+0000", 2024-09-18T04:57:14.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6078e-05 2024-09-18T04:57:14.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.328243+0000", 2024-09-18T04:57:14.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2781999999999997e-05 2024-09-18T04:57:14.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.328250+0000", 2024-09-18T04:57:14.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7549999999999997e-06 2024-09-18T04:57:14.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.329128+0000", 2024-09-18T04:57:14.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00087802699999999997 2024-09-18T04:57:14.210 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.210 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.210 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.210 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "MOSDPGPush(3.1a 60/59 [PushOp(3:5f22f35c:::rbd_data.11ac7a27a554.0000000000000018:head, version: 32'57, data_included: [0~1048576], data_size: 1048576, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(3:5f22f35c:::rbd_data.11ac7a27a554.0000000000000018:head@32'57, size: 1048576, copy_subset: [0~1048576], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1048576, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-09-18T04:57:14.210 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:06.339467+0000", 2024-09-18T04:57:14.210 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.8261632130000001, 2024-09-18T04:57:14.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021495609999999999, 2024-09-18T04:57:14.210 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.210 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:14.210 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.339467+0000", 2024-09-18T04:57:14.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.339467+0000", 2024-09-18T04:57:14.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.339466+0000", 2024-09-18T04:57:14.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:14.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.340457+0000", 2024-09-18T04:57:14.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00099054000000000004 2024-09-18T04:57:14.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.340477+0000", 2024-09-18T04:57:14.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9500999999999998e-05 2024-09-18T04:57:14.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.340491+0000", 2024-09-18T04:57:14.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4968e-05 2024-09-18T04:57:14.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.340555+0000", 2024-09-18T04:57:14.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3110999999999999e-05 2024-09-18T04:57:14.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.340562+0000", 2024-09-18T04:57:14.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.61e-06 2024-09-18T04:57:14.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.341617+0000", 2024-09-18T04:57:14.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010545140000000001 2024-09-18T04:57:14.214 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.214 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.214 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.215 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:104253 3.0 3:014cd031:::rbd_data.11acf685aece.0000000000000091:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T04:57:14.215 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:06.635820+0000", 2024-09-18T04:57:14.215 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.5298099550000002, 2024-09-18T04:57:14.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021358190000000002, 2024-09-18T04:57:14.215 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.215 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:14.216 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.216 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.216 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.216 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104253 2024-09-18T04:57:14.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.216 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635820+0000", 2024-09-18T04:57:14.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635820+0000", 2024-09-18T04:57:14.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635819+0000", 2024-09-18T04:57:14.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999981 2024-09-18T04:57:14.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635829+0000", 2024-09-18T04:57:14.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.066e-05 2024-09-18T04:57:14.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635846+0000", 2024-09-18T04:57:14.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7058000000000002e-05 2024-09-18T04:57:14.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635855+0000", 2024-09-18T04:57:14.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6649999999999992e-06 2024-09-18T04:57:14.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635909+0000", 2024-09-18T04:57:14.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4156999999999997e-05 2024-09-18T04:57:14.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635993+0000", 2024-09-18T04:57:14.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3363999999999997e-05 2024-09-18T04:57:14.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-18T04:57:14.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.636106+0000", 2024-09-18T04:57:14.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000113682 2024-09-18T04:57:14.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:57:14.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.636749+0000", 2024-09-18T04:57:14.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00064277099999999999 2024-09-18T04:57:14.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:14.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.637800+0000", 2024-09-18T04:57:14.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0010504900000000001 2024-09-18T04:57:14.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:14.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.637854+0000", 2024-09-18T04:57:14.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4676999999999998e-05 2024-09-18T04:57:14.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.637956+0000", 2024-09-18T04:57:14.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010197800000000001 2024-09-18T04:57:14.221 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.221 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.221 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.221 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:104256 3.4 3:22227238:::rbd_data.11acf685aece.0000000000000094:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e60)", 2024-09-18T04:57:14.221 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:06.635869+0000", 2024-09-18T04:57:14.221 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.5297616359999999, 2024-09-18T04:57:14.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0019687379999999998, 2024-09-18T04:57:14.221 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.221 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:14.221 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.221 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.222 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.222 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 104256 2024-09-18T04:57:14.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.222 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635869+0000", 2024-09-18T04:57:14.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635869+0000", 2024-09-18T04:57:14.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635868+0000", 2024-09-18T04:57:14.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:14.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635874+0000", 2024-09-18T04:57:14.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6180000000000003e-06 2024-09-18T04:57:14.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635881+0000", 2024-09-18T04:57:14.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5839999999999997e-06 2024-09-18T04:57:14.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635887+0000", 2024-09-18T04:57:14.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9320000000000001e-06 2024-09-18T04:57:14.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.635940+0000", 2024-09-18T04:57:14.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2954000000000002e-05 2024-09-18T04:57:14.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.636028+0000", 2024-09-18T04:57:14.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7862999999999995e-05 2024-09-18T04:57:14.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-18T04:57:14.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.636143+0000", 2024-09-18T04:57:14.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011521100000000001 2024-09-18T04:57:14.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:57:14.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.636970+0000", 2024-09-18T04:57:14.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00082689899999999997 2024-09-18T04:57:14.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:14.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.637690+0000", 2024-09-18T04:57:14.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00071944100000000003 2024-09-18T04:57:14.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:14.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.637776+0000", 2024-09-18T04:57:14.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5902000000000005e-05 2024-09-18T04:57:14.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.637837+0000", 2024-09-18T04:57:14.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1797999999999993e-05 2024-09-18T04:57:14.227 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.227 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.227 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.227 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:104394 3.1a e60/59 3:5d3c2665:::rbd_data.11acf685aece.0000000000000002:head v 60'167, mlcod=60'165)", 2024-09-18T04:57:14.227 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:06.663312+0000", 2024-09-18T04:57:14.227 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.5023180700000003, 2024-09-18T04:57:14.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022438900000000001, 2024-09-18T04:57:14.227 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.227 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:14.227 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.663312+0000", 2024-09-18T04:57:14.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.663312+0000", 2024-09-18T04:57:14.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.663311+0000", 2024-09-18T04:57:14.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:57:14.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.664199+0000", 2024-09-18T04:57:14.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00088800900000000004 2024-09-18T04:57:14.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.664222+0000", 2024-09-18T04:57:14.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3734999999999999e-05 2024-09-18T04:57:14.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.664235+0000", 2024-09-18T04:57:14.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2617e-05 2024-09-18T04:57:14.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.664308+0000", 2024-09-18T04:57:14.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2632000000000005e-05 2024-09-18T04:57:14.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.664332+0000", 2024-09-18T04:57:14.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4341999999999998e-05 2024-09-18T04:57:14.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:14.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.665489+0000", 2024-09-18T04:57:14.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0011568640000000001 2024-09-18T04:57:14.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:06.665556+0000", 2024-09-18T04:57:14.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7286999999999995e-05 2024-09-18T04:57:14.231 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.231 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.232 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.232 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:105545 3.1a e63/61 3:5f592997:::rbd_data.11ac491a2851.0000000000000000:head v 63'172, mlcod=62'170)", 2024-09-18T04:57:14.232 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:09.668693+0000", 2024-09-18T04:57:14.232 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.4969373030000002, 2024-09-18T04:57:14.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003007455, 2024-09-18T04:57:14.232 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.232 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:14.232 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.668693+0000", 2024-09-18T04:57:14.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.668693+0000", 2024-09-18T04:57:14.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.668691+0000", 2024-09-18T04:57:14.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999981 2024-09-18T04:57:14.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.670050+0000", 2024-09-18T04:57:14.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001358529 2024-09-18T04:57:14.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.670067+0000", 2024-09-18T04:57:14.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7050000000000001e-05 2024-09-18T04:57:14.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.670075+0000", 2024-09-18T04:57:14.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1030000000000002e-06 2024-09-18T04:57:14.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.670124+0000", 2024-09-18T04:57:14.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9228999999999999e-05 2024-09-18T04:57:14.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.670143+0000", 2024-09-18T04:57:14.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9097999999999999e-05 2024-09-18T04:57:14.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:14.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671662+0000", 2024-09-18T04:57:14.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015189929999999999 2024-09-18T04:57:14.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671701+0000", 2024-09-18T04:57:14.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8262000000000001e-05 2024-09-18T04:57:14.236 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.236 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.236 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.236 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop(client.4524.0:105546 3.9 e63/19 3:94add86d:::rbd_data.11ac491a2851.0000000000000003:head v 63'270, mlcod=57'268)", 2024-09-18T04:57:14.236 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:09.671085+0000", 2024-09-18T04:57:14.236 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.4945454150000002, 2024-09-18T04:57:14.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0025510419999999999, 2024-09-18T04:57:14.236 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.237 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:14.237 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671085+0000", 2024-09-18T04:57:14.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671085+0000", 2024-09-18T04:57:14.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671084+0000", 2024-09-18T04:57:14.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:14.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671945+0000", 2024-09-18T04:57:14.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00086074600000000002 2024-09-18T04:57:14.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671959+0000", 2024-09-18T04:57:14.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.3708e-05 2024-09-18T04:57:14.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671965+0000", 2024-09-18T04:57:14.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5590000000000002e-06 2024-09-18T04:57:14.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.672010+0000", 2024-09-18T04:57:14.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4591e-05 2024-09-18T04:57:14.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.672028+0000", 2024-09-18T04:57:14.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.7731e-05 2024-09-18T04:57:14.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:14.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.673599+0000", 2024-09-18T04:57:14.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0015717940000000001 2024-09-18T04:57:14.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.673636+0000", 2024-09-18T04:57:14.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6647000000000003e-05 2024-09-18T04:57:14.240 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.240 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.240 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.241 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:105549 3.0 3:041c4dc5:::rbd_data.11ac491a2851.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 7340032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T04:57:14.241 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:09.671361+0000", 2024-09-18T04:57:14.241 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.4942696140000002, 2024-09-18T04:57:14.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0026345079999999998, 2024-09-18T04:57:14.241 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.241 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:14.241 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.241 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.241 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.241 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 105549 2024-09-18T04:57:14.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.241 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671361+0000", 2024-09-18T04:57:14.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671361+0000", 2024-09-18T04:57:14.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671359+0000", 2024-09-18T04:57:14.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999981 2024-09-18T04:57:14.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671411+0000", 2024-09-18T04:57:14.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1700999999999998e-05 2024-09-18T04:57:14.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671437+0000", 2024-09-18T04:57:14.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6795999999999999e-05 2024-09-18T04:57:14.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671448+0000", 2024-09-18T04:57:14.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0124e-05 2024-09-18T04:57:14.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671502+0000", 2024-09-18T04:57:14.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4768999999999997e-05 2024-09-18T04:57:14.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671682+0000", 2024-09-18T04:57:14.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00018011000000000001 2024-09-18T04:57:14.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-18T04:57:14.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.671897+0000", 2024-09-18T04:57:14.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000214112 2024-09-18T04:57:14.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:57:14.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.672805+0000", 2024-09-18T04:57:14.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00090863399999999996 2024-09-18T04:57:14.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:14.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.673922+0000", 2024-09-18T04:57:14.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001117067 2024-09-18T04:57:14.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:14.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.673971+0000", 2024-09-18T04:57:14.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8814999999999999e-05 2024-09-18T04:57:14.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.673995+0000", 2024-09-18T04:57:14.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4207999999999999e-05 2024-09-18T04:57:14.246 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.246 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.246 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.246 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:105660 3.0 3:041c4dc5:::rbd_data.11ac491a2851.0000000000000006:head [stat,truncate 5242880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T04:57:14.247 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:09.802588+0000", 2024-09-18T04:57:14.247 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.363042074, 2024-09-18T04:57:14.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001891818, 2024-09-18T04:57:14.247 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.247 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:14.247 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.247 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.247 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.247 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 105660 2024-09-18T04:57:14.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.247 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.802588+0000", 2024-09-18T04:57:14.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.802588+0000", 2024-09-18T04:57:14.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.802586+0000", 2024-09-18T04:57:14.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999976 2024-09-18T04:57:14.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.802596+0000", 2024-09-18T04:57:14.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0019e-05 2024-09-18T04:57:14.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.802619+0000", 2024-09-18T04:57:14.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2734999999999998e-05 2024-09-18T04:57:14.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.802631+0000", 2024-09-18T04:57:14.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2218999999999999e-05 2024-09-18T04:57:14.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.802692+0000", 2024-09-18T04:57:14.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1080000000000005e-05 2024-09-18T04:57:14.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.802806+0000", 2024-09-18T04:57:14.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000113688 2024-09-18T04:57:14.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-18T04:57:14.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.802941+0000", 2024-09-18T04:57:14.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000135196 2024-09-18T04:57:14.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:57:14.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.803610+0000", 2024-09-18T04:57:14.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00066944400000000005 2024-09-18T04:57:14.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:14.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.804151+0000", 2024-09-18T04:57:14.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00054033100000000001 2024-09-18T04:57:14.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:14.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.804442+0000", 2024-09-18T04:57:14.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00029086100000000001 2024-09-18T04:57:14.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:09.804480+0000", 2024-09-18T04:57:14.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8577e-05 2024-09-18T04:57:14.252 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.252 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.252 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.253 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:105911 3.0 3:041c4dc5:::rbd_data.11ac491a2851.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T04:57:14.253 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:10.056416+0000", 2024-09-18T04:57:14.253 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.1092142909999998, 2024-09-18T04:57:14.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017911119999999999, 2024-09-18T04:57:14.253 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.253 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:14.253 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.253 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.254 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.254 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 105911 2024-09-18T04:57:14.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.254 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.056416+0000", 2024-09-18T04:57:14.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.056416+0000", 2024-09-18T04:57:14.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.056414+0000", 2024-09-18T04:57:14.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999981 2024-09-18T04:57:14.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.056426+0000", 2024-09-18T04:57:14.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1584e-05 2024-09-18T04:57:14.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.056442+0000", 2024-09-18T04:57:14.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6297e-05 2024-09-18T04:57:14.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.056449+0000", 2024-09-18T04:57:14.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9070000000000001e-06 2024-09-18T04:57:14.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.056491+0000", 2024-09-18T04:57:14.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1885000000000003e-05 2024-09-18T04:57:14.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.056546+0000", 2024-09-18T04:57:14.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4614999999999998e-05 2024-09-18T04:57:14.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-18T04:57:14.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.059571+0000", 2024-09-18T04:57:14.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030249389999999999 2024-09-18T04:57:14.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:14.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.074240+0000", 2024-09-18T04:57:14.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014668976 2024-09-18T04:57:14.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:57:14.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.074250+0000", 2024-09-18T04:57:14.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0461e-05 2024-09-18T04:57:14.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:14.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.074285+0000", 2024-09-18T04:57:14.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4541e-05 2024-09-18T04:57:14.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.074327+0000", 2024-09-18T04:57:14.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2626000000000002e-05 2024-09-18T04:57:14.259 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.259 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.259 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.259 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_repop_reply(client.4524.0:105911 3.0 e63/61 ondisk, result = 0)", 2024-09-18T04:57:14.259 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:10.069182+0000", 2024-09-18T04:57:14.259 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.0964486300000003, 2024-09-18T04:57:14.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0050614400000000004, 2024-09-18T04:57:14.260 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.260 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:14.260 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.069182+0000", 2024-09-18T04:57:14.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.069182+0000", 2024-09-18T04:57:14.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.069180+0000", 2024-09-18T04:57:14.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:57:14.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.069188+0000", 2024-09-18T04:57:14.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6569999999999997e-06 2024-09-18T04:57:14.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.069208+0000", 2024-09-18T04:57:14.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9775e-05 2024-09-18T04:57:14.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.069220+0000", 2024-09-18T04:57:14.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.242e-05 2024-09-18T04:57:14.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.074226+0000", 2024-09-18T04:57:14.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0050062099999999997 2024-09-18T04:57:14.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.074235+0000", 2024-09-18T04:57:14.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6959999999999994e-06 2024-09-18T04:57:14.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.074243+0000", 2024-09-18T04:57:14.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0709999999999995e-06 2024-09-18T04:57:14.263 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.263 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.263 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.263 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:106489 3.4 3:235f2ef8:::rbd_data.11acc690427d.000000000000009d:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T04:57:14.263 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:10.675471+0000", 2024-09-18T04:57:14.263 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.490159566, 2024-09-18T04:57:14.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.004686544, 2024-09-18T04:57:14.264 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.264 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:14.264 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.264 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.264 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.264 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106489 2024-09-18T04:57:14.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.264 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.675471+0000", 2024-09-18T04:57:14.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.675471+0000", 2024-09-18T04:57:14.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.675470+0000", 2024-09-18T04:57:14.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999986 2024-09-18T04:57:14.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.675477+0000", 2024-09-18T04:57:14.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4930000000000001e-06 2024-09-18T04:57:14.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.675493+0000", 2024-09-18T04:57:14.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6337e-05 2024-09-18T04:57:14.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.675502+0000", 2024-09-18T04:57:14.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2009999999999999e-06 2024-09-18T04:57:14.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.675563+0000", 2024-09-18T04:57:14.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0974999999999998e-05 2024-09-18T04:57:14.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.675712+0000", 2024-09-18T04:57:14.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00014978100000000001 2024-09-18T04:57:14.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-18T04:57:14.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.675907+0000", 2024-09-18T04:57:14.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000194274 2024-09-18T04:57:14.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:57:14.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.676524+0000", 2024-09-18T04:57:14.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00061781599999999998 2024-09-18T04:57:14.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:14.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.680084+0000", 2024-09-18T04:57:14.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.003560003 2024-09-18T04:57:14.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:14.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.680122+0000", 2024-09-18T04:57:14.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7499000000000001e-05 2024-09-18T04:57:14.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.680157+0000", 2024-09-18T04:57:14.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5370000000000002e-05 2024-09-18T04:57:14.269 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.269 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.269 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.269 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:106696 3.0 3:04466edb:::rbd_data.11acc690427d.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e64)", 2024-09-18T04:57:14.269 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:10.833452+0000", 2024-09-18T04:57:14.269 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 3.3321783269999998, 2024-09-18T04:57:14.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0021179110000000001, 2024-09-18T04:57:14.270 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.270 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:14.270 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.270 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.270 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.270 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 106696 2024-09-18T04:57:14.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.270 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.833452+0000", 2024-09-18T04:57:14.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.833452+0000", 2024-09-18T04:57:14.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.833450+0000", 2024-09-18T04:57:14.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999981 2024-09-18T04:57:14.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.833620+0000", 2024-09-18T04:57:14.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016985599999999999 2024-09-18T04:57:14.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.833637+0000", 2024-09-18T04:57:14.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6835999999999999e-05 2024-09-18T04:57:14.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.833646+0000", 2024-09-18T04:57:14.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6349999999999995e-06 2024-09-18T04:57:14.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.833683+0000", 2024-09-18T04:57:14.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7758e-05 2024-09-18T04:57:14.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.833794+0000", 2024-09-18T04:57:14.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00011047700000000001 2024-09-18T04:57:14.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-18T04:57:14.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.833934+0000", 2024-09-18T04:57:14.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00013985199999999999 2024-09-18T04:57:14.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:57:14.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.835080+0000", 2024-09-18T04:57:14.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001146563 2024-09-18T04:57:14.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:14.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.835482+0000", 2024-09-18T04:57:14.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00040172100000000001 2024-09-18T04:57:14.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:14.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.835535+0000", 2024-09-18T04:57:14.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3298999999999999e-05 2024-09-18T04:57:14.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.835570+0000", 2024-09-18T04:57:14.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4718000000000002e-05 2024-09-18T04:57:14.275 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.275 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.275 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.275 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.275 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.275 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:14.276 INFO:teuthology.orchestra.run.smithi002.stdout:3902 writesame 0x2e77664 thru 0x2e79a9d data_size 0x1(0x243a bytes) 2024-09-18T04:57:14.277 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:14.274+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:14.278 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:57:14.279 INFO:teuthology.orchestra.run.smithi002.stdout:3903 trunc from 0xc5c24f8 to 0x829a911 2024-09-18T04:57:14.279 INFO:teuthology.orchestra.run.smithi002.stdout:3904 read 0x6ba39bc thru 0x6bb2791 (0xedd6 bytes) 2024-09-18T04:57:14.280 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:14.280 INFO:teuthology.orchestra.run.smithi002.stdout:3907 write 0x4a4cd39 thru 0x4a568e0 (0x9ba8 bytes) 2024-09-18T04:57:14.280 INFO:teuthology.orchestra.run.smithi002.stdout:3908 read 0x42b390 thru 0x42ee73 (0x3ae4 bytes) 2024-09-18T04:57:14.280 INFO:teuthology.orchestra.run.smithi002.stdout:3909 read 0x272fe73 thru 0x273fc7f (0xfe0d bytes) 2024-09-18T04:57:14.280 INFO:teuthology.orchestra.run.smithi002.stdout:3910 write 0xa2fe884 thru 0xa30bbbe (0xd33b bytes) 2024-09-18T04:57:14.280 INFO:teuthology.orchestra.run.smithi002.stdout:3912 write 0x7b4a571 thru 0x7b59ab6 (0xf546 bytes) 2024-09-18T04:57:14.280 INFO:teuthology.orchestra.run.smithi002.stdout:3914 write 0x75dc937 thru 0x75e4f2c (0x85f6 bytes) 2024-09-18T04:57:14.280 INFO:teuthology.orchestra.run.smithi002.stdout:3915 trunc from 0xa30bbbf to 0xe49add8 2024-09-18T04:57:14.280 INFO:teuthology.orchestra.run.smithi002.stdout:3917 trunc from 0xe49add8 to 0xcdfe970 2024-09-18T04:57:14.280 INFO:teuthology.orchestra.run.smithi002.stdout:3918 writesame 0x450690 thru 0x458252 data_size 0x3b(0x7bc3 bytes) 2024-09-18T04:57:14.281 INFO:teuthology.orchestra.run.smithi002.stdout:3919 read 0x9e2d4cc thru 0x9e35922 (0x8457 bytes) 2024-09-18T04:57:14.281 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:14.281 INFO:teuthology.orchestra.run.smithi002.stdout:3922 read 0x691a20e thru 0x692185f (0x7652 bytes) 2024-09-18T04:57:14.281 INFO:teuthology.orchestra.run.smithi002.stdout:3924 read 0x4572c96 thru 0x45796a1 (0x6a0c bytes) 2024-09-18T04:57:14.281 INFO:teuthology.orchestra.run.smithi002.stdout:3925 read 0x2dccd9a thru 0x2dd77b7 (0xaa1e bytes) 2024-09-18T04:57:14.281 INFO:teuthology.orchestra.run.smithi002.stdout:3926 trunc from 0xcdfe970 to 0xe750cd9 2024-09-18T04:57:14.281 INFO:teuthology.orchestra.run.smithi002.stdout:3927 writesame 0xb307019 thru 0xb30a6ea data_size 0x3(0x36d2 bytes) 2024-09-18T04:57:14.281 INFO:teuthology.orchestra.run.smithi002.stdout:3928 writesame 0x40e1ea thru 0x41de77 data_size 0x1(0xfc8e bytes) 2024-09-18T04:57:14.285 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:14.298 INFO:teuthology.orchestra.run.smithi002.stdout:3930 trunc from 0xe750cd9 to 0x2597e4a 2024-09-18T04:57:14.315 INFO:teuthology.orchestra.run.smithi002.stdout:3931 read 0x17dbe02 thru 0x17dd70e (0x190d bytes) 2024-09-18T04:57:14.316 INFO:teuthology.orchestra.run.smithi002.stdout:3932 trunc from 0x2597e4a to 0x38a23fb 2024-09-18T04:57:14.340 INFO:teuthology.orchestra.run.smithi002.stdout:3933 write 0xe16282e thru 0xe17130a (0xeadd bytes) 2024-09-18T04:57:14.346 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-18T04:57:14.350 INFO:teuthology.orchestra.run.smithi002.stdout:3934 write 0x59b2662 thru 0x59ba0fc (0x7a9b bytes) 2024-09-18T04:57:14.361 INFO:teuthology.orchestra.run.smithi002.stdout:3935 trunc from 0xe17130b to 0x9126f3e 2024-09-18T04:57:14.371 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:14.371 INFO:teuthology.orchestra.run.smithi002.stdout:3937 read 0x84a93b5 thru 0x84b8d67 (0xf9b3 bytes) 2024-09-18T04:57:14.371 INFO:teuthology.orchestra.run.smithi002.stdout:3938 punch from 0x35e1835 to 0x35e1f3f, (0x70a bytes) 2024-09-18T04:57:14.371 INFO:teuthology.orchestra.run.smithi002.stdout:3939 write 0x10438c3 thru 0x10457ec (0x1f2a bytes) 2024-09-18T04:57:14.375 INFO:teuthology.orchestra.run.smithi002.stdout:3940 write 0x1c70353 thru 0x1c7e685 (0xe333 bytes) 2024-09-18T04:57:14.377 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:14.374+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:14.381 INFO:teuthology.orchestra.run.smithi002.stdout:3941 read 0x89dc548 thru 0x89e705f (0xab18 bytes) 2024-09-18T04:57:14.385 INFO:teuthology.orchestra.run.smithi002.stdout:3942 trunc from 0x9126f3e to 0x5dcef97 2024-09-18T04:57:14.391 INFO:teuthology.orchestra.run.smithi002.stdout:3943 read 0x1461249 thru 0x146f63d (0xe3f5 bytes) 2024-09-18T04:57:14.391 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:14.391 INFO:teuthology.orchestra.run.smithi002.stdout:3945 writesame 0x48b30dc thru 0x48bb1e8 data_size 0x1(0x810d bytes) 2024-09-18T04:57:14.411 INFO:teuthology.orchestra.run.smithi002.stdout:3946 writesame 0x6dd19ba thru 0x6dd63a6 data_size 0x5(0x49ed bytes) 2024-09-18T04:57:14.411 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:14.411 INFO:teuthology.orchestra.run.smithi002.stdout:3949 write 0x4ab658 thru 0x4b54f3 (0x9e9c bytes) 2024-09-18T04:57:14.415 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:109751 3.b 3:d66255a2:::rbd_object_map.11acf951905a:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68)", 2024-09-18T04:57:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:14.412196+0000", 2024-09-18T04:57:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.0010028400000000001, 2024-09-18T04:57:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.001034716, 2024-09-18T04:57:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.415 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-18T04:57:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 109751 2024-09-18T04:57:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:14.412196+0000", 2024-09-18T04:57:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:14.412196+0000", 2024-09-18T04:57:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:14.412195+0000", 2024-09-18T04:57:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:14.412201+0000", 2024-09-18T04:57:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1410000000000002e-06 2024-09-18T04:57:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:14.412213+0000", 2024-09-18T04:57:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2724999999999999e-05 2024-09-18T04:57:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:14.412221+0000", 2024-09-18T04:57:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4080000000000001e-06 2024-09-18T04:57:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:14.412263+0000", 2024-09-18T04:57:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2114999999999999e-05 2024-09-18T04:57:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:14.412316+0000", 2024-09-18T04:57:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3464000000000002e-05 2024-09-18T04:57:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:57:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:14.412567+0000", 2024-09-18T04:57:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00025026900000000001 2024-09-18T04:57:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:57:14.421 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 1 2024-09-18T04:57:14.421 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:14.424 INFO:teuthology.orchestra.run.smithi002.stdout:3950 write 0x75e26c8 thru 0x75ea979 (0x82b2 bytes) 2024-09-18T04:57:14.426 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:57:14.429 INFO:teuthology.orchestra.run.smithi002.stdout:3952 punch from 0x1e3f2ac to 0x1e44187, (0x4edb bytes) 2024-09-18T04:57:14.433 INFO:teuthology.orchestra.run.smithi002.stdout:3953 trunc from 0x75ea97a to 0xb0a6db5 2024-09-18T04:57:14.445 INFO:teuthology.orchestra.run.smithi002.stdout:3954 write 0xb597b21 thru 0xb5a1bb0 (0xa090 bytes) 2024-09-18T04:57:14.449 INFO:teuthology.orchestra.run.smithi002.stdout:3956 write 0xa6a73bc thru 0xa6ac400 (0x5045 bytes) 2024-09-18T04:57:14.463 INFO:teuthology.orchestra.run.smithi002.stdout:3957 write 0xec20c15 thru 0xec28fa2 (0x838e bytes) 2024-09-18T04:57:14.466 INFO:teuthology.orchestra.run.smithi002.stdout:3958 writesame 0x24a86c9 thru 0x24b1489 data_size 0x1(0x8dc1 bytes) 2024-09-18T04:57:14.477 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:14.475+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:14.483 INFO:teuthology.orchestra.run.smithi002.stdout:3960 read 0x4a28305 thru 0x4a2b323 (0x301f bytes) 2024-09-18T04:57:14.490 INFO:teuthology.orchestra.run.smithi002.stdout:3961 trunc from 0xec28fa3 to 0x8b425f9 2024-09-18T04:57:14.499 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T04:57:14.499 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-18T04:57:14.502 INFO:teuthology.orchestra.run.smithi002.stdout:3962 write 0xf8a5d1 thru 0xf8e17a (0x3baa bytes) 2024-09-18T04:57:14.505 INFO:teuthology.orchestra.run.smithi002.stdout:3963 writesame 0x704724 thru 0x70cd11 data_size 0x4f(0x85ee bytes) 2024-09-18T04:57:14.507 INFO:teuthology.orchestra.run.smithi002.stdout:3965 writesame 0x469bb76 thru 0x46ab43d data_size 0x4c(0xf8c8 bytes) 2024-09-18T04:57:14.514 INFO:teuthology.orchestra.run.smithi002.stdout:3966 read 0x25ad021 thru 0x25b4dfd (0x7ddd bytes) 2024-09-18T04:57:14.514 INFO:teuthology.orchestra.run.smithi002.stdout:3967 punch from 0xfcfb21 to 0xfd95ba, (0x9a99 bytes) 2024-09-18T04:57:14.515 INFO:teuthology.orchestra.run.smithi002.stdout:3968 punch from 0x7cef00d to 0x7cfc12e, (0xd121 bytes) 2024-09-18T04:57:14.515 INFO:teuthology.orchestra.run.smithi002.stdout:3969 punch from 0x20f108e to 0x20fd335, (0xc2a7 bytes) 2024-09-18T04:57:14.516 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:14.526 INFO:teuthology.orchestra.run.smithi002.stdout:3972 writesame 0xe04b315 thru 0xe04e703 data_size 0x1(0x33ef bytes) 2024-09-18T04:57:14.526 INFO:teuthology.orchestra.run.smithi002.stdout:3973 punch from 0xd673b7c to 0xd6779c3, (0x3e47 bytes) 2024-09-18T04:57:14.531 INFO:teuthology.orchestra.run.smithi002.stdout:3974 trunc from 0xe04e704 to 0x94bab7a 2024-09-18T04:57:14.544 INFO:teuthology.orchestra.run.smithi002.stdout:3975 write 0x9fce0d0 thru 0x9fd39a6 (0x58d7 bytes) 2024-09-18T04:57:14.548 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:14.549 INFO:teuthology.orchestra.run.smithi002.stdout:3978 read 0x41e60ac thru 0x41e6632 (0x587 bytes) 2024-09-18T04:57:14.549 INFO:teuthology.orchestra.run.smithi002.stdout:3979 read 0x8660d94 thru 0x866f24c (0xe4b9 bytes) 2024-09-18T04:57:14.549 INFO:teuthology.orchestra.run.smithi002.stdout:3980 read 0x45a7ad7 thru 0x45b0069 (0x8593 bytes) 2024-09-18T04:57:14.554 INFO:teuthology.orchestra.run.smithi002.stdout:3981 trunc from 0x9fd39a7 to 0x50de52b 2024-09-18T04:57:14.556 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:14.556 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:14.556 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:14.556 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:14.556 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:14.568 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:57:14.577 INFO:teuthology.orchestra.run.smithi002.stdout:3982 trunc from 0x50de52b to 0xdc5ba63 2024-09-18T04:57:14.578 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:14.575+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:14.582 INFO:teuthology.orchestra.run.smithi002.stdout:3983 write 0x3589eb7 thru 0x358d201 (0x334b bytes) 2024-09-18T04:57:14.586 INFO:teuthology.orchestra.run.smithi002.stdout:3984 write 0x35f928c thru 0x360904a (0xfdbf bytes) 2024-09-18T04:57:14.590 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:14.590 INFO:teuthology.orchestra.run.smithi002.stdout:3987 read 0x177580f thru 0x178060c (0xadfe bytes) 2024-09-18T04:57:14.591 INFO:teuthology.orchestra.run.smithi002.stdout:3989 trunc from 0xdc5ba63 to 0xde6c4dc 2024-09-18T04:57:14.597 INFO:teuthology.orchestra.run.smithi002.stdout:3990 trunc from 0xde6c4dc to 0xed2011a 2024-09-18T04:57:14.603 INFO:teuthology.orchestra.run.smithi002.stdout:3992 punch from 0x1d39c38 to 0x1d48e8e, (0xf256 bytes) 2024-09-18T04:57:14.603 INFO:teuthology.orchestra.run.smithi002.stdout:3993 read 0x7905d0e thru 0x7909610 (0x3903 bytes) 2024-09-18T04:57:14.607 INFO:teuthology.orchestra.run.smithi002.stdout:3994 trunc from 0xed2011a to 0xb18fa0c 2024-09-18T04:57:14.621 INFO:teuthology.orchestra.run.smithi002.stdout:3995 write 0xd41b27f thru 0xd422406 (0x7188 bytes) 2024-09-18T04:57:14.625 INFO:teuthology.orchestra.run.smithi002.stdout:3996 write 0x8e7062b thru 0x8e76f4a (0x6920 bytes) 2024-09-18T04:57:14.629 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:14.641 INFO:teuthology.orchestra.run.smithi002.stdout:3998 trunc from 0xd422407 to 0x2b14ca9 2024-09-18T04:57:14.655 INFO:teuthology.orchestra.run.smithi002.stdout:3999 punch from 0x27ec298 to 0x27f5f9f, (0x9d07 bytes) 2024-09-18T04:57:14.668 INFO:teuthology.orchestra.run.smithi002.stdout:4000 write 0x8affd9f thru 0x8b085ce (0x8830 bytes) 2024-09-18T04:57:14.673 INFO:teuthology.orchestra.run.smithi002.stdout:4001 writesame 0x75ef8ca thru 0x75f5cbd data_size 0x2(0x63f4 bytes) 2024-09-18T04:57:14.678 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:14.675+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:14.683 INFO:teuthology.orchestra.run.smithi002.stdout:4002 write 0x4fab857 thru 0x4fb2b02 (0x72ac bytes) 2024-09-18T04:57:14.698 INFO:teuthology.orchestra.run.smithi002.stdout:4003 writesame 0xa4e5e8c thru 0xa4e9127 data_size 0x52(0x329c bytes) 2024-09-18T04:57:14.698 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:14.698 INFO:teuthology.orchestra.run.smithi002.stdout:4010 read 0xa3c30bf thru 0xa3d0566 (0xd4a8 bytes) 2024-09-18T04:57:14.698 INFO:teuthology.orchestra.run.smithi002.stdout:4011 writesame 0x559f76e thru 0x55a08ed data_size 0x40(0x1180 bytes) 2024-09-18T04:57:14.701 INFO:teuthology.orchestra.run.smithi002.stdout:4012 write 0xa4bfe11 thru 0xa4c0e29 (0x1019 bytes) 2024-09-18T04:57:14.710 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:14.711 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:14.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:14.711 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:14.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.711 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11151 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1802240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:14.711 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889328+0000", 2024-09-18T04:57:14.711 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.818242105000003, 2024-09-18T04:57:14.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040037528000000003, 2024-09-18T04:57:14.711 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.711 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:14.711 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.711 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.712 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.712 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11151 2024-09-18T04:57:14.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.712 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889328+0000", 2024-09-18T04:57:14.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889328+0000", 2024-09-18T04:57:14.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889328+0000", 2024-09-18T04:57:14.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:14.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889330+0000", 2024-09-18T04:57:14.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.148e-06 2024-09-18T04:57:14.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889336+0000", 2024-09-18T04:57:14.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8220000000000003e-06 2024-09-18T04:57:14.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889340+0000", 2024-09-18T04:57:14.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.318e-06 2024-09-18T04:57:14.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929171+0000", 2024-09-18T04:57:14.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039830466000000002 2024-09-18T04:57:14.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929268+0000", 2024-09-18T04:57:14.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7569e-05 2024-09-18T04:57:14.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929366+0000", 2024-09-18T04:57:14.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7573999999999998e-05 2024-09-18T04:57:14.715 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.715 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.715 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.716 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11152 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1867776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:14.716 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889344+0000", 2024-09-18T04:57:14.716 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.818226520000003, 2024-09-18T04:57:14.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040227974, 2024-09-18T04:57:14.716 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.716 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:14.716 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.716 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.716 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.716 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11152 2024-09-18T04:57:14.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.716 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889344+0000", 2024-09-18T04:57:14.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889344+0000", 2024-09-18T04:57:14.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889344+0000", 2024-09-18T04:57:14.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:14.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889346+0000", 2024-09-18T04:57:14.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.99e-06 2024-09-18T04:57:14.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889351+0000", 2024-09-18T04:57:14.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4260000000000002e-06 2024-09-18T04:57:14.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889370+0000", 2024-09-18T04:57:14.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9262999999999998e-05 2024-09-18T04:57:14.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929397+0000", 2024-09-18T04:57:14.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040026679000000003 2024-09-18T04:57:14.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929479+0000", 2024-09-18T04:57:14.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2467999999999997e-05 2024-09-18T04:57:14.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929572+0000", 2024-09-18T04:57:14.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2519e-05 2024-09-18T04:57:14.721 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.721 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.721 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.721 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11153 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1916928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:14.722 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889377+0000", 2024-09-18T04:57:14.722 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.818193546000003, 2024-09-18T04:57:14.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040407884999999998, 2024-09-18T04:57:14.722 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.722 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:14.722 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.722 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.722 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.722 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11153 2024-09-18T04:57:14.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.723 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889377+0000", 2024-09-18T04:57:14.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889377+0000", 2024-09-18T04:57:14.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889376+0000", 2024-09-18T04:57:14.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:14.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889385+0000", 2024-09-18T04:57:14.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1030000000000002e-06 2024-09-18T04:57:14.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889392+0000", 2024-09-18T04:57:14.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1799999999999999e-06 2024-09-18T04:57:14.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889396+0000", 2024-09-18T04:57:14.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6890000000000003e-06 2024-09-18T04:57:14.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929603+0000", 2024-09-18T04:57:14.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040206652000000002 2024-09-18T04:57:14.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929688+0000", 2024-09-18T04:57:14.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5327000000000005e-05 2024-09-18T04:57:14.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929785+0000", 2024-09-18T04:57:14.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6362999999999999e-05 2024-09-18T04:57:14.728 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.728 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.728 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.728 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11154 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:14.728 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:57:14.728 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.818169716, 2024-09-18T04:57:14.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040591919999999997, 2024-09-18T04:57:14.728 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.728 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:14.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.729 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.729 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11154 2024-09-18T04:57:14.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.729 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:57:14.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:57:14.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889400+0000", 2024-09-18T04:57:14.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:14.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889403+0000", 2024-09-18T04:57:14.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3999999999999999e-06 2024-09-18T04:57:14.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889409+0000", 2024-09-18T04:57:14.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3380000000000001e-06 2024-09-18T04:57:14.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889414+0000", 2024-09-18T04:57:14.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5789999999999997e-06 2024-09-18T04:57:14.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929811+0000", 2024-09-18T04:57:14.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040397486000000003 2024-09-18T04:57:14.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929898+0000", 2024-09-18T04:57:14.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6370000000000001e-05 2024-09-18T04:57:14.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.732 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.732 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929993+0000", 2024-09-18T04:57:14.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5136000000000005e-05 2024-09-18T04:57:14.732 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.732 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.732 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.733 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11155 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2015232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:14.733 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:57:14.733 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.818152538, 2024-09-18T04:57:14.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040769156000000001, 2024-09-18T04:57:14.733 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.733 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:14.733 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.733 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.733 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.733 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11155 2024-09-18T04:57:14.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.733 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:57:14.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:57:14.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:57:14.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:14.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889420+0000", 2024-09-18T04:57:14.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.181e-06 2024-09-18T04:57:14.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889426+0000", 2024-09-18T04:57:14.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8599999999999998e-06 2024-09-18T04:57:14.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889430+0000", 2024-09-18T04:57:14.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4399999999999998e-06 2024-09-18T04:57:14.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930020+0000", 2024-09-18T04:57:14.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040589698 2024-09-18T04:57:14.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930105+0000", 2024-09-18T04:57:14.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4812e-05 2024-09-18T04:57:14.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.736 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930187+0000", 2024-09-18T04:57:14.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2540999999999995e-05 2024-09-18T04:57:14.737 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.737 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.737 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.737 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11156 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2064384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:14.737 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:57:14.737 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.818112243999998, 2024-09-18T04:57:14.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040934419, 2024-09-18T04:57:14.738 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.738 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:14.738 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.738 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.738 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.738 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11156 2024-09-18T04:57:14.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.738 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:57:14.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:57:14.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889457+0000", 2024-09-18T04:57:14.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:14.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889462+0000", 2024-09-18T04:57:14.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8350000000000003e-06 2024-09-18T04:57:14.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889481+0000", 2024-09-18T04:57:14.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8845999999999999e-05 2024-09-18T04:57:14.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889495+0000", 2024-09-18T04:57:14.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4229e-05 2024-09-18T04:57:14.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930216+0000", 2024-09-18T04:57:14.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040721081999999999 2024-09-18T04:57:14.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930299+0000", 2024-09-18T04:57:14.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2995000000000001e-05 2024-09-18T04:57:14.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930393+0000", 2024-09-18T04:57:14.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3185999999999999e-05 2024-09-18T04:57:14.742 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.742 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.742 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.743 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11402 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1495040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:14.743 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896220+0000", 2024-09-18T04:57:14.743 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.811350273999999, 2024-09-18T04:57:14.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040050388999999999, 2024-09-18T04:57:14.743 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.743 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:14.743 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.743 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.743 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.743 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11402 2024-09-18T04:57:14.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.743 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896220+0000", 2024-09-18T04:57:14.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896220+0000", 2024-09-18T04:57:14.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896220+0000", 2024-09-18T04:57:14.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:14.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896222+0000", 2024-09-18T04:57:14.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0049999999999999e-06 2024-09-18T04:57:14.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896227+0000", 2024-09-18T04:57:14.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6130000000000003e-06 2024-09-18T04:57:14.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896232+0000", 2024-09-18T04:57:14.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4100000000000001e-06 2024-09-18T04:57:14.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936072+0000", 2024-09-18T04:57:14.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039840253999999999 2024-09-18T04:57:14.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936175+0000", 2024-09-18T04:57:14.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103037 2024-09-18T04:57:14.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936271+0000", 2024-09-18T04:57:14.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5383999999999993e-05 2024-09-18T04:57:14.747 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.747 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.747 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.747 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11403 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1544192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:14.748 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896250+0000", 2024-09-18T04:57:14.748 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.811320895000001, 2024-09-18T04:57:14.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040244017999999999, 2024-09-18T04:57:14.748 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.748 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:14.748 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.748 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.748 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.748 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11403 2024-09-18T04:57:14.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.748 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896250+0000", 2024-09-18T04:57:14.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896250+0000", 2024-09-18T04:57:14.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896249+0000", 2024-09-18T04:57:14.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:14.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896251+0000", 2024-09-18T04:57:14.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.136e-06 2024-09-18T04:57:14.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896256+0000", 2024-09-18T04:57:14.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9350000000000002e-06 2024-09-18T04:57:14.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896259+0000", 2024-09-18T04:57:14.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8839999999999998e-06 2024-09-18T04:57:14.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936320+0000", 2024-09-18T04:57:14.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040061173999999998 2024-09-18T04:57:14.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936401+0000", 2024-09-18T04:57:14.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0987000000000002e-05 2024-09-18T04:57:14.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936494+0000", 2024-09-18T04:57:14.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2281999999999995e-05 2024-09-18T04:57:14.752 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.752 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.752 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.752 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11404 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1609728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:14.752 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:57:14.752 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.811308081999996, 2024-09-18T04:57:14.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040669012999999997, 2024-09-18T04:57:14.752 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.752 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:14.752 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.752 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.752 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.753 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11404 2024-09-18T04:57:14.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.753 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:57:14.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:57:14.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:57:14.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:57:14.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896263+0000", 2024-09-18T04:57:14.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2729999999999999e-06 2024-09-18T04:57:14.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896266+0000", 2024-09-18T04:57:14.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8200000000000001e-06 2024-09-18T04:57:14.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896269+0000", 2024-09-18T04:57:14.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3199999999999998e-06 2024-09-18T04:57:14.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936538+0000", 2024-09-18T04:57:14.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040269394999999999 2024-09-18T04:57:14.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936657+0000", 2024-09-18T04:57:14.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119498 2024-09-18T04:57:14.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936931+0000", 2024-09-18T04:57:14.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027391900000000002 2024-09-18T04:57:14.758 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.758 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.758 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.758 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11405 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1658880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:14.758 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:57:14.758 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.811299824000002, 2024-09-18T04:57:14.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040891034999999999, 2024-09-18T04:57:14.758 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.758 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:14.758 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.758 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.759 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.759 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11405 2024-09-18T04:57:14.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.759 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:57:14.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:57:14.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896270+0000", 2024-09-18T04:57:14.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:57:14.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896272+0000", 2024-09-18T04:57:14.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.124e-06 2024-09-18T04:57:14.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896274+0000", 2024-09-18T04:57:14.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8660000000000002e-06 2024-09-18T04:57:14.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896285+0000", 2024-09-18T04:57:14.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0101e-05 2024-09-18T04:57:14.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936986+0000", 2024-09-18T04:57:14.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040701213999999999 2024-09-18T04:57:14.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937070+0000", 2024-09-18T04:57:14.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4211999999999999e-05 2024-09-18T04:57:14.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937162+0000", 2024-09-18T04:57:14.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1728e-05 2024-09-18T04:57:14.762 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.762 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.762 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.763 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11406 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1708032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:14.763 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:57:14.763 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.811283654999997, 2024-09-18T04:57:14.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041085251000000003, 2024-09-18T04:57:14.763 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.763 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:14.763 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.763 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.763 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.763 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11406 2024-09-18T04:57:14.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.763 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:57:14.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:57:14.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:57:14.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:57:14.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896288+0000", 2024-09-18T04:57:14.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2440000000000001e-06 2024-09-18T04:57:14.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896291+0000", 2024-09-18T04:57:14.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8289999999999999e-06 2024-09-18T04:57:14.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896293+0000", 2024-09-18T04:57:14.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5399999999999998e-06 2024-09-18T04:57:14.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937201+0000", 2024-09-18T04:57:14.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040907418000000001 2024-09-18T04:57:14.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937283+0000", 2024-09-18T04:57:14.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2460999999999995e-05 2024-09-18T04:57:14.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937372+0000", 2024-09-18T04:57:14.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8987999999999996e-05 2024-09-18T04:57:14.767 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.767 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.767 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.768 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11407 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:14.768 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:57:14.768 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.811273835000001, 2024-09-18T04:57:14.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041438580000000003, 2024-09-18T04:57:14.768 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.768 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:14.768 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.768 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.768 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.769 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11407 2024-09-18T04:57:14.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.769 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:57:14.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:57:14.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896296+0000", 2024-09-18T04:57:14.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:14.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896298+0000", 2024-09-18T04:57:14.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:57:14.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896304+0000", 2024-09-18T04:57:14.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6570000000000002e-06 2024-09-18T04:57:14.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896308+0000", 2024-09-18T04:57:14.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0940000000000001e-06 2024-09-18T04:57:14.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937540+0000", 2024-09-18T04:57:14.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041231469 2024-09-18T04:57:14.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937625+0000", 2024-09-18T04:57:14.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5767999999999996e-05 2024-09-18T04:57:14.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937735+0000", 2024-09-18T04:57:14.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010974 2024-09-18T04:57:14.772 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.772 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.772 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.772 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11408 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:14.772 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:57:14.773 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.811258195999997, 2024-09-18T04:57:14.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041653055000000001, 2024-09-18T04:57:14.773 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.773 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:14.773 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.773 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.773 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.773 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11408 2024-09-18T04:57:14.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.773 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:57:14.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:57:14.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:57:14.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:14.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896314+0000", 2024-09-18T04:57:14.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0140000000000001e-06 2024-09-18T04:57:14.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896320+0000", 2024-09-18T04:57:14.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-18T04:57:14.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896324+0000", 2024-09-18T04:57:14.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4410000000000003e-06 2024-09-18T04:57:14.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937769+0000", 2024-09-18T04:57:14.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041445243 2024-09-18T04:57:14.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937861+0000", 2024-09-18T04:57:14.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1471000000000006e-05 2024-09-18T04:57:14.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937965+0000", 2024-09-18T04:57:14.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104516 2024-09-18T04:57:14.776 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.776 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.776 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.777 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11409 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:14.777 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:57:14.777 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.811242139999997, 2024-09-18T04:57:14.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041840467999999999, 2024-09-18T04:57:14.777 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.777 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:14.777 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.777 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.777 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.777 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11409 2024-09-18T04:57:14.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.777 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:57:14.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:57:14.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:57:14.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:14.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896331+0000", 2024-09-18T04:57:14.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5950000000000001e-06 2024-09-18T04:57:14.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896336+0000", 2024-09-18T04:57:14.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7259999999999999e-06 2024-09-18T04:57:14.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896341+0000", 2024-09-18T04:57:14.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5619999999999997e-06 2024-09-18T04:57:14.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937995+0000", 2024-09-18T04:57:14.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041654060999999999 2024-09-18T04:57:14.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938078+0000", 2024-09-18T04:57:14.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2869999999999998e-05 2024-09-18T04:57:14.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938169+0000", 2024-09-18T04:57:14.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1040999999999998e-05 2024-09-18T04:57:14.781 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.781 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.781 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.781 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11410 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1921024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:14.781 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:57:14.782 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.811225657000001, 2024-09-18T04:57:14.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042060342000000001, 2024-09-18T04:57:14.782 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.782 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:14.782 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.783 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.783 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.783 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11410 2024-09-18T04:57:14.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.783 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:57:14.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:57:14.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896344+0000", 2024-09-18T04:57:14.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:14.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896360+0000", 2024-09-18T04:57:14.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5068e-05 2024-09-18T04:57:14.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896366+0000", 2024-09-18T04:57:14.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5119999999999997e-06 2024-09-18T04:57:14.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896371+0000", 2024-09-18T04:57:14.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:57:14.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938198+0000", 2024-09-18T04:57:14.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041826579000000003 2024-09-18T04:57:14.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938297+0000", 2024-09-18T04:57:14.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9067000000000005e-05 2024-09-18T04:57:14.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938405+0000", 2024-09-18T04:57:14.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108492 2024-09-18T04:57:14.786 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.786 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.786 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.786 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11411 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1970176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:14.787 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:57:14.787 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.811194815, 2024-09-18T04:57:14.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042259191000000002, 2024-09-18T04:57:14.787 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.787 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:14.787 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.787 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.787 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.787 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11411 2024-09-18T04:57:14.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.787 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:57:14.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:57:14.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896375+0000", 2024-09-18T04:57:14.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:14.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896377+0000", 2024-09-18T04:57:14.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2699999999999999e-06 2024-09-18T04:57:14.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896383+0000", 2024-09-18T04:57:14.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6740000000000002e-06 2024-09-18T04:57:14.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896388+0000", 2024-09-18T04:57:14.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:57:14.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938435+0000", 2024-09-18T04:57:14.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042047169000000002 2024-09-18T04:57:14.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938534+0000", 2024-09-18T04:57:14.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8939999999999998e-05 2024-09-18T04:57:14.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938635+0000", 2024-09-18T04:57:14.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101208 2024-09-18T04:57:14.791 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.791 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.791 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.791 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11412 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:14.791 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:57:14.791 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.811178833, 2024-09-18T04:57:14.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042828079999999998, 2024-09-18T04:57:14.791 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.791 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:14.791 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.791 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.791 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.792 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11412 2024-09-18T04:57:14.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.792 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:57:14.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:57:14.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896391+0000", 2024-09-18T04:57:14.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:14.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896393+0000", 2024-09-18T04:57:14.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0810000000000001e-06 2024-09-18T04:57:14.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896399+0000", 2024-09-18T04:57:14.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7230000000000001e-06 2024-09-18T04:57:14.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896403+0000", 2024-09-18T04:57:14.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3470000000000001e-06 2024-09-18T04:57:14.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939043+0000", 2024-09-18T04:57:14.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042639829999999997 2024-09-18T04:57:14.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939124+0000", 2024-09-18T04:57:14.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0809000000000005e-05 2024-09-18T04:57:14.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939220+0000", 2024-09-18T04:57:14.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5659e-05 2024-09-18T04:57:14.795 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.795 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.795 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.795 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11413 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2068480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:14.795 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:57:14.795 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.811163137999998, 2024-09-18T04:57:14.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043042108000000003, 2024-09-18T04:57:14.796 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.796 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:14.796 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.796 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.796 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.796 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11413 2024-09-18T04:57:14.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.796 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:57:14.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:57:14.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:57:14.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:14.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896409+0000", 2024-09-18T04:57:14.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.165e-06 2024-09-18T04:57:14.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896415+0000", 2024-09-18T04:57:14.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6149999999999996e-06 2024-09-18T04:57:14.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896419+0000", 2024-09-18T04:57:14.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2649999999999998e-06 2024-09-18T04:57:14.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939262+0000", 2024-09-18T04:57:14.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042843444000000001 2024-09-18T04:57:14.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939351+0000", 2024-09-18T04:57:14.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8944999999999996e-05 2024-09-18T04:57:14.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939449+0000", 2024-09-18T04:57:14.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8029000000000006e-05 2024-09-18T04:57:14.799 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.799 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.799 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.800 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11669 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2023424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:14.800 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.907990+0000", 2024-09-18T04:57:14.800 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.799580243999998, 2024-09-18T04:57:14.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040045942000000001, 2024-09-18T04:57:14.800 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.800 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:14.800 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.800 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.800 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.800 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11669 2024-09-18T04:57:14.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.800 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.907990+0000", 2024-09-18T04:57:14.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.907990+0000", 2024-09-18T04:57:14.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.907989+0000", 2024-09-18T04:57:14.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:14.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.907993+0000", 2024-09-18T04:57:14.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6109999999999998e-06 2024-09-18T04:57:14.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908003+0000", 2024-09-18T04:57:14.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7610000000000002e-06 2024-09-18T04:57:14.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908009+0000", 2024-09-18T04:57:14.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7660000000000004e-06 2024-09-18T04:57:14.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.947929+0000", 2024-09-18T04:57:14.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039920297 2024-09-18T04:57:14.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.947982+0000", 2024-09-18T04:57:14.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3176e-05 2024-09-18T04:57:14.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.948036+0000", 2024-09-18T04:57:14.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4052999999999999e-05 2024-09-18T04:57:14.804 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.804 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.804 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.804 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11670 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2072576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:14.804 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.908014+0000", 2024-09-18T04:57:14.804 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.799556363000001, 2024-09-18T04:57:14.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040179074000000002, 2024-09-18T04:57:14.804 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:14.804 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:14.804 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:14.804 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:14.805 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:14.805 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11670 2024-09-18T04:57:14.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.805 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:14.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:14.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908014+0000", 2024-09-18T04:57:14.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:14.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908014+0000", 2024-09-18T04:57:14.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:14.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:14.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908014+0000", 2024-09-18T04:57:14.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:14.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:14.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908016+0000", 2024-09-18T04:57:14.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3889999999999999e-06 2024-09-18T04:57:14.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:14.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908023+0000", 2024-09-18T04:57:14.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5599999999999999e-06 2024-09-18T04:57:14.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:14.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908028+0000", 2024-09-18T04:57:14.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.242e-06 2024-09-18T04:57:14.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:14.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.948075+0000", 2024-09-18T04:57:14.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040047117 2024-09-18T04:57:14.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:14.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.948128+0000", 2024-09-18T04:57:14.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3127999999999997e-05 2024-09-18T04:57:14.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:14.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:14.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:14.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.948193+0000", 2024-09-18T04:57:14.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5042000000000005e-05 2024-09-18T04:57:14.808 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.808 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.808 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.808 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:14.808 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:14.808 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:14.809 INFO:teuthology.orchestra.run.smithi002.stdout:4014 writesame 0xad21ed9 thru 0xad2ccbf data_size 0x1(0xade7 bytes) 2024-09-18T04:57:14.809 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:14.810 INFO:teuthology.orchestra.run.smithi002.stdout:4018 punch from 0x378e707 to 0x37936d2, (0x4fcb bytes) 2024-09-18T04:57:14.811 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:57:14.812 INFO:teuthology.orchestra.run.smithi002.stdout:4020 trunc from 0xad2ccc0 to 0x1242d80 2024-09-18T04:57:14.812 INFO:teuthology.orchestra.run.smithi002.stdout:4021 read 0x119270d thru 0x119dcb5 (0xb5a9 bytes) 2024-09-18T04:57:14.812 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:14.812 INFO:teuthology.orchestra.run.smithi002.stdout:4023 read 0x1107d76 thru 0x111354f (0xb7da bytes) 2024-09-18T04:57:14.812 INFO:teuthology.orchestra.run.smithi002.stdout:4025 writesame 0x781e4c thru 0x78952d data_size 0x1(0x76e2 bytes) 2024-09-18T04:57:14.812 INFO:teuthology.orchestra.run.smithi002.stdout:4026 write 0x91bed69 thru 0x91ce1de (0xf476 bytes) 2024-09-18T04:57:14.812 INFO:teuthology.orchestra.run.smithi002.stdout:4027 trunc from 0x91ce1df to 0xbb8c815 2024-09-18T04:57:14.812 INFO:teuthology.orchestra.run.smithi002.stdout:4028 trunc from 0xbb8c815 to 0x1642dbf 2024-09-18T04:57:14.812 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:14.806+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:14.821 INFO:teuthology.orchestra.run.smithi002.stdout:4029 write 0x92b886b thru 0x92c4e18 (0xc5ae bytes) 2024-09-18T04:57:14.826 INFO:teuthology.orchestra.run.smithi002.stdout:4030 writesame 0x78bc106 thru 0x78cbfb0 data_size 0x55(0xfeab bytes) 2024-09-18T04:57:14.831 INFO:teuthology.orchestra.run.smithi002.stdout:4031 punch from 0xee6242 to 0xef3229, (0xcfe7 bytes) 2024-09-18T04:57:14.831 INFO:teuthology.orchestra.run.smithi002.stdout:4032 write 0x4fd73d1 thru 0x4fdbb1e (0x474e bytes) 2024-09-18T04:57:14.841 INFO:teuthology.orchestra.run.smithi002.stdout:4033 trunc from 0x92c4e19 to 0x348aff3 2024-09-18T04:57:14.854 INFO:teuthology.orchestra.run.smithi002.stdout:4036 writesame 0x30ac10f thru 0x30afbf9 data_size 0x1(0x3aeb bytes) 2024-09-18T04:57:14.876 INFO:teuthology.orchestra.run.smithi002.stdout:4037 trunc from 0x348aff3 to 0xe8a223a 2024-09-18T04:57:14.881 INFO:teuthology.orchestra.run.smithi002.stdout:4038 read 0xa5d22f thru 0xa68bd3 (0xb9a5 bytes) 2024-09-18T04:57:14.881 INFO:teuthology.orchestra.run.smithi002.stdout:4039 write 0x8957b49 thru 0x895a245 (0x26fd bytes) 2024-09-18T04:57:14.885 INFO:teuthology.orchestra.run.smithi002.stdout:4041 punch from 0x66e9ced to 0x66f8e1e, (0xf131 bytes) 2024-09-18T04:57:14.895 INFO:teuthology.orchestra.run.smithi002.stdout:4042 trunc from 0xe8a223a to 0x539df5f 2024-09-18T04:57:14.909 INFO:teuthology.orchestra.run.smithi002.stdout:4043 write 0x150281d thru 0x150c5fa (0x9dde bytes) 2024-09-18T04:57:14.911 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:14.908+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:14.914 INFO:teuthology.orchestra.run.smithi002.stdout:4044 trunc from 0x539df5f to 0x722fb99 2024-09-18T04:57:14.918 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:14.918 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:14.918 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:57:14.918 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:14.920 INFO:teuthology.orchestra.run.smithi002.stdout:4045 read 0x6af0fbd thru 0x6af9fbd (0x9001 bytes) 2024-09-18T04:57:14.921 INFO:teuthology.orchestra.run.smithi002.stdout:4046 write 0x4a61a60 thru 0x4a6b675 (0x9c16 bytes) 2024-09-18T04:57:14.924 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:14.924 INFO:teuthology.orchestra.run.smithi002.stdout:4050 writesame 0x1dd23ee thru 0x1dde874 data_size 0x1(0xc487 bytes) 2024-09-18T04:57:14.929 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:57:14.939 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:14.940 INFO:teuthology.orchestra.run.smithi002.stdout:4055 read 0xbfcecb thru 0xbffea6 (0x2fdc bytes) 2024-09-18T04:57:14.944 INFO:teuthology.orchestra.run.smithi002.stdout:4056 trunc from 0x722fb99 to 0x2e8e400 2024-09-18T04:57:14.965 INFO:teuthology.orchestra.run.smithi002.stdout:4057 trunc from 0x2e8e400 to 0xaa69e1f 2024-09-18T04:57:14.971 INFO:teuthology.orchestra.run.smithi002.stdout:4058 punch from 0x272d806 to 0x272e785, (0xf7f bytes) 2024-09-18T04:57:14.971 INFO:teuthology.orchestra.run.smithi002.stdout:4059 read 0x56fed90 thru 0x570c366 (0xd5d7 bytes) 2024-09-18T04:57:14.971 INFO:teuthology.orchestra.run.smithi002.stdout:4060 write 0x8f0a257 thru 0x8f1059e (0x6348 bytes) 2024-09-18T04:57:14.975 INFO:teuthology.orchestra.run.smithi002.stdout:4062 read 0x92d1a6a thru 0x92e0e21 (0xf3b8 bytes) 2024-09-18T04:57:14.975 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:14.975 INFO:teuthology.orchestra.run.smithi002.stdout:4064 write 0x26a49d1 thru 0x26a6aa2 (0x20d2 bytes) 2024-09-18T04:57:14.978 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:14.978 INFO:teuthology.orchestra.run.smithi002.stdout:4066 write 0x7d25653 thru 0x7d2ec58 (0x9606 bytes) 2024-09-18T04:57:14.993 INFO:teuthology.orchestra.run.smithi002.stdout:4067 write 0xd0d9553 thru 0xd0e518a (0xbc38 bytes) 2024-09-18T04:57:14.997 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:14.997 INFO:teuthology.orchestra.run.smithi002.stdout:4069 punch from 0xc6c607 to 0xc72c04, (0x65fd bytes) 2024-09-18T04:57:14.997 INFO:teuthology.orchestra.run.smithi002.stdout:4072 write 0x8ecb976 thru 0x8ed9b3c (0xe1c7 bytes) 2024-09-18T04:57:15.011 INFO:teuthology.orchestra.run.smithi002.stdout:4073 trunc from 0xd0e518b to 0x6ffe4b1 2024-09-18T04:57:15.014 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:15.011+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:15.035 INFO:teuthology.orchestra.run.smithi002.stdout:4075 writesame 0xbb9dbd6 thru 0xbbab2ee data_size 0x5(0xd719 bytes) 2024-09-18T04:57:15.043 INFO:teuthology.orchestra.run.smithi002.stdout:4078 writesame 0xda66096 thru 0xda67ba4 data_size 0x1(0x1b0f bytes) 2024-09-18T04:57:15.043 INFO:teuthology.orchestra.run.smithi002.stdout:4079 writesame 0x1e6c30b thru 0x1e7ae38 data_size 0x1(0xeb2e bytes) 2024-09-18T04:57:15.068 INFO:teuthology.orchestra.run.smithi002.stdout:4080 write 0xe84ea38 thru 0xe85aa5a (0xc023 bytes) 2024-09-18T04:57:15.072 INFO:teuthology.orchestra.run.smithi002.stdout:4081 write 0x6ca8486 thru 0x6cac797 (0x4312 bytes) 2024-09-18T04:57:15.076 INFO:teuthology.orchestra.run.smithi002.stdout:4082 punch from 0x71172f9 to 0x7121875, (0xa57c bytes) 2024-09-18T04:57:15.076 INFO:teuthology.orchestra.run.smithi002.stdout:4083 writesame 0xe60a53 thru 0xe6c645 data_size 0x1(0xbbf3 bytes) 2024-09-18T04:57:15.077 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:15.077 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:15.077 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:57:15.077 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:57:15.078 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:15.087 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:57:15.091 INFO:teuthology.orchestra.run.smithi002.stdout:4084 read 0x5c824e9 thru 0x5c87268 (0x4d80 bytes) 2024-09-18T04:57:15.091 INFO:teuthology.orchestra.run.smithi002.stdout:4086 writesame 0x5a3e2b9 thru 0x5a44a41 data_size 0x5f(0x6789 bytes) 2024-09-18T04:57:15.109 INFO:teuthology.orchestra.run.smithi002.stdout:4089 trunc from 0xe85aa5b to 0x2a1c2f5 2024-09-18T04:57:15.121 INFO:teuthology.orchestra.run.smithi002.stdout:4090 trunc from 0x2a1c2f5 to 0x25ea575 2024-09-18T04:57:15.128 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:15.126+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:15.134 INFO:teuthology.orchestra.run.smithi002.stdout:4091 trunc from 0x25ea575 to 0x816a4d0 2024-09-18T04:57:15.139 INFO:teuthology.orchestra.run.smithi002.stdout:4092 punch from 0x7df6b85 to 0x7e058c8, (0xed43 bytes) 2024-09-18T04:57:15.139 INFO:teuthology.orchestra.run.smithi002.stdout:4093 read 0x5a015cc thru 0x5a06c1d (0x5652 bytes) 2024-09-18T04:57:15.139 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:15.140 INFO:teuthology.orchestra.run.smithi002.stdout:4096 punch from 0x5883c3a to 0x588c7d7, (0x8b9d bytes) 2024-09-18T04:57:15.140 INFO:teuthology.orchestra.run.smithi002.stdout:4098 write 0x5be152e thru 0x5bf024c (0xed1f bytes) 2024-09-18T04:57:15.144 INFO:teuthology.orchestra.run.smithi002.stdout:4099 write 0x424857c thru 0x4251e21 (0x98a6 bytes) 2024-09-18T04:57:15.148 INFO:teuthology.orchestra.run.smithi002.stdout:4100 read 0x61efdb9 thru 0x61fec77 (0xeebf bytes) 2024-09-18T04:57:15.148 INFO:teuthology.orchestra.run.smithi002.stdout:4103 punch from 0x5b20a88 to 0x5b252a6, (0x481e bytes) 2024-09-18T04:57:15.148 INFO:teuthology.orchestra.run.smithi002.stdout:4104 writesame 0x5e4d7ff thru 0x5e54138 data_size 0x1(0x693a bytes) 2024-09-18T04:57:15.158 INFO:teuthology.orchestra.run.smithi002.stdout:4107 read 0x4811b74 thru 0x4816607 (0x4a94 bytes) 2024-09-18T04:57:15.160 INFO:teuthology.orchestra.run.smithi002.stdout:4109 trunc from 0x816a4d0 to 0x9b7d15d 2024-09-18T04:57:15.175 INFO:teuthology.orchestra.run.smithi002.stdout:4111 trunc from 0x9b7d15d to 0x138a3c6 2024-09-18T04:57:15.196 INFO:teuthology.orchestra.run.smithi002.stdout:4112 read 0x264ce7 thru 0x2725be (0xd8d8 bytes) 2024-09-18T04:57:15.197 INFO:teuthology.orchestra.run.smithi002.stdout:4113 read 0x73568 thru 0x7f5ec (0xc085 bytes) 2024-09-18T04:57:15.197 INFO:teuthology.orchestra.run.smithi002.stdout:4114 punch from 0x5fbf8e to 0x600ed8, (0x4f4a bytes) 2024-09-18T04:57:15.209 INFO:teuthology.orchestra.run.smithi002.stdout:4115 writesame 0x67affeb thru 0x67b8244 data_size 0xa(0x825a bytes) 2024-09-18T04:57:15.209 INFO:teuthology.orchestra.run.smithi002.stdout:4116 writesame 0x340cc29 thru 0x34144e1 data_size 0x7(0x78b9 bytes) 2024-09-18T04:57:15.213 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:15.211+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:15.214 INFO:teuthology.orchestra.run.smithi002.stdout:4118 read 0x1413a44 thru 0x1416acb (0x3088 bytes) 2024-09-18T04:57:15.229 INFO:teuthology.orchestra.run.smithi002.stdout:4119 write 0xe48bd8c thru 0xe48ddd2 (0x2047 bytes) 2024-09-18T04:57:15.232 INFO:teuthology.orchestra.run.smithi002.stdout:4120 read 0x37c4d41 thru 0x37cda64 (0x8d24 bytes) 2024-09-18T04:57:15.232 INFO:teuthology.orchestra.run.smithi002.stdout:4121 write 0xb2080f2 thru 0xb211722 (0x9631 bytes) 2024-09-18T04:57:15.236 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:15.237 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:15.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:15.237 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:15.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.237 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72135 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 394240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:15.237 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:57:15.237 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.279527986000002, 2024-09-18T04:57:15.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048559739999999997, 2024-09-18T04:57:15.237 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.237 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.237 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.238 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.238 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.238 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72135 2024-09-18T04:57:15.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.238 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:57:15.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:57:15.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:57:15.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954005+0000", 2024-09-18T04:57:15.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5629999999999999e-06 2024-09-18T04:57:15.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954008+0000", 2024-09-18T04:57:15.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-18T04:57:15.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954012+0000", 2024-09-18T04:57:15.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4790000000000001e-06 2024-09-18T04:57:15.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002418+0000", 2024-09-18T04:57:15.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048406049 2024-09-18T04:57:15.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002460+0000", 2024-09-18T04:57:15.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2188999999999999e-05 2024-09-18T04:57:15.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002564+0000", 2024-09-18T04:57:15.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010377 2024-09-18T04:57:15.242 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.242 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.242 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.242 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72136 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 409600~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:15.242 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:57:15.243 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.279423039000001, 2024-09-18T04:57:15.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048670191000000002, 2024-09-18T04:57:15.243 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.243 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.243 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.243 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.243 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.243 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72136 2024-09-18T04:57:15.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.243 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:57:15.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:57:15.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954019+0000", 2024-09-18T04:57:15.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999099 2024-09-18T04:57:15.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954112+0000", 2024-09-18T04:57:15.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3273e-05 2024-09-18T04:57:15.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954121+0000", 2024-09-18T04:57:15.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0289999999999994e-06 2024-09-18T04:57:15.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954128+0000", 2024-09-18T04:57:15.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6189999999999996e-06 2024-09-18T04:57:15.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002604+0000", 2024-09-18T04:57:15.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048476048000000001 2024-09-18T04:57:15.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002682+0000", 2024-09-18T04:57:15.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7955999999999997e-05 2024-09-18T04:57:15.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002779+0000", 2024-09-18T04:57:15.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7372000000000002e-05 2024-09-18T04:57:15.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.247 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.248 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72138 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 440320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:15.248 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:57:15.248 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.278989187000001, 2024-09-18T04:57:15.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048549430999999997, 2024-09-18T04:57:15.248 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.248 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.248 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72138 2024-09-18T04:57:15.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.248 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:57:15.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:57:15.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954542+0000", 2024-09-18T04:57:15.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954545+0000", 2024-09-18T04:57:15.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4360000000000001e-06 2024-09-18T04:57:15.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954551+0000", 2024-09-18T04:57:15.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4949999999999996e-06 2024-09-18T04:57:15.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954557+0000", 2024-09-18T04:57:15.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5099999999999998e-06 2024-09-18T04:57:15.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002993+0000", 2024-09-18T04:57:15.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048436358999999998 2024-09-18T04:57:15.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003039+0000", 2024-09-18T04:57:15.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5884000000000003e-05 2024-09-18T04:57:15.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003092+0000", 2024-09-18T04:57:15.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3208000000000003e-05 2024-09-18T04:57:15.252 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.253 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.253 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.253 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72139 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 455680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:15.253 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:57:15.253 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.278970379, 2024-09-18T04:57:15.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048718773, 2024-09-18T04:57:15.253 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.253 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.254 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.254 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.254 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.254 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72139 2024-09-18T04:57:15.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.254 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:57:15.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:57:15.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954561+0000", 2024-09-18T04:57:15.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954564+0000", 2024-09-18T04:57:15.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4099999999999998e-06 2024-09-18T04:57:15.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954570+0000", 2024-09-18T04:57:15.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.178e-06 2024-09-18T04:57:15.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954575+0000", 2024-09-18T04:57:15.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0560000000000004e-06 2024-09-18T04:57:15.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003114+0000", 2024-09-18T04:57:15.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048539212999999998 2024-09-18T04:57:15.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003176+0000", 2024-09-18T04:57:15.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2154000000000002e-05 2024-09-18T04:57:15.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003280+0000", 2024-09-18T04:57:15.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010421600000000001 2024-09-18T04:57:15.259 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.259 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.259 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.259 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72140 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 471040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:15.260 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:57:15.260 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.278714438000002, 2024-09-18T04:57:15.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048670819999999997, 2024-09-18T04:57:15.260 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.260 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.260 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.260 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.260 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.260 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72140 2024-09-18T04:57:15.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.261 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:57:15.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:57:15.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954579+0000", 2024-09-18T04:57:15.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997611 2024-09-18T04:57:15.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954821+0000", 2024-09-18T04:57:15.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024241000000000001 2024-09-18T04:57:15.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954831+0000", 2024-09-18T04:57:15.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8770000000000005e-06 2024-09-18T04:57:15.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954838+0000", 2024-09-18T04:57:15.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0690000000000004e-06 2024-09-18T04:57:15.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003317+0000", 2024-09-18T04:57:15.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048478796999999997 2024-09-18T04:57:15.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003392+0000", 2024-09-18T04:57:15.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4411000000000003e-05 2024-09-18T04:57:15.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003488+0000", 2024-09-18T04:57:15.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6927000000000002e-05 2024-09-18T04:57:15.269 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.269 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.269 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.270 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72141 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 486400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:15.270 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:57:15.270 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.278686955000001, 2024-09-18T04:57:15.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048808114, 2024-09-18T04:57:15.270 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.270 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.271 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72141 2024-09-18T04:57:15.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.272 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:57:15.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:57:15.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:57:15.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954847+0000", 2024-09-18T04:57:15.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3800000000000001e-06 2024-09-18T04:57:15.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954854+0000", 2024-09-18T04:57:15.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6080000000000001e-06 2024-09-18T04:57:15.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954859+0000", 2024-09-18T04:57:15.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:57:15.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003526+0000", 2024-09-18T04:57:15.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048666433000000002 2024-09-18T04:57:15.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003599+0000", 2024-09-18T04:57:15.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2854000000000004e-05 2024-09-18T04:57:15.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003653+0000", 2024-09-18T04:57:15.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4511e-05 2024-09-18T04:57:15.280 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.280 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.280 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.280 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72142 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 501760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:15.281 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:57:15.281 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.278648152999999, 2024-09-18T04:57:15.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048885500999999998, 2024-09-18T04:57:15.281 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.281 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.281 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.282 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.282 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.282 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72142 2024-09-18T04:57:15.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.282 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:57:15.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:57:15.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954864+0000", 2024-09-18T04:57:15.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999804 2024-09-18T04:57:15.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954886+0000", 2024-09-18T04:57:15.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2042999999999999e-05 2024-09-18T04:57:15.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954893+0000", 2024-09-18T04:57:15.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.255e-06 2024-09-18T04:57:15.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954899+0000", 2024-09-18T04:57:15.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8780000000000003e-06 2024-09-18T04:57:15.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003678+0000", 2024-09-18T04:57:15.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048778834 2024-09-18T04:57:15.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003717+0000", 2024-09-18T04:57:15.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8860999999999999e-05 2024-09-18T04:57:15.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003769+0000", 2024-09-18T04:57:15.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2389000000000003e-05 2024-09-18T04:57:15.288 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.288 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.288 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.288 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72143 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 517120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:15.289 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:57:15.289 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.278522527, 2024-09-18T04:57:15.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048951738000000002, 2024-09-18T04:57:15.289 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.289 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.290 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.290 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.290 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.290 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72143 2024-09-18T04:57:15.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.290 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:57:15.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:57:15.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954904+0000", 2024-09-18T04:57:15.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998946 2024-09-18T04:57:15.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955013+0000", 2024-09-18T04:57:15.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108977 2024-09-18T04:57:15.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955021+0000", 2024-09-18T04:57:15.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7570000000000004e-06 2024-09-18T04:57:15.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955025+0000", 2024-09-18T04:57:15.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1219999999999997e-06 2024-09-18T04:57:15.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003792+0000", 2024-09-18T04:57:15.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048767379 2024-09-18T04:57:15.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003860+0000", 2024-09-18T04:57:15.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8033000000000004e-05 2024-09-18T04:57:15.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003961+0000", 2024-09-18T04:57:15.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010103300000000001 2024-09-18T04:57:15.296 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.296 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.296 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.296 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72161 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 13312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:15.296 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:57:15.297 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.27825889, 2024-09-18T04:57:15.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048891539999999997, 2024-09-18T04:57:15.297 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.297 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.297 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.297 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.297 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.297 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72161 2024-09-18T04:57:15.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.297 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:57:15.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:57:15.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955028+0000", 2024-09-18T04:57:15.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997549 2024-09-18T04:57:15.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955278+0000", 2024-09-18T04:57:15.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024973299999999998 2024-09-18T04:57:15.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955289+0000", 2024-09-18T04:57:15.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0812000000000001e-05 2024-09-18T04:57:15.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955296+0000", 2024-09-18T04:57:15.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2570000000000001e-06 2024-09-18T04:57:15.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004002+0000", 2024-09-18T04:57:15.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048705904000000001 2024-09-18T04:57:15.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004076+0000", 2024-09-18T04:57:15.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4086999999999997e-05 2024-09-18T04:57:15.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004165+0000", 2024-09-18T04:57:15.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8712e-05 2024-09-18T04:57:15.303 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.303 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.303 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.303 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72163 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 44032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:15.303 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:57:15.303 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.277633452, 2024-09-18T04:57:15.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048567011, 2024-09-18T04:57:15.303 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.303 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.303 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.304 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.304 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.304 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72163 2024-09-18T04:57:15.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.304 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:57:15.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:57:15.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955898+0000", 2024-09-18T04:57:15.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955901+0000", 2024-09-18T04:57:15.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.706e-06 2024-09-18T04:57:15.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955907+0000", 2024-09-18T04:57:15.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3559999999999998e-06 2024-09-18T04:57:15.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955912+0000", 2024-09-18T04:57:15.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9640000000000002e-06 2024-09-18T04:57:15.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004377+0000", 2024-09-18T04:57:15.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048465293999999999 2024-09-18T04:57:15.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004418+0000", 2024-09-18T04:57:15.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0163000000000002e-05 2024-09-18T04:57:15.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004466+0000", 2024-09-18T04:57:15.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8004000000000003e-05 2024-09-18T04:57:15.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.308 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.309 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72164 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:15.309 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:57:15.309 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.277615179000001, 2024-09-18T04:57:15.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048707253999999998, 2024-09-18T04:57:15.310 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.310 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.310 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.310 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.310 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.310 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72164 2024-09-18T04:57:15.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.310 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:57:15.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:57:15.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:57:15.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955919+0000", 2024-09-18T04:57:15.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2520000000000002e-06 2024-09-18T04:57:15.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955925+0000", 2024-09-18T04:57:15.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3740000000000003e-06 2024-09-18T04:57:15.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955930+0000", 2024-09-18T04:57:15.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:57:15.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004489+0000", 2024-09-18T04:57:15.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048558581000000003 2024-09-18T04:57:15.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004529+0000", 2024-09-18T04:57:15.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9722999999999999e-05 2024-09-18T04:57:15.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004624+0000", 2024-09-18T04:57:15.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5402999999999994e-05 2024-09-18T04:57:15.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.314 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.315 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72165 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 74752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:15.315 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:57:15.315 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.27734032, 2024-09-18T04:57:15.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048633223000000003, 2024-09-18T04:57:15.315 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.315 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.316 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.316 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.316 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.316 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72165 2024-09-18T04:57:15.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.316 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:57:15.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:57:15.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955935+0000", 2024-09-18T04:57:15.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999743 2024-09-18T04:57:15.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956199+0000", 2024-09-18T04:57:15.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000263906 2024-09-18T04:57:15.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956207+0000", 2024-09-18T04:57:15.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2519999999999999e-06 2024-09-18T04:57:15.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956214+0000", 2024-09-18T04:57:15.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7610000000000002e-06 2024-09-18T04:57:15.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004661+0000", 2024-09-18T04:57:15.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048447967000000002 2024-09-18T04:57:15.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004735+0000", 2024-09-18T04:57:15.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3789999999999997e-05 2024-09-18T04:57:15.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004825+0000", 2024-09-18T04:57:15.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9724999999999999e-05 2024-09-18T04:57:15.321 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.321 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.321 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.321 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72166 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 90112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:15.321 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:57:15.321 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.277086786000002, 2024-09-18T04:57:15.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048595737999999999, 2024-09-18T04:57:15.322 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.322 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.322 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.322 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.323 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.323 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72166 2024-09-18T04:57:15.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.323 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:57:15.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:57:15.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956221+0000", 2024-09-18T04:57:15.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997759 2024-09-18T04:57:15.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956448+0000", 2024-09-18T04:57:15.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022685500000000001 2024-09-18T04:57:15.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956456+0000", 2024-09-18T04:57:15.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0099999999999995e-06 2024-09-18T04:57:15.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956462+0000", 2024-09-18T04:57:15.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7200000000000003e-06 2024-09-18T04:57:15.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004862+0000", 2024-09-18T04:57:15.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048399795000000002 2024-09-18T04:57:15.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004941+0000", 2024-09-18T04:57:15.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9414999999999998e-05 2024-09-18T04:57:15.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005041+0000", 2024-09-18T04:57:15.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9942e-05 2024-09-18T04:57:15.327 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.328 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.328 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.328 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72167 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:15.328 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:57:15.328 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.276999748000001, 2024-09-18T04:57:15.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048622894999999999, 2024-09-18T04:57:15.328 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.328 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.329 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.329 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.329 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72167 2024-09-18T04:57:15.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.329 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:57:15.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:57:15.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956468+0000", 2024-09-18T04:57:15.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999352 2024-09-18T04:57:15.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956535+0000", 2024-09-18T04:57:15.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7440999999999993e-05 2024-09-18T04:57:15.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956544+0000", 2024-09-18T04:57:15.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1039999999999996e-06 2024-09-18T04:57:15.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956550+0000", 2024-09-18T04:57:15.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9850000000000003e-06 2024-09-18T04:57:15.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005066+0000", 2024-09-18T04:57:15.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048515614999999998 2024-09-18T04:57:15.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005107+0000", 2024-09-18T04:57:15.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0843999999999997e-05 2024-09-18T04:57:15.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005155+0000", 2024-09-18T04:57:15.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8686000000000001e-05 2024-09-18T04:57:15.333 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.333 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.333 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.333 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72168 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:15.333 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:57:15.333 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.276838685000001, 2024-09-18T04:57:15.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048585179999999999, 2024-09-18T04:57:15.334 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.334 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.334 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.334 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.334 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.334 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72168 2024-09-18T04:57:15.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.334 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:57:15.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:57:15.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956555+0000", 2024-09-18T04:57:15.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998612 2024-09-18T04:57:15.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956696+0000", 2024-09-18T04:57:15.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000140922 2024-09-18T04:57:15.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956707+0000", 2024-09-18T04:57:15.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1555e-05 2024-09-18T04:57:15.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956714+0000", 2024-09-18T04:57:15.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2709999999999998e-06 2024-09-18T04:57:15.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005177+0000", 2024-09-18T04:57:15.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048463421 2024-09-18T04:57:15.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005214+0000", 2024-09-18T04:57:15.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.676e-05 2024-09-18T04:57:15.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005279+0000", 2024-09-18T04:57:15.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4899000000000003e-05 2024-09-18T04:57:15.338 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.339 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.339 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.339 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72172 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 441344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:15.339 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.957556+0000", 2024-09-18T04:57:15.339 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.275976284999999, 2024-09-18T04:57:15.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048545056000000003, 2024-09-18T04:57:15.339 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.340 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.340 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.340 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.340 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.340 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72172 2024-09-18T04:57:15.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.340 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957556+0000", 2024-09-18T04:57:15.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957556+0000", 2024-09-18T04:57:15.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957555+0000", 2024-09-18T04:57:15.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957558+0000", 2024-09-18T04:57:15.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2900000000000001e-06 2024-09-18T04:57:15.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957564+0000", 2024-09-18T04:57:15.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0700000000000003e-06 2024-09-18T04:57:15.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957569+0000", 2024-09-18T04:57:15.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7870000000000001e-06 2024-09-18T04:57:15.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005922+0000", 2024-09-18T04:57:15.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048353259000000003 2024-09-18T04:57:15.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005999+0000", 2024-09-18T04:57:15.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6889000000000002e-05 2024-09-18T04:57:15.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.006101+0000", 2024-09-18T04:57:15.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102136 2024-09-18T04:57:15.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.345 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.345 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72199 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:15.345 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:57:15.346 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.274306793000001, 2024-09-18T04:57:15.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048626057, 2024-09-18T04:57:15.346 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.346 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.346 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.346 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.346 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.346 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72199 2024-09-18T04:57:15.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.346 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:57:15.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:57:15.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959195+0000", 2024-09-18T04:57:15.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999969 2024-09-18T04:57:15.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959228+0000", 2024-09-18T04:57:15.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3609000000000003e-05 2024-09-18T04:57:15.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959236+0000", 2024-09-18T04:57:15.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.157e-06 2024-09-18T04:57:15.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959243+0000", 2024-09-18T04:57:15.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5910000000000001e-06 2024-09-18T04:57:15.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007684+0000", 2024-09-18T04:57:15.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048440745 2024-09-18T04:57:15.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007762+0000", 2024-09-18T04:57:15.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8360000000000002e-05 2024-09-18T04:57:15.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007851+0000", 2024-09-18T04:57:15.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9388000000000005e-05 2024-09-18T04:57:15.350 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.350 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.350 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.350 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72210 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 503808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:15.351 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.961205+0000", 2024-09-18T04:57:15.351 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.272326802999999, 2024-09-18T04:57:15.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048502545000000001, 2024-09-18T04:57:15.351 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.351 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.351 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.351 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.351 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.351 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72210 2024-09-18T04:57:15.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.351 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.961205+0000", 2024-09-18T04:57:15.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.961205+0000", 2024-09-18T04:57:15.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.961205+0000", 2024-09-18T04:57:15.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.961207+0000", 2024-09-18T04:57:15.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.261e-06 2024-09-18T04:57:15.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.961213+0000", 2024-09-18T04:57:15.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6169999999999998e-06 2024-09-18T04:57:15.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.961217+0000", 2024-09-18T04:57:15.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6539999999999998e-06 2024-09-18T04:57:15.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.009550+0000", 2024-09-18T04:57:15.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048332561000000003 2024-09-18T04:57:15.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.009621+0000", 2024-09-18T04:57:15.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1599000000000003e-05 2024-09-18T04:57:15.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.009708+0000", 2024-09-18T04:57:15.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6372999999999994e-05 2024-09-18T04:57:15.356 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.356 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.356 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.356 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72239 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 412672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:15.356 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.963222+0000", 2024-09-18T04:57:15.356 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.270310517999999, 2024-09-18T04:57:15.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048529088999999997, 2024-09-18T04:57:15.357 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.357 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.357 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.357 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.357 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.357 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72239 2024-09-18T04:57:15.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.357 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963222+0000", 2024-09-18T04:57:15.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963222+0000", 2024-09-18T04:57:15.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963221+0000", 2024-09-18T04:57:15.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963223+0000", 2024-09-18T04:57:15.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.136e-06 2024-09-18T04:57:15.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963231+0000", 2024-09-18T04:57:15.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7349999999999996e-06 2024-09-18T04:57:15.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963236+0000", 2024-09-18T04:57:15.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3550000000000004e-06 2024-09-18T04:57:15.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011591+0000", 2024-09-18T04:57:15.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048354772999999997 2024-09-18T04:57:15.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011661+0000", 2024-09-18T04:57:15.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9759999999999996e-05 2024-09-18T04:57:15.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011751+0000", 2024-09-18T04:57:15.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9689000000000001e-05 2024-09-18T04:57:15.361 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.361 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.361 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.362 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72240 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 428032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:15.362 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:57:15.362 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.270213482999999, 2024-09-18T04:57:15.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048643767999999997, 2024-09-18T04:57:15.362 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.362 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.362 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.362 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.362 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.362 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72240 2024-09-18T04:57:15.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.362 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:57:15.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:57:15.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963240+0000", 2024-09-18T04:57:15.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999218 2024-09-18T04:57:15.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963323+0000", 2024-09-18T04:57:15.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2375999999999999e-05 2024-09-18T04:57:15.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963332+0000", 2024-09-18T04:57:15.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1700000000000003e-06 2024-09-18T04:57:15.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963338+0000", 2024-09-18T04:57:15.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2149999999999998e-06 2024-09-18T04:57:15.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011785+0000", 2024-09-18T04:57:15.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048446493 2024-09-18T04:57:15.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011861+0000", 2024-09-18T04:57:15.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6472999999999998e-05 2024-09-18T04:57:15.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011962+0000", 2024-09-18T04:57:15.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101167 2024-09-18T04:57:15.366 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.366 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.366 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.366 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.366 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.366 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:15.369 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:57:15.370 INFO:teuthology.orchestra.run.smithi002.stdout:4122 writesame 0x64a6471 thru 0x64ae1d5 data_size 0x1(0x7d65 bytes) 2024-09-18T04:57:15.370 INFO:teuthology.orchestra.run.smithi002.stdout:4123 trunc from 0xe48ddd3 to 0x902c02a 2024-09-18T04:57:15.370 INFO:teuthology.orchestra.run.smithi002.stdout:4124 write 0x727b41d thru 0x7288dd1 (0xd9b5 bytes) 2024-09-18T04:57:15.371 INFO:teuthology.orchestra.run.smithi002.stdout:4125 trunc from 0x902c02a to 0x300510a 2024-09-18T04:57:15.371 INFO:teuthology.orchestra.run.smithi002.stdout:4127 read 0x8c4ba6 thru 0x8ce0a4 (0x94ff bytes) 2024-09-18T04:57:15.371 INFO:teuthology.orchestra.run.smithi002.stdout:4129 write 0x969ceae thru 0x96a4292 (0x73e5 bytes) 2024-09-18T04:57:15.371 INFO:teuthology.orchestra.run.smithi002.stdout:4130 punch from 0x155f3b3 to 0x156c4e7, (0xd134 bytes) 2024-09-18T04:57:15.371 INFO:teuthology.orchestra.run.smithi002.stdout:4131 read 0x125a177 thru 0x1264738 (0xa5c2 bytes) 2024-09-18T04:57:15.371 INFO:teuthology.orchestra.run.smithi002.stdout:4132 write 0xa5ccded thru 0xa5db105 (0xe319 bytes) 2024-09-18T04:57:15.371 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:15.371 INFO:teuthology.orchestra.run.smithi002.stdout:4134 read 0x1b0d0b3 thru 0x1b18a53 (0xb9a1 bytes) 2024-09-18T04:57:15.371 INFO:teuthology.orchestra.run.smithi002.stdout:4135 read 0x1d77899 thru 0x1d7c16f (0x48d7 bytes) 2024-09-18T04:57:15.371 INFO:teuthology.orchestra.run.smithi002.stdout:4137 read 0x6041ccd thru 0x6049518 (0x784c bytes) 2024-09-18T04:57:15.371 INFO:teuthology.orchestra.run.smithi002.stdout:4140 punch from 0x557741e to 0x5585ac8, (0xe6aa bytes) 2024-09-18T04:57:15.371 INFO:teuthology.orchestra.run.smithi002.stdout:4141 write 0x5c6c010 thru 0x5c7416a (0x815b bytes) 2024-09-18T04:57:15.371 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:15.372 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:15.372 INFO:teuthology.orchestra.run.smithi002.stdout:4144 writesame 0x670e6be thru 0x6711f53 data_size 0x1(0x3896 bytes) 2024-09-18T04:57:15.372 INFO:teuthology.orchestra.run.smithi002.stdout:4145 punch from 0x9db29aa to 0x9dbec63, (0xc2b9 bytes) 2024-09-18T04:57:15.372 INFO:teuthology.orchestra.run.smithi002.stdout:4146 writesame 0x49860dc thru 0x498634b data_size 0x8(0x270 bytes) 2024-09-18T04:57:15.372 INFO:teuthology.orchestra.run.smithi002.stdout:4147 write 0x2394058 thru 0x2398bf2 (0x4b9b bytes) 2024-09-18T04:57:15.372 INFO:teuthology.orchestra.run.smithi002.stdout:4148 trunc from 0xa5db106 to 0x676af27 2024-09-18T04:57:15.373 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.0 to 1 (10000) 2024-09-18T04:57:15.373 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-18T04:57:15.374 INFO:teuthology.orchestra.run.smithi002.stdout:4149 writesame 0x78e7ef2 thru 0x78ea247 data_size 0x1(0x2356 bytes) 2024-09-18T04:57:15.374 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:15.374 INFO:teuthology.orchestra.run.smithi002.stdout:4151 punch from 0x25f4a6a to 0x25f8fcc, (0x4562 bytes) 2024-09-18T04:57:15.374 INFO:teuthology.orchestra.run.smithi002.stdout:4152 read 0x2a8d6 thru 0x2d839 (0x2f64 bytes) 2024-09-18T04:57:15.375 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:15.364+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:15.375 INFO:teuthology.orchestra.run.smithi002.stdout:4154 trunc from 0x78ea248 to 0xca5cb49 2024-09-18T04:57:15.377 INFO:teuthology.orchestra.run.smithi002.stdout:4155 read 0xadea7e5 thru 0xadf1fc6 (0x77e2 bytes) 2024-09-18T04:57:15.377 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:15.385 INFO:teuthology.orchestra.run.smithi002.stdout:4157 write 0xe3caab6 thru 0xe3d7ff0 (0xd53b bytes) 2024-09-18T04:57:15.389 INFO:teuthology.orchestra.run.smithi002.stdout:4158 write 0x1b45746 thru 0x1b54428 (0xece3 bytes) 2024-09-18T04:57:15.394 INFO:teuthology.orchestra.run.smithi002.stdout:4160 read 0x3734d9f thru 0x373da52 (0x8cb4 bytes) 2024-09-18T04:57:15.394 INFO:teuthology.orchestra.run.smithi002.stdout:4161 read 0x3c91258 thru 0x3c9bf33 (0xacdc bytes) 2024-09-18T04:57:15.394 INFO:teuthology.orchestra.run.smithi002.stdout:4162 read 0x3e1f561 thru 0x3e26de4 (0x7884 bytes) 2024-09-18T04:57:15.394 INFO:teuthology.orchestra.run.smithi002.stdout:4163 writesame 0xbc6237e thru 0xbc65c3b data_size 0x1b(0x38be bytes) 2024-09-18T04:57:15.397 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:15.397 INFO:teuthology.orchestra.run.smithi002.stdout:4165 read 0xe24af30 thru 0xe24dd9b (0x2e6c bytes) 2024-09-18T04:57:15.397 INFO:teuthology.orchestra.run.smithi002.stdout:4166 write 0x3a26e10 thru 0x3a36cc9 (0xfeba bytes) 2024-09-18T04:57:15.403 INFO:teuthology.orchestra.run.smithi002.stdout:4167 write 0xa3105da thru 0xa313002 (0x2a29 bytes) 2024-09-18T04:57:15.410 INFO:teuthology.orchestra.run.smithi002.stdout:4169 trunc from 0xe3d7ff1 to 0xafe3b39 2024-09-18T04:57:15.421 INFO:teuthology.orchestra.run.smithi002.stdout:4170 punch from 0x83b3a19 to 0x83c1b6b, (0xe152 bytes) 2024-09-18T04:57:15.431 INFO:teuthology.orchestra.run.smithi002.stdout:4171 writesame 0xe066d28 thru 0xe070edb data_size 0x83(0xa1b4 bytes) 2024-09-18T04:57:15.439 INFO:teuthology.orchestra.run.smithi002.stdout:4173 trunc from 0xe070edc to 0x65c1b10 2024-09-18T04:57:15.458 INFO:teuthology.orchestra.run.smithi002.stdout:4174 write 0x7e15f25 thru 0x7e21021 (0xb0fd bytes) 2024-09-18T04:57:15.463 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:15.464 INFO:teuthology.orchestra.run.smithi002.stdout:4176 punch from 0x7d9969 to 0x7e1f8c, (0x8623 bytes) 2024-09-18T04:57:15.464 INFO:teuthology.orchestra.run.smithi002.stdout:4177 read 0x1136e2d thru 0x11404de (0x96b2 bytes) 2024-09-18T04:57:15.469 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:15.466+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:15.476 INFO:teuthology.orchestra.run.smithi002.stdout:4179 write 0xd15c9e0 thru 0xd16102b (0x464c bytes) 2024-09-18T04:57:15.480 INFO:teuthology.orchestra.run.smithi002.stdout:4182 read 0x80006a5 thru 0x80055c2 (0x4f1e bytes) 2024-09-18T04:57:15.481 INFO:teuthology.orchestra.run.smithi002.stdout:4183 punch from 0x64d97f8 to 0x64e1fbd, (0x87c5 bytes) 2024-09-18T04:57:15.481 INFO:teuthology.orchestra.run.smithi002.stdout:4185 write 0x5c49df6 thru 0x5c57306 (0xd511 bytes) 2024-09-18T04:57:15.485 INFO:teuthology.orchestra.run.smithi002.stdout:4186 read 0x3009ad5 thru 0x300dfc4 (0x44f0 bytes) 2024-09-18T04:57:15.485 INFO:teuthology.orchestra.run.smithi002.stdout:4187 read 0x8eaef5b thru 0x8eb4b00 (0x5ba6 bytes) 2024-09-18T04:57:15.485 INFO:teuthology.orchestra.run.smithi002.stdout:4188 read 0xaf0ff6a thru 0xaf13f9c (0x4033 bytes) 2024-09-18T04:57:15.485 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:15.485 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:15.485 INFO:teuthology.orchestra.run.smithi002.stdout:4193 writesame 0x327b029 thru 0x32859cb data_size 0x1(0xa9a3 bytes) 2024-09-18T04:57:15.506 INFO:teuthology.orchestra.run.smithi002.stdout:4194 write 0xe96ee5b thru 0xe97e591 (0xf737 bytes) 2024-09-18T04:57:15.514 INFO:teuthology.orchestra.run.smithi002.stdout:4196 read 0xabeab80 thru 0xabfa957 (0xfdd8 bytes) 2024-09-18T04:57:15.514 INFO:teuthology.orchestra.run.smithi002.stdout:4197 writesame 0x5ad8ca4 thru 0x5ae3ad7 data_size 0x2(0xae34 bytes) 2024-09-18T04:57:15.519 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:15.519 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:15.519 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:57:15.519 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:15.529 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:57:15.530 INFO:teuthology.orchestra.run.smithi002.stdout:4198 writesame 0xecf14cb thru 0xecfbbcb data_size 0x1(0xa701 bytes) 2024-09-18T04:57:15.530 INFO:teuthology.orchestra.run.smithi002.stdout:4199 punch from 0xb8b5f00 to 0xb8bd0d4, (0x71d4 bytes) 2024-09-18T04:57:15.531 INFO:teuthology.orchestra.run.smithi002.stdout:4200 read 0x95ecf3e thru 0x95ee8e3 (0x19a6 bytes) 2024-09-18T04:57:15.531 INFO:teuthology.orchestra.run.smithi002.stdout:4201 punch from 0x14b83f1 to 0x14b9e94, (0x1aa3 bytes) 2024-09-18T04:57:15.531 INFO:teuthology.orchestra.run.smithi002.stdout:4202 write 0x609dc07 thru 0x60acf9d (0xf397 bytes) 2024-09-18T04:57:15.535 INFO:teuthology.orchestra.run.smithi002.stdout:4203 punch from 0x86c15b7 to 0x86c42db, (0x2d24 bytes) 2024-09-18T04:57:15.535 INFO:teuthology.orchestra.run.smithi002.stdout:4205 read 0xa295464 thru 0xa29a530 (0x50cd bytes) 2024-09-18T04:57:15.535 INFO:teuthology.orchestra.run.smithi002.stdout:4206 write 0x652abec thru 0x65335f6 (0x8a0b bytes) 2024-09-18T04:57:15.537 INFO:teuthology.orchestra.run.smithi002.stdout:4207 write 0x3d93c4d thru 0x3d95a59 (0x1e0d bytes) 2024-09-18T04:57:15.541 INFO:teuthology.orchestra.run.smithi002.stdout:4208 write 0x25be389 thru 0x25c1fab (0x3c23 bytes) 2024-09-18T04:57:15.546 INFO:teuthology.orchestra.run.smithi002.stdout:4209 read 0xb7eaeb1 thru 0xb7f83dc (0xd52c bytes) 2024-09-18T04:57:15.546 INFO:teuthology.orchestra.run.smithi002.stdout:4212 writesame 0x780a6b1 thru 0x781454c data_size 0x2(0x9e9c bytes) 2024-09-18T04:57:15.554 INFO:teuthology.orchestra.run.smithi002.stdout:4213 read 0x3c6e055 thru 0x3c775d7 (0x9583 bytes) 2024-09-18T04:57:15.554 INFO:teuthology.orchestra.run.smithi002.stdout:4214 read 0x2319813 thru 0x231f159 (0x5947 bytes) 2024-09-18T04:57:15.555 INFO:teuthology.orchestra.run.smithi002.stdout:4215 write 0xeef033 thru 0xef04f3 (0x14c1 bytes) 2024-09-18T04:57:15.559 INFO:teuthology.orchestra.run.smithi002.stdout:4217 trunc from 0xecfbbcc to 0xcb9fa62 2024-09-18T04:57:15.570 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:15.568+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:15.573 INFO:teuthology.orchestra.run.smithi002.stdout:4218 read 0x371e3f thru 0x371ee8 (0xaa bytes) 2024-09-18T04:57:15.573 INFO:teuthology.orchestra.run.smithi002.stdout:4219 punch from 0xc1c0c14 to 0xc1c71e5, (0x65d1 bytes) 2024-09-18T04:57:15.574 INFO:teuthology.orchestra.run.smithi002.stdout:4220 write 0xc205fe7 thru 0xc206836 (0x850 bytes) 2024-09-18T04:57:15.576 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:15.576 INFO:teuthology.orchestra.run.smithi002.stdout:4224 write 0x5db578a thru 0x5db67a7 (0x101e bytes) 2024-09-18T04:57:15.578 INFO:teuthology.orchestra.run.smithi002.stdout:4227 read 0x8a9c1d9 thru 0x8a9f97e (0x37a6 bytes) 2024-09-18T04:57:15.578 INFO:teuthology.orchestra.run.smithi002.stdout:4228 read 0x9dafd0b thru 0x9db6ffd (0x72f3 bytes) 2024-09-18T04:57:15.578 INFO:teuthology.orchestra.run.smithi002.stdout:4229 read 0x579d910 thru 0x579eade (0x11cf bytes) 2024-09-18T04:57:15.578 INFO:teuthology.orchestra.run.smithi002.stdout:4230 write 0x47f5bd4 thru 0x47f99c0 (0x3ded bytes) 2024-09-18T04:57:15.582 INFO:teuthology.orchestra.run.smithi002.stdout:4231 read 0x9cd4d33 thru 0x9cd57dc (0xaaa bytes) 2024-09-18T04:57:15.582 INFO:teuthology.orchestra.run.smithi002.stdout:4233 read 0x44f21d6 thru 0x44fdbef (0xba1a bytes) 2024-09-18T04:57:15.583 INFO:teuthology.orchestra.run.smithi002.stdout:4234 read 0x86c338f thru 0x86ca9c1 (0x7633 bytes) 2024-09-18T04:57:15.583 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:15.583 INFO:teuthology.orchestra.run.smithi002.stdout:4240 write 0xa06af0b thru 0xa070661 (0x5757 bytes) 2024-09-18T04:57:15.592 INFO:teuthology.orchestra.run.smithi002.stdout:4242 trunc from 0xcb9fa62 to 0x7282a4b 2024-09-18T04:57:15.608 INFO:teuthology.orchestra.run.smithi002.stdout:4243 writesame 0x851b07 thru 0x85799f data_size 0x1(0x5e99 bytes) 2024-09-18T04:57:15.626 INFO:teuthology.orchestra.run.smithi002.stdout:4245 write 0x7b2b614 thru 0x7b32a60 (0x744d bytes) 2024-09-18T04:57:15.629 INFO:teuthology.orchestra.run.smithi002.stdout:4246 read 0x5c76503 thru 0x5c78a7a (0x2578 bytes) 2024-09-18T04:57:15.630 INFO:teuthology.orchestra.run.smithi002.stdout:4247 write 0x3d67ce9 thru 0x3d77a37 (0xfd4f bytes) 2024-09-18T04:57:15.635 INFO:teuthology.orchestra.run.smithi002.stdout:4249 writesame 0x7d8cd5 thru 0x7e6d9e data_size 0x45(0xe0ca bytes) 2024-09-18T04:57:15.648 INFO:teuthology.orchestra.run.smithi002.stdout:4251 writesame 0xc423e65 thru 0xc43277e data_size 0x1(0xe91a bytes) 2024-09-18T04:57:15.648 INFO:teuthology.orchestra.run.smithi002.stdout:4252 read 0x7317a73 thru 0x731955d (0x1aeb bytes) 2024-09-18T04:57:15.648 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:15.648 INFO:teuthology.orchestra.run.smithi002.stdout:4254 read 0x4132065 thru 0x4133a36 (0x19d2 bytes) 2024-09-18T04:57:15.648 INFO:teuthology.orchestra.run.smithi002.stdout:4256 read 0x3387d85 thru 0x3388f23 (0x119f bytes) 2024-09-18T04:57:15.656 INFO:teuthology.orchestra.run.smithi002.stdout:4258 write 0xe66b581 thru 0xe67a843 (0xf2c3 bytes) 2024-09-18T04:57:15.663 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:15.670 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:15.668+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:15.677 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:15.677 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:15.678 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:57:15.678 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:57:15.678 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:15.679 INFO:teuthology.orchestra.run.smithi002.stdout:4260 trunc from 0xe67a844 to 0x9b0371 2024-09-18T04:57:15.688 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:57:15.705 INFO:teuthology.orchestra.run.smithi002.stdout:4261 punch from 0x4b519d to 0x4b6d0b, (0x1b6e bytes) 2024-09-18T04:57:15.705 INFO:teuthology.orchestra.run.smithi002.stdout:4263 punch from 0x431d4b to 0x4344e9, (0x279e bytes) 2024-09-18T04:57:15.723 INFO:teuthology.orchestra.run.smithi002.stdout:4265 write 0xcaae208 thru 0xcab38e7 (0x56e0 bytes) 2024-09-18T04:57:15.727 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:15.727 INFO:teuthology.orchestra.run.smithi002.stdout:4267 punch from 0x820a847 to 0x82191b4, (0xe96d bytes) 2024-09-18T04:57:15.728 INFO:teuthology.orchestra.run.smithi002.stdout:4268 punch from 0xad8e30b to 0xad9a498, (0xc18d bytes) 2024-09-18T04:57:15.728 INFO:teuthology.orchestra.run.smithi002.stdout:4269 read 0xc07b994 thru 0xc08af50 (0xf5bd bytes) 2024-09-18T04:57:15.728 INFO:teuthology.orchestra.run.smithi002.stdout:4270 trunc from 0xcab38e8 to 0xdabb079 2024-09-18T04:57:15.735 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:15.735 INFO:teuthology.orchestra.run.smithi002.stdout:4272 read 0xcb69692 thru 0xcb6de7d (0x47ec bytes) 2024-09-18T04:57:15.735 INFO:teuthology.orchestra.run.smithi002.stdout:4273 write 0x38264a1 thru 0x382682b (0x38b bytes) 2024-09-18T04:57:15.737 INFO:teuthology.orchestra.run.smithi002.stdout:4274 write 0x354707b thru 0x35510c1 (0xa047 bytes) 2024-09-18T04:57:15.740 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:57:15.749 INFO:teuthology.orchestra.run.smithi002.stdout:4275 trunc from 0xdabb079 to 0x6308ca3 2024-09-18T04:57:15.760 INFO:teuthology.orchestra.run.smithi002.stdout:4276 read 0x183fbbc thru 0x184518f (0x55d4 bytes) 2024-09-18T04:57:15.764 INFO:teuthology.orchestra.run.smithi002.stdout:4277 trunc from 0x6308ca3 to 0x393e6a5 2024-09-18T04:57:15.771 INFO:teuthology.orchestra.run.smithi002.stdout:4278 read 0x1f6f2e4 thru 0x1f7bf46 (0xcc63 bytes) 2024-09-18T04:57:15.771 INFO:teuthology.orchestra.run.smithi002.stdout:4279 read 0x2e4e265 thru 0x2e5b1f6 (0xcf92 bytes) 2024-09-18T04:57:15.771 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:15.768+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:15.772 INFO:teuthology.orchestra.run.smithi002.stdout:4280 read 0xfe7e7d thru 0xfeb1d8 (0x335c bytes) 2024-09-18T04:57:15.789 INFO:teuthology.orchestra.run.smithi002.stdout:4281 write 0xde86aae thru 0xde8a88f (0x3de2 bytes) 2024-09-18T04:57:15.793 INFO:teuthology.orchestra.run.smithi002.stdout:4282 writesame 0x599683f thru 0x59a400c data_size 0x17(0xd7ce bytes) 2024-09-18T04:57:15.798 INFO:teuthology.orchestra.run.smithi002.stdout:4283 read 0xc0789d thru 0xc0e1b2 (0x6916 bytes) 2024-09-18T04:57:15.799 INFO:teuthology.orchestra.run.smithi002.stdout:4284 writesame 0x6b6a4a3 thru 0x6b73a73 data_size 0x1(0x95d1 bytes) 2024-09-18T04:57:15.815 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:15.815 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:15.815 INFO:teuthology.orchestra.run.smithi002.stdout:4287 writesame 0x4dda090 thru 0x4de7479 data_size 0x23(0xd3ea bytes) 2024-09-18T04:57:15.820 INFO:teuthology.orchestra.run.smithi002.stdout:4289 punch from 0x3b783d9 to 0x3b7fc47, (0x786e bytes) 2024-09-18T04:57:15.820 INFO:teuthology.orchestra.run.smithi002.stdout:4292 read 0x657db8c thru 0x658129e (0x3713 bytes) 2024-09-18T04:57:15.820 INFO:teuthology.orchestra.run.smithi002.stdout:4293 punch from 0x84d115b to 0x84e044a, (0xf2ef bytes) 2024-09-18T04:57:15.820 INFO:teuthology.orchestra.run.smithi002.stdout:4294 write 0xa0453f3 thru 0xa04696f (0x157d bytes) 2024-09-18T04:57:15.823 INFO:teuthology.orchestra.run.smithi002.stdout:4295 flatten 2024-09-18T04:57:15.841 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:15.841 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:15.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:15.841 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:15.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.841 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60344 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1456128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:15.841 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:15.841 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.725355633, 2024-09-18T04:57:15.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044026324999999998, 2024-09-18T04:57:15.841 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.842 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.842 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.842 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.842 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.842 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60344 2024-09-18T04:57:15.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.842 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:15.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:15.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:15.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112224+0000", 2024-09-18T04:57:15.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1320000000000001e-06 2024-09-18T04:57:15.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112230+0000", 2024-09-18T04:57:15.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9619999999999998e-06 2024-09-18T04:57:15.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112234+0000", 2024-09-18T04:57:15.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7890000000000002e-06 2024-09-18T04:57:15.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156105+0000", 2024-09-18T04:57:15.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043870531999999997 2024-09-18T04:57:15.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156170+0000", 2024-09-18T04:57:15.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5036000000000005e-05 2024-09-18T04:57:15.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156248+0000", 2024-09-18T04:57:15.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8295999999999997e-05 2024-09-18T04:57:15.847 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.847 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.847 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.847 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60345 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1471488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:15.847 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:57:15.847 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.725338868000001, 2024-09-18T04:57:15.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044180090999999998, 2024-09-18T04:57:15.848 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.848 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.848 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.848 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.848 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.848 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60345 2024-09-18T04:57:15.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.849 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:57:15.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:57:15.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112238+0000", 2024-09-18T04:57:15.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112240+0000", 2024-09-18T04:57:15.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9910000000000001e-06 2024-09-18T04:57:15.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112246+0000", 2024-09-18T04:57:15.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0859999999999998e-06 2024-09-18T04:57:15.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112251+0000", 2024-09-18T04:57:15.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5020000000000003e-06 2024-09-18T04:57:15.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156274+0000", 2024-09-18T04:57:15.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044023063000000001 2024-09-18T04:57:15.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156337+0000", 2024-09-18T04:57:15.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3077000000000006e-05 2024-09-18T04:57:15.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156419+0000", 2024-09-18T04:57:15.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1699000000000005e-05 2024-09-18T04:57:15.854 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.854 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.854 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.855 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60346 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1486848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:15.855 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:57:15.855 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.725321087000001, 2024-09-18T04:57:15.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044336331999999999, 2024-09-18T04:57:15.855 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.855 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.855 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.855 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.855 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.855 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60346 2024-09-18T04:57:15.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.855 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:57:15.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:57:15.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112256+0000", 2024-09-18T04:57:15.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112258+0000", 2024-09-18T04:57:15.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1720000000000001e-06 2024-09-18T04:57:15.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112265+0000", 2024-09-18T04:57:15.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4280000000000001e-06 2024-09-18T04:57:15.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112269+0000", 2024-09-18T04:57:15.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.741e-06 2024-09-18T04:57:15.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156445+0000", 2024-09-18T04:57:15.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044175341 2024-09-18T04:57:15.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156512+0000", 2024-09-18T04:57:15.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6723000000000005e-05 2024-09-18T04:57:15.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156593+0000", 2024-09-18T04:57:15.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1284000000000003e-05 2024-09-18T04:57:15.859 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.860 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.860 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.860 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60347 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1502208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:15.860 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:57:15.860 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.725292675999999, 2024-09-18T04:57:15.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044482560999999997, 2024-09-18T04:57:15.860 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.860 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.861 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.861 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.861 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.861 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60347 2024-09-18T04:57:15.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.861 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:57:15.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:57:15.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112284+0000", 2024-09-18T04:57:15.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112286+0000", 2024-09-18T04:57:15.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9709999999999998e-06 2024-09-18T04:57:15.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112290+0000", 2024-09-18T04:57:15.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9199999999999997e-06 2024-09-18T04:57:15.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112293+0000", 2024-09-18T04:57:15.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870000000000001e-06 2024-09-18T04:57:15.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156617+0000", 2024-09-18T04:57:15.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044323960000000003 2024-09-18T04:57:15.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156687+0000", 2024-09-18T04:57:15.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0141000000000005e-05 2024-09-18T04:57:15.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156767+0000", 2024-09-18T04:57:15.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0140000000000002e-05 2024-09-18T04:57:15.865 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.866 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.866 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.866 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60348 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1517568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:15.866 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:15.866 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.725281672000001, 2024-09-18T04:57:15.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044645562, 2024-09-18T04:57:15.866 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.866 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.866 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.866 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.867 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60348 2024-09-18T04:57:15.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.867 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:15.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:15.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:15.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112297+0000", 2024-09-18T04:57:15.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.384e-06 2024-09-18T04:57:15.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112301+0000", 2024-09-18T04:57:15.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0570000000000003e-06 2024-09-18T04:57:15.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112304+0000", 2024-09-18T04:57:15.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6000000000000001e-06 2024-09-18T04:57:15.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156793+0000", 2024-09-18T04:57:15.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044489494999999997 2024-09-18T04:57:15.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156859+0000", 2024-09-18T04:57:15.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5483999999999998e-05 2024-09-18T04:57:15.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156941+0000", 2024-09-18T04:57:15.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2856999999999996e-05 2024-09-18T04:57:15.872 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.872 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.872 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.872 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60349 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1532928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:15.873 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:15.873 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.725271696, 2024-09-18T04:57:15.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044832167999999999, 2024-09-18T04:57:15.873 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.873 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.873 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.874 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.874 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.874 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60349 2024-09-18T04:57:15.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.874 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:15.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:15.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:15.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112307+0000", 2024-09-18T04:57:15.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.308e-06 2024-09-18T04:57:15.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112310+0000", 2024-09-18T04:57:15.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-18T04:57:15.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112312+0000", 2024-09-18T04:57:15.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4889999999999998e-06 2024-09-18T04:57:15.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156977+0000", 2024-09-18T04:57:15.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044664598999999999 2024-09-18T04:57:15.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157053+0000", 2024-09-18T04:57:15.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6014999999999996e-05 2024-09-18T04:57:15.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157138+0000", 2024-09-18T04:57:15.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5162999999999998e-05 2024-09-18T04:57:15.880 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.880 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.880 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.880 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60350 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1548288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:15.880 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:15.881 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.725262782000001, 2024-09-18T04:57:15.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045011858000000002, 2024-09-18T04:57:15.881 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.881 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.881 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.881 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.881 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.881 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60350 2024-09-18T04:57:15.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.882 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:15.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:15.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:15.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112316+0000", 2024-09-18T04:57:15.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7579999999999999e-06 2024-09-18T04:57:15.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112321+0000", 2024-09-18T04:57:15.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1649999999999997e-06 2024-09-18T04:57:15.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112325+0000", 2024-09-18T04:57:15.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.726e-06 2024-09-18T04:57:15.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157172+0000", 2024-09-18T04:57:15.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.04484718 2024-09-18T04:57:15.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157242+0000", 2024-09-18T04:57:15.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9675e-05 2024-09-18T04:57:15.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157327+0000", 2024-09-18T04:57:15.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4614e-05 2024-09-18T04:57:15.886 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.886 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.886 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.886 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60351 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1563648~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:15.886 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:15.887 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.725249366, 2024-09-18T04:57:15.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045132113000000001, 2024-09-18T04:57:15.887 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.887 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.887 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.887 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.887 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.887 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60351 2024-09-18T04:57:15.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.887 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:15.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:15.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:15.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112330+0000", 2024-09-18T04:57:15.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1660000000000001e-06 2024-09-18T04:57:15.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.890 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112336+0000", 2024-09-18T04:57:15.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5940000000000001e-06 2024-09-18T04:57:15.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112340+0000", 2024-09-18T04:57:15.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5589999999999999e-06 2024-09-18T04:57:15.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157365+0000", 2024-09-18T04:57:15.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045024675 2024-09-18T04:57:15.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157409+0000", 2024-09-18T04:57:15.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4356e-05 2024-09-18T04:57:15.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157460+0000", 2024-09-18T04:57:15.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1072000000000002e-05 2024-09-18T04:57:15.892 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.892 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.892 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.893 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60501 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1494016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:15.893 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:15.893 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.719188670000001, 2024-09-18T04:57:15.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044007728000000003, 2024-09-18T04:57:15.893 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.893 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.893 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.893 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.893 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.894 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60501 2024-09-18T04:57:15.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.894 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:15.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:15.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:15.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118391+0000", 2024-09-18T04:57:15.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:57:15.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118396+0000", 2024-09-18T04:57:15.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2680000000000002e-06 2024-09-18T04:57:15.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118400+0000", 2024-09-18T04:57:15.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2080000000000002e-06 2024-09-18T04:57:15.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162317+0000", 2024-09-18T04:57:15.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043916753000000003 2024-09-18T04:57:15.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162351+0000", 2024-09-18T04:57:15.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4372000000000002e-05 2024-09-18T04:57:15.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162397+0000", 2024-09-18T04:57:15.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5207999999999998e-05 2024-09-18T04:57:15.897 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.897 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.897 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.898 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60502 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1509376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:15.898 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:15.898 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.719173193, 2024-09-18T04:57:15.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044092438999999997, 2024-09-18T04:57:15.898 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.898 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.898 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.898 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.898 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.898 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60502 2024-09-18T04:57:15.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.898 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:15.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:15.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:15.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118406+0000", 2024-09-18T04:57:15.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:57:15.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118412+0000", 2024-09-18T04:57:15.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.524e-06 2024-09-18T04:57:15.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118416+0000", 2024-09-18T04:57:15.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:57:15.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162418+0000", 2024-09-18T04:57:15.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001508000000002 2024-09-18T04:57:15.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162451+0000", 2024-09-18T04:57:15.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3662999999999999e-05 2024-09-18T04:57:15.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162497+0000", 2024-09-18T04:57:15.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5550000000000003e-05 2024-09-18T04:57:15.902 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.902 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.902 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.903 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60503 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1524736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:15.903 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:57:15.903 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.719157034999999, 2024-09-18T04:57:15.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044178780000000001, 2024-09-18T04:57:15.903 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.903 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.903 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.903 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.903 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.903 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60503 2024-09-18T04:57:15.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.903 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:57:15.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:57:15.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118420+0000", 2024-09-18T04:57:15.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118422+0000", 2024-09-18T04:57:15.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.08e-06 2024-09-18T04:57:15.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118428+0000", 2024-09-18T04:57:15.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.964e-06 2024-09-18T04:57:15.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118433+0000", 2024-09-18T04:57:15.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.425e-06 2024-09-18T04:57:15.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162519+0000", 2024-09-18T04:57:15.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044086082999999998 2024-09-18T04:57:15.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162552+0000", 2024-09-18T04:57:15.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3719e-05 2024-09-18T04:57:15.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162599+0000", 2024-09-18T04:57:15.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6873999999999999e-05 2024-09-18T04:57:15.907 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.907 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.907 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.907 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60504 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1540096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:15.908 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:57:15.908 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.719140930000002, 2024-09-18T04:57:15.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044261921000000003, 2024-09-18T04:57:15.908 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.908 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.908 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.908 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.908 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.908 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60504 2024-09-18T04:57:15.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.908 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:57:15.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:57:15.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118436+0000", 2024-09-18T04:57:15.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118438+0000", 2024-09-18T04:57:15.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0250000000000001e-06 2024-09-18T04:57:15.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118444+0000", 2024-09-18T04:57:15.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4360000000000001e-06 2024-09-18T04:57:15.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118448+0000", 2024-09-18T04:57:15.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.357e-06 2024-09-18T04:57:15.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162620+0000", 2024-09-18T04:57:15.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044171821 2024-09-18T04:57:15.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162653+0000", 2024-09-18T04:57:15.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3479999999999998e-05 2024-09-18T04:57:15.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162699+0000", 2024-09-18T04:57:15.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5113e-05 2024-09-18T04:57:15.913 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.913 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.914 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.914 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60584 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1196032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:15.914 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:57:15.914 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.717347913000001, 2024-09-18T04:57:15.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044048727000000003, 2024-09-18T04:57:15.914 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.915 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.915 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.915 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.915 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.915 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60584 2024-09-18T04:57:15.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.915 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:57:15.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:57:15.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120229+0000", 2024-09-18T04:57:15.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120231+0000", 2024-09-18T04:57:15.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:57:15.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120237+0000", 2024-09-18T04:57:15.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8379999999999998e-06 2024-09-18T04:57:15.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120242+0000", 2024-09-18T04:57:15.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3889999999999998e-06 2024-09-18T04:57:15.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164187+0000", 2024-09-18T04:57:15.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043945045000000002 2024-09-18T04:57:15.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164227+0000", 2024-09-18T04:57:15.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0627999999999998e-05 2024-09-18T04:57:15.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164278+0000", 2024-09-18T04:57:15.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1146000000000001e-05 2024-09-18T04:57:15.920 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.920 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.920 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.920 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60585 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1211392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:15.920 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:57:15.920 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.717331805000001, 2024-09-18T04:57:15.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044389814, 2024-09-18T04:57:15.921 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.921 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.921 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.921 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.921 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.921 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60585 2024-09-18T04:57:15.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.921 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:57:15.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:57:15.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120245+0000", 2024-09-18T04:57:15.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120247+0000", 2024-09-18T04:57:15.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0150000000000002e-06 2024-09-18T04:57:15.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120255+0000", 2024-09-18T04:57:15.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.858e-06 2024-09-18T04:57:15.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120260+0000", 2024-09-18T04:57:15.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6530000000000002e-06 2024-09-18T04:57:15.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164489+0000", 2024-09-18T04:57:15.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044229483999999999 2024-09-18T04:57:15.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164555+0000", 2024-09-18T04:57:15.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.533e-05 2024-09-18T04:57:15.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164636+0000", 2024-09-18T04:57:15.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0819e-05 2024-09-18T04:57:15.926 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.926 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.927 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.927 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60586 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1226752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:15.927 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:15.927 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.717313354000002, 2024-09-18T04:57:15.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044578214999999997, 2024-09-18T04:57:15.927 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.927 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.927 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.927 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.927 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.928 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60586 2024-09-18T04:57:15.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.928 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:15.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:15.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:15.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120266+0000", 2024-09-18T04:57:15.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0660000000000002e-06 2024-09-18T04:57:15.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120272+0000", 2024-09-18T04:57:15.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-18T04:57:15.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120276+0000", 2024-09-18T04:57:15.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.476e-06 2024-09-18T04:57:15.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164673+0000", 2024-09-18T04:57:15.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044396672999999998 2024-09-18T04:57:15.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164749+0000", 2024-09-18T04:57:15.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5811999999999999e-05 2024-09-18T04:57:15.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164842+0000", 2024-09-18T04:57:15.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3560000000000005e-05 2024-09-18T04:57:15.931 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.931 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.931 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.932 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60587 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1242112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:15.932 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:57:15.932 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.717264255, 2024-09-18T04:57:15.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044634794999999998, 2024-09-18T04:57:15.933 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.933 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.933 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.933 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.933 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.933 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60587 2024-09-18T04:57:15.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.933 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:57:15.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:57:15.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120280+0000", 2024-09-18T04:57:15.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999671 2024-09-18T04:57:15.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120316+0000", 2024-09-18T04:57:15.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5728000000000001e-05 2024-09-18T04:57:15.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120329+0000", 2024-09-18T04:57:15.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2914e-05 2024-09-18T04:57:15.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120335+0000", 2024-09-18T04:57:15.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8289999999999996e-06 2024-09-18T04:57:15.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164858+0000", 2024-09-18T04:57:15.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044523417000000003 2024-09-18T04:57:15.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164896+0000", 2024-09-18T04:57:15.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7969999999999997e-05 2024-09-18T04:57:15.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164948+0000", 2024-09-18T04:57:15.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1996000000000002e-05 2024-09-18T04:57:15.937 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.938 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.938 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.938 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60595 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1364992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:15.938 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:57:15.938 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.715421038999999, 2024-09-18T04:57:15.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001360000000003, 2024-09-18T04:57:15.938 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.938 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.938 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.938 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.939 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.939 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60595 2024-09-18T04:57:15.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.939 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:57:15.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:57:15.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122156+0000", 2024-09-18T04:57:15.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122158+0000", 2024-09-18T04:57:15.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0040000000000002e-06 2024-09-18T04:57:15.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122164+0000", 2024-09-18T04:57:15.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8719999999999999e-06 2024-09-18T04:57:15.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122169+0000", 2024-09-18T04:57:15.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5090000000000004e-06 2024-09-18T04:57:15.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166040+0000", 2024-09-18T04:57:15.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043871226999999999 2024-09-18T04:57:15.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166096+0000", 2024-09-18T04:57:15.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6495e-05 2024-09-18T04:57:15.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166158+0000", 2024-09-18T04:57:15.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1626999999999998e-05 2024-09-18T04:57:15.942 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.942 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.942 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.942 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60596 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1380352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:15.943 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:15.943 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.715404717999998, 2024-09-18T04:57:15.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044120800000000002, 2024-09-18T04:57:15.943 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.943 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.943 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.943 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.943 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.943 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60596 2024-09-18T04:57:15.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.943 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:15.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:15.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:15.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122175+0000", 2024-09-18T04:57:15.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:57:15.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122182+0000", 2024-09-18T04:57:15.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3509999999999996e-06 2024-09-18T04:57:15.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122187+0000", 2024-09-18T04:57:15.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7940000000000002e-06 2024-09-18T04:57:15.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166184+0000", 2024-09-18T04:57:15.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043997236000000002 2024-09-18T04:57:15.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166227+0000", 2024-09-18T04:57:15.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2561000000000001e-05 2024-09-18T04:57:15.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166294+0000", 2024-09-18T04:57:15.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7068000000000002e-05 2024-09-18T04:57:15.947 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.947 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.947 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.947 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60597 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1395712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:15.947 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:15.947 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.715386689999999, 2024-09-18T04:57:15.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044242166999999999, 2024-09-18T04:57:15.947 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.947 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.947 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.947 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.948 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.948 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60597 2024-09-18T04:57:15.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.948 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:15.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:15.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:15.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122193+0000", 2024-09-18T04:57:15.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:57:15.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122199+0000", 2024-09-18T04:57:15.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6449999999999999e-06 2024-09-18T04:57:15.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122204+0000", 2024-09-18T04:57:15.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4819999999999996e-06 2024-09-18T04:57:15.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166329+0000", 2024-09-18T04:57:15.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044125059000000001 2024-09-18T04:57:15.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166370+0000", 2024-09-18T04:57:15.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1012e-05 2024-09-18T04:57:15.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166433+0000", 2024-09-18T04:57:15.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3178999999999999e-05 2024-09-18T04:57:15.952 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.952 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.952 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.952 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60608 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1564672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:15.952 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:57:15.952 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.713388827999999, 2024-09-18T04:57:15.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044082723999999997, 2024-09-18T04:57:15.952 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:15.953 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:15.953 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:15.953 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:15.953 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:15.953 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60608 2024-09-18T04:57:15.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.953 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:15.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:15.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:57:15.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:15.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:57:15.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:15.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:15.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124188+0000", 2024-09-18T04:57:15.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:15.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:15.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124190+0000", 2024-09-18T04:57:15.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0329999999999998e-06 2024-09-18T04:57:15.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:15.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124196+0000", 2024-09-18T04:57:15.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.575e-06 2024-09-18T04:57:15.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:15.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124201+0000", 2024-09-18T04:57:15.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5000000000000001e-06 2024-09-18T04:57:15.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:15.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168166+0000", 2024-09-18T04:57:15.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043965763999999997 2024-09-18T04:57:15.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:15.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168218+0000", 2024-09-18T04:57:15.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1785e-05 2024-09-18T04:57:15.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:15.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:15.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:15.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168271+0000", 2024-09-18T04:57:15.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3429e-05 2024-09-18T04:57:15.957 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.957 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.957 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.958 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:15.958 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:15.958 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:15.960 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:57:15.960 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:15.957+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:15.961 INFO:teuthology.orchestra.run.smithi002.stdout:4298 read 0x5849c32 thru 0x584e811 (0x4be0 bytes) 2024-09-18T04:57:15.961 INFO:teuthology.orchestra.run.smithi002.stdout:4299 punch from 0x606050b to 0x606af6e, (0xaa63 bytes) 2024-09-18T04:57:15.961 INFO:teuthology.orchestra.run.smithi002.stdout:4300 write 0x8e90776 thru 0x8e98bed (0x8478 bytes) 2024-09-18T04:57:15.961 INFO:teuthology.orchestra.run.smithi002.stdout:4302 writesame 0x5ea9165 thru 0x5ead55b data_size 0x1(0x43f7 bytes) 2024-09-18T04:57:15.961 INFO:teuthology.orchestra.run.smithi002.stdout:4303 read 0x985f99b thru 0x9865e50 (0x64b6 bytes) 2024-09-18T04:57:15.961 INFO:teuthology.orchestra.run.smithi002.stdout:4304 writesame 0x60fd3fe thru 0x6104925 data_size 0x2e(0x7528 bytes) 2024-09-18T04:57:15.962 INFO:teuthology.orchestra.run.smithi002.stdout:4306 read 0x7e1211b thru 0x7e186b6 (0x659c bytes) 2024-09-18T04:57:15.962 INFO:teuthology.orchestra.run.smithi002.stdout:4308 writesame 0x48537eb thru 0x4853824 data_size 0x1(0x3a bytes) 2024-09-18T04:57:15.962 INFO:teuthology.orchestra.run.smithi002.stdout:4311 writesame 0x4133843 thru 0x413fb2c data_size 0x3d(0xc2ea bytes) 2024-09-18T04:57:15.962 INFO:teuthology.orchestra.run.smithi002.stdout:4312 writesame 0xe8ffe48 thru 0xe906046 data_size 0x1(0x61ff bytes) 2024-09-18T04:57:15.962 INFO:teuthology.orchestra.run.smithi002.stdout:4314 trunc from 0xe906047 to 0x8ee4e44 2024-09-18T04:57:15.962 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:57:15.962 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:57:15.963 INFO:teuthology.orchestra.run.smithi002.stdout:4316 write 0x9794b1f thru 0x979b1ed (0x66cf bytes) 2024-09-18T04:57:15.963 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:57:15.963 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:57:15.964 INFO:teuthology.orchestra.run.smithi002.stdout:4317 writesame 0x1cec766 thru 0x1cfb242 data_size 0x7d(0xeadd bytes) 2024-09-18T04:57:15.964 INFO:teuthology.orchestra.run.smithi002.stdout:4318 read 0x922a583 thru 0x9234c1c (0xa69a bytes) 2024-09-18T04:57:15.964 INFO:teuthology.orchestra.run.smithi002.stdout:4319 read 0x29dcc67 thru 0x29e81ab (0xb545 bytes) 2024-09-18T04:57:15.964 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:15.964 INFO:teuthology.orchestra.run.smithi002.stdout:4321 punch from 0x181a885 to 0x1823a9b, (0x9216 bytes) 2024-09-18T04:57:15.964 INFO:teuthology.orchestra.run.smithi002.stdout:4322 trunc from 0x979b1ee to 0xc05956d 2024-09-18T04:57:15.964 INFO:teuthology.orchestra.run.smithi002.stdout:4324 trunc from 0xc05956d to 0xe8db258 2024-09-18T04:57:15.965 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:57:15.965 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:57:15.965 INFO:teuthology.orchestra.run.smithi002.stdout:4325 write 0x8b157d0 thru 0x8b1c904 (0x7135 bytes) 2024-09-18T04:57:15.965 INFO:teuthology.orchestra.run.smithi002.stdout:4327 writesame 0xc930a51 thru 0xc93f542 data_size 0x3d(0xeaf2 bytes) 2024-09-18T04:57:15.965 INFO:teuthology.orchestra.run.smithi002.stdout:4328 read 0x3575a8f thru 0x358266e (0xcbe0 bytes) 2024-09-18T04:57:15.966 INFO:teuthology.orchestra.run.smithi002.stdout:4329 writesame 0x12499c0 thru 0x124fc74 data_size 0x1(0x62b5 bytes) 2024-09-18T04:57:15.966 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:57:15.966 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:57:15.967 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:15.967 INFO:teuthology.orchestra.run.smithi002.stdout:4332 write 0xaee2b9 thru 0xaf1df6 (0x3b3e bytes) 2024-09-18T04:57:15.967 INFO:teuthology.orchestra.run.smithi002.stdout:4333 write 0x4e9890a thru 0x4ea8243 (0xf93a bytes) 2024-09-18T04:57:15.967 INFO:teuthology.orchestra.run.smithi002.stdout:4334 trunc from 0xe8db258 to 0xd241019 2024-09-18T04:57:15.967 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:57:15.967 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:57:15.968 INFO:teuthology.orchestra.run.smithi002.stdout:4335 punch from 0x661bb48 to 0x66296d4, (0xdb8c bytes) 2024-09-18T04:57:15.968 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:15.968 INFO:teuthology.orchestra.run.smithi002.stdout:4340 punch from 0x430fc08 to 0x4313301, (0x36f9 bytes) 2024-09-18T04:57:15.968 INFO:teuthology.orchestra.run.smithi002.stdout:4341 writesame 0x8318fe7 thru 0x8324a3f data_size 0x2f(0xba59 bytes) 2024-09-18T04:57:15.968 INFO:teuthology.orchestra.run.smithi002.stdout:4344 punch from 0xf0e644 to 0xf15b22, (0x74de bytes) 2024-09-18T04:57:15.969 INFO:teuthology.orchestra.run.smithi002.stdout:4345 writesame 0xa25c4d0 thru 0xa265c8b data_size 0x9c(0x97bc bytes) 2024-09-18T04:57:15.977 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:57:15.978 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:57:15.986 INFO:teuthology.orchestra.run.smithi002.stdout:4347 trunc from 0xd241019 to 0xe01ae8 2024-09-18T04:57:15.998 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:57:15.998 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:57:16.015 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:16.015 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:16.015 INFO:teuthology.orchestra.run.smithi002.stdout:4350 clone 33 order 19 su 32768 sc 5 2024-09-18T04:57:16.018 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:57:16.019 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:57:16.071 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:16.068+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:16.105 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:16.105 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:16.113 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:57:16.161 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:16.158+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:16.214 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:16.214 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:16.223 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:57:16.266 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:16.264+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:16.325 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:16.325 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:16.333 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:57:16.351 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.2 to 1 (10000) 2024-09-18T04:57:16.363 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T04:57:16.364 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:16.360+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:16.366 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-18T04:57:16.442 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-09-18T04:57:16.464 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:16.461+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:16.478 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:16.479 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:16.489 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:57:16.572 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:16.570+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:16.630 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:16.630 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:16.639 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:57:16.670 INFO:teuthology.orchestra.run.smithi002.stdout:4352 punch from 0x8481b8 to 0x8548d0, (0xc718 bytes) 2024-09-18T04:57:16.674 INFO:teuthology.orchestra.run.smithi002.stdout:4353 read 0xc9918e thru 0xca87b3 (0xf626 bytes) 2024-09-18T04:57:16.679 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:16.677+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:16.694 INFO:teuthology.orchestra.run.smithi002.stdout:4354 writesame 0xb74f890 thru 0xb75b793 data_size 0x4b(0xbf04 bytes) 2024-09-18T04:57:16.695 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:16.695 INFO:teuthology.orchestra.run.smithi002.stdout:4357 punch from 0x9068ab5 to 0x9075d8d, (0xd2d8 bytes) 2024-09-18T04:57:16.695 INFO:teuthology.orchestra.run.smithi002.stdout:4358 write 0x41b26ea thru 0x41befe2 (0xc8f9 bytes) 2024-09-18T04:57:16.698 INFO:teuthology.orchestra.run.smithi002.stdout:4359 writesame 0x34f6e7c thru 0x34ff291 data_size 0x3a(0x8416 bytes) 2024-09-18T04:57:16.709 INFO:teuthology.orchestra.run.smithi002.stdout:4360 trunc from 0xb75b794 to 0x5ec33e1 2024-09-18T04:57:16.715 INFO:teuthology.orchestra.run.smithi002.stdout:4361 punch from 0x136bd44 to 0x137261a, (0x68d6 bytes) 2024-09-18T04:57:16.715 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:16.715 INFO:teuthology.orchestra.run.smithi002.stdout:4363 read 0x67ca9d thru 0x68144b (0x49af bytes) 2024-09-18T04:57:16.727 INFO:teuthology.orchestra.run.smithi002.stdout:4364 write 0xab58c9e thru 0xab5e3e7 (0x574a bytes) 2024-09-18T04:57:16.730 INFO:teuthology.orchestra.run.smithi002.stdout:4365 punch from 0x2727436 to 0x272a3f7, (0x2fc1 bytes) 2024-09-18T04:57:16.730 INFO:teuthology.orchestra.run.smithi002.stdout:4366 read 0x4558d56 thru 0x4559e9d (0x1148 bytes) 2024-09-18T04:57:16.731 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:16.731 INFO:teuthology.orchestra.run.smithi002.stdout:4368 write 0x85e41bb thru 0x85e7244 (0x308a bytes) 2024-09-18T04:57:16.733 INFO:teuthology.orchestra.run.smithi002.stdout:4369 write 0x3c28c34 thru 0x3c2cf69 (0x4336 bytes) 2024-09-18T04:57:16.736 INFO:teuthology.orchestra.run.smithi002.stdout:4370 read 0x6c59c7d thru 0x6c5f3ba (0x573e bytes) 2024-09-18T04:57:16.736 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:16.746 INFO:teuthology.orchestra.run.smithi002.stdout:4372 write 0xec7c4c0 thru 0xec80a02 (0x4543 bytes) 2024-09-18T04:57:16.749 INFO:teuthology.orchestra.run.smithi002.stdout:4373 read 0xe08e355 thru 0xe092822 (0x44ce bytes) 2024-09-18T04:57:16.749 INFO:teuthology.orchestra.run.smithi002.stdout:4374 write 0x4970747 thru 0x4977bbd (0x7477 bytes) 2024-09-18T04:57:16.752 INFO:teuthology.orchestra.run.smithi002.stdout:4375 write 0x6c00819 thru 0x6c06ebd (0x66a5 bytes) 2024-09-18T04:57:16.767 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:16.764+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:16.768 INFO:teuthology.orchestra.run.smithi002.stdout:4376 trunc from 0xec80a03 to 0x41ea570 2024-09-18T04:57:16.787 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:16.788 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:16.791 INFO:teuthology.orchestra.run.smithi002.stdout:4377 writesame 0xa9c81ee thru 0xa9cbe58 data_size 0x1(0x3c6b bytes) 2024-09-18T04:57:16.791 INFO:teuthology.orchestra.run.smithi002.stdout:4378 read 0x189b652 thru 0x18a48e5 (0x9294 bytes) 2024-09-18T04:57:16.792 INFO:teuthology.orchestra.run.smithi002.stdout:4379 punch from 0x35c81d3 to 0x35cdeff, (0x5d2c bytes) 2024-09-18T04:57:16.792 INFO:teuthology.orchestra.run.smithi002.stdout:4380 writesame 0x6606aad thru 0x660fbc4 data_size 0x4(0x9118 bytes) 2024-09-18T04:57:16.796 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:57:16.798 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:16.798 INFO:teuthology.orchestra.run.smithi002.stdout:4383 read 0x6c126db thru 0x6c2194e (0xf274 bytes) 2024-09-18T04:57:16.798 INFO:teuthology.orchestra.run.smithi002.stdout:4384 read 0xa7bc3e7 thru 0xa7c99d5 (0xd5ef bytes) 2024-09-18T04:57:16.798 INFO:teuthology.orchestra.run.smithi002.stdout:4386 write 0x1d7679b thru 0x1d76c78 (0x4de bytes) 2024-09-18T04:57:16.801 INFO:teuthology.orchestra.run.smithi002.stdout:4387 writesame 0x2610fb3 thru 0x2613f7d data_size 0x1(0x2fcb bytes) 2024-09-18T04:57:16.807 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:16.808 INFO:teuthology.orchestra.run.smithi002.stdout:4389 writesame 0x1629d08 thru 0x162e031 data_size 0x1(0x432a bytes) 2024-09-18T04:57:16.816 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:16.816 INFO:teuthology.orchestra.run.smithi002.stdout:4391 write 0xa589d thru 0xaaa78 (0x51dc bytes) 2024-09-18T04:57:16.823 INFO:teuthology.orchestra.run.smithi002.stdout:4393 writesame 0x8ff8f6a thru 0x9008f43 data_size 0x1(0xffda bytes) 2024-09-18T04:57:16.842 INFO:teuthology.orchestra.run.smithi002.stdout:4396 read 0x95a46e7 thru 0x95aa411 (0x5d2b bytes) 2024-09-18T04:57:16.842 INFO:teuthology.orchestra.run.smithi002.stdout:4397 write 0x1364734 thru 0x13666ed (0x1fba bytes) 2024-09-18T04:57:16.845 INFO:teuthology.orchestra.run.smithi002.stdout:4399 punch from 0x9775b81 to 0x97810e6, (0xb565 bytes) 2024-09-18T04:57:16.845 INFO:teuthology.orchestra.run.smithi002.stdout:4400 writesame 0x6363217 thru 0x636ceb7 data_size 0x1(0x9ca1 bytes) 2024-09-18T04:57:16.864 INFO:teuthology.orchestra.run.smithi002.stdout:4401 trunc from 0xa9cbe59 to 0x73615b7 2024-09-18T04:57:16.868 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:16.865+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:16.872 INFO:teuthology.orchestra.run.smithi002.stdout:4402 read 0x47e730b thru 0x47ee8be (0x75b4 bytes) 2024-09-18T04:57:16.885 INFO:teuthology.orchestra.run.smithi002.stdout:4405 write 0xe067424 thru 0xe070ad7 (0x96b4 bytes) 2024-09-18T04:57:16.889 INFO:teuthology.orchestra.run.smithi002.stdout:4406 read 0xc6bf32c thru 0xc6c3717 (0x43ec bytes) 2024-09-18T04:57:16.889 INFO:teuthology.orchestra.run.smithi002.stdout:4409 write 0xaeabf9b thru 0xaeaf204 (0x326a bytes) 2024-09-18T04:57:16.892 INFO:teuthology.orchestra.run.smithi002.stdout:4410 write 0xc6245a5 thru 0xc6314b3 (0xcf0f bytes) 2024-09-18T04:57:16.899 INFO:teuthology.orchestra.run.smithi002.stdout:4411 writesame 0x7ae666d thru 0x7aeeae7 data_size 0x77(0x847b bytes) 2024-09-18T04:57:16.904 INFO:teuthology.orchestra.run.smithi002.stdout:4413 punch from 0x842dc55 to 0x843c969, (0xed14 bytes) 2024-09-18T04:57:16.905 INFO:teuthology.orchestra.run.smithi002.stdout:4415 writesame 0x8e0bd40 thru 0x8e0f3e7 data_size 0x35(0x36a8 bytes) 2024-09-18T04:57:16.911 INFO:teuthology.orchestra.run.smithi002.stdout:4417 writesame 0x2dec3df thru 0x2df4410 data_size 0x3d(0x8032 bytes) 2024-09-18T04:57:16.914 INFO:teuthology.orchestra.run.smithi002.stdout:4418 punch from 0x6e7e26 to 0x6f160d, (0x97e7 bytes) 2024-09-18T04:57:16.915 INFO:teuthology.orchestra.run.smithi002.stdout:4419 write 0xb83bca1 thru 0xb8437ea (0x7b4a bytes) 2024-09-18T04:57:16.918 INFO:teuthology.orchestra.run.smithi002.stdout:4420 read 0x401f4ca thru 0x402e167 (0xec9e bytes) 2024-09-18T04:57:16.918 INFO:teuthology.orchestra.run.smithi002.stdout:4421 punch from 0x7f75452 to 0x7f82c07, (0xd7b5 bytes) 2024-09-18T04:57:16.925 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:16.926 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:16.932 INFO:teuthology.orchestra.run.smithi002.stdout:4422 trunc from 0xe070ad8 to 0x1afa96b 2024-09-18T04:57:16.941 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:57:16.956 INFO:teuthology.orchestra.run.smithi002.stdout:4423 write 0x4f89780 thru 0x4f927d6 (0x9057 bytes) 2024-09-18T04:57:16.960 INFO:teuthology.orchestra.run.smithi002.stdout:4424 punch from 0xaf3f96 to 0xb031b7, (0xf221 bytes) 2024-09-18T04:57:16.969 INFO:teuthology.orchestra.run.smithi002.stdout:4425 trunc from 0x4f927d7 to 0xc7a330f 2024-09-18T04:57:16.969 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:16.967+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:16.981 INFO:teuthology.orchestra.run.smithi002.stdout:4426 write 0xe114289 thru 0xe115446 (0x11be bytes) 2024-09-18T04:57:16.984 INFO:teuthology.orchestra.run.smithi002.stdout:4428 punch from 0x30d41ff to 0x30e0159, (0xbf5a bytes) 2024-09-18T04:57:16.996 INFO:teuthology.orchestra.run.smithi002.stdout:4429 trunc from 0xe115447 to 0x3a55cd0 2024-09-18T04:57:17.006 INFO:teuthology.orchestra.run.smithi002.stdout:4430 read 0xe38eea thru 0xe40723 (0x783a bytes) 2024-09-18T04:57:17.009 INFO:teuthology.orchestra.run.smithi002.stdout:4431 trunc from 0x3a55cd0 to 0x1168cf3 2024-09-18T04:57:17.021 INFO:teuthology.orchestra.run.smithi002.stdout:4433 clone 34 order 23 su 32768 sc 8 2024-09-18T04:57:17.070 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:17.070 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:17.071 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:17.068+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:17.079 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:57:17.173 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:17.168+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:17.227 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:17.227 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:17.235 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:57:17.276 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:17.274+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:17.358 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.3 to 1 (10000) 2024-09-18T04:57:17.358 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-18T04:57:17.372 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:17.370+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:17.374 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-18T04:57:17.393 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:17.393 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:17.403 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:57:17.420 INFO:teuthology.orchestra.run.smithi002.stdout:leaving image image_client.0-clone33 intact 2024-09-18T04:57:17.465 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-09-18T04:57:17.473 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:17.470+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:17.500 INFO:teuthology.orchestra.run.smithi002.stdout:4434 trunc from 0x1168cf3 to 0xbf20631 2024-09-18T04:57:17.509 INFO:teuthology.orchestra.run.smithi002.stdout:4435 read 0xb687784 thru 0xb68e31f (0x6b9c bytes) 2024-09-18T04:57:17.509 INFO:teuthology.orchestra.run.smithi002.stdout:4436 punch from 0x97ba4a6 to 0x97c57d7, (0xb331 bytes) 2024-09-18T04:57:17.509 INFO:teuthology.orchestra.run.smithi002.stdout:4439 writesame 0x4366c62 thru 0x4374b16 data_size 0x47(0xdeb5 bytes) 2024-09-18T04:57:17.516 INFO:teuthology.orchestra.run.smithi002.stdout:4441 trunc from 0xbf20631 to 0xa3d0f46 2024-09-18T04:57:17.521 INFO:teuthology.orchestra.run.smithi002.stdout:4442 writesame 0x5ec61b5 thru 0x5ed42ce data_size 0x1(0xe11a bytes) 2024-09-18T04:57:17.540 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:17.540 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:17.541 INFO:teuthology.orchestra.run.smithi002.stdout:4444 write 0x3b2ff0 thru 0x3b8810 (0x5821 bytes) 2024-09-18T04:57:17.553 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:57:17.556 INFO:teuthology.orchestra.run.smithi002.stdout:4445 write 0x3154290 thru 0x315a4eb (0x625c bytes) 2024-09-18T04:57:17.573 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:17.571+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:17.574 INFO:teuthology.orchestra.run.smithi002.stdout:4446 read 0x8f6d1ec thru 0x8f78de3 (0xbbf8 bytes) 2024-09-18T04:57:17.574 INFO:teuthology.orchestra.run.smithi002.stdout:4447 writesame 0x23a2a50 thru 0x23ac89a data_size 0x1(0x9e4b bytes) 2024-09-18T04:57:17.608 INFO:teuthology.orchestra.run.smithi002.stdout:4449 write 0xee5a2ce thru 0xee6645e (0xc191 bytes) 2024-09-18T04:57:17.612 INFO:teuthology.orchestra.run.smithi002.stdout:4451 writesame 0x62a8643 thru 0x62aa389 data_size 0x1(0x1d47 bytes) 2024-09-18T04:57:17.615 INFO:teuthology.orchestra.run.smithi002.stdout:4452 writesame 0x198b098 thru 0x1993297 data_size 0x34(0x8200 bytes) 2024-09-18T04:57:17.643 INFO:teuthology.orchestra.run.smithi002.stdout:4453 trunc from 0xee6645f to 0x1bc46b4 2024-09-18T04:57:17.666 INFO:teuthology.orchestra.run.smithi002.stdout:4454 punch from 0x7d6b42 to 0x7e6a5b, (0xff19 bytes) 2024-09-18T04:57:17.673 INFO:teuthology.orchestra.run.smithi002.stdout:4455 trunc from 0x1bc46b4 to 0x81be486 2024-09-18T04:57:17.674 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:17.672+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:17.678 INFO:teuthology.orchestra.run.smithi002.stdout:4457 read 0x1d6652d thru 0x1d720ae (0xbb82 bytes) 2024-09-18T04:57:17.678 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:17.686 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:17.687 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:17.691 INFO:teuthology.orchestra.run.smithi002.stdout:4460 writesame 0xe9f167c thru 0xe9fbaa4 data_size 0x5(0xa429 bytes) 2024-09-18T04:57:17.691 INFO:teuthology.orchestra.run.smithi002.stdout:4461 writesame 0x5fdec0e thru 0x5fe4ba0 data_size 0x1(0x5f93 bytes) 2024-09-18T04:57:17.696 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:57:17.704 INFO:teuthology.orchestra.run.smithi002.stdout:4463 read 0xbe29043 thru 0xbe2ea7a (0x5a38 bytes) 2024-09-18T04:57:17.704 INFO:teuthology.orchestra.run.smithi002.stdout:4464 writesame 0x10ed447 thru 0x10ee543 data_size 0x1(0x10fd bytes) 2024-09-18T04:57:17.706 INFO:teuthology.orchestra.run.smithi002.stdout:4467 write 0x5a034c8 thru 0x5a061c7 (0x2d00 bytes) 2024-09-18T04:57:17.711 INFO:teuthology.orchestra.run.smithi002.stdout:4468 punch from 0x9c56a43 to 0x9c5b46d, (0x4a2a bytes) 2024-09-18T04:57:17.711 INFO:teuthology.orchestra.run.smithi002.stdout:4469 write 0x8b924f4 thru 0x8b94d9d (0x28aa bytes) 2024-09-18T04:57:17.715 INFO:teuthology.orchestra.run.smithi002.stdout:4470 read 0x1928862 thru 0x1937722 (0xeec1 bytes) 2024-09-18T04:57:17.715 INFO:teuthology.orchestra.run.smithi002.stdout:4471 read 0x3957349 thru 0x3959593 (0x224b bytes) 2024-09-18T04:57:17.716 INFO:teuthology.orchestra.run.smithi002.stdout:4472 writesame 0x6cc170e thru 0x6cc1dc3 data_size 0x1(0x6b6 bytes) 2024-09-18T04:57:17.717 INFO:teuthology.orchestra.run.smithi002.stdout:4473 punch from 0xbe0a1fa to 0xbe102bb, (0x60c1 bytes) 2024-09-18T04:57:17.718 INFO:teuthology.orchestra.run.smithi002.stdout:4474 write 0xd73004 thru 0xd771ac (0x41a9 bytes) 2024-09-18T04:57:17.731 INFO:teuthology.orchestra.run.smithi002.stdout:4475 trunc from 0xe9fbaa5 to 0x5ba7865 2024-09-18T04:57:17.747 INFO:teuthology.orchestra.run.smithi002.stdout:4477 trunc from 0x5ba7865 to 0x706c1bb 2024-09-18T04:57:17.753 INFO:teuthology.orchestra.run.smithi002.stdout:4478 read 0x114701f thru 0x114d88b (0x686d bytes) 2024-09-18T04:57:17.753 INFO:teuthology.orchestra.run.smithi002.stdout:4479 punch from 0x23edb91 to 0x23fd0ae, (0xf51d bytes) 2024-09-18T04:57:17.754 INFO:teuthology.orchestra.run.smithi002.stdout:4481 read 0xe32aef thru 0xe3d228 (0xa73a bytes) 2024-09-18T04:57:17.760 INFO:teuthology.orchestra.run.smithi002.stdout:4482 trunc from 0x706c1bb to 0x1764a4f 2024-09-18T04:57:17.775 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:17.773+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:17.776 INFO:teuthology.orchestra.run.smithi002.stdout:4483 read 0x6ffb1 thru 0x72bee (0x2c3e bytes) 2024-09-18T04:57:17.785 INFO:teuthology.orchestra.run.smithi002.stdout:4484 trunc from 0x1764a4f to 0x8447d13 2024-09-18T04:57:17.790 INFO:teuthology.orchestra.run.smithi002.stdout:4485 punch from 0x4f0e99d to 0x4f1033c, (0x199f bytes) 2024-09-18T04:57:17.790 INFO:teuthology.orchestra.run.smithi002.stdout:4486 read 0x54b5abf thru 0x54ba48b (0x49cd bytes) 2024-09-18T04:57:17.790 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:17.790 INFO:teuthology.orchestra.run.smithi002.stdout:4489 write 0x97ec36 thru 0x9818a0 (0x2c6b bytes) 2024-09-18T04:57:17.791 INFO:teuthology.orchestra.run.smithi002.stdout:4493 read 0x2655dce thru 0x266204e (0xc281 bytes) 2024-09-18T04:57:17.792 INFO:teuthology.orchestra.run.smithi002.stdout:4494 read 0x14c1d43 thru 0x14c5b2a (0x3de8 bytes) 2024-09-18T04:57:17.792 INFO:teuthology.orchestra.run.smithi002.stdout:4495 read 0x20202de thru 0x2027152 (0x6e75 bytes) 2024-09-18T04:57:17.792 INFO:teuthology.orchestra.run.smithi002.stdout:4496 read 0x4f508f0 thru 0x4f5906b (0x877c bytes) 2024-09-18T04:57:17.804 INFO:teuthology.orchestra.run.smithi002.stdout:4497 write 0xd987837 thru 0xd98bbf8 (0x43c2 bytes) 2024-09-18T04:57:17.808 INFO:teuthology.orchestra.run.smithi002.stdout:4498 write 0xcde3d7b thru 0xcdf13a7 (0xd62d bytes) 2024-09-18T04:57:17.812 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:17.813 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:17.813 INFO:teuthology.orchestra.run.smithi002.stdout:4503 write 0x7f3ecc thru 0x7f9e8b (0x5fc0 bytes) 2024-09-18T04:57:17.814 INFO:teuthology.orchestra.run.smithi002.stdout:4506 writesame 0x3577588 thru 0x357c827 data_size 0x10(0x52a0 bytes) 2024-09-18T04:57:17.816 INFO:teuthology.orchestra.run.smithi002.stdout:4507 write 0x463706 thru 0x464dc4 (0x16bf bytes) 2024-09-18T04:57:17.817 INFO:teuthology.orchestra.run.smithi002.stdout:4508 write 0x6839d5f thru 0x684836e (0xe610 bytes) 2024-09-18T04:57:17.822 INFO:teuthology.orchestra.run.smithi002.stdout:4509 punch from 0x9399dde to 0x93a2b93, (0x8db5 bytes) 2024-09-18T04:57:17.822 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:17.822 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:17.822 INFO:teuthology.orchestra.run.smithi002.stdout:4513 write 0xb7f5f35 thru 0xb80565a (0xf726 bytes) 2024-09-18T04:57:17.831 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:17.831 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:17.833 INFO:teuthology.orchestra.run.smithi002.stdout:4514 trunc from 0xd98bbf9 to 0x781fd29 2024-09-18T04:57:17.843 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:57:17.848 INFO:teuthology.orchestra.run.smithi002.stdout:4515 trunc from 0x781fd29 to 0x92cdcd7 2024-09-18T04:57:17.853 INFO:teuthology.orchestra.run.smithi002.stdout:4516 punch from 0x895383f to 0x8954b82, (0x1343 bytes) 2024-09-18T04:57:17.863 INFO:teuthology.orchestra.run.smithi002.stdout:4517 trunc from 0x92cdcd7 to 0x69b7fa 2024-09-18T04:57:17.876 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:17.874+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:17.884 INFO:teuthology.orchestra.run.smithi002.stdout:4518 write 0x35946ad thru 0x3598444 (0x3d98 bytes) 2024-09-18T04:57:17.893 INFO:teuthology.orchestra.run.smithi002.stdout:4519 writesame 0x55a6afa thru 0x55adb93 data_size 0x1d(0x709a bytes) 2024-09-18T04:57:17.893 INFO:teuthology.orchestra.run.smithi002.stdout:4520 write 0xdcba85 thru 0xdcdbbe (0x213a bytes) 2024-09-18T04:57:17.900 INFO:teuthology.orchestra.run.smithi002.stdout:4521 write 0x62b62b9 thru 0x62bd799 (0x74e1 bytes) 2024-09-18T04:57:17.907 INFO:teuthology.orchestra.run.smithi002.stdout:4523 trunc from 0x62bd79a to 0x36326b3 2024-09-18T04:57:17.919 INFO:teuthology.orchestra.run.smithi002.stdout:4524 punch from 0x34de90c to 0x34e7aab, (0x919f bytes) 2024-09-18T04:57:17.920 INFO:teuthology.orchestra.run.smithi002.stdout:4525 read 0x25e793d thru 0x25ea6e9 (0x2dad bytes) 2024-09-18T04:57:17.920 INFO:teuthology.orchestra.run.smithi002.stdout:4527 punch from 0x1cf66b4 to 0x1d0228b, (0xbbd7 bytes) 2024-09-18T04:57:17.920 INFO:teuthology.orchestra.run.smithi002.stdout:4528 read 0x24c5a9e thru 0x24cc09e (0x6601 bytes) 2024-09-18T04:57:17.935 INFO:teuthology.orchestra.run.smithi002.stdout:4529 write 0xc124ab0 thru 0xc12a6ec (0x5c3d bytes) 2024-09-18T04:57:17.939 INFO:teuthology.orchestra.run.smithi002.stdout:4530 read 0x33dd074 thru 0x33ec091 (0xf01e bytes) 2024-09-18T04:57:17.940 INFO:teuthology.orchestra.run.smithi002.stdout:4531 read 0xaffb7a9 thru 0xb000db3 (0x560b bytes) 2024-09-18T04:57:17.940 INFO:teuthology.orchestra.run.smithi002.stdout:4532 read 0x302c1ec thru 0x303a6d1 (0xe4e6 bytes) 2024-09-18T04:57:17.940 INFO:teuthology.orchestra.run.smithi002.stdout:4533 punch from 0x7f9eaf4 to 0x7fa4fbe, (0x64ca bytes) 2024-09-18T04:57:17.941 INFO:teuthology.orchestra.run.smithi002.stdout:4534 trunc from 0xc12a6ed to 0xd303701 2024-09-18T04:57:17.947 INFO:teuthology.orchestra.run.smithi002.stdout:4535 write 0x32aa9db thru 0x32b9c16 (0xf23c bytes) 2024-09-18T04:57:17.948 INFO:teuthology.orchestra.run.smithi002.stdout:4537 read 0x810ac6a thru 0x811a4ff (0xf896 bytes) 2024-09-18T04:57:17.949 INFO:teuthology.orchestra.run.smithi002.stdout:4538 writesame 0x8b8ae01 thru 0x8b97378 data_size 0x8e(0xc578 bytes) 2024-09-18T04:57:17.952 INFO:teuthology.orchestra.run.smithi002.stdout:4540 punch from 0x196b14e to 0x1970470, (0x5322 bytes) 2024-09-18T04:57:17.958 INFO:teuthology.orchestra.run.smithi002.stdout:4542 trunc from 0xd303701 to 0x848adb2 2024-09-18T04:57:17.973 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:17.973 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:17.977 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:17.975+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:17.980 INFO:teuthology.orchestra.run.smithi002.stdout:4543 write 0xd5b2e15 thru 0xd5c1426 (0xe612 bytes) 2024-09-18T04:57:17.984 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:57:17.989 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:17.989 INFO:teuthology.orchestra.run.smithi002.stdout:4546 write 0x93f0733 thru 0x93fcd80 (0xc64e bytes) 2024-09-18T04:57:17.994 INFO:teuthology.orchestra.run.smithi002.stdout:4548 write 0x212a4d0 thru 0x2138bbe (0xe6ef bytes) 2024-09-18T04:57:17.996 INFO:teuthology.orchestra.run.smithi002.stdout:4549 write 0xa7a9c82 thru 0xa7b7f50 (0xe2cf bytes) 2024-09-18T04:57:17.999 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:17.999 INFO:teuthology.orchestra.run.smithi002.stdout:4552 punch from 0x70c0f9a to 0x70c79d6, (0x6a3c bytes) 2024-09-18T04:57:17.999 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:17.999 INFO:teuthology.orchestra.run.smithi002.stdout:4554 write 0x4c6d3d4 thru 0x4c70ebd (0x3aea bytes) 2024-09-18T04:57:18.002 INFO:teuthology.orchestra.run.smithi002.stdout:4555 write 0x241e9d4 thru 0x242b3fa (0xca27 bytes) 2024-09-18T04:57:18.004 INFO:teuthology.orchestra.run.smithi002.stdout:4556 punch from 0xa091a14 to 0xa09ce14, (0xb400 bytes) 2024-09-18T04:57:18.015 INFO:teuthology.orchestra.run.smithi002.stdout:4557 trunc from 0xd5c1427 to 0x3b23f79 2024-09-18T04:57:18.030 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:57:18.034 INFO:teuthology.orchestra.run.smithi002.stdout:4560 read 0x385ccd9 thru 0x386b494 (0xe7bc bytes) 2024-09-18T04:57:18.034 INFO:teuthology.orchestra.run.smithi002.stdout:4562 punch from 0x1711d24 to 0x1718f2e, (0x720a bytes) 2024-09-18T04:57:18.034 INFO:teuthology.orchestra.run.smithi002.stdout:4564 read 0x25b0485 thru 0x25b1030 (0xbac bytes) 2024-09-18T04:57:18.035 INFO:teuthology.orchestra.run.smithi002.stdout:4565 read 0x8faff0 thru 0x909514 (0xe525 bytes) 2024-09-18T04:57:18.035 INFO:teuthology.orchestra.run.smithi002.stdout:4567 write 0x22db74a thru 0x22e4b6c (0x9423 bytes) 2024-09-18T04:57:18.048 INFO:teuthology.orchestra.run.smithi002.stdout:4570 write 0x89e2cd4 thru 0x89ec042 (0x936f bytes) 2024-09-18T04:57:18.053 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:18.053 INFO:teuthology.orchestra.run.smithi002.stdout:4572 write 0xe06a31 thru 0xe11bd2 (0xb1a2 bytes) 2024-09-18T04:57:18.054 INFO:teuthology.orchestra.run.smithi002.stdout:4573 punch from 0x2feb6ae to 0x2ff3aab, (0x83fd bytes) 2024-09-18T04:57:18.062 INFO:teuthology.orchestra.run.smithi002.stdout:4575 write 0xb0f0e8e thru 0xb0f21d9 (0x134c bytes) 2024-09-18T04:57:18.065 INFO:teuthology.orchestra.run.smithi002.stdout:4576 punch from 0x4fd41b0 to 0x4fd7d4f, (0x3b9f bytes) 2024-09-18T04:57:18.065 INFO:teuthology.orchestra.run.smithi002.stdout:4577 read 0xabb0212 thru 0xabb99cb (0x97ba bytes) 2024-09-18T04:57:18.066 INFO:teuthology.orchestra.run.smithi002.stdout:4579 clone 35 order 20 su 262144 sc 8 2024-09-18T04:57:18.078 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:18.076+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:18.107 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:18.107 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:18.117 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:57:18.160 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:57:18.161 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:57:18.177 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:57:18.177 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:57:18.179 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:18.177+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:18.195 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:57:18.195 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:57:18.209 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:57:18.210 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:57:18.226 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:57:18.226 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:57:18.243 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:57:18.243 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:57:18.259 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:57:18.260 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:57:18.264 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:18.265 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:18.265 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:18.265 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:18.273 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:57:18.276 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:57:18.276 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:57:18.279 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:18.276+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:18.362 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.4 to 1 (10000) 2024-09-18T04:57:18.378 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-18T04:57:18.379 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:18.377+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:18.386 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:18.387 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:18.387 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:18.387 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:18.387 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:18.399 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:57:18.434 INFO:teuthology.orchestra.run.smithi002.stdout:leaving image image_client.0-clone34 intact 2024-09-18T04:57:18.481 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:18.478+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:18.553 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11151 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1802240~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889328+0000", 2024-09-18T04:57:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.660174615000003, 2024-09-18T04:57:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040037528000000003, 2024-09-18T04:57:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11151 2024-09-18T04:57:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889328+0000", 2024-09-18T04:57:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889328+0000", 2024-09-18T04:57:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889328+0000", 2024-09-18T04:57:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889330+0000", 2024-09-18T04:57:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.148e-06 2024-09-18T04:57:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889336+0000", 2024-09-18T04:57:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8220000000000003e-06 2024-09-18T04:57:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889340+0000", 2024-09-18T04:57:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.318e-06 2024-09-18T04:57:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929171+0000", 2024-09-18T04:57:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039830466000000002 2024-09-18T04:57:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929268+0000", 2024-09-18T04:57:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7569e-05 2024-09-18T04:57:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929366+0000", 2024-09-18T04:57:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7573999999999998e-05 2024-09-18T04:57:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11152 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1867776~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889344+0000", 2024-09-18T04:57:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.660159030000003, 2024-09-18T04:57:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040227974, 2024-09-18T04:57:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11152 2024-09-18T04:57:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889344+0000", 2024-09-18T04:57:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889344+0000", 2024-09-18T04:57:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889344+0000", 2024-09-18T04:57:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889346+0000", 2024-09-18T04:57:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.99e-06 2024-09-18T04:57:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889351+0000", 2024-09-18T04:57:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4260000000000002e-06 2024-09-18T04:57:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889370+0000", 2024-09-18T04:57:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9262999999999998e-05 2024-09-18T04:57:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929397+0000", 2024-09-18T04:57:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040026679000000003 2024-09-18T04:57:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929479+0000", 2024-09-18T04:57:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2467999999999997e-05 2024-09-18T04:57:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929572+0000", 2024-09-18T04:57:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2519e-05 2024-09-18T04:57:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11153 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1916928~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889377+0000", 2024-09-18T04:57:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.660126056000003, 2024-09-18T04:57:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040407884999999998, 2024-09-18T04:57:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11153 2024-09-18T04:57:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889377+0000", 2024-09-18T04:57:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889377+0000", 2024-09-18T04:57:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889376+0000", 2024-09-18T04:57:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889385+0000", 2024-09-18T04:57:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1030000000000002e-06 2024-09-18T04:57:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889392+0000", 2024-09-18T04:57:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1799999999999999e-06 2024-09-18T04:57:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889396+0000", 2024-09-18T04:57:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6890000000000003e-06 2024-09-18T04:57:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929603+0000", 2024-09-18T04:57:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040206652000000002 2024-09-18T04:57:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929688+0000", 2024-09-18T04:57:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5327000000000005e-05 2024-09-18T04:57:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929785+0000", 2024-09-18T04:57:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6362999999999999e-05 2024-09-18T04:57:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11154 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:57:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.660102225999999, 2024-09-18T04:57:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040591919999999997, 2024-09-18T04:57:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11154 2024-09-18T04:57:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:57:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:57:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889400+0000", 2024-09-18T04:57:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889403+0000", 2024-09-18T04:57:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3999999999999999e-06 2024-09-18T04:57:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889409+0000", 2024-09-18T04:57:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3380000000000001e-06 2024-09-18T04:57:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889414+0000", 2024-09-18T04:57:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5789999999999997e-06 2024-09-18T04:57:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929811+0000", 2024-09-18T04:57:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040397486000000003 2024-09-18T04:57:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929898+0000", 2024-09-18T04:57:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6370000000000001e-05 2024-09-18T04:57:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929993+0000", 2024-09-18T04:57:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5136000000000005e-05 2024-09-18T04:57:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11155 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2015232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:57:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.660085047999999, 2024-09-18T04:57:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040769156000000001, 2024-09-18T04:57:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11155 2024-09-18T04:57:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:57:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:57:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:57:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889420+0000", 2024-09-18T04:57:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.181e-06 2024-09-18T04:57:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889426+0000", 2024-09-18T04:57:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8599999999999998e-06 2024-09-18T04:57:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889430+0000", 2024-09-18T04:57:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4399999999999998e-06 2024-09-18T04:57:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930020+0000", 2024-09-18T04:57:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040589698 2024-09-18T04:57:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930105+0000", 2024-09-18T04:57:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4812e-05 2024-09-18T04:57:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930187+0000", 2024-09-18T04:57:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2540999999999995e-05 2024-09-18T04:57:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11156 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2064384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:57:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.660044753999998, 2024-09-18T04:57:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040934419, 2024-09-18T04:57:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11156 2024-09-18T04:57:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:57:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:57:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889457+0000", 2024-09-18T04:57:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889462+0000", 2024-09-18T04:57:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8350000000000003e-06 2024-09-18T04:57:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889481+0000", 2024-09-18T04:57:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8845999999999999e-05 2024-09-18T04:57:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889495+0000", 2024-09-18T04:57:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4229e-05 2024-09-18T04:57:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930216+0000", 2024-09-18T04:57:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040721081999999999 2024-09-18T04:57:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930299+0000", 2024-09-18T04:57:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2995000000000001e-05 2024-09-18T04:57:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930393+0000", 2024-09-18T04:57:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3185999999999999e-05 2024-09-18T04:57:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11402 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1495040~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896220+0000", 2024-09-18T04:57:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.653282783999998, 2024-09-18T04:57:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040050388999999999, 2024-09-18T04:57:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11402 2024-09-18T04:57:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896220+0000", 2024-09-18T04:57:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896220+0000", 2024-09-18T04:57:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896220+0000", 2024-09-18T04:57:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896222+0000", 2024-09-18T04:57:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0049999999999999e-06 2024-09-18T04:57:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896227+0000", 2024-09-18T04:57:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6130000000000003e-06 2024-09-18T04:57:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896232+0000", 2024-09-18T04:57:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4100000000000001e-06 2024-09-18T04:57:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936072+0000", 2024-09-18T04:57:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039840253999999999 2024-09-18T04:57:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936175+0000", 2024-09-18T04:57:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000103037 2024-09-18T04:57:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936271+0000", 2024-09-18T04:57:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5383999999999993e-05 2024-09-18T04:57:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11403 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1544192~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896250+0000", 2024-09-18T04:57:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.653253405000001, 2024-09-18T04:57:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040244017999999999, 2024-09-18T04:57:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11403 2024-09-18T04:57:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896250+0000", 2024-09-18T04:57:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896250+0000", 2024-09-18T04:57:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896249+0000", 2024-09-18T04:57:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896251+0000", 2024-09-18T04:57:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.136e-06 2024-09-18T04:57:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896256+0000", 2024-09-18T04:57:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9350000000000002e-06 2024-09-18T04:57:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896259+0000", 2024-09-18T04:57:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8839999999999998e-06 2024-09-18T04:57:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936320+0000", 2024-09-18T04:57:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040061173999999998 2024-09-18T04:57:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936401+0000", 2024-09-18T04:57:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0987000000000002e-05 2024-09-18T04:57:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936494+0000", 2024-09-18T04:57:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2281999999999995e-05 2024-09-18T04:57:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11404 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1609728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:57:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.653240592000003, 2024-09-18T04:57:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040669012999999997, 2024-09-18T04:57:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11404 2024-09-18T04:57:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:57:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:18.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:57:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:57:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:57:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896263+0000", 2024-09-18T04:57:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2729999999999999e-06 2024-09-18T04:57:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896266+0000", 2024-09-18T04:57:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8200000000000001e-06 2024-09-18T04:57:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896269+0000", 2024-09-18T04:57:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3199999999999998e-06 2024-09-18T04:57:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936538+0000", 2024-09-18T04:57:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040269394999999999 2024-09-18T04:57:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936657+0000", 2024-09-18T04:57:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119498 2024-09-18T04:57:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936931+0000", 2024-09-18T04:57:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027391900000000002 2024-09-18T04:57:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11405 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1658880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:57:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.653232334000002, 2024-09-18T04:57:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040891034999999999, 2024-09-18T04:57:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:18.602 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:18.602 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:18.602 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:18.602 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11405 2024-09-18T04:57:18.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.602 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:18.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:57:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:57:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896270+0000", 2024-09-18T04:57:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:57:18.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:18.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896272+0000", 2024-09-18T04:57:18.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.124e-06 2024-09-18T04:57:18.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:18.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896274+0000", 2024-09-18T04:57:18.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8660000000000002e-06 2024-09-18T04:57:18.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:18.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896285+0000", 2024-09-18T04:57:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0101e-05 2024-09-18T04:57:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936986+0000", 2024-09-18T04:57:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040701213999999999 2024-09-18T04:57:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937070+0000", 2024-09-18T04:57:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4211999999999999e-05 2024-09-18T04:57:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937162+0000", 2024-09-18T04:57:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1728e-05 2024-09-18T04:57:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11406 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1708032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:57:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.653216165000003, 2024-09-18T04:57:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041085251000000003, 2024-09-18T04:57:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:18.606 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11406 2024-09-18T04:57:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:57:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:18.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:57:18.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:18.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:57:18.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:57:18.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:18.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896288+0000", 2024-09-18T04:57:18.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2440000000000001e-06 2024-09-18T04:57:18.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896291+0000", 2024-09-18T04:57:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8289999999999999e-06 2024-09-18T04:57:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896293+0000", 2024-09-18T04:57:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5399999999999998e-06 2024-09-18T04:57:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937201+0000", 2024-09-18T04:57:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040907418000000001 2024-09-18T04:57:18.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:18.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937283+0000", 2024-09-18T04:57:18.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2460999999999995e-05 2024-09-18T04:57:18.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:18.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937372+0000", 2024-09-18T04:57:18.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8987999999999996e-05 2024-09-18T04:57:18.610 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11407 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:57:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.653206345000001, 2024-09-18T04:57:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041438580000000003, 2024-09-18T04:57:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:18.611 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11407 2024-09-18T04:57:18.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.612 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:18.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:18.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:57:18.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:18.612 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:57:18.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:18.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896296+0000", 2024-09-18T04:57:18.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:18.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:18.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896298+0000", 2024-09-18T04:57:18.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:57:18.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:18.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896304+0000", 2024-09-18T04:57:18.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6570000000000002e-06 2024-09-18T04:57:18.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:18.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896308+0000", 2024-09-18T04:57:18.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0940000000000001e-06 2024-09-18T04:57:18.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:18.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937540+0000", 2024-09-18T04:57:18.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041231469 2024-09-18T04:57:18.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:18.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937625+0000", 2024-09-18T04:57:18.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5767999999999996e-05 2024-09-18T04:57:18.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:18.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937735+0000", 2024-09-18T04:57:18.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010974 2024-09-18T04:57:18.615 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.615 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:18.616 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.616 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11408 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:18.616 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:57:18.616 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.653190705999997, 2024-09-18T04:57:18.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041653055000000001, 2024-09-18T04:57:18.616 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:18.616 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:18.616 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:18.616 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:18.617 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:18.617 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11408 2024-09-18T04:57:18.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.617 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:18.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:18.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:57:18.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:18.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:57:18.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:18.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:57:18.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:18.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:18.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896314+0000", 2024-09-18T04:57:18.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0140000000000001e-06 2024-09-18T04:57:18.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.618 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.618 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:18.618 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896320+0000", 2024-09-18T04:57:18.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-18T04:57:18.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:18.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896324+0000", 2024-09-18T04:57:18.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4410000000000003e-06 2024-09-18T04:57:18.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:18.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937769+0000", 2024-09-18T04:57:18.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041445243 2024-09-18T04:57:18.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:18.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937861+0000", 2024-09-18T04:57:18.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1471000000000006e-05 2024-09-18T04:57:18.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:18.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937965+0000", 2024-09-18T04:57:18.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104516 2024-09-18T04:57:18.620 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.620 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:18.620 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.621 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11409 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:18.621 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:57:18.621 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.653174649999997, 2024-09-18T04:57:18.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041840467999999999, 2024-09-18T04:57:18.621 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:18.621 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:18.621 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:18.621 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:18.621 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:18.621 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11409 2024-09-18T04:57:18.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.621 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:18.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:18.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:57:18.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:18.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:57:18.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:18.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:57:18.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:18.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:18.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896331+0000", 2024-09-18T04:57:18.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5950000000000001e-06 2024-09-18T04:57:18.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.623 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:18.623 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896336+0000", 2024-09-18T04:57:18.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7259999999999999e-06 2024-09-18T04:57:18.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:18.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896341+0000", 2024-09-18T04:57:18.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5619999999999997e-06 2024-09-18T04:57:18.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:18.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937995+0000", 2024-09-18T04:57:18.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041654060999999999 2024-09-18T04:57:18.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:18.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938078+0000", 2024-09-18T04:57:18.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2869999999999998e-05 2024-09-18T04:57:18.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:18.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938169+0000", 2024-09-18T04:57:18.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1040999999999998e-05 2024-09-18T04:57:18.625 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.625 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:18.625 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.626 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11410 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1921024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:18.626 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:57:18.626 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.653158167000001, 2024-09-18T04:57:18.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042060342000000001, 2024-09-18T04:57:18.626 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:18.626 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:18.626 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:18.626 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:18.626 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:18.626 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11410 2024-09-18T04:57:18.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.626 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:18.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:18.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:57:18.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:18.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:57:18.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:18.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896344+0000", 2024-09-18T04:57:18.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:18.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:18.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896360+0000", 2024-09-18T04:57:18.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5068e-05 2024-09-18T04:57:18.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:18.628 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896366+0000", 2024-09-18T04:57:18.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5119999999999997e-06 2024-09-18T04:57:18.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.628 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:18.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896371+0000", 2024-09-18T04:57:18.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:57:18.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:18.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938198+0000", 2024-09-18T04:57:18.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041826579000000003 2024-09-18T04:57:18.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:18.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938297+0000", 2024-09-18T04:57:18.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9067000000000005e-05 2024-09-18T04:57:18.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:18.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938405+0000", 2024-09-18T04:57:18.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108492 2024-09-18T04:57:18.630 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.630 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:18.630 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.630 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11411 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1970176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:18.630 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:57:18.630 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.653127325, 2024-09-18T04:57:18.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042259191000000002, 2024-09-18T04:57:18.630 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:18.631 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:18.631 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:18.631 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:18.631 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:18.631 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11411 2024-09-18T04:57:18.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.631 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:18.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:18.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:57:18.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:18.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:57:18.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:18.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896375+0000", 2024-09-18T04:57:18.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:18.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:18.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896377+0000", 2024-09-18T04:57:18.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2699999999999999e-06 2024-09-18T04:57:18.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:18.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896383+0000", 2024-09-18T04:57:18.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6740000000000002e-06 2024-09-18T04:57:18.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:18.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896388+0000", 2024-09-18T04:57:18.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:57:18.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:18.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938435+0000", 2024-09-18T04:57:18.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042047169000000002 2024-09-18T04:57:18.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:18.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938534+0000", 2024-09-18T04:57:18.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8939999999999998e-05 2024-09-18T04:57:18.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:18.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938635+0000", 2024-09-18T04:57:18.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101208 2024-09-18T04:57:18.634 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.634 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:18.634 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.635 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11412 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:18.635 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:57:18.635 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.653111342999999, 2024-09-18T04:57:18.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042828079999999998, 2024-09-18T04:57:18.635 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:18.635 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:18.635 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:18.635 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:18.635 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:18.635 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11412 2024-09-18T04:57:18.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.636 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:18.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:18.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:57:18.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:18.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:57:18.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:18.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896391+0000", 2024-09-18T04:57:18.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:18.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:18.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896393+0000", 2024-09-18T04:57:18.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0810000000000001e-06 2024-09-18T04:57:18.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:18.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896399+0000", 2024-09-18T04:57:18.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7230000000000001e-06 2024-09-18T04:57:18.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:18.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896403+0000", 2024-09-18T04:57:18.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3470000000000001e-06 2024-09-18T04:57:18.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:18.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939043+0000", 2024-09-18T04:57:18.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042639829999999997 2024-09-18T04:57:18.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:18.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939124+0000", 2024-09-18T04:57:18.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0809000000000005e-05 2024-09-18T04:57:18.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:18.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939220+0000", 2024-09-18T04:57:18.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5659e-05 2024-09-18T04:57:18.639 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.639 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:18.639 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.640 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11413 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2068480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:18.640 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:57:18.640 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.653095647999997, 2024-09-18T04:57:18.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043042108000000003, 2024-09-18T04:57:18.640 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:18.640 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:18.640 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:18.640 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:18.640 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:18.640 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11413 2024-09-18T04:57:18.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.640 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:18.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:18.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:57:18.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:18.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:57:18.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:18.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:57:18.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:18.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:18.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896409+0000", 2024-09-18T04:57:18.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.165e-06 2024-09-18T04:57:18.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:18.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896415+0000", 2024-09-18T04:57:18.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6149999999999996e-06 2024-09-18T04:57:18.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:18.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896419+0000", 2024-09-18T04:57:18.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2649999999999998e-06 2024-09-18T04:57:18.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:18.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939262+0000", 2024-09-18T04:57:18.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042843444000000001 2024-09-18T04:57:18.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:18.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939351+0000", 2024-09-18T04:57:18.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8944999999999996e-05 2024-09-18T04:57:18.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:18.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939449+0000", 2024-09-18T04:57:18.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8029000000000006e-05 2024-09-18T04:57:18.644 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.644 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:18.644 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.644 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11669 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2023424~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:18.644 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.907990+0000", 2024-09-18T04:57:18.644 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.641512753999997, 2024-09-18T04:57:18.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040045942000000001, 2024-09-18T04:57:18.645 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:18.645 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:18.645 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:18.645 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:18.645 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:18.645 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11669 2024-09-18T04:57:18.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.645 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:18.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:18.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.907990+0000", 2024-09-18T04:57:18.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:18.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.907990+0000", 2024-09-18T04:57:18.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:18.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.907989+0000", 2024-09-18T04:57:18.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:18.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:18.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.907993+0000", 2024-09-18T04:57:18.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.6109999999999998e-06 2024-09-18T04:57:18.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:18.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908003+0000", 2024-09-18T04:57:18.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7610000000000002e-06 2024-09-18T04:57:18.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:18.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908009+0000", 2024-09-18T04:57:18.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7660000000000004e-06 2024-09-18T04:57:18.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:18.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.947929+0000", 2024-09-18T04:57:18.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039920297 2024-09-18T04:57:18.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:18.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.947982+0000", 2024-09-18T04:57:18.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3176e-05 2024-09-18T04:57:18.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:18.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.948036+0000", 2024-09-18T04:57:18.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4052999999999999e-05 2024-09-18T04:57:18.649 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.649 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:18.649 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.650 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11670 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2072576~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:18.650 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.908014+0000", 2024-09-18T04:57:18.650 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.641488873, 2024-09-18T04:57:18.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040179074000000002, 2024-09-18T04:57:18.650 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:18.650 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:18.650 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:18.650 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:18.650 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:18.650 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11670 2024-09-18T04:57:18.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.651 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:18.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:18.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908014+0000", 2024-09-18T04:57:18.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:18.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908014+0000", 2024-09-18T04:57:18.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:18.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:18.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908014+0000", 2024-09-18T04:57:18.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:18.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:18.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908016+0000", 2024-09-18T04:57:18.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3889999999999999e-06 2024-09-18T04:57:18.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:18.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908023+0000", 2024-09-18T04:57:18.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5599999999999999e-06 2024-09-18T04:57:18.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:18.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.908028+0000", 2024-09-18T04:57:18.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.242e-06 2024-09-18T04:57:18.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:18.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.948075+0000", 2024-09-18T04:57:18.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040047117 2024-09-18T04:57:18.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:18.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.948128+0000", 2024-09-18T04:57:18.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3127999999999997e-05 2024-09-18T04:57:18.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:18.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:18.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:18.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.948193+0000", 2024-09-18T04:57:18.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5042000000000005e-05 2024-09-18T04:57:18.654 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.654 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:18.654 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.654 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:18.654 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:18.654 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:18.656 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:57:18.657 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-09-18T04:57:18.657 INFO:teuthology.orchestra.run.smithi002.stdout:4580 punch from 0x3abac7e to 0x3ac6f25, (0xc2a7 bytes) 2024-09-18T04:57:18.657 INFO:teuthology.orchestra.run.smithi002.stdout:4581 writesame 0x1154deb thru 0x115a62a data_size 0x20(0x5840 bytes) 2024-09-18T04:57:18.657 INFO:teuthology.orchestra.run.smithi002.stdout:4583 punch from 0x931673 to 0x93452d, (0x2eba bytes) 2024-09-18T04:57:18.657 INFO:teuthology.orchestra.run.smithi002.stdout:4585 punch from 0x7ee35a2 to 0x7eeae25, (0x7883 bytes) 2024-09-18T04:57:18.657 INFO:teuthology.orchestra.run.smithi002.stdout:4586 write 0xd31a82f thru 0xd3231ce (0x89a0 bytes) 2024-09-18T04:57:18.658 INFO:teuthology.orchestra.run.smithi002.stdout:4587 trunc from 0xd3231cf to 0x53c0507 2024-09-18T04:57:18.658 INFO:teuthology.orchestra.run.smithi002.stdout:4588 read 0xf95040 thru 0xf9e842 (0x9803 bytes) 2024-09-18T04:57:18.658 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:18.653+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:18.667 INFO:teuthology.orchestra.run.smithi002.stdout:4591 writesame 0xc800380 thru 0xc80a645 data_size 0x2d(0xa2c6 bytes) 2024-09-18T04:57:18.667 INFO:teuthology.orchestra.run.smithi002.stdout:4592 write 0x4e2eda6 thru 0x4e3105c (0x22b7 bytes) 2024-09-18T04:57:18.672 INFO:teuthology.orchestra.run.smithi002.stdout:4593 trunc from 0xc80a646 to 0xb4ec406 2024-09-18T04:57:18.679 INFO:teuthology.orchestra.run.smithi002.stdout:4594 trunc from 0xb4ec406 to 0xa07ab41 2024-09-18T04:57:18.684 INFO:teuthology.orchestra.run.smithi002.stdout:4595 read 0x8a822b4 thru 0x8a8e46b (0xc1b8 bytes) 2024-09-18T04:57:18.684 INFO:teuthology.orchestra.run.smithi002.stdout:4596 writesame 0x3d5979 thru 0x3e268d data_size 0x1(0xcd15 bytes) 2024-09-18T04:57:18.725 INFO:teuthology.orchestra.run.smithi002.stdout:4597 write 0x93a3336 thru 0x93aa2e2 (0x6fad bytes) 2024-09-18T04:57:18.729 INFO:teuthology.orchestra.run.smithi002.stdout:4598 read 0x8834a10 thru 0x883c431 (0x7a22 bytes) 2024-09-18T04:57:18.734 INFO:teuthology.orchestra.run.smithi002.stdout:4600 trunc from 0xa07ab41 to 0x5fe7ddd 2024-09-18T04:57:18.743 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:18.744 INFO:teuthology.orchestra.run.smithi002.stdout:4603 read 0x2b2b841 thru 0x2b2bca2 (0x462 bytes) 2024-09-18T04:57:18.744 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:18.744 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:18.753 INFO:teuthology.orchestra.run.smithi002.stdout:4606 writesame 0x97ca1ac thru 0x97ccc0d data_size 0x32(0x2a62 bytes) 2024-09-18T04:57:18.753 INFO:teuthology.orchestra.run.smithi002.stdout:4607 read 0x4c53106 thru 0x4c5fa66 (0xc961 bytes) 2024-09-18T04:57:18.753 INFO:teuthology.orchestra.run.smithi002.stdout:4608 write 0x96bdca7 thru 0x96c9ad6 (0xbe30 bytes) 2024-09-18T04:57:18.756 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:18.754+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:18.759 INFO:teuthology.orchestra.run.smithi002.stdout:4609 read 0xc09ddd thru 0xc123b1 (0x85d5 bytes) 2024-09-18T04:57:18.768 INFO:teuthology.orchestra.run.smithi002.stdout:4610 writesame 0xcb5d2e3 thru 0xcb678dc data_size 0x23(0xa5fa bytes) 2024-09-18T04:57:18.768 INFO:teuthology.orchestra.run.smithi002.stdout:4611 punch from 0x1f2fb8b to 0x1f3e8ef, (0xed64 bytes) 2024-09-18T04:57:18.771 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:18.771 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:18.771 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:57:18.771 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:18.780 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:57:18.781 INFO:teuthology.orchestra.run.smithi002.stdout:4613 trunc from 0xcb678dd to 0xb37a2c 2024-09-18T04:57:18.822 INFO:teuthology.orchestra.run.smithi002.stdout:4615 trunc from 0xb37a2c to 0x4bdc8bb 2024-09-18T04:57:18.828 INFO:teuthology.orchestra.run.smithi002.stdout:4616 punch from 0x4baeb87 to 0x4bbaf8b, (0xc404 bytes) 2024-09-18T04:57:18.832 INFO:teuthology.orchestra.run.smithi002.stdout:4617 trunc from 0x4bdc8bb to 0xbee3c2 2024-09-18T04:57:18.858 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:18.856+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:18.862 INFO:teuthology.orchestra.run.smithi002.stdout:4620 writesame 0xe23ccfc thru 0xe23ec49 data_size 0x1(0x1f4e bytes) 2024-09-18T04:57:18.863 INFO:teuthology.orchestra.run.smithi002.stdout:4621 write 0xb4aae49 thru 0xb4b73cc (0xc584 bytes) 2024-09-18T04:57:18.866 INFO:teuthology.orchestra.run.smithi002.stdout:4622 writesame 0x1eb14f7 thru 0x1ebd714 data_size 0x1(0xc21e bytes) 2024-09-18T04:57:18.891 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:18.892 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:18.892 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:57:18.892 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:57:18.892 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:18.892 INFO:teuthology.orchestra.run.smithi002.stdout:4623 write 0x7f20697 thru 0x7f25819 (0x5183 bytes) 2024-09-18T04:57:18.895 INFO:teuthology.orchestra.run.smithi002.stdout:4624 write 0xb3b73b6 thru 0xb3b8126 (0xd71 bytes) 2024-09-18T04:57:18.898 INFO:teuthology.orchestra.run.smithi002.stdout:4625 write 0x8b9e012 thru 0x8ba0352 (0x2341 bytes) 2024-09-18T04:57:18.901 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:57:18.902 INFO:teuthology.orchestra.run.smithi002.stdout:4626 write 0x5fad5d6 thru 0x5fba064 (0xca8f bytes) 2024-09-18T04:57:18.911 INFO:teuthology.orchestra.run.smithi002.stdout:4628 trunc from 0xe23ec4a to 0x8562602 2024-09-18T04:57:18.922 INFO:teuthology.orchestra.run.smithi002.stdout:4629 read 0x182262f thru 0x182f44a (0xce1c bytes) 2024-09-18T04:57:18.923 INFO:teuthology.orchestra.run.smithi002.stdout:4630 punch from 0x4dcfbb to 0x4ddd51, (0xd96 bytes) 2024-09-18T04:57:18.923 INFO:teuthology.orchestra.run.smithi002.stdout:4632 read 0x1c63c2c thru 0x1c6979e (0x5b73 bytes) 2024-09-18T04:57:18.923 INFO:teuthology.orchestra.run.smithi002.stdout:4633 read 0x97c071 thru 0x97f51b (0x34ab bytes) 2024-09-18T04:57:18.923 INFO:teuthology.orchestra.run.smithi002.stdout:4635 read 0x57dd4ac thru 0x57e53dd (0x7f32 bytes) 2024-09-18T04:57:18.923 INFO:teuthology.orchestra.run.smithi002.stdout:4636 read 0x5aa38ca thru 0x5aa6fbe (0x36f5 bytes) 2024-09-18T04:57:18.923 INFO:teuthology.orchestra.run.smithi002.stdout:4639 read 0x5b67d3e thru 0x5b70183 (0x8446 bytes) 2024-09-18T04:57:18.923 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:18.923 INFO:teuthology.orchestra.run.smithi002.stdout:4642 write 0x25d9373 thru 0x25e5659 (0xc2e7 bytes) 2024-09-18T04:57:18.925 INFO:teuthology.orchestra.run.smithi002.stdout:4644 read 0x74970ab thru 0x74a0615 (0x956b bytes) 2024-09-18T04:57:18.925 INFO:teuthology.orchestra.run.smithi002.stdout:4645 write 0x408bf75 thru 0x408e0da (0x2166 bytes) 2024-09-18T04:57:18.937 INFO:teuthology.orchestra.run.smithi002.stdout:4646 writesame 0xb671b27 thru 0xb68185f data_size 0x5(0xfd39 bytes) 2024-09-18T04:57:18.937 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:18.944 INFO:teuthology.orchestra.run.smithi002.stdout:4649 trunc from 0xb681860 to 0x48a1192 2024-09-18T04:57:18.959 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:18.957+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:18.960 INFO:teuthology.orchestra.run.smithi002.stdout:4650 write 0x54a89f4 thru 0x54b08c3 (0x7ed0 bytes) 2024-09-18T04:57:18.963 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:18.963 INFO:teuthology.orchestra.run.smithi002.stdout:4652 punch from 0x4c3103f to 0x4c3340e, (0x23cf bytes) 2024-09-18T04:57:18.963 INFO:teuthology.orchestra.run.smithi002.stdout:4654 read 0x1d2af0d thru 0x1d34c8d (0x9d81 bytes) 2024-09-18T04:57:18.963 INFO:teuthology.orchestra.run.smithi002.stdout:4655 punch from 0x121ecbb to 0x1224e18, (0x615d bytes) 2024-09-18T04:57:18.971 INFO:teuthology.orchestra.run.smithi002.stdout:4656 writesame 0x84ce95d thru 0x84d77a2 data_size 0x1(0x8e46 bytes) 2024-09-18T04:57:18.972 INFO:teuthology.orchestra.run.smithi002.stdout:4657 punch from 0x1e4154f to 0x1e4b1cf, (0x9c80 bytes) 2024-09-18T04:57:18.980 INFO:teuthology.orchestra.run.smithi002.stdout:4659 write 0xacf81bb thru 0xad07d68 (0xfbae bytes) 2024-09-18T04:57:18.990 INFO:teuthology.orchestra.run.smithi002.stdout:4663 write 0xc04a314 thru 0xc051c46 (0x7933 bytes) 2024-09-18T04:57:18.993 INFO:teuthology.orchestra.run.smithi002.stdout:4666 writesame 0x254e4bc thru 0x2556811 data_size 0x1(0x8356 bytes) 2024-09-18T04:57:19.009 INFO:teuthology.orchestra.run.smithi002.stdout:4667 punch from 0x6fde290 to 0x6fdec7a, (0x9ea bytes) 2024-09-18T04:57:19.009 INFO:teuthology.orchestra.run.smithi002.stdout:4669 read 0x6cd6276 thru 0x6cdb7b0 (0x553b bytes) 2024-09-18T04:57:19.010 INFO:teuthology.orchestra.run.smithi002.stdout:4670 writesame 0x8d17e87 thru 0x8d1b4ba data_size 0x2(0x3634 bytes) 2024-09-18T04:57:19.014 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:19.014 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:19.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:19.015 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:19.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.015 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72135 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 394240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:19.015 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:57:19.015 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.057414606999998, 2024-09-18T04:57:19.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048559739999999997, 2024-09-18T04:57:19.015 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.015 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.015 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.016 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72135 2024-09-18T04:57:19.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.016 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:57:19.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:57:19.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:57:19.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954005+0000", 2024-09-18T04:57:19.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5629999999999999e-06 2024-09-18T04:57:19.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954008+0000", 2024-09-18T04:57:19.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-18T04:57:19.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954012+0000", 2024-09-18T04:57:19.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4790000000000001e-06 2024-09-18T04:57:19.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002418+0000", 2024-09-18T04:57:19.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048406049 2024-09-18T04:57:19.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002460+0000", 2024-09-18T04:57:19.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2188999999999999e-05 2024-09-18T04:57:19.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002564+0000", 2024-09-18T04:57:19.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010377 2024-09-18T04:57:19.019 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.019 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.020 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.020 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72136 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 409600~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:19.020 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:57:19.020 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.057309660000001, 2024-09-18T04:57:19.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048670191000000002, 2024-09-18T04:57:19.020 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.020 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.020 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72136 2024-09-18T04:57:19.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.021 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:57:19.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:57:19.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954019+0000", 2024-09-18T04:57:19.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999099 2024-09-18T04:57:19.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954112+0000", 2024-09-18T04:57:19.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3273e-05 2024-09-18T04:57:19.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954121+0000", 2024-09-18T04:57:19.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0289999999999994e-06 2024-09-18T04:57:19.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954128+0000", 2024-09-18T04:57:19.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6189999999999996e-06 2024-09-18T04:57:19.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002604+0000", 2024-09-18T04:57:19.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048476048000000001 2024-09-18T04:57:19.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002682+0000", 2024-09-18T04:57:19.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7955999999999997e-05 2024-09-18T04:57:19.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002779+0000", 2024-09-18T04:57:19.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7372000000000002e-05 2024-09-18T04:57:19.024 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.024 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.024 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.024 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72138 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 440320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:19.025 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:57:19.025 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.056875808000001, 2024-09-18T04:57:19.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048549430999999997, 2024-09-18T04:57:19.025 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.025 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.025 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.025 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.025 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.025 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72138 2024-09-18T04:57:19.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.025 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:57:19.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:57:19.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954542+0000", 2024-09-18T04:57:19.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954545+0000", 2024-09-18T04:57:19.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4360000000000001e-06 2024-09-18T04:57:19.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954551+0000", 2024-09-18T04:57:19.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4949999999999996e-06 2024-09-18T04:57:19.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954557+0000", 2024-09-18T04:57:19.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5099999999999998e-06 2024-09-18T04:57:19.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002993+0000", 2024-09-18T04:57:19.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048436358999999998 2024-09-18T04:57:19.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003039+0000", 2024-09-18T04:57:19.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5884000000000003e-05 2024-09-18T04:57:19.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003092+0000", 2024-09-18T04:57:19.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3208000000000003e-05 2024-09-18T04:57:19.029 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.029 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.029 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.029 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72139 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 455680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:19.029 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:57:19.029 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.056857000000001, 2024-09-18T04:57:19.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048718773, 2024-09-18T04:57:19.029 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.029 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.030 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.030 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.030 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.030 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72139 2024-09-18T04:57:19.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.030 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:57:19.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:57:19.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954561+0000", 2024-09-18T04:57:19.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954564+0000", 2024-09-18T04:57:19.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4099999999999998e-06 2024-09-18T04:57:19.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954570+0000", 2024-09-18T04:57:19.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.178e-06 2024-09-18T04:57:19.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954575+0000", 2024-09-18T04:57:19.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0560000000000004e-06 2024-09-18T04:57:19.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003114+0000", 2024-09-18T04:57:19.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048539212999999998 2024-09-18T04:57:19.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003176+0000", 2024-09-18T04:57:19.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2154000000000002e-05 2024-09-18T04:57:19.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003280+0000", 2024-09-18T04:57:19.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010421600000000001 2024-09-18T04:57:19.033 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.033 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.033 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.033 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72140 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 471040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:19.034 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:57:19.034 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.056601058999998, 2024-09-18T04:57:19.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048670819999999997, 2024-09-18T04:57:19.034 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.034 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.034 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.034 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.034 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.034 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72140 2024-09-18T04:57:19.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.034 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:57:19.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:57:19.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954579+0000", 2024-09-18T04:57:19.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997611 2024-09-18T04:57:19.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954821+0000", 2024-09-18T04:57:19.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024241000000000001 2024-09-18T04:57:19.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954831+0000", 2024-09-18T04:57:19.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8770000000000005e-06 2024-09-18T04:57:19.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954838+0000", 2024-09-18T04:57:19.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0690000000000004e-06 2024-09-18T04:57:19.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003317+0000", 2024-09-18T04:57:19.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048478796999999997 2024-09-18T04:57:19.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003392+0000", 2024-09-18T04:57:19.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4411000000000003e-05 2024-09-18T04:57:19.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003488+0000", 2024-09-18T04:57:19.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6927000000000002e-05 2024-09-18T04:57:19.038 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.038 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.038 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.038 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72141 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 486400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:19.038 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:57:19.038 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.056573576000002, 2024-09-18T04:57:19.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048808114, 2024-09-18T04:57:19.038 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.038 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.038 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.039 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.039 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.039 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72141 2024-09-18T04:57:19.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.039 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:57:19.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:57:19.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:57:19.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954847+0000", 2024-09-18T04:57:19.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3800000000000001e-06 2024-09-18T04:57:19.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954854+0000", 2024-09-18T04:57:19.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6080000000000001e-06 2024-09-18T04:57:19.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954859+0000", 2024-09-18T04:57:19.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:57:19.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003526+0000", 2024-09-18T04:57:19.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048666433000000002 2024-09-18T04:57:19.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003599+0000", 2024-09-18T04:57:19.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2854000000000004e-05 2024-09-18T04:57:19.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003653+0000", 2024-09-18T04:57:19.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4511e-05 2024-09-18T04:57:19.042 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.042 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.042 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.042 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72142 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 501760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:19.043 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:57:19.043 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.056534773999999, 2024-09-18T04:57:19.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048885500999999998, 2024-09-18T04:57:19.043 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.043 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.043 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.043 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.043 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.043 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72142 2024-09-18T04:57:19.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.043 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:57:19.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:57:19.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954864+0000", 2024-09-18T04:57:19.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999804 2024-09-18T04:57:19.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954886+0000", 2024-09-18T04:57:19.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2042999999999999e-05 2024-09-18T04:57:19.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954893+0000", 2024-09-18T04:57:19.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.255e-06 2024-09-18T04:57:19.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954899+0000", 2024-09-18T04:57:19.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8780000000000003e-06 2024-09-18T04:57:19.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003678+0000", 2024-09-18T04:57:19.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048778834 2024-09-18T04:57:19.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003717+0000", 2024-09-18T04:57:19.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8860999999999999e-05 2024-09-18T04:57:19.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003769+0000", 2024-09-18T04:57:19.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2389000000000003e-05 2024-09-18T04:57:19.047 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.047 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.047 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.047 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72143 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 517120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:19.048 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:57:19.048 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.056409148, 2024-09-18T04:57:19.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048951738000000002, 2024-09-18T04:57:19.048 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.048 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.048 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.048 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.048 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.048 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72143 2024-09-18T04:57:19.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.048 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:57:19.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:57:19.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954904+0000", 2024-09-18T04:57:19.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998946 2024-09-18T04:57:19.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955013+0000", 2024-09-18T04:57:19.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108977 2024-09-18T04:57:19.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955021+0000", 2024-09-18T04:57:19.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7570000000000004e-06 2024-09-18T04:57:19.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955025+0000", 2024-09-18T04:57:19.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1219999999999997e-06 2024-09-18T04:57:19.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003792+0000", 2024-09-18T04:57:19.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048767379 2024-09-18T04:57:19.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003860+0000", 2024-09-18T04:57:19.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8033000000000004e-05 2024-09-18T04:57:19.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003961+0000", 2024-09-18T04:57:19.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010103300000000001 2024-09-18T04:57:19.052 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.052 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.052 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.053 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72161 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 13312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:19.053 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:57:19.053 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.056145511, 2024-09-18T04:57:19.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048891539999999997, 2024-09-18T04:57:19.053 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.053 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.053 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.053 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.053 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.053 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72161 2024-09-18T04:57:19.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.053 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:57:19.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:57:19.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955028+0000", 2024-09-18T04:57:19.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997549 2024-09-18T04:57:19.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955278+0000", 2024-09-18T04:57:19.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024973299999999998 2024-09-18T04:57:19.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955289+0000", 2024-09-18T04:57:19.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0812000000000001e-05 2024-09-18T04:57:19.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955296+0000", 2024-09-18T04:57:19.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2570000000000001e-06 2024-09-18T04:57:19.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004002+0000", 2024-09-18T04:57:19.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048705904000000001 2024-09-18T04:57:19.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004076+0000", 2024-09-18T04:57:19.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4086999999999997e-05 2024-09-18T04:57:19.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004165+0000", 2024-09-18T04:57:19.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8712e-05 2024-09-18T04:57:19.057 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.057 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.057 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.057 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72163 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 44032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:19.057 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:57:19.057 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.055520073, 2024-09-18T04:57:19.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048567011, 2024-09-18T04:57:19.057 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.057 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.058 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.058 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.058 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.058 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72163 2024-09-18T04:57:19.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.058 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:57:19.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:57:19.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955898+0000", 2024-09-18T04:57:19.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955901+0000", 2024-09-18T04:57:19.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.706e-06 2024-09-18T04:57:19.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955907+0000", 2024-09-18T04:57:19.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3559999999999998e-06 2024-09-18T04:57:19.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955912+0000", 2024-09-18T04:57:19.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9640000000000002e-06 2024-09-18T04:57:19.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004377+0000", 2024-09-18T04:57:19.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048465293999999999 2024-09-18T04:57:19.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004418+0000", 2024-09-18T04:57:19.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0163000000000002e-05 2024-09-18T04:57:19.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004466+0000", 2024-09-18T04:57:19.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8004000000000003e-05 2024-09-18T04:57:19.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.061 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.062 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72164 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:19.062 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:57:19.062 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.055501799999998, 2024-09-18T04:57:19.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048707253999999998, 2024-09-18T04:57:19.062 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.062 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.062 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.062 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72164 2024-09-18T04:57:19.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.062 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:57:19.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:57:19.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:57:19.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955919+0000", 2024-09-18T04:57:19.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2520000000000002e-06 2024-09-18T04:57:19.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955925+0000", 2024-09-18T04:57:19.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3740000000000003e-06 2024-09-18T04:57:19.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955930+0000", 2024-09-18T04:57:19.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:57:19.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004489+0000", 2024-09-18T04:57:19.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048558581000000003 2024-09-18T04:57:19.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004529+0000", 2024-09-18T04:57:19.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9722999999999999e-05 2024-09-18T04:57:19.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004624+0000", 2024-09-18T04:57:19.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5402999999999994e-05 2024-09-18T04:57:19.066 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.066 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.066 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.066 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72165 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 74752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:19.066 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:57:19.066 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.055226941000001, 2024-09-18T04:57:19.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048633223000000003, 2024-09-18T04:57:19.066 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.066 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.066 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.067 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.067 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.067 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72165 2024-09-18T04:57:19.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.067 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:57:19.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:57:19.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955935+0000", 2024-09-18T04:57:19.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999743 2024-09-18T04:57:19.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956199+0000", 2024-09-18T04:57:19.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000263906 2024-09-18T04:57:19.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.068 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.068 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956207+0000", 2024-09-18T04:57:19.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2519999999999999e-06 2024-09-18T04:57:19.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956214+0000", 2024-09-18T04:57:19.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7610000000000002e-06 2024-09-18T04:57:19.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004661+0000", 2024-09-18T04:57:19.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048447967000000002 2024-09-18T04:57:19.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004735+0000", 2024-09-18T04:57:19.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3789999999999997e-05 2024-09-18T04:57:19.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004825+0000", 2024-09-18T04:57:19.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9724999999999999e-05 2024-09-18T04:57:19.070 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.070 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.070 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.070 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72166 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 90112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:19.070 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:57:19.071 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.054973406999999, 2024-09-18T04:57:19.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048595737999999999, 2024-09-18T04:57:19.071 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.071 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.071 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.071 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.071 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.071 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72166 2024-09-18T04:57:19.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.071 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:57:19.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:57:19.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956221+0000", 2024-09-18T04:57:19.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997759 2024-09-18T04:57:19.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956448+0000", 2024-09-18T04:57:19.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022685500000000001 2024-09-18T04:57:19.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956456+0000", 2024-09-18T04:57:19.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0099999999999995e-06 2024-09-18T04:57:19.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956462+0000", 2024-09-18T04:57:19.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7200000000000003e-06 2024-09-18T04:57:19.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004862+0000", 2024-09-18T04:57:19.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048399795000000002 2024-09-18T04:57:19.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004941+0000", 2024-09-18T04:57:19.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9414999999999998e-05 2024-09-18T04:57:19.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005041+0000", 2024-09-18T04:57:19.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9942e-05 2024-09-18T04:57:19.074 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.075 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.075 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.075 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72167 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:19.075 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:57:19.075 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.054886368999998, 2024-09-18T04:57:19.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048622894999999999, 2024-09-18T04:57:19.075 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.075 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.075 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.075 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.075 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.075 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72167 2024-09-18T04:57:19.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.076 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:57:19.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:57:19.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956468+0000", 2024-09-18T04:57:19.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999352 2024-09-18T04:57:19.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956535+0000", 2024-09-18T04:57:19.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7440999999999993e-05 2024-09-18T04:57:19.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956544+0000", 2024-09-18T04:57:19.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1039999999999996e-06 2024-09-18T04:57:19.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956550+0000", 2024-09-18T04:57:19.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9850000000000003e-06 2024-09-18T04:57:19.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005066+0000", 2024-09-18T04:57:19.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048515614999999998 2024-09-18T04:57:19.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005107+0000", 2024-09-18T04:57:19.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0843999999999997e-05 2024-09-18T04:57:19.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005155+0000", 2024-09-18T04:57:19.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8686000000000001e-05 2024-09-18T04:57:19.079 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.079 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.079 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.079 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72168 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:19.080 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:57:19.080 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.054725306000002, 2024-09-18T04:57:19.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048585179999999999, 2024-09-18T04:57:19.080 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.080 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.080 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.080 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.080 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.080 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72168 2024-09-18T04:57:19.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.080 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:57:19.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:57:19.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956555+0000", 2024-09-18T04:57:19.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998612 2024-09-18T04:57:19.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956696+0000", 2024-09-18T04:57:19.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000140922 2024-09-18T04:57:19.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956707+0000", 2024-09-18T04:57:19.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1555e-05 2024-09-18T04:57:19.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956714+0000", 2024-09-18T04:57:19.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2709999999999998e-06 2024-09-18T04:57:19.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005177+0000", 2024-09-18T04:57:19.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048463421 2024-09-18T04:57:19.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005214+0000", 2024-09-18T04:57:19.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.676e-05 2024-09-18T04:57:19.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005279+0000", 2024-09-18T04:57:19.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4899000000000003e-05 2024-09-18T04:57:19.084 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.084 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.084 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.084 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72172 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 441344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:19.084 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.957556+0000", 2024-09-18T04:57:19.084 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.053862905999999, 2024-09-18T04:57:19.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048545056000000003, 2024-09-18T04:57:19.084 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.084 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.084 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.084 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.085 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.085 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72172 2024-09-18T04:57:19.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.085 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957556+0000", 2024-09-18T04:57:19.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957556+0000", 2024-09-18T04:57:19.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957555+0000", 2024-09-18T04:57:19.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957558+0000", 2024-09-18T04:57:19.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2900000000000001e-06 2024-09-18T04:57:19.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957564+0000", 2024-09-18T04:57:19.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0700000000000003e-06 2024-09-18T04:57:19.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957569+0000", 2024-09-18T04:57:19.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7870000000000001e-06 2024-09-18T04:57:19.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005922+0000", 2024-09-18T04:57:19.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048353259000000003 2024-09-18T04:57:19.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005999+0000", 2024-09-18T04:57:19.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6889000000000002e-05 2024-09-18T04:57:19.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.006101+0000", 2024-09-18T04:57:19.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102136 2024-09-18T04:57:19.088 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.088 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.088 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.088 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72199 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:19.089 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:57:19.089 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.052193414000001, 2024-09-18T04:57:19.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048626057, 2024-09-18T04:57:19.089 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.089 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.089 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.089 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.089 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.089 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72199 2024-09-18T04:57:19.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.089 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:57:19.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:57:19.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959195+0000", 2024-09-18T04:57:19.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999969 2024-09-18T04:57:19.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959228+0000", 2024-09-18T04:57:19.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3609000000000003e-05 2024-09-18T04:57:19.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959236+0000", 2024-09-18T04:57:19.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.157e-06 2024-09-18T04:57:19.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959243+0000", 2024-09-18T04:57:19.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5910000000000001e-06 2024-09-18T04:57:19.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007684+0000", 2024-09-18T04:57:19.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048440745 2024-09-18T04:57:19.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007762+0000", 2024-09-18T04:57:19.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8360000000000002e-05 2024-09-18T04:57:19.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007851+0000", 2024-09-18T04:57:19.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9388000000000005e-05 2024-09-18T04:57:19.093 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.093 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.093 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.093 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72210 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 503808~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:19.093 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.961205+0000", 2024-09-18T04:57:19.093 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.050213423999999, 2024-09-18T04:57:19.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048502545000000001, 2024-09-18T04:57:19.093 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.093 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.094 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.094 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.094 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.094 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72210 2024-09-18T04:57:19.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.094 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.961205+0000", 2024-09-18T04:57:19.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.961205+0000", 2024-09-18T04:57:19.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.961205+0000", 2024-09-18T04:57:19.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.961207+0000", 2024-09-18T04:57:19.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.261e-06 2024-09-18T04:57:19.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.961213+0000", 2024-09-18T04:57:19.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6169999999999998e-06 2024-09-18T04:57:19.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.961217+0000", 2024-09-18T04:57:19.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6539999999999998e-06 2024-09-18T04:57:19.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.009550+0000", 2024-09-18T04:57:19.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048332561000000003 2024-09-18T04:57:19.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.009621+0000", 2024-09-18T04:57:19.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1599000000000003e-05 2024-09-18T04:57:19.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.009708+0000", 2024-09-18T04:57:19.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.6372999999999994e-05 2024-09-18T04:57:19.097 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.097 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.097 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.098 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72239 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 412672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:19.098 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.963222+0000", 2024-09-18T04:57:19.098 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.048197138999999, 2024-09-18T04:57:19.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048529088999999997, 2024-09-18T04:57:19.098 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.098 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.098 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.098 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.098 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.098 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72239 2024-09-18T04:57:19.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.098 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963222+0000", 2024-09-18T04:57:19.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963222+0000", 2024-09-18T04:57:19.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963221+0000", 2024-09-18T04:57:19.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963223+0000", 2024-09-18T04:57:19.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.136e-06 2024-09-18T04:57:19.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963231+0000", 2024-09-18T04:57:19.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7349999999999996e-06 2024-09-18T04:57:19.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963236+0000", 2024-09-18T04:57:19.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3550000000000004e-06 2024-09-18T04:57:19.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011591+0000", 2024-09-18T04:57:19.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048354772999999997 2024-09-18T04:57:19.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011661+0000", 2024-09-18T04:57:19.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9759999999999996e-05 2024-09-18T04:57:19.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011751+0000", 2024-09-18T04:57:19.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9689000000000001e-05 2024-09-18T04:57:19.102 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.102 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.102 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.102 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72240 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 428032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:19.102 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:57:19.102 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 22.048100104, 2024-09-18T04:57:19.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048643767999999997, 2024-09-18T04:57:19.102 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.102 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.103 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.103 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.103 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.103 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72240 2024-09-18T04:57:19.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.103 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:57:19.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:57:19.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963240+0000", 2024-09-18T04:57:19.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999218 2024-09-18T04:57:19.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963323+0000", 2024-09-18T04:57:19.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2375999999999999e-05 2024-09-18T04:57:19.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963332+0000", 2024-09-18T04:57:19.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1700000000000003e-06 2024-09-18T04:57:19.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963338+0000", 2024-09-18T04:57:19.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2149999999999998e-06 2024-09-18T04:57:19.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011785+0000", 2024-09-18T04:57:19.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048446493 2024-09-18T04:57:19.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011861+0000", 2024-09-18T04:57:19.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6472999999999998e-05 2024-09-18T04:57:19.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011962+0000", 2024-09-18T04:57:19.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101167 2024-09-18T04:57:19.106 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.106 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.106 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.106 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.106 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.107 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:19.109 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:57:19.109 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:19.105+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:19.110 INFO:teuthology.orchestra.run.smithi002.stdout:4672 read 0xbc13d18 thru 0xbc22a08 (0xecf1 bytes) 2024-09-18T04:57:19.110 INFO:teuthology.orchestra.run.smithi002.stdout:4675 write 0x90dc891 thru 0x90e9cb0 (0xd420 bytes) 2024-09-18T04:57:19.110 INFO:teuthology.orchestra.run.smithi002.stdout:4678 writesame 0x505e918 thru 0x505ee2f data_size 0x4(0x518 bytes) 2024-09-18T04:57:19.110 INFO:teuthology.orchestra.run.smithi002.stdout:4679 writesame 0x200d14 thru 0x20c30e data_size 0x35(0xb5fb bytes) 2024-09-18T04:57:19.110 INFO:teuthology.orchestra.run.smithi002.stdout:4680 write 0x2dbca7 thru 0x2ddeb1 (0x220b bytes) 2024-09-18T04:57:19.110 INFO:teuthology.orchestra.run.smithi002.stdout:4681 writesame 0x304ee thru 0x334b7 data_size 0x3(0x2fca bytes) 2024-09-18T04:57:19.110 INFO:teuthology.orchestra.run.smithi002.stdout:4682 punch from 0x9582cca to 0x958eada, (0xbe10 bytes) 2024-09-18T04:57:19.110 INFO:teuthology.orchestra.run.smithi002.stdout:4684 trunc from 0xc051c47 to 0x3e0d6e8 2024-09-18T04:57:19.110 INFO:teuthology.orchestra.run.smithi002.stdout:4685 read 0x17789e2 thru 0x1783a4d (0xb06c bytes) 2024-09-18T04:57:19.111 INFO:teuthology.orchestra.run.smithi002.stdout:4687 write 0xdc4fa0b thru 0xdc583c5 (0x89bb bytes) 2024-09-18T04:57:19.111 INFO:teuthology.orchestra.run.smithi002.stdout:4688 write 0xcc998aa thru 0xcca6641 (0xcd98 bytes) 2024-09-18T04:57:19.111 INFO:teuthology.orchestra.run.smithi002.stdout:4690 trunc from 0xdc583c6 to 0xb89f825 2024-09-18T04:57:19.111 INFO:teuthology.orchestra.run.smithi002.stdout:4691 punch from 0x2bc7ba9 to 0x2bc932f, (0x1786 bytes) 2024-09-18T04:57:19.111 INFO:teuthology.orchestra.run.smithi002.stdout:4692 write 0x6eccbed thru 0x6ecd61b (0xa2f bytes) 2024-09-18T04:57:19.111 INFO:teuthology.orchestra.run.smithi002.stdout:4694 write 0xe1f98d1 thru 0xe2040cd (0xa7fd bytes) 2024-09-18T04:57:19.115 INFO:teuthology.orchestra.run.smithi002.stdout:4696 read 0x39a47d7 thru 0x39a526c (0xa96 bytes) 2024-09-18T04:57:19.115 INFO:teuthology.orchestra.run.smithi002.stdout:4698 read 0xb09627c thru 0xb09ab8f (0x4914 bytes) 2024-09-18T04:57:19.116 INFO:teuthology.orchestra.run.smithi002.stdout:4699 read 0x70da95f thru 0x70df5d4 (0x4c76 bytes) 2024-09-18T04:57:19.116 INFO:teuthology.orchestra.run.smithi002.stdout:4700 write 0x2546c5 thru 0x25b068 (0x69a4 bytes) 2024-09-18T04:57:19.122 INFO:teuthology.orchestra.run.smithi002.stdout:4701 write 0x6c450c9 thru 0x6c4ef52 (0x9e8a bytes) 2024-09-18T04:57:19.127 INFO:teuthology.orchestra.run.smithi002.stdout:4702 trunc from 0xe2040ce to 0xc3b4a63 2024-09-18T04:57:19.136 INFO:teuthology.orchestra.run.smithi002.stdout:4703 read 0xa11f95a thru 0xa1272d1 (0x7978 bytes) 2024-09-18T04:57:19.136 INFO:teuthology.orchestra.run.smithi002.stdout:4704 write 0x589de13 thru 0x58ab3cb (0xd5b9 bytes) 2024-09-18T04:57:19.140 INFO:teuthology.orchestra.run.smithi002.stdout:4706 read 0x63cd8cb thru 0x63d3ffb (0x6731 bytes) 2024-09-18T04:57:19.140 INFO:teuthology.orchestra.run.smithi002.stdout:4707 writesame 0x2afc372 thru 0x2afe862 data_size 0x7(0x24f1 bytes) 2024-09-18T04:57:19.145 INFO:teuthology.orchestra.run.smithi002.stdout:4708 read 0x906256f thru 0x9064207 (0x1c99 bytes) 2024-09-18T04:57:19.145 INFO:teuthology.orchestra.run.smithi002.stdout:4711 punch from 0x50960ad to 0x50a0eff, (0xae52 bytes) 2024-09-18T04:57:19.151 INFO:teuthology.orchestra.run.smithi002.stdout:4713 trunc from 0xc3b4a63 to 0x7a9744a 2024-09-18T04:57:19.156 INFO:teuthology.orchestra.run.smithi002.stdout:4714 read 0x250a286 thru 0x2516b4a (0xc8c5 bytes) 2024-09-18T04:57:19.156 INFO:teuthology.orchestra.run.smithi002.stdout:4715 punch from 0x3eed73c to 0x3ef60ae, (0x8972 bytes) 2024-09-18T04:57:19.156 INFO:teuthology.orchestra.run.smithi002.stdout:4716 write 0x4c02277 thru 0x4c0e9db (0xc765 bytes) 2024-09-18T04:57:19.159 INFO:teuthology.orchestra.run.smithi002.stdout:4717 read 0x2bdb1e0 thru 0x2be1e53 (0x6c74 bytes) 2024-09-18T04:57:19.159 INFO:teuthology.orchestra.run.smithi002.stdout:4718 write 0x58fd148 thru 0x590411d (0x6fd6 bytes) 2024-09-18T04:57:19.176 INFO:teuthology.orchestra.run.smithi002.stdout:4719 writesame 0xda1de32 thru 0xda2d0a3 data_size 0x1(0xf272 bytes) 2024-09-18T04:57:19.176 INFO:teuthology.orchestra.run.smithi002.stdout:4720 read 0x88f350a thru 0x88fdc7e (0xa775 bytes) 2024-09-18T04:57:19.176 INFO:teuthology.orchestra.run.smithi002.stdout:4721 read 0x2cbe9b8 thru 0x2ccc625 (0xdc6e bytes) 2024-09-18T04:57:19.176 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:19.176 INFO:teuthology.orchestra.run.smithi002.stdout:4727 write 0xa54ae75 thru 0xa552794 (0x7920 bytes) 2024-09-18T04:57:19.179 INFO:teuthology.orchestra.run.smithi002.stdout:4728 writesame 0x7de6546 thru 0x7de8642 data_size 0x5(0x20fd bytes) 2024-09-18T04:57:19.183 INFO:teuthology.orchestra.run.smithi002.stdout:4729 writesame 0x848505e thru 0x8488ec6 data_size 0x1(0x3e69 bytes) 2024-09-18T04:57:19.192 INFO:teuthology.orchestra.run.smithi002.stdout:4730 writesame 0x21a03b2 thru 0x21a0f55 data_size 0xa(0xba4 bytes) 2024-09-18T04:57:19.195 INFO:teuthology.orchestra.run.smithi002.stdout:4732 writesame 0xc978c00 thru 0xc98860c data_size 0x1(0xfa0d bytes) 2024-09-18T04:57:19.209 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:19.206+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:19.219 INFO:teuthology.orchestra.run.smithi002.stdout:4733 trunc from 0xda2d0a4 to 0xd1ece9f 2024-09-18T04:57:19.224 INFO:teuthology.orchestra.run.smithi002.stdout:4734 read 0x464f4ec thru 0x465ec8d (0xf7a2 bytes) 2024-09-18T04:57:19.224 INFO:teuthology.orchestra.run.smithi002.stdout:4735 write 0x39a8c3e thru 0x39abe0c (0x31cf bytes) 2024-09-18T04:57:19.228 INFO:teuthology.orchestra.run.smithi002.stdout:4736 write 0x84d8d56 thru 0x84df040 (0x62eb bytes) 2024-09-18T04:57:19.232 INFO:teuthology.orchestra.run.smithi002.stdout:4737 read 0x6a680b8 thru 0x6a71f2a (0x9e73 bytes) 2024-09-18T04:57:19.232 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:19.232 INFO:teuthology.orchestra.run.smithi002.stdout:4739 read 0x7cfa61f thru 0x7d03f5b (0x993d bytes) 2024-09-18T04:57:19.232 INFO:teuthology.orchestra.run.smithi002.stdout:4740 write 0xd1bffd4 thru 0xd1c9d4f (0x9d7c bytes) 2024-09-18T04:57:19.236 INFO:teuthology.orchestra.run.smithi002.stdout:4741 write 0x54ac8ca thru 0x54bbc19 (0xf350 bytes) 2024-09-18T04:57:19.240 INFO:teuthology.orchestra.run.smithi002.stdout:4743 punch from 0xc6cacde to 0xc6d0d1f, (0x6041 bytes) 2024-09-18T04:57:19.240 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:19.240 INFO:teuthology.orchestra.run.smithi002.stdout:4745 read 0x83e98f6 thru 0x83eaa44 (0x114f bytes) 2024-09-18T04:57:19.240 INFO:teuthology.orchestra.run.smithi002.stdout:4749 clone 36 order 19 su 8192 sc 4 2024-09-18T04:57:19.260 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:19.260 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:19.260 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:57:19.260 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:19.269 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:57:19.309 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:19.307+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:19.368 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.5 to 1 (10000) 2024-09-18T04:57:19.379 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T04:57:19.383 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-18T04:57:19.410 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:19.408+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:19.418 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:19.418 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:19.418 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:57:19.418 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:57:19.419 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:19.428 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:57:19.440 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone35 from 0xd1ece9f (overlap 0xb37a2c) to 0x59bd16 2024-09-18T04:57:19.512 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:19.509+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:19.553 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-09-18T04:57:19.578 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:19.578 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:19.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:19.578 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:19.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.578 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60344 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1456128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:19.578 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:19.578 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.462873971, 2024-09-18T04:57:19.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044026324999999998, 2024-09-18T04:57:19.579 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.579 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.579 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.579 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.579 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.579 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60344 2024-09-18T04:57:19.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.579 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:19.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:19.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:19.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112224+0000", 2024-09-18T04:57:19.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1320000000000001e-06 2024-09-18T04:57:19.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112230+0000", 2024-09-18T04:57:19.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9619999999999998e-06 2024-09-18T04:57:19.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112234+0000", 2024-09-18T04:57:19.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7890000000000002e-06 2024-09-18T04:57:19.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156105+0000", 2024-09-18T04:57:19.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043870531999999997 2024-09-18T04:57:19.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156170+0000", 2024-09-18T04:57:19.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5036000000000005e-05 2024-09-18T04:57:19.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156248+0000", 2024-09-18T04:57:19.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8295999999999997e-05 2024-09-18T04:57:19.584 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.584 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.584 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.585 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60345 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1471488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:19.585 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:57:19.585 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.462857205999999, 2024-09-18T04:57:19.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044180090999999998, 2024-09-18T04:57:19.585 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.585 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.585 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.585 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.585 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.585 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60345 2024-09-18T04:57:19.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.585 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:57:19.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:57:19.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112238+0000", 2024-09-18T04:57:19.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112240+0000", 2024-09-18T04:57:19.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9910000000000001e-06 2024-09-18T04:57:19.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112246+0000", 2024-09-18T04:57:19.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0859999999999998e-06 2024-09-18T04:57:19.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112251+0000", 2024-09-18T04:57:19.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5020000000000003e-06 2024-09-18T04:57:19.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156274+0000", 2024-09-18T04:57:19.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044023063000000001 2024-09-18T04:57:19.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156337+0000", 2024-09-18T04:57:19.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3077000000000006e-05 2024-09-18T04:57:19.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156419+0000", 2024-09-18T04:57:19.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1699000000000005e-05 2024-09-18T04:57:19.589 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.590 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.590 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.591 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60346 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1486848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:19.592 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:57:19.592 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.462839424999999, 2024-09-18T04:57:19.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044336331999999999, 2024-09-18T04:57:19.592 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.592 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.592 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.592 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.592 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.592 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60346 2024-09-18T04:57:19.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.593 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:57:19.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:57:19.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112256+0000", 2024-09-18T04:57:19.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112258+0000", 2024-09-18T04:57:19.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1720000000000001e-06 2024-09-18T04:57:19.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112265+0000", 2024-09-18T04:57:19.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4280000000000001e-06 2024-09-18T04:57:19.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112269+0000", 2024-09-18T04:57:19.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.741e-06 2024-09-18T04:57:19.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156445+0000", 2024-09-18T04:57:19.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044175341 2024-09-18T04:57:19.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.596 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156512+0000", 2024-09-18T04:57:19.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6723000000000005e-05 2024-09-18T04:57:19.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156593+0000", 2024-09-18T04:57:19.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1284000000000003e-05 2024-09-18T04:57:19.598 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.598 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.598 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.598 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60347 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1502208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:19.598 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:57:19.598 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.462811014, 2024-09-18T04:57:19.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044482560999999997, 2024-09-18T04:57:19.599 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.599 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.599 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.599 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.599 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.599 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60347 2024-09-18T04:57:19.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.600 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:57:19.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:57:19.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112284+0000", 2024-09-18T04:57:19.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.602 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112286+0000", 2024-09-18T04:57:19.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9709999999999998e-06 2024-09-18T04:57:19.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112290+0000", 2024-09-18T04:57:19.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9199999999999997e-06 2024-09-18T04:57:19.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112293+0000", 2024-09-18T04:57:19.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870000000000001e-06 2024-09-18T04:57:19.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156617+0000", 2024-09-18T04:57:19.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044323960000000003 2024-09-18T04:57:19.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156687+0000", 2024-09-18T04:57:19.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0141000000000005e-05 2024-09-18T04:57:19.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156767+0000", 2024-09-18T04:57:19.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0140000000000002e-05 2024-09-18T04:57:19.605 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.605 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.605 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.606 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60348 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1517568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:19.606 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:19.606 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.462800009999999, 2024-09-18T04:57:19.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044645562, 2024-09-18T04:57:19.606 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.606 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.606 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.606 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.606 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.606 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60348 2024-09-18T04:57:19.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.607 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:19.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:19.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:19.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112297+0000", 2024-09-18T04:57:19.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.384e-06 2024-09-18T04:57:19.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112301+0000", 2024-09-18T04:57:19.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0570000000000003e-06 2024-09-18T04:57:19.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112304+0000", 2024-09-18T04:57:19.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6000000000000001e-06 2024-09-18T04:57:19.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156793+0000", 2024-09-18T04:57:19.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044489494999999997 2024-09-18T04:57:19.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156859+0000", 2024-09-18T04:57:19.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5483999999999998e-05 2024-09-18T04:57:19.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156941+0000", 2024-09-18T04:57:19.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2856999999999996e-05 2024-09-18T04:57:19.612 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.613 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.613 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.613 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60349 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1532928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:19.613 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:19.613 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.462790034000001, 2024-09-18T04:57:19.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044832167999999999, 2024-09-18T04:57:19.613 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.613 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.613 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.613 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.614 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.614 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60349 2024-09-18T04:57:19.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.614 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:19.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:19.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:19.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112307+0000", 2024-09-18T04:57:19.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.308e-06 2024-09-18T04:57:19.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112310+0000", 2024-09-18T04:57:19.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-18T04:57:19.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112312+0000", 2024-09-18T04:57:19.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4889999999999998e-06 2024-09-18T04:57:19.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156977+0000", 2024-09-18T04:57:19.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044664598999999999 2024-09-18T04:57:19.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157053+0000", 2024-09-18T04:57:19.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6014999999999996e-05 2024-09-18T04:57:19.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157138+0000", 2024-09-18T04:57:19.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5162999999999998e-05 2024-09-18T04:57:19.618 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.618 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.618 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.618 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60350 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1548288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:19.618 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:19.618 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.462781119999999, 2024-09-18T04:57:19.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045011858000000002, 2024-09-18T04:57:19.618 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.619 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.619 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.619 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.619 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.619 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60350 2024-09-18T04:57:19.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.619 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:19.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:19.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:19.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112316+0000", 2024-09-18T04:57:19.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7579999999999999e-06 2024-09-18T04:57:19.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112321+0000", 2024-09-18T04:57:19.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1649999999999997e-06 2024-09-18T04:57:19.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112325+0000", 2024-09-18T04:57:19.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.726e-06 2024-09-18T04:57:19.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157172+0000", 2024-09-18T04:57:19.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.04484718 2024-09-18T04:57:19.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157242+0000", 2024-09-18T04:57:19.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9675e-05 2024-09-18T04:57:19.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157327+0000", 2024-09-18T04:57:19.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4614e-05 2024-09-18T04:57:19.623 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.623 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.623 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.623 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60351 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1563648~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:19.623 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:19.624 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.462767704000001, 2024-09-18T04:57:19.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045132113000000001, 2024-09-18T04:57:19.624 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.624 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.624 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.624 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.624 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.625 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60351 2024-09-18T04:57:19.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.625 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:19.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.625 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.625 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.625 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:19.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.626 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.626 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:19.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.626 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112330+0000", 2024-09-18T04:57:19.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1660000000000001e-06 2024-09-18T04:57:19.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112336+0000", 2024-09-18T04:57:19.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5940000000000001e-06 2024-09-18T04:57:19.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112340+0000", 2024-09-18T04:57:19.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5589999999999999e-06 2024-09-18T04:57:19.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157365+0000", 2024-09-18T04:57:19.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045024675 2024-09-18T04:57:19.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157409+0000", 2024-09-18T04:57:19.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4356e-05 2024-09-18T04:57:19.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157460+0000", 2024-09-18T04:57:19.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1072000000000002e-05 2024-09-18T04:57:19.629 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.630 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.630 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.630 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60501 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1494016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:19.630 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:19.630 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.456707007999999, 2024-09-18T04:57:19.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044007728000000003, 2024-09-18T04:57:19.630 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.630 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.631 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.631 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.631 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.631 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60501 2024-09-18T04:57:19.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.631 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.631 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.631 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:19.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.632 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.632 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:19.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:19.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118391+0000", 2024-09-18T04:57:19.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:57:19.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118396+0000", 2024-09-18T04:57:19.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2680000000000002e-06 2024-09-18T04:57:19.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118400+0000", 2024-09-18T04:57:19.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2080000000000002e-06 2024-09-18T04:57:19.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162317+0000", 2024-09-18T04:57:19.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043916753000000003 2024-09-18T04:57:19.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162351+0000", 2024-09-18T04:57:19.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4372000000000002e-05 2024-09-18T04:57:19.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162397+0000", 2024-09-18T04:57:19.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5207999999999998e-05 2024-09-18T04:57:19.637 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.637 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.637 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.638 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60502 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1509376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:19.638 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:19.638 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.456691531000001, 2024-09-18T04:57:19.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044092438999999997, 2024-09-18T04:57:19.638 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.638 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.638 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.638 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.638 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.638 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60502 2024-09-18T04:57:19.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.639 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:19.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:19.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:19.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118406+0000", 2024-09-18T04:57:19.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:57:19.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118412+0000", 2024-09-18T04:57:19.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.524e-06 2024-09-18T04:57:19.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118416+0000", 2024-09-18T04:57:19.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:57:19.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162418+0000", 2024-09-18T04:57:19.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001508000000002 2024-09-18T04:57:19.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162451+0000", 2024-09-18T04:57:19.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3662999999999999e-05 2024-09-18T04:57:19.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162497+0000", 2024-09-18T04:57:19.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5550000000000003e-05 2024-09-18T04:57:19.642 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.643 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.643 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.643 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.643 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60503 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1524736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:19.643 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:57:19.643 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.456675372999999, 2024-09-18T04:57:19.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044178780000000001, 2024-09-18T04:57:19.643 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.643 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.643 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.643 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.644 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.644 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60503 2024-09-18T04:57:19.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.644 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:57:19.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:57:19.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118420+0000", 2024-09-18T04:57:19.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118422+0000", 2024-09-18T04:57:19.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.08e-06 2024-09-18T04:57:19.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118428+0000", 2024-09-18T04:57:19.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.964e-06 2024-09-18T04:57:19.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118433+0000", 2024-09-18T04:57:19.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.425e-06 2024-09-18T04:57:19.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162519+0000", 2024-09-18T04:57:19.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044086082999999998 2024-09-18T04:57:19.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162552+0000", 2024-09-18T04:57:19.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3719e-05 2024-09-18T04:57:19.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162599+0000", 2024-09-18T04:57:19.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6873999999999999e-05 2024-09-18T04:57:19.652 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.652 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.652 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.652 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60504 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1540096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:19.652 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:57:19.653 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.456659267999999, 2024-09-18T04:57:19.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044261921000000003, 2024-09-18T04:57:19.653 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.653 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.653 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.653 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.653 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.653 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60504 2024-09-18T04:57:19.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.653 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:57:19.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:57:19.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118436+0000", 2024-09-18T04:57:19.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118438+0000", 2024-09-18T04:57:19.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0250000000000001e-06 2024-09-18T04:57:19.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118444+0000", 2024-09-18T04:57:19.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4360000000000001e-06 2024-09-18T04:57:19.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118448+0000", 2024-09-18T04:57:19.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.357e-06 2024-09-18T04:57:19.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162620+0000", 2024-09-18T04:57:19.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044171821 2024-09-18T04:57:19.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162653+0000", 2024-09-18T04:57:19.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3479999999999998e-05 2024-09-18T04:57:19.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162699+0000", 2024-09-18T04:57:19.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5113e-05 2024-09-18T04:57:19.657 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.657 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.657 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.658 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60584 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1196032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:19.658 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:57:19.658 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.454866250999999, 2024-09-18T04:57:19.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044048727000000003, 2024-09-18T04:57:19.658 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.658 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.658 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.658 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.658 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.658 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60584 2024-09-18T04:57:19.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.659 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:57:19.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:57:19.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120229+0000", 2024-09-18T04:57:19.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120231+0000", 2024-09-18T04:57:19.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:57:19.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120237+0000", 2024-09-18T04:57:19.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8379999999999998e-06 2024-09-18T04:57:19.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120242+0000", 2024-09-18T04:57:19.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3889999999999998e-06 2024-09-18T04:57:19.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164187+0000", 2024-09-18T04:57:19.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043945045000000002 2024-09-18T04:57:19.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164227+0000", 2024-09-18T04:57:19.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0627999999999998e-05 2024-09-18T04:57:19.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164278+0000", 2024-09-18T04:57:19.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1146000000000001e-05 2024-09-18T04:57:19.664 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.664 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.664 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.664 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60585 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1211392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:19.664 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:57:19.665 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.454850143000002, 2024-09-18T04:57:19.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044389814, 2024-09-18T04:57:19.665 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.665 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.665 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.665 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.665 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.665 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60585 2024-09-18T04:57:19.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.665 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:57:19.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:57:19.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120245+0000", 2024-09-18T04:57:19.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120247+0000", 2024-09-18T04:57:19.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0150000000000002e-06 2024-09-18T04:57:19.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120255+0000", 2024-09-18T04:57:19.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.858e-06 2024-09-18T04:57:19.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120260+0000", 2024-09-18T04:57:19.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6530000000000002e-06 2024-09-18T04:57:19.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164489+0000", 2024-09-18T04:57:19.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044229483999999999 2024-09-18T04:57:19.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164555+0000", 2024-09-18T04:57:19.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.533e-05 2024-09-18T04:57:19.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164636+0000", 2024-09-18T04:57:19.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0819e-05 2024-09-18T04:57:19.669 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.669 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.669 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.670 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60586 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1226752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:19.670 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:19.670 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.454831691999999, 2024-09-18T04:57:19.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044578214999999997, 2024-09-18T04:57:19.670 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.670 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.670 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.670 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.670 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.670 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60586 2024-09-18T04:57:19.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.671 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:19.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:19.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:19.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120266+0000", 2024-09-18T04:57:19.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0660000000000002e-06 2024-09-18T04:57:19.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120272+0000", 2024-09-18T04:57:19.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-18T04:57:19.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120276+0000", 2024-09-18T04:57:19.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.476e-06 2024-09-18T04:57:19.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164673+0000", 2024-09-18T04:57:19.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044396672999999998 2024-09-18T04:57:19.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164749+0000", 2024-09-18T04:57:19.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5811999999999999e-05 2024-09-18T04:57:19.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164842+0000", 2024-09-18T04:57:19.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3560000000000005e-05 2024-09-18T04:57:19.674 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.674 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.674 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.674 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60587 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1242112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:19.675 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:57:19.675 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.454782593000001, 2024-09-18T04:57:19.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044634794999999998, 2024-09-18T04:57:19.675 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.675 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.675 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.675 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.675 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.675 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60587 2024-09-18T04:57:19.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.675 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:57:19.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:57:19.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120280+0000", 2024-09-18T04:57:19.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999671 2024-09-18T04:57:19.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120316+0000", 2024-09-18T04:57:19.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5728000000000001e-05 2024-09-18T04:57:19.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120329+0000", 2024-09-18T04:57:19.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2914e-05 2024-09-18T04:57:19.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120335+0000", 2024-09-18T04:57:19.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8289999999999996e-06 2024-09-18T04:57:19.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164858+0000", 2024-09-18T04:57:19.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044523417000000003 2024-09-18T04:57:19.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164896+0000", 2024-09-18T04:57:19.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7969999999999997e-05 2024-09-18T04:57:19.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164948+0000", 2024-09-18T04:57:19.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1996000000000002e-05 2024-09-18T04:57:19.679 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.679 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.679 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.679 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60595 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1364992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:19.679 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:57:19.679 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.452939377, 2024-09-18T04:57:19.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001360000000003, 2024-09-18T04:57:19.679 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.679 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.680 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.680 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.680 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60595 2024-09-18T04:57:19.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.680 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:57:19.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:57:19.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122156+0000", 2024-09-18T04:57:19.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122158+0000", 2024-09-18T04:57:19.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0040000000000002e-06 2024-09-18T04:57:19.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122164+0000", 2024-09-18T04:57:19.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8719999999999999e-06 2024-09-18T04:57:19.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122169+0000", 2024-09-18T04:57:19.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5090000000000004e-06 2024-09-18T04:57:19.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166040+0000", 2024-09-18T04:57:19.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043871226999999999 2024-09-18T04:57:19.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166096+0000", 2024-09-18T04:57:19.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6495e-05 2024-09-18T04:57:19.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166158+0000", 2024-09-18T04:57:19.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1626999999999998e-05 2024-09-18T04:57:19.684 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.684 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.684 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.684 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60596 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1380352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:19.684 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:19.684 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.452923055999999, 2024-09-18T04:57:19.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044120800000000002, 2024-09-18T04:57:19.684 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.684 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.684 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.685 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.685 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.685 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60596 2024-09-18T04:57:19.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.685 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:19.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:19.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:19.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122175+0000", 2024-09-18T04:57:19.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:57:19.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122182+0000", 2024-09-18T04:57:19.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3509999999999996e-06 2024-09-18T04:57:19.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122187+0000", 2024-09-18T04:57:19.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7940000000000002e-06 2024-09-18T04:57:19.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166184+0000", 2024-09-18T04:57:19.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043997236000000002 2024-09-18T04:57:19.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166227+0000", 2024-09-18T04:57:19.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2561000000000001e-05 2024-09-18T04:57:19.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166294+0000", 2024-09-18T04:57:19.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7068000000000002e-05 2024-09-18T04:57:19.688 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.688 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.688 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.688 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60597 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1395712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:19.689 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:19.689 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.452905028, 2024-09-18T04:57:19.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044242166999999999, 2024-09-18T04:57:19.689 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.689 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.689 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.689 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.689 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.689 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60597 2024-09-18T04:57:19.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.689 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:19.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:19.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:19.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122193+0000", 2024-09-18T04:57:19.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:57:19.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122199+0000", 2024-09-18T04:57:19.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6449999999999999e-06 2024-09-18T04:57:19.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122204+0000", 2024-09-18T04:57:19.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4819999999999996e-06 2024-09-18T04:57:19.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166329+0000", 2024-09-18T04:57:19.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044125059000000001 2024-09-18T04:57:19.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166370+0000", 2024-09-18T04:57:19.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1012e-05 2024-09-18T04:57:19.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166433+0000", 2024-09-18T04:57:19.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3178999999999999e-05 2024-09-18T04:57:19.693 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.693 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.693 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.693 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60608 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1564672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:19.693 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:57:19.693 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.450907166, 2024-09-18T04:57:19.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044082723999999997, 2024-09-18T04:57:19.693 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:19.694 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:19.694 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:19.694 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:19.694 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:19.694 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60608 2024-09-18T04:57:19.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.694 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:19.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:19.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:57:19.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:19.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:57:19.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:19.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:19.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124188+0000", 2024-09-18T04:57:19.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:19.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:19.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124190+0000", 2024-09-18T04:57:19.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0329999999999998e-06 2024-09-18T04:57:19.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:19.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124196+0000", 2024-09-18T04:57:19.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.575e-06 2024-09-18T04:57:19.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:19.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124201+0000", 2024-09-18T04:57:19.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5000000000000001e-06 2024-09-18T04:57:19.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:19.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168166+0000", 2024-09-18T04:57:19.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043965763999999997 2024-09-18T04:57:19.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:19.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168218+0000", 2024-09-18T04:57:19.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1785e-05 2024-09-18T04:57:19.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:19.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:19.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:19.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168271+0000", 2024-09-18T04:57:19.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3429e-05 2024-09-18T04:57:19.697 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.697 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.697 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.697 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:19.697 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:19.697 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:19.699 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:57:19.699 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:19.697+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:19.753 INFO:teuthology.orchestra.run.smithi002.stdout:4750 read 0x67c4a8b thru 0x67d3834 (0xedaa bytes) 2024-09-18T04:57:19.765 INFO:teuthology.orchestra.run.smithi002.stdout:4751 write 0xedc3582 thru 0xedcc062 (0x8ae1 bytes) 2024-09-18T04:57:19.772 INFO:teuthology.orchestra.run.smithi002.stdout:4752 read 0x64d874 thru 0x65241d (0x4baa bytes) 2024-09-18T04:57:19.773 INFO:teuthology.orchestra.run.smithi002.stdout:4753 read 0x56ba542 thru 0x56c12c1 (0x6d80 bytes) 2024-09-18T04:57:19.774 INFO:teuthology.orchestra.run.smithi002.stdout:4754 write 0xec1a500 thru 0xec22bbc (0x86bd bytes) 2024-09-18T04:57:19.778 INFO:teuthology.orchestra.run.smithi002.stdout:4755 writesame 0xd9f3ef thru 0xdac98d data_size 0x1(0xd59f bytes) 2024-09-18T04:57:19.799 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:19.797+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:19.802 INFO:teuthology.orchestra.run.smithi002.stdout:4756 trunc from 0xedcc063 to 0xa37c844 2024-09-18T04:57:19.833 INFO:teuthology.orchestra.run.smithi002.stdout:4757 trunc from 0xa37c844 to 0x288a7b 2024-09-18T04:57:19.858 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:19.858 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:19.858 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:57:19.859 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:19.867 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:57:19.871 INFO:teuthology.orchestra.run.smithi002.stdout:4758 writesame 0x49a1fc6 thru 0x49b1f4a data_size 0x1(0xff85 bytes) 2024-09-18T04:57:19.881 INFO:teuthology.orchestra.run.smithi002.stdout:4760 write 0x8808acb thru 0x88094da (0xa10 bytes) 2024-09-18T04:57:19.895 INFO:teuthology.orchestra.run.smithi002.stdout:4761 writesame 0xd190e96 thru 0xd1977fc data_size 0xb(0x6967 bytes) 2024-09-18T04:57:19.895 INFO:teuthology.orchestra.run.smithi002.stdout:4762 write 0x48f8f02 thru 0x4904ccf (0xbdce bytes) 2024-09-18T04:57:19.903 INFO:teuthology.orchestra.run.smithi002.stdout:4763 punch from 0x831c0 to 0x8901e, (0x5e5e bytes) 2024-09-18T04:57:19.903 INFO:teuthology.orchestra.run.smithi002.stdout:4764 punch from 0x566d8c9 to 0x5672901, (0x5038 bytes) 2024-09-18T04:57:19.903 INFO:teuthology.orchestra.run.smithi002.stdout:4765 read 0x9c5ffcc thru 0x9c68808 (0x883d bytes) 2024-09-18T04:57:19.903 INFO:teuthology.orchestra.run.smithi002.stdout:4767 read 0xad221e1 thru 0xad264a3 (0x42c3 bytes) 2024-09-18T04:57:19.903 INFO:teuthology.orchestra.run.smithi002.stdout:4768 writesame 0xb78d49d thru 0xb79a98d data_size 0x1b(0xd4f1 bytes) 2024-09-18T04:57:19.909 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:19.906+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:19.910 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:19.910 INFO:teuthology.orchestra.run.smithi002.stdout:4770 punch from 0xd063914 to 0xd0687b4, (0x4ea0 bytes) 2024-09-18T04:57:19.910 INFO:teuthology.orchestra.run.smithi002.stdout:4771 read 0x9dd6bb5 thru 0x9de1dc6 (0xb212 bytes) 2024-09-18T04:57:19.910 INFO:teuthology.orchestra.run.smithi002.stdout:4772 writesame 0x1e55f8b thru 0x1e60cad data_size 0x1(0xad23 bytes) 2024-09-18T04:57:19.922 INFO:teuthology.orchestra.run.smithi002.stdout:4774 read 0xaeece09 thru 0xaef80d5 (0xb2cd bytes) 2024-09-18T04:57:19.922 INFO:teuthology.orchestra.run.smithi002.stdout:4775 punch from 0x2ee5140 to 0x2ef01bf, (0xb07f bytes) 2024-09-18T04:57:19.922 INFO:teuthology.orchestra.run.smithi002.stdout:4776 writesame 0x288a815 thru 0x2897125 data_size 0x1(0xc911 bytes) 2024-09-18T04:57:19.939 INFO:teuthology.orchestra.run.smithi002.stdout:4777 punch from 0x7a59c1f to 0x7a6737f, (0xd760 bytes) 2024-09-18T04:57:19.939 INFO:teuthology.orchestra.run.smithi002.stdout:4779 read 0x914de01 thru 0x9157bfc (0x9dfc bytes) 2024-09-18T04:57:19.939 INFO:teuthology.orchestra.run.smithi002.stdout:4781 read 0x8a26df thru 0x8b04cc (0xddee bytes) 2024-09-18T04:57:19.946 INFO:teuthology.orchestra.run.smithi002.stdout:4782 trunc from 0xd1977fd to 0x6fa15e1 2024-09-18T04:57:19.955 INFO:teuthology.orchestra.run.smithi002.stdout:4784 punch from 0x52e70f2 to 0x52f5fa6, (0xeeb4 bytes) 2024-09-18T04:57:19.955 INFO:teuthology.orchestra.run.smithi002.stdout:4787 writesame 0x2165179 thru 0x216d1ad data_size 0x1(0x8035 bytes) 2024-09-18T04:57:19.968 INFO:teuthology.orchestra.run.smithi002.stdout:4788 punch from 0x2b5d4b3 to 0x2b6d2f0, (0xfe3d bytes) 2024-09-18T04:57:19.981 INFO:teuthology.orchestra.run.smithi002.stdout:4789 writesame 0xd954d74 thru 0xd961df0 data_size 0x1(0xd07d bytes) 2024-09-18T04:57:19.981 INFO:teuthology.orchestra.run.smithi002.stdout:4790 read 0x76620ab thru 0x766ebdf (0xcb35 bytes) 2024-09-18T04:57:19.981 INFO:teuthology.orchestra.run.smithi002.stdout:4791 writesame 0x2faebe thru 0x2fdfe0 data_size 0x7(0x3123 bytes) 2024-09-18T04:57:19.983 INFO:teuthology.orchestra.run.smithi002.stdout:4792 read 0xbe9ce3d thru 0xbea20c8 (0x528c bytes) 2024-09-18T04:57:19.983 INFO:teuthology.orchestra.run.smithi002.stdout:4793 write 0x2976419 thru 0x298569f (0xf287 bytes) 2024-09-18T04:57:19.985 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:19.985 INFO:teuthology.orchestra.run.smithi002.stdout:4797 write 0xc599163 thru 0xc59a48a (0x1328 bytes) 2024-09-18T04:57:19.993 INFO:teuthology.orchestra.run.smithi002.stdout:4798 writesame 0xddae45d thru 0xddb4877 data_size 0x7(0x641b bytes) 2024-09-18T04:57:19.993 INFO:teuthology.orchestra.run.smithi002.stdout:4799 read 0x4b0991e thru 0x4b0e242 (0x4925 bytes) 2024-09-18T04:57:19.993 INFO:teuthology.orchestra.run.smithi002.stdout:4800 writesame 0xbd81c52 thru 0xbd89f71 data_size 0x10(0x8320 bytes) 2024-09-18T04:57:19.998 INFO:teuthology.orchestra.run.smithi002.stdout:4801 punch from 0x2d85dfe to 0x2d8631b, (0x51d bytes) 2024-09-18T04:57:19.999 INFO:teuthology.orchestra.run.smithi002.stdout:4802 writesame 0x675429f thru 0x6763fc2 data_size 0x22(0xfd24 bytes) 2024-09-18T04:57:20.001 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:19.997+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:20.005 INFO:teuthology.orchestra.run.smithi002.stdout:4803 read 0xd10c411 thru 0xd112376 (0x5f66 bytes) 2024-09-18T04:57:20.006 INFO:teuthology.orchestra.run.smithi002.stdout:4804 read 0x95ef04c thru 0x95f97ea (0xa79f bytes) 2024-09-18T04:57:20.006 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:20.006 INFO:teuthology.orchestra.run.smithi002.stdout:4807 write 0x53bb595 thru 0x53c4ade (0x954a bytes) 2024-09-18T04:57:20.009 INFO:teuthology.orchestra.run.smithi002.stdout:4808 read 0x2a56822 thru 0x2a5db54 (0x7333 bytes) 2024-09-18T04:57:20.009 INFO:teuthology.orchestra.run.smithi002.stdout:4809 read 0x7ac28 thru 0x879ed (0xcdc6 bytes) 2024-09-18T04:57:20.010 INFO:teuthology.orchestra.run.smithi002.stdout:4810 writesame 0x7de9944 thru 0x7df96d2 data_size 0x4d(0xfd8f bytes) 2024-09-18T04:57:20.017 INFO:teuthology.orchestra.run.smithi002.stdout:4811 read 0x27206c4 thru 0x2723d69 (0x36a6 bytes) 2024-09-18T04:57:20.017 INFO:teuthology.orchestra.run.smithi002.stdout:4813 write 0xc90bb60 thru 0xc910e68 (0x5309 bytes) 2024-09-18T04:57:20.022 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:20.022 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:20.022 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:57:20.023 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:57:20.023 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:20.024 INFO:teuthology.orchestra.run.smithi002.stdout:4815 writesame 0xcee3b0 thru 0xcf9647 data_size 0x7f(0xb298 bytes) 2024-09-18T04:57:20.032 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:57:20.033 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:20.033 INFO:teuthology.orchestra.run.smithi002.stdout:4817 punch from 0xa2cdaa3 to 0xa2dcb8d, (0xf0ea bytes) 2024-09-18T04:57:20.033 INFO:teuthology.orchestra.run.smithi002.stdout:4820 write 0xacd90c7 thru 0xace30e6 (0xa020 bytes) 2024-09-18T04:57:20.040 INFO:teuthology.orchestra.run.smithi002.stdout:4821 punch from 0xb2943e to 0xb2fa09, (0x65cb bytes) 2024-09-18T04:57:20.040 INFO:teuthology.orchestra.run.smithi002.stdout:4822 writesame 0x4e3d6a6 thru 0x4e44e61 data_size 0x1(0x77bc bytes) 2024-09-18T04:57:20.048 INFO:teuthology.orchestra.run.smithi002.stdout:4823 read 0x58944bc thru 0x5899377 (0x4ebc bytes) 2024-09-18T04:57:20.049 INFO:teuthology.orchestra.run.smithi002.stdout:4824 punch from 0x87e41c5 to 0x87ec034, (0x7e6f bytes) 2024-09-18T04:57:20.049 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:20.049 INFO:teuthology.orchestra.run.smithi002.stdout:4827 write 0xac82229 thru 0xac84048 (0x1e20 bytes) 2024-09-18T04:57:20.055 INFO:teuthology.orchestra.run.smithi002.stdout:4829 write 0xe9630e0 thru 0xe964732 (0x1653 bytes) 2024-09-18T04:57:20.064 INFO:teuthology.orchestra.run.smithi002.stdout:4830 trunc from 0xe964733 to 0x9070ef8 2024-09-18T04:57:20.078 INFO:teuthology.orchestra.run.smithi002.stdout:4833 write 0x4fa26dd thru 0x4fa77d6 (0x50fa bytes) 2024-09-18T04:57:20.079 INFO:teuthology.orchestra.run.smithi002.stdout:4836 writesame 0x7a17e4a thru 0x7a26d39 data_size 0x8(0xeef0 bytes) 2024-09-18T04:57:20.087 INFO:teuthology.orchestra.run.smithi002.stdout:4837 write 0x26c04a2 thru 0x26c7d70 (0x78cf bytes) 2024-09-18T04:57:20.098 INFO:teuthology.orchestra.run.smithi002.stdout:4838 write 0x9a27afb thru 0x9a2b107 (0x360d bytes) 2024-09-18T04:57:20.112 INFO:teuthology.orchestra.run.smithi002.stdout:4839 trunc from 0x9a2b108 to 0xfc8445 2024-09-18T04:57:20.116 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:20.113+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:20.129 INFO:teuthology.orchestra.run.smithi002.stdout:4840 read 0xccd55c thru 0xccf3f3 (0x1e98 bytes) 2024-09-18T04:57:20.143 INFO:teuthology.orchestra.run.smithi002.stdout:4841 write 0x8918100 thru 0x891dcde (0x5bdf bytes) 2024-09-18T04:57:20.149 INFO:teuthology.orchestra.run.smithi002.stdout:4842 write 0x5cb0161 thru 0x5cbc80a (0xc6aa bytes) 2024-09-18T04:57:20.155 INFO:teuthology.orchestra.run.smithi002.stdout:4843 write 0x4733ff5 thru 0x4737ef0 (0x3efc bytes) 2024-09-18T04:57:20.171 INFO:teuthology.orchestra.run.smithi002.stdout:4844 writesame 0xd69cd90 thru 0xd6a532b data_size 0x22(0x859c bytes) 2024-09-18T04:57:20.171 INFO:teuthology.orchestra.run.smithi002.stdout:4845 read 0x88738d7 thru 0x8880325 (0xca4f bytes) 2024-09-18T04:57:20.172 INFO:teuthology.orchestra.run.smithi002.stdout:4846 write 0x200ea13 thru 0x20100c0 (0x16ae bytes) 2024-09-18T04:57:20.176 INFO:teuthology.orchestra.run.smithi002.stdout:4847 write 0xb03d9d6 thru 0xb046187 (0x87b2 bytes) 2024-09-18T04:57:20.182 INFO:teuthology.orchestra.run.smithi002.stdout:4848 write 0x479db82 thru 0x47a1004 (0x3483 bytes) 2024-09-18T04:57:20.185 INFO:teuthology.orchestra.run.smithi002.stdout:4849 punch from 0x50efd86 to 0x50fd655, (0xd8cf bytes) 2024-09-18T04:57:20.185 INFO:teuthology.orchestra.run.smithi002.stdout:4851 read 0x199f2b6 thru 0x19a6f4f (0x7c9a bytes) 2024-09-18T04:57:20.185 INFO:teuthology.orchestra.run.smithi002.stdout:4853 read 0x13f0a0 thru 0x143f49 (0x4eaa bytes) 2024-09-18T04:57:20.189 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:20.189 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:20.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:20.189 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:20.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.189 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26189 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 237568~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:20.189 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:20.189 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 39.546375169000001, 2024-09-18T04:57:20.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022263431, 2024-09-18T04:57:20.190 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:20.190 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:20.190 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:20.190 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:20.190 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:20.190 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26189 2024-09-18T04:57:20.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.190 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:20.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:20.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:20.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:20.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:20.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:20.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:20.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:20.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:20.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639787+0000", 2024-09-18T04:57:20.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9759999999999998e-06 2024-09-18T04:57:20.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:20.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639792+0000", 2024-09-18T04:57:20.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5829999999999997e-06 2024-09-18T04:57:20.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:20.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639796+0000", 2024-09-18T04:57:20.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0690000000000003e-06 2024-09-18T04:57:20.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:20.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.661944+0000", 2024-09-18T04:57:20.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022147463999999999 2024-09-18T04:57:20.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:20.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.661985+0000", 2024-09-18T04:57:20.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1360999999999999e-05 2024-09-18T04:57:20.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:20.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662048+0000", 2024-09-18T04:57:20.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3298000000000003e-05 2024-09-18T04:57:20.193 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.193 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:20.194 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.194 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26199 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 4096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:20.194 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:20.194 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 39.546359883000001, 2024-09-18T04:57:20.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022348776000000001, 2024-09-18T04:57:20.194 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:20.194 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:20.194 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:20.194 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:20.194 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:20.194 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26199 2024-09-18T04:57:20.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.195 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:20.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:20.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:20.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:20.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:20.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:20.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:20.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:20.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:20.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639806+0000", 2024-09-18T04:57:20.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1539999999999999e-06 2024-09-18T04:57:20.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:20.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639812+0000", 2024-09-18T04:57:20.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5520000000000004e-06 2024-09-18T04:57:20.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:20.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639816+0000", 2024-09-18T04:57:20.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1840000000000001e-06 2024-09-18T04:57:20.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:20.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662063+0000", 2024-09-18T04:57:20.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022247125999999999 2024-09-18T04:57:20.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:20.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662098+0000", 2024-09-18T04:57:20.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4968000000000001e-05 2024-09-18T04:57:20.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:20.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662149+0000", 2024-09-18T04:57:20.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1122000000000003e-05 2024-09-18T04:57:20.198 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.198 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:20.198 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.198 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26200 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 32768~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:20.198 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:20.198 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 39.546340358000002, 2024-09-18T04:57:20.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022501118000000001, 2024-09-18T04:57:20.198 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:20.199 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:20.199 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:20.199 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:20.199 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:20.199 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26200 2024-09-18T04:57:20.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.199 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:20.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:20.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:20.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:20.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:20.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:20.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:20.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:20.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:20.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639822+0000", 2024-09-18T04:57:20.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.007e-06 2024-09-18T04:57:20.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:20.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639827+0000", 2024-09-18T04:57:20.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5709999999999997e-06 2024-09-18T04:57:20.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:20.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639831+0000", 2024-09-18T04:57:20.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2819999999999998e-06 2024-09-18T04:57:20.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:20.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662181+0000", 2024-09-18T04:57:20.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022349878 2024-09-18T04:57:20.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:20.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662252+0000", 2024-09-18T04:57:20.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0542000000000003e-05 2024-09-18T04:57:20.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:20.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662321+0000", 2024-09-18T04:57:20.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9167999999999999e-05 2024-09-18T04:57:20.203 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.203 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:20.203 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.203 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26201 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 61440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:20.203 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:20.203 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 39.546325066999998, 2024-09-18T04:57:20.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022569783, 2024-09-18T04:57:20.203 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:20.204 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:20.204 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:20.204 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:20.204 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:20.204 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26201 2024-09-18T04:57:20.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.205 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:20.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:20.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:20.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:20.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:20.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:20.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:20.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:20.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:20.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639837+0000", 2024-09-18T04:57:20.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.928e-06 2024-09-18T04:57:20.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:20.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639843+0000", 2024-09-18T04:57:20.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8130000000000001e-06 2024-09-18T04:57:20.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:20.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639847+0000", 2024-09-18T04:57:20.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0999999999999997e-06 2024-09-18T04:57:20.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:20.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662335+0000", 2024-09-18T04:57:20.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022488207999999999 2024-09-18T04:57:20.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:20.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662370+0000", 2024-09-18T04:57:20.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5292e-05 2024-09-18T04:57:20.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:20.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662405+0000", 2024-09-18T04:57:20.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4765999999999999e-05 2024-09-18T04:57:20.208 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.208 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:20.208 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.208 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26202 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 90112~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:20.209 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:57:20.209 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 39.546309649999998, 2024-09-18T04:57:20.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022655937000000001, 2024-09-18T04:57:20.209 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:20.209 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:20.209 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:20.209 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:20.209 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:20.209 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26202 2024-09-18T04:57:20.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.209 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:20.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:20.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:57:20.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:20.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:57:20.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.210 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:20.210 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639850+0000", 2024-09-18T04:57:20.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:20.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:20.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639852+0000", 2024-09-18T04:57:20.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9700000000000002e-06 2024-09-18T04:57:20.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:20.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639858+0000", 2024-09-18T04:57:20.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.434e-06 2024-09-18T04:57:20.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:20.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639862+0000", 2024-09-18T04:57:20.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3499999999999999e-06 2024-09-18T04:57:20.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:20.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662429+0000", 2024-09-18T04:57:20.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022567409 2024-09-18T04:57:20.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:20.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662469+0000", 2024-09-18T04:57:20.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9801000000000001e-05 2024-09-18T04:57:20.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:20.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662507+0000", 2024-09-18T04:57:20.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7311000000000002e-05 2024-09-18T04:57:20.213 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.213 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:20.213 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.213 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26203 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 118784~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:20.213 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:57:20.213 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 39.546294494999998, 2024-09-18T04:57:20.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022730358999999999, 2024-09-18T04:57:20.213 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:20.213 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:20.213 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:20.213 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:20.214 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:20.214 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26203 2024-09-18T04:57:20.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.214 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:20.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:20.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:57:20.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:20.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:57:20.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:20.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639865+0000", 2024-09-18T04:57:20.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:20.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:20.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639867+0000", 2024-09-18T04:57:20.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0889999999999998e-06 2024-09-18T04:57:20.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.215 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:20.215 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639873+0000", 2024-09-18T04:57:20.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4979999999999997e-06 2024-09-18T04:57:20.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:20.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639877+0000", 2024-09-18T04:57:20.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4510000000000002e-06 2024-09-18T04:57:20.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:20.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662528+0000", 2024-09-18T04:57:20.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022650791999999999 2024-09-18T04:57:20.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:20.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662563+0000", 2024-09-18T04:57:20.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4530000000000003e-05 2024-09-18T04:57:20.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:20.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662596+0000", 2024-09-18T04:57:20.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3463000000000001e-05 2024-09-18T04:57:20.217 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.217 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:20.217 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.217 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26204 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 147456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:20.218 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:20.218 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 39.546279103000003, 2024-09-18T04:57:20.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022803233999999999, 2024-09-18T04:57:20.218 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:20.218 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:20.218 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:20.218 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:20.218 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:20.218 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26204 2024-09-18T04:57:20.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.218 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:20.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:20.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:20.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:20.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:20.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:20.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:20.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:20.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:20.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639883+0000", 2024-09-18T04:57:20.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6520000000000002e-06 2024-09-18T04:57:20.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:20.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639888+0000", 2024-09-18T04:57:20.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2870000000000004e-06 2024-09-18T04:57:20.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:20.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639893+0000", 2024-09-18T04:57:20.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3839999999999999e-06 2024-09-18T04:57:20.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:20.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662617+0000", 2024-09-18T04:57:20.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022723876 2024-09-18T04:57:20.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:20.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662651+0000", 2024-09-18T04:57:20.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3797000000000002e-05 2024-09-18T04:57:20.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:20.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662684+0000", 2024-09-18T04:57:20.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3828999999999997e-05 2024-09-18T04:57:20.222 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.222 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:20.222 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.222 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26205 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 176128~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:20.222 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:20.222 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 39.546256141999997, 2024-09-18T04:57:20.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022919108000000001, 2024-09-18T04:57:20.223 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:20.223 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:20.223 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:20.223 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:20.223 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:20.223 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26205 2024-09-18T04:57:20.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.223 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:20.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:20.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:20.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:20.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:20.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:20.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:20.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:20.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:20.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639906+0000", 2024-09-18T04:57:20.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.531e-06 2024-09-18T04:57:20.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:20.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639913+0000", 2024-09-18T04:57:20.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.331e-06 2024-09-18T04:57:20.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:20.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639920+0000", 2024-09-18T04:57:20.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5399999999999998e-06 2024-09-18T04:57:20.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:20.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662706+0000", 2024-09-18T04:57:20.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.02278587 2024-09-18T04:57:20.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:20.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662764+0000", 2024-09-18T04:57:20.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7630000000000002e-05 2024-09-18T04:57:20.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:20.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662823+0000", 2024-09-18T04:57:20.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9648000000000002e-05 2024-09-18T04:57:20.226 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.226 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:20.227 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.227 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26206 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 204800~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:20.227 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:20.227 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 39.546235742999997, 2024-09-18T04:57:20.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023047517, 2024-09-18T04:57:20.227 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:20.227 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:20.227 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:20.227 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:20.227 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:20.227 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26206 2024-09-18T04:57:20.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.228 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:20.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:20.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:20.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:20.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:20.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:20.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:20.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:20.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:20.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639926+0000", 2024-09-18T04:57:20.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3120000000000001e-06 2024-09-18T04:57:20.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:20.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639932+0000", 2024-09-18T04:57:20.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4890000000000003e-06 2024-09-18T04:57:20.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:20.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639936+0000", 2024-09-18T04:57:20.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.07e-06 2024-09-18T04:57:20.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:20.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662850+0000", 2024-09-18T04:57:20.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022913579999999999 2024-09-18T04:57:20.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:20.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662903+0000", 2024-09-18T04:57:20.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3176e-05 2024-09-18T04:57:20.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:20.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662972+0000", 2024-09-18T04:57:20.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9270999999999995e-05 2024-09-18T04:57:20.233 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.233 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:20.234 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.234 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26207 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 233472~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:20.234 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:57:20.234 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 39.546220599000002, 2024-09-18T04:57:20.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023139541999999999, 2024-09-18T04:57:20.234 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:20.234 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:20.234 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:20.234 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:20.234 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:20.234 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26207 2024-09-18T04:57:20.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.235 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:20.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:20.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:57:20.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:20.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:57:20.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:20.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639939+0000", 2024-09-18T04:57:20.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:20.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:20.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639941+0000", 2024-09-18T04:57:20.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0080000000000001e-06 2024-09-18T04:57:20.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:20.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639947+0000", 2024-09-18T04:57:20.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.485e-06 2024-09-18T04:57:20.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:20.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639951+0000", 2024-09-18T04:57:20.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4549999999999997e-06 2024-09-18T04:57:20.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:20.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663005+0000", 2024-09-18T04:57:20.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023053569999999999 2024-09-18T04:57:20.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:20.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663042+0000", 2024-09-18T04:57:20.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7245e-05 2024-09-18T04:57:20.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:20.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663079+0000", 2024-09-18T04:57:20.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7119000000000002e-05 2024-09-18T04:57:20.238 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.238 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:20.238 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.238 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26208 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 524288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:20.238 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:20.239 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 39.546203851000001, 2024-09-18T04:57:20.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023283564999999999, 2024-09-18T04:57:20.239 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:20.239 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:20.239 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:20.239 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:20.239 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:20.239 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26208 2024-09-18T04:57:20.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.239 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:20.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:20.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:20.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:20.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:20.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:20.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:20.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:20.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:20.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639958+0000", 2024-09-18T04:57:20.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0899999999999999e-06 2024-09-18T04:57:20.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:20.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639963+0000", 2024-09-18T04:57:20.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2209999999999996e-06 2024-09-18T04:57:20.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:20.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639967+0000", 2024-09-18T04:57:20.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1169999999999997e-06 2024-09-18T04:57:20.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:20.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663114+0000", 2024-09-18T04:57:20.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023146102000000002 2024-09-18T04:57:20.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:20.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663183+0000", 2024-09-18T04:57:20.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9686999999999999e-05 2024-09-18T04:57:20.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:20.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663240+0000", 2024-09-18T04:57:20.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6728000000000003e-05 2024-09-18T04:57:20.242 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.242 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:20.243 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.243 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26209 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 552960~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:20.243 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:20.243 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 39.546186075000001, 2024-09-18T04:57:20.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023440966000000001, 2024-09-18T04:57:20.243 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:20.243 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:20.243 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:20.243 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:20.243 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:20.243 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26209 2024-09-18T04:57:20.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.244 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:20.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:20.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:20.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:20.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:20.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:20.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:20.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:20.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:20.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639976+0000", 2024-09-18T04:57:20.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4940000000000002e-06 2024-09-18T04:57:20.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:20.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639982+0000", 2024-09-18T04:57:20.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0560000000000001e-06 2024-09-18T04:57:20.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:20.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639987+0000", 2024-09-18T04:57:20.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-18T04:57:20.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:20.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663278+0000", 2024-09-18T04:57:20.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023290789999999999 2024-09-18T04:57:20.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:20.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663348+0000", 2024-09-18T04:57:20.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0837e-05 2024-09-18T04:57:20.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663415+0000", 2024-09-18T04:57:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6736000000000006e-05 2024-09-18T04:57:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26210 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 581632~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 39.546169067000001, 2024-09-18T04:57:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023616437000000001, 2024-09-18T04:57:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:20.247 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:20.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:20.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:20.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:20.248 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26210 2024-09-18T04:57:20.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.248 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:20.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:20.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:20.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639993+0000", 2024-09-18T04:57:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.419e-06 2024-09-18T04:57:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:20.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640006+0000", 2024-09-18T04:57:20.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2918000000000001e-05 2024-09-18T04:57:20.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:20.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640011+0000", 2024-09-18T04:57:20.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9320000000000004e-06 2024-09-18T04:57:20.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:20.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663454+0000", 2024-09-18T04:57:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023443035000000001 2024-09-18T04:57:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663532+0000", 2024-09-18T04:57:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7443999999999999e-05 2024-09-18T04:57:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663608+0000", 2024-09-18T04:57:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6093000000000005e-05 2024-09-18T04:57:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:20.251 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26211 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 610304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:57:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 39.546144515000002, 2024-09-18T04:57:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023774337999999999, 2024-09-18T04:57:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26211 2024-09-18T04:57:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.252 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:57:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:57:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640015+0000", 2024-09-18T04:57:20.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:20.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:20.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640023+0000", 2024-09-18T04:57:20.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3900000000000004e-06 2024-09-18T04:57:20.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:20.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640036+0000", 2024-09-18T04:57:20.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3091999999999999e-05 2024-09-18T04:57:20.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:20.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640041+0000", 2024-09-18T04:57:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6219999999999997e-06 2024-09-18T04:57:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663644+0000", 2024-09-18T04:57:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023602643999999999 2024-09-18T04:57:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663718+0000", 2024-09-18T04:57:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4481999999999996e-05 2024-09-18T04:57:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663790+0000", 2024-09-18T04:57:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1589999999999997e-05 2024-09-18T04:57:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26212 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 638976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:57:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 39.546111475000004, 2024-09-18T04:57:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023924748999999999, 2024-09-18T04:57:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:20.256 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:20.257 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:20.257 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:20.257 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26212 2024-09-18T04:57:20.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.257 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:20.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:20.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:57:20.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:20.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:57:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640048+0000", 2024-09-18T04:57:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640051+0000", 2024-09-18T04:57:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.095e-06 2024-09-18T04:57:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640060+0000", 2024-09-18T04:57:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8929999999999994e-06 2024-09-18T04:57:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640065+0000", 2024-09-18T04:57:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0270000000000003e-06 2024-09-18T04:57:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663824+0000", 2024-09-18T04:57:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023758873999999999 2024-09-18T04:57:20.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:20.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663904+0000", 2024-09-18T04:57:20.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9856000000000003e-05 2024-09-18T04:57:20.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:20.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663974+0000", 2024-09-18T04:57:20.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9450999999999996e-05 2024-09-18T04:57:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26213 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 667648~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 39.546090304000003, 2024-09-18T04:57:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024059812999999999, 2024-09-18T04:57:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:20.261 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26213 2024-09-18T04:57:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640072+0000", 2024-09-18T04:57:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4200000000000001e-06 2024-09-18T04:57:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640078+0000", 2024-09-18T04:57:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0179999999999998e-06 2024-09-18T04:57:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:20.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640083+0000", 2024-09-18T04:57:20.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:57:20.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:20.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664008+0000", 2024-09-18T04:57:20.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023925287 2024-09-18T04:57:20.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:20.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664059+0000", 2024-09-18T04:57:20.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0730999999999999e-05 2024-09-18T04:57:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664130+0000", 2024-09-18T04:57:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0801000000000002e-05 2024-09-18T04:57:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26214 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 696320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 39.546073333999999, 2024-09-18T04:57:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024223738000000002, 2024-09-18T04:57:20.265 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26214 2024-09-18T04:57:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640089+0000", 2024-09-18T04:57:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0849999999999999e-06 2024-09-18T04:57:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640095+0000", 2024-09-18T04:57:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9279999999999998e-06 2024-09-18T04:57:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640099+0000", 2024-09-18T04:57:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:57:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664166+0000", 2024-09-18T04:57:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024066513000000001 2024-09-18T04:57:20.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664239+0000", 2024-09-18T04:57:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3201999999999995e-05 2024-09-18T04:57:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664311+0000", 2024-09-18T04:57:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1434999999999996e-05 2024-09-18T04:57:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26215 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 724992~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:57:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 39.546056573000001, 2024-09-18T04:57:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024302284, 2024-09-18T04:57:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26215 2024-09-18T04:57:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.270 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:57:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:57:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640103+0000", 2024-09-18T04:57:20.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640106+0000", 2024-09-18T04:57:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2120000000000002e-06 2024-09-18T04:57:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640111+0000", 2024-09-18T04:57:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7880000000000003e-06 2024-09-18T04:57:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:20.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640116+0000", 2024-09-18T04:57:20.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6099999999999999e-06 2024-09-18T04:57:20.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:20.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664327+0000", 2024-09-18T04:57:20.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024210941999999999 2024-09-18T04:57:20.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:20.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664367+0000", 2024-09-18T04:57:20.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9666000000000003e-05 2024-09-18T04:57:20.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664406+0000", 2024-09-18T04:57:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9402e-05 2024-09-18T04:57:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26216 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 753664~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 39.546040079000001, 2024-09-18T04:57:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024452854999999999, 2024-09-18T04:57:20.274 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:20.275 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:20.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:20.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:20.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:20.275 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26216 2024-09-18T04:57:20.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.275 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:20.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:20.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:20.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640122+0000", 2024-09-18T04:57:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3599999999999999e-06 2024-09-18T04:57:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640128+0000", 2024-09-18T04:57:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6339999999999998e-06 2024-09-18T04:57:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640132+0000", 2024-09-18T04:57:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:57:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664436+0000", 2024-09-18T04:57:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024303585999999999 2024-09-18T04:57:20.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:20.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664507+0000", 2024-09-18T04:57:20.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1046999999999999e-05 2024-09-18T04:57:20.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:20.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664573+0000", 2024-09-18T04:57:20.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6217000000000007e-05 2024-09-18T04:57:20.278 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.278 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:20.278 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26217 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 782336~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 39.546023845000001, 2024-09-18T04:57:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024526287000000001, 2024-09-18T04:57:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26217 2024-09-18T04:57:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:20.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:20.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640140+0000", 2024-09-18T04:57:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2989999999999998e-06 2024-09-18T04:57:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640146+0000", 2024-09-18T04:57:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9569999999999999e-06 2024-09-18T04:57:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640151+0000", 2024-09-18T04:57:20.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:57:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664590+0000", 2024-09-18T04:57:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0244389 2024-09-18T04:57:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664628+0000", 2024-09-18T04:57:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7910999999999997e-05 2024-09-18T04:57:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:20.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664663+0000", 2024-09-18T04:57:20.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5024999999999998e-05 2024-09-18T04:57:20.283 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.283 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:20.283 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.283 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:20.283 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:20.283 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:20.285 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:57:20.287 INFO:teuthology.orchestra.run.smithi002.stdout:4854 trunc from 0xd6a532c to 0x93e9ab2 2024-09-18T04:57:20.287 INFO:teuthology.orchestra.run.smithi002.stdout:4855 read 0x7be2bb4 thru 0x7be84ce (0x591b bytes) 2024-09-18T04:57:20.287 INFO:teuthology.orchestra.run.smithi002.stdout:4858 write 0xd968d84 thru 0xd9707d8 (0x7a55 bytes) 2024-09-18T04:57:20.287 INFO:teuthology.orchestra.run.smithi002.stdout:4859 write 0xc710f0 thru 0xc76fab (0x5ebc bytes) 2024-09-18T04:57:20.287 INFO:teuthology.orchestra.run.smithi002.stdout:4860 read 0x4dbac0b thru 0x4dbddfb (0x31f1 bytes) 2024-09-18T04:57:20.287 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:20.287 INFO:teuthology.orchestra.run.smithi002.stdout:4863 writesame 0x9066403 thru 0x9068018 data_size 0x5(0x1c16 bytes) 2024-09-18T04:57:20.287 INFO:teuthology.orchestra.run.smithi002.stdout:4864 punch from 0xb3ff1b8 to 0xb3ff2c6, (0x10e bytes) 2024-09-18T04:57:20.287 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:20.287 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:20.287 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:20.287 INFO:teuthology.orchestra.run.smithi002.stdout:4869 write 0x7599087 thru 0x75993e0 (0x35a bytes) 2024-09-18T04:57:20.287 INFO:teuthology.orchestra.run.smithi002.stdout:4870 trunc from 0xd9707d9 to 0x7f43446 2024-09-18T04:57:20.288 INFO:teuthology.orchestra.run.smithi002.stdout:4871 punch from 0x772c231 to 0x773017e, (0x3f4d bytes) 2024-09-18T04:57:20.288 INFO:teuthology.orchestra.run.smithi002.stdout:4872 read 0x2cbff96 thru 0x2cc9a1e (0x9a89 bytes) 2024-09-18T04:57:20.288 INFO:teuthology.orchestra.run.smithi002.stdout:4873 read 0x5505036 thru 0x5511dca (0xcd95 bytes) 2024-09-18T04:57:20.288 INFO:teuthology.orchestra.run.smithi002.stdout:4874 punch from 0x4cef2d1 to 0x4cfd261, (0xdf90 bytes) 2024-09-18T04:57:20.288 INFO:teuthology.orchestra.run.smithi002.stdout:4876 write 0xec1a5e5 thru 0xec24ff6 (0xaa12 bytes) 2024-09-18T04:57:20.288 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:20.288 INFO:teuthology.orchestra.run.smithi002.stdout:4879 punch from 0x4230277 to 0x423197e, (0x1707 bytes) 2024-09-18T04:57:20.288 INFO:teuthology.orchestra.run.smithi002.stdout:4880 write 0x5bc6ba5 thru 0x5bc7ce0 (0x113c bytes) 2024-09-18T04:57:20.288 INFO:teuthology.orchestra.run.smithi002.stdout:4881 write 0x3425283 thru 0x3426fac (0x1d2a bytes) 2024-09-18T04:57:20.288 INFO:teuthology.orchestra.run.smithi002.stdout:4882 write 0x743ee64 thru 0x7447962 (0x8aff bytes) 2024-09-18T04:57:20.288 INFO:teuthology.orchestra.run.smithi002.stdout:4883 punch from 0x2f1072a to 0x2f1b713, (0xafe9 bytes) 2024-09-18T04:57:20.288 INFO:teuthology.orchestra.run.smithi002.stdout:4884 write 0x7a9d161 thru 0x7a9ea7b (0x191b bytes) 2024-09-18T04:57:20.289 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:20.282+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:20.289 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:20.289 INFO:teuthology.orchestra.run.smithi002.stdout:4887 writesame 0x1b218b thru 0x1bb51b data_size 0x1(0x9391 bytes) 2024-09-18T04:57:20.289 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:57:20.308 INFO:teuthology.orchestra.run.smithi002.stdout:4888 writesame 0x21ce1f0 thru 0x21d024f data_size 0x8(0x2060 bytes) 2024-09-18T04:57:20.309 INFO:teuthology.orchestra.run.smithi002.stdout:4889 read 0x630555d thru 0x6311243 (0xbce7 bytes) 2024-09-18T04:57:20.309 INFO:teuthology.orchestra.run.smithi002.stdout:4891 write 0xdbf8a41 thru 0xdc06e18 (0xe3d8 bytes) 2024-09-18T04:57:20.319 INFO:teuthology.orchestra.run.smithi002.stdout:4892 writesame 0xc34eb6e thru 0xc353db6 data_size 0xb(0x5249 bytes) 2024-09-18T04:57:20.324 INFO:teuthology.orchestra.run.smithi002.stdout:4893 writesame 0x98c7b4e thru 0x98d42da data_size 0x11(0xc78d bytes) 2024-09-18T04:57:20.329 INFO:teuthology.orchestra.run.smithi002.stdout:4894 writesame 0xd345b07 thru 0xd350d96 data_size 0x8(0xb290 bytes) 2024-09-18T04:57:20.336 INFO:teuthology.orchestra.run.smithi002.stdout:4895 write 0x52e6692 thru 0x52eda44 (0x73b3 bytes) 2024-09-18T04:57:20.341 INFO:teuthology.orchestra.run.smithi002.stdout:4898 writesame 0x6c8504b thru 0x6c8e3fb data_size 0x3(0x93b1 bytes) 2024-09-18T04:57:20.349 INFO:teuthology.orchestra.run.smithi002.stdout:4899 writesame 0x7d5988c thru 0x7d621ff data_size 0x26(0x8974 bytes) 2024-09-18T04:57:20.355 INFO:teuthology.orchestra.run.smithi002.stdout:4902 punch from 0x29b93db to 0x29c10d0, (0x7cf5 bytes) 2024-09-18T04:57:20.355 INFO:teuthology.orchestra.run.smithi002.stdout:4903 read 0x8da8d58 thru 0x8db8133 (0xf3dc bytes) 2024-09-18T04:57:20.355 INFO:teuthology.orchestra.run.smithi002.stdout:4906 write 0xad3e834 thru 0xad46559 (0x7d26 bytes) 2024-09-18T04:57:20.361 INFO:teuthology.orchestra.run.smithi002.stdout:4907 write 0xa07bf47 thru 0xa082832 (0x68ec bytes) 2024-09-18T04:57:20.366 INFO:teuthology.orchestra.run.smithi002.stdout:4908 write 0xcd48832 thru 0xcd54776 (0xbf45 bytes) 2024-09-18T04:57:20.373 INFO:teuthology.orchestra.run.smithi002.stdout:4912 read 0x79f470d thru 0x79f84f8 (0x3dec bytes) 2024-09-18T04:57:20.373 INFO:teuthology.orchestra.run.smithi002.stdout:4913 write 0xd602779 thru 0xd60b041 (0x88c9 bytes) 2024-09-18T04:57:20.379 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-18T04:57:20.379 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.6 to 1 (10000) 2024-09-18T04:57:20.385 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:20.383+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:20.386 INFO:teuthology.orchestra.run.smithi002.stdout:4914 punch from 0xb87174e to 0xb87a5f3, (0x8ea5 bytes) 2024-09-18T04:57:20.400 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-18T04:57:20.401 INFO:teuthology.orchestra.run.smithi002.stdout:4915 trunc from 0xec24ff7 to 0x33627ec 2024-09-18T04:57:20.437 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:20.445 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:20.445 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:20.445 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:57:20.445 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:20.448 INFO:teuthology.orchestra.run.smithi002.stdout:4918 trunc from 0x33627ec to 0xe1f173e 2024-09-18T04:57:20.454 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:57:20.459 INFO:teuthology.orchestra.run.smithi002.stdout:4919 writesame 0xea6834d thru 0xea68dc2 data_size 0xd(0xa76 bytes) 2024-09-18T04:57:20.459 INFO:teuthology.orchestra.run.smithi002.stdout:4920 write 0xb286de1 thru 0xb28a4a3 (0x36c3 bytes) 2024-09-18T04:57:20.467 INFO:teuthology.orchestra.run.smithi002.stdout:4921 read 0x85fff21 thru 0x860da03 (0xdae3 bytes) 2024-09-18T04:57:20.467 INFO:teuthology.orchestra.run.smithi002.stdout:4922 writesame 0x6305c5b thru 0x6306c3d data_size 0x7(0xfe3 bytes) 2024-09-18T04:57:20.469 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:57:20.470 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:57:20.470 INFO:teuthology.orchestra.run.smithi002.stdout:4923 writesame 0x473881f thru 0x473f441 data_size 0x1f(0x6c23 bytes) 2024-09-18T04:57:20.476 INFO:teuthology.orchestra.run.smithi002.stdout:4925 read 0x4050dcf thru 0x4056bc4 (0x5df6 bytes) 2024-09-18T04:57:20.476 INFO:teuthology.orchestra.run.smithi002.stdout:4926 punch from 0x36c211b to 0x36c5812, (0x36f7 bytes) 2024-09-18T04:57:20.476 INFO:teuthology.orchestra.run.smithi002.stdout:4929 clone 37 order 18 su 131072 sc 14 2024-09-18T04:57:20.488 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:57:20.489 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:57:20.495 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:20.493+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:20.505 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:57:20.505 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:57:20.521 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:57:20.521 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:57:20.541 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:57:20.541 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:57:20.557 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:57:20.557 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:57:20.573 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:57:20.574 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:57:20.588 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:20.585+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:20.589 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:57:20.589 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:57:20.599 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:20.599 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:20.608 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:57:20.691 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:20.689+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:20.750 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:20.751 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:20.759 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:57:20.799 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:20.797+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:20.867 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:20.867 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:20.876 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:57:20.890 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:20.888+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:20.982 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:20.983 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:20.991 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:57:20.992 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:20.988+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:21.092 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:21.090+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:21.097 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:21.097 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:21.106 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:57:21.193 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:21.190+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:21.212 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:21.213 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:21.221 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:57:21.293 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:21.291+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:21.327 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:21.327 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:21.336 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:57:21.382 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.7 to 1 (10000) 2024-09-18T04:57:21.395 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:21.392+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:21.397 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-18T04:57:21.397 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T04:57:21.465 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone36 from 0xea68dc3 (overlap 0x288a7b) to 0x14453d 2024-09-18T04:57:21.496 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:21.493+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:21.498 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:21.498 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:21.509 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:57:21.573 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-09-18T04:57:21.585 INFO:teuthology.orchestra.run.smithi002.stdout:4932 read 0x489c32e thru 0x48a791a (0xb5ed bytes) 2024-09-18T04:57:21.586 INFO:teuthology.orchestra.run.smithi002.stdout:4933 write 0x68a2d03 thru 0x68aa292 (0x7590 bytes) 2024-09-18T04:57:21.595 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:21.595 INFO:teuthology.orchestra.run.smithi002.stdout:4935 writesame 0xe5f146a thru 0xe5fae20 data_size 0xd(0x99b7 bytes) 2024-09-18T04:57:21.596 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:21.594+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:21.599 INFO:teuthology.orchestra.run.smithi002.stdout:4936 read 0x654ac27 thru 0x6553bdc (0x8fb6 bytes) 2024-09-18T04:57:21.600 INFO:teuthology.orchestra.run.smithi002.stdout:4937 read 0xba13ce9 thru 0xba1a6c6 (0x69de bytes) 2024-09-18T04:57:21.600 INFO:teuthology.orchestra.run.smithi002.stdout:4938 punch from 0xdc00c1f to 0xdc02e25, (0x2206 bytes) 2024-09-18T04:57:21.600 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:21.600 INFO:teuthology.orchestra.run.smithi002.stdout:4940 writesame 0xd172934 thru 0xd181c80 data_size 0x1(0xf34d bytes) 2024-09-18T04:57:21.629 INFO:teuthology.orchestra.run.smithi002.stdout:4941 trunc from 0xea68dc3 to 0xdc4a477 2024-09-18T04:57:21.646 INFO:teuthology.orchestra.run.smithi002.stdout:4942 write 0x5a6a0ac thru 0x5a6f557 (0x54ac bytes) 2024-09-18T04:57:21.650 INFO:teuthology.orchestra.run.smithi002.stdout:4943 trunc from 0xdc4a477 to 0xcdfc459 2024-09-18T04:57:21.667 INFO:teuthology.orchestra.run.smithi002.stdout:4944 write 0x8e81737 thru 0x8e83d0f (0x25d9 bytes) 2024-09-18T04:57:21.671 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:21.671 INFO:teuthology.orchestra.run.smithi002.stdout:4946 write 0x13eddd8 thru 0x13f76d7 (0x9900 bytes) 2024-09-18T04:57:21.674 INFO:teuthology.orchestra.run.smithi002.stdout:4947 punch from 0x40d765d to 0x40dd6bc, (0x605f bytes) 2024-09-18T04:57:21.674 INFO:teuthology.orchestra.run.smithi002.stdout:4949 read 0x72a5310 thru 0x72b0ebb (0xbbac bytes) 2024-09-18T04:57:21.674 INFO:teuthology.orchestra.run.smithi002.stdout:4952 punch from 0x7fa06f5 to 0x7fa83e6, (0x7cf1 bytes) 2024-09-18T04:57:21.674 INFO:teuthology.orchestra.run.smithi002.stdout:4953 write 0x7985c41 thru 0x798c9d5 (0x6d95 bytes) 2024-09-18T04:57:21.677 INFO:teuthology.orchestra.run.smithi002.stdout:4954 write 0x4536a8f thru 0x45454c4 (0xea36 bytes) 2024-09-18T04:57:21.679 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:21.679 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:21.681 INFO:teuthology.orchestra.run.smithi002.stdout:4955 write 0xa43838a thru 0xa446b69 (0xe7e0 bytes) 2024-09-18T04:57:21.686 INFO:teuthology.orchestra.run.smithi002.stdout:4956 writesame 0xae17d72 thru 0xae22005 data_size 0xa(0xa294 bytes) 2024-09-18T04:57:21.691 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:57:21.694 INFO:teuthology.orchestra.run.smithi002.stdout:4957 punch from 0x1282b0e to 0x1290802, (0xdcf4 bytes) 2024-09-18T04:57:21.694 INFO:teuthology.orchestra.run.smithi002.stdout:4959 read 0x8b83f93 thru 0x8b8653e (0x25ac bytes) 2024-09-18T04:57:21.694 INFO:teuthology.orchestra.run.smithi002.stdout:4961 read 0x62197b2 thru 0x6225f71 (0xc7c0 bytes) 2024-09-18T04:57:21.696 INFO:teuthology.orchestra.run.smithi002.stdout:4962 write 0x5e3443c thru 0x5e359ce (0x1593 bytes) 2024-09-18T04:57:21.699 INFO:teuthology.orchestra.run.smithi002.stdout:4963 flatten 2024-09-18T04:57:21.699 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:21.697+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:21.763 INFO:teuthology.orchestra.run.smithi002.stdout:4966 writesame 0xd195e13 thru 0xd1a2a82 data_size 0x8(0xcc70 bytes) 2024-09-18T04:57:21.764 INFO:teuthology.orchestra.run.smithi002.stdout:4967 write 0xcba52b2 thru 0xcbb3c05 (0xe954 bytes) 2024-09-18T04:57:21.767 INFO:teuthology.orchestra.run.smithi002.stdout:4969 clone 38 order 19 su 32768 sc 5 2024-09-18T04:57:21.795 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:57:21.796 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-18T04:57:21.798 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:21.796+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:21.806 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-09-18T04:57:21.379145+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1394288833,"num_objects":1156,"num_object_clones":252,"num_object_copies":2312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1156,"num_whiteouts":216,"num_read":147282,"num_read_kb":524364,"num_write":16446,"num_write_kb":308814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":419,"num_bytes_recovered":34820157,"num_keys_recovered":65,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12469,"ondisk_log_size":12469,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":333560,"kb_used_data":57496,"kb_used_omap":10,"kb_used_meta":275957,"kb_avail":660269320,"statfs":{"total":676457349120,"available":676115783680,"internally_reserved":0,"allocated":58875904,"data_stored":364007663,"data_compressed":15172643,"data_compressed_allocated":32751616,"data_compressed_original":344818305,"omap_allocated":11149,"internal_metadata":282580083},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":88843622,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":8085,"num_read_kb":56825,"num_write":2075,"num_write_kb":29934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.024921"},"pg_stats":[{"pgid":"3.1f","version":"72'445","reported_seq":3242,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:18.904055+0000","last_change":"2024-09-18T04:56:23.130295+0000","last_active":"2024-09-18T04:57:18.904055+0000","last_peered":"2024-09-18T04:57:18.904055+0000","last_clean":"2024-09-18T04:57:18.904055+0000","last_became_active":"2024-09-18T04:56:21.132342+0000","last_became_peered":"2024-09-18T04:56:21.132342+0000","last_unstale":"2024-09-18T04:57:18.904055+0000","last_undegraded":"2024-09-18T04:57:18.904055+0000","last_fullsized":"2024-09-18T04:57:18.904055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:43:47.128460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011070279999999999,"stat_sum":{"num_bytes":49363979,"num_objects":39,"num_object_clones":6,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":4,"num_read":4656,"num_read_kb":16410,"num_write":555,"num_write_kb":8423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"72'565","reported_seq":962,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.078717+0000","last_change":"2024-09-18T04:56:23.582447+0000","last_active":"2024-09-18T04:57:19.078717+0000","last_peered":"2024-09-18T04:57:19.078717+0000","last_clean":"2024-09-18T04:57:19.078717+0000","last_became_active":"2024-09-18T04:56:21.126431+0000","last_became_peered":"2024-09-18T04:56:21.126431+0000","last_unstale":"2024-09-18T04:57:19.078717+0000","last_undegraded":"2024-09-18T04:57:19.078717+0000","last_fullsized":"2024-09-18T04:57:19.078717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:59.671751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062644600000000001,"stat_sum":{"num_bytes":55973624,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":6,"num_read":2228,"num_read_kb":12318,"num_write":846,"num_write_kb":6341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"70'252","reported_seq":8375,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-18T04:57:17.359739+0000","last_change":"2024-09-18T04:56:23.137064+0000","last_active":"2024-09-18T04:57:17.359739+0000","last_peered":"2024-09-18T04:57:17.359739+0000","last_clean":"2024-09-18T04:57:17.359739+0000","last_became_active":"2024-09-18T04:56:21.131613+0000","last_became_peered":"2024-09-18T04:56:21.131613+0000","last_unstale":"2024-09-18T04:57:17.359739+0000","last_undegraded":"2024-09-18T04:57:17.359739+0000","last_fullsized":"2024-09-18T04:57:17.359739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:15:27.419409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000455248,"stat_sum":{"num_bytes":56110006,"num_objects":40,"num_object_clones":11,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":5,"num_read":8178,"num_read_kb":25621,"num_write":305,"num_write_kb":21539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"72'557","reported_seq":6022,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.233331+0000","last_change":"2024-09-18T04:56:23.137422+0000","last_active":"2024-09-18T04:57:19.233331+0000","last_peered":"2024-09-18T04:57:19.233331+0000","last_clean":"2024-09-18T04:57:19.233331+0000","last_became_active":"2024-09-18T04:56:21.129551+0000","last_became_peered":"2024-09-18T04:56:21.129551+0000","last_unstale":"2024-09-18T04:57:19.233331+0000","last_undegraded":"2024-09-18T04:57:19.233331+0000","last_fullsized":"2024-09-18T04:57:19.233331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:38:38.231233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081678499999999999,"stat_sum":{"num_bytes":10773715,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":5,"num_read":7443,"num_read_kb":7698,"num_write":821,"num_write_kb":4603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"72'200","reported_seq":554,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.157961+0000","last_change":"2024-09-18T04:56:23.130841+0000","last_active":"2024-09-18T04:57:19.157961+0000","last_peered":"2024-09-18T04:57:19.157961+0000","last_clean":"2024-09-18T04:57:19.157961+0000","last_became_active":"2024-09-18T04:56:21.129660+0000","last_became_peered":"2024-09-18T04:56:21.129660+0000","last_unstale":"2024-09-18T04:57:19.157961+0000","last_undegraded":"2024-09-18T04:57:19.157961+0000","last_fullsized":"2024-09-18T04:57:19.157961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:43.007677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014550170000000001,"stat_sum":{"num_bytes":7144998,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":7,"num_read":379,"num_read_kb":6956,"num_write":221,"num_write_kb":4955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"72'299","reported_seq":2133,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.173670+0000","last_change":"2024-09-18T04:57:08.324870+0000","last_active":"2024-09-18T04:57:19.173670+0000","last_peered":"2024-09-18T04:57:19.173670+0000","last_clean":"2024-09-18T04:57:19.173670+0000","last_became_active":"2024-09-18T04:57:08.323638+0000","last_became_peered":"2024-09-18T04:57:08.323638+0000","last_unstale":"2024-09-18T04:57:19.173670+0000","last_undegraded":"2024-09-18T04:57:19.173670+0000","last_fullsized":"2024-09-18T04:57:19.173670+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:55:00.155452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087073300000000005,"stat_sum":{"num_bytes":34050594,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":9,"num_read":2065,"num_read_kb":9990,"num_write":355,"num_write_kb":17778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":2691477,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"72'226","reported_seq":2985,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.020124+0000","last_change":"2024-09-18T04:57:08.326645+0000","last_active":"2024-09-18T04:57:19.020124+0000","last_peered":"2024-09-18T04:57:19.020124+0000","last_clean":"2024-09-18T04:57:19.020124+0000","last_became_active":"2024-09-18T04:57:08.325917+0000","last_became_peered":"2024-09-18T04:57:08.325917+0000","last_unstale":"2024-09-18T04:57:19.020124+0000","last_undegraded":"2024-09-18T04:57:19.020124+0000","last_fullsized":"2024-09-18T04:57:19.020124+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:03:57.377160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012370669999999999,"stat_sum":{"num_bytes":39941352,"num_objects":47,"num_object_clones":13,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":8,"num_read":2922,"num_read_kb":23260,"num_write":265,"num_write_kb":11203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":8557650,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"72'281","reported_seq":2790,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.161602+0000","last_change":"2024-09-18T04:56:23.582074+0000","last_active":"2024-09-18T04:57:19.161602+0000","last_peered":"2024-09-18T04:57:19.161602+0000","last_clean":"2024-09-18T04:57:19.161602+0000","last_became_active":"2024-09-18T04:56:21.127959+0000","last_became_peered":"2024-09-18T04:56:21.127959+0000","last_unstale":"2024-09-18T04:57:19.161602+0000","last_undegraded":"2024-09-18T04:57:19.161602+0000","last_fullsized":"2024-09-18T04:57:19.161602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:17:32.019845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053040699999999999,"stat_sum":{"num_bytes":45914219,"num_objects":45,"num_object_clones":8,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":9,"num_read":2807,"num_read_kb":14765,"num_write":358,"num_write_kb":12257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"72'199","reported_seq":2965,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.052385+0000","last_change":"2024-09-18T04:56:23.130832+0000","last_active":"2024-09-18T04:57:19.052385+0000","last_peered":"2024-09-18T04:57:19.052385+0000","last_clean":"2024-09-18T04:57:19.052385+0000","last_became_active":"2024-09-18T04:56:21.130316+0000","last_became_peered":"2024-09-18T04:56:21.130316+0000","last_unstale":"2024-09-18T04:57:19.052385+0000","last_undegraded":"2024-09-18T04:57:19.052385+0000","last_fullsized":"2024-09-18T04:57:19.052385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:12:05.411040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001612023,"stat_sum":{"num_bytes":33035339,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":5,"num_read":3381,"num_read_kb":10356,"num_write":238,"num_write_kb":5321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"72'542","reported_seq":3300,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.036793+0000","last_change":"2024-09-18T04:56:23.136846+0000","last_active":"2024-09-18T04:57:19.036793+0000","last_peered":"2024-09-18T04:57:19.036793+0000","last_clean":"2024-09-18T04:57:19.036793+0000","last_became_active":"2024-09-18T04:56:21.131133+0000","last_became_peered":"2024-09-18T04:56:21.131133+0000","last_unstale":"2024-09-18T04:57:19.036793+0000","last_undegraded":"2024-09-18T04:57:19.036793+0000","last_fullsized":"2024-09-18T04:57:19.036793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:01:57.036904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025847199999999999,"stat_sum":{"num_bytes":63625902,"num_objects":47,"num_object_clones":13,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":13,"num_read":4640,"num_read_kb":22037,"num_write":852,"num_write_kb":10939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"72'376","reported_seq":2877,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.115457+0000","last_change":"2024-09-18T04:56:23.142657+0000","last_active":"2024-09-18T04:57:19.115457+0000","last_peered":"2024-09-18T04:57:19.115457+0000","last_clean":"2024-09-18T04:57:19.115457+0000","last_became_active":"2024-09-18T04:56:21.131979+0000","last_became_peered":"2024-09-18T04:56:21.131979+0000","last_unstale":"2024-09-18T04:57:19.115457+0000","last_undegraded":"2024-09-18T04:57:19.115457+0000","last_fullsized":"2024-09-18T04:57:19.115457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:16:43.303511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029355000000000001,"stat_sum":{"num_bytes":93323204,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":9,"num_read":3972,"num_read_kb":21850,"num_write":380,"num_write_kb":6078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"72'295","reported_seq":1936,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.123815+0000","last_change":"2024-09-18T04:56:23.132876+0000","last_active":"2024-09-18T04:57:19.123815+0000","last_peered":"2024-09-18T04:57:19.123815+0000","last_clean":"2024-09-18T04:57:19.123815+0000","last_became_active":"2024-09-18T04:56:21.134979+0000","last_became_peered":"2024-09-18T04:56:21.134979+0000","last_unstale":"2024-09-18T04:57:19.123815+0000","last_undegraded":"2024-09-18T04:57:19.123815+0000","last_fullsized":"2024-09-18T04:57:19.123815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:36:24.881782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038004100000000002,"stat_sum":{"num_bytes":25704269,"num_objects":33,"num_object_clones":5,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":5,"num_read":2776,"num_read_kb":26480,"num_write":401,"num_write_kb":11068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":71,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-18T04:57:17.411081+0000","last_change":"2024-09-18T04:56:12.390222+0000","last_active":"2024-09-18T04:57:17.411081+0000","last_peered":"2024-09-18T04:57:17.411081+0000","last_clean":"2024-09-18T04:57:17.411081+0000","last_became_active":"2024-09-18T04:56:10.401272+0000","last_became_peered":"2024-09-18T04:56:10.401272+0000","last_unstale":"2024-09-18T04:57:17.411081+0000","last_undegraded":"2024-09-18T04:57:17.411081+0000","last_fullsized":"2024-09-18T04:57:17.411081+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:33:50.692096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063436799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"72'290","reported_seq":5867,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.115215+0000","last_change":"2024-09-18T04:56:23.137022+0000","last_active":"2024-09-18T04:57:19.115215+0000","last_peered":"2024-09-18T04:57:19.115215+0000","last_clean":"2024-09-18T04:57:19.115215+0000","last_became_active":"2024-09-18T04:56:21.129964+0000","last_became_peered":"2024-09-18T04:56:21.129964+0000","last_unstale":"2024-09-18T04:57:19.115215+0000","last_undegraded":"2024-09-18T04:57:19.115215+0000","last_fullsized":"2024-09-18T04:57:19.115215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:29:32.843883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043204099999999998,"stat_sum":{"num_bytes":56178906,"num_objects":41,"num_object_clones":7,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":7,"num_read":6281,"num_read_kb":20527,"num_write":409,"num_write_kb":14396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-18T04:57:17.411041+0000","last_change":"2024-09-18T04:56:12.389971+0000","last_active":"2024-09-18T04:57:17.411041+0000","last_peered":"2024-09-18T04:57:17.411041+0000","last_clean":"2024-09-18T04:57:17.411041+0000","last_became_active":"2024-09-18T04:56:10.412595+0000","last_became_peered":"2024-09-18T04:56:10.412595+0000","last_unstale":"2024-09-18T04:57:17.411041+0000","last_undegraded":"2024-09-18T04:57:17.411041+0000","last_fullsized":"2024-09-18T04:57:17.411041+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:45.638420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038576799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"72'547","reported_seq":5692,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.115425+0000","last_change":"2024-09-18T04:57:08.322165+0000","last_active":"2024-09-18T04:57:19.115425+0000","last_peered":"2024-09-18T04:57:19.115425+0000","last_clean":"2024-09-18T04:57:19.115425+0000","last_became_active":"2024-09-18T04:57:08.321093+0000","last_became_peered":"2024-09-18T04:57:08.321093+0000","last_unstale":"2024-09-18T04:57:19.115425+0000","last_undegraded":"2024-09-18T04:57:19.115425+0000","last_fullsized":"2024-09-18T04:57:19.115425+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:36:01.358492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072197,"stat_sum":{"num_bytes":54552768,"num_objects":49,"num_object_clones":14,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":10,"num_read":6896,"num_read_kb":22852,"num_write":822,"num_write_kb":9476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":5575777,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-18T04:57:17.372102+0000","last_change":"2024-09-18T04:56:12.412298+0000","last_active":"2024-09-18T04:57:17.372102+0000","last_peered":"2024-09-18T04:57:17.372102+0000","last_clean":"2024-09-18T04:57:17.372102+0000","last_became_active":"2024-09-18T04:56:10.399718+0000","last_became_peered":"2024-09-18T04:56:10.399718+0000","last_unstale":"2024-09-18T04:57:17.372102+0000","last_undegraded":"2024-09-18T04:57:17.372102+0000","last_fullsized":"2024-09-18T04:57:17.372102+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:41:31.956685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024059099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"72'529","reported_seq":2881,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:18.585012+0000","last_change":"2024-09-18T04:56:23.582182+0000","last_active":"2024-09-18T04:57:18.585012+0000","last_peered":"2024-09-18T04:57:18.585012+0000","last_clean":"2024-09-18T04:57:18.585012+0000","last_became_active":"2024-09-18T04:56:21.125785+0000","last_became_peered":"2024-09-18T04:56:21.125785+0000","last_unstale":"2024-09-18T04:57:18.585012+0000","last_undegraded":"2024-09-18T04:57:18.585012+0000","last_fullsized":"2024-09-18T04:57:18.585012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:35:40.684580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048263400000000001,"stat_sum":{"num_bytes":30875762,"num_objects":38,"num_object_clones":6,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":10,"num_read":4215,"num_read_kb":14675,"num_write":724,"num_write_kb":2366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-18T04:57:17.411022+0000","last_change":"2024-09-18T04:56:46.254241+0000","last_active":"2024-09-18T04:57:17.411022+0000","last_peered":"2024-09-18T04:57:17.411022+0000","last_clean":"2024-09-18T04:57:17.411022+0000","last_became_active":"2024-09-18T04:56:46.253894+0000","last_became_peered":"2024-09-18T04:56:46.253894+0000","last_unstale":"2024-09-18T04:57:17.411022+0000","last_undegraded":"2024-09-18T04:57:17.411022+0000","last_fullsized":"2024-09-18T04:57:17.411022+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:58:42.098099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045144100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"72'209","reported_seq":4047,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.116103+0000","last_change":"2024-09-18T04:57:08.325361+0000","last_active":"2024-09-18T04:57:19.116103+0000","last_peered":"2024-09-18T04:57:19.116103+0000","last_clean":"2024-09-18T04:57:19.116103+0000","last_became_active":"2024-09-18T04:57:08.324375+0000","last_became_peered":"2024-09-18T04:57:08.324375+0000","last_unstale":"2024-09-18T04:57:19.116103+0000","last_undegraded":"2024-09-18T04:57:19.116103+0000","last_fullsized":"2024-09-18T04:57:19.116103+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:54.994870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039168900000000002,"stat_sum":{"num_bytes":56794934,"num_objects":35,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":1,"num_read":3780,"num_read_kb":28001,"num_write":250,"num_write_kb":34486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":9876200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"71'194","reported_seq":7678,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:18.586717+0000","last_change":"2024-09-18T04:56:23.130445+0000","last_active":"2024-09-18T04:57:18.586717+0000","last_peered":"2024-09-18T04:57:18.586717+0000","last_clean":"2024-09-18T04:57:18.586717+0000","last_became_active":"2024-09-18T04:56:21.130514+0000","last_became_peered":"2024-09-18T04:56:21.130514+0000","last_unstale":"2024-09-18T04:57:18.586717+0000","last_undegraded":"2024-09-18T04:57:18.586717+0000","last_fullsized":"2024-09-18T04:57:18.586717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:06:37.491009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001360497,"stat_sum":{"num_bytes":71656766,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":5,"num_read":7511,"num_read_kb":21981,"num_write":228,"num_write_kb":11603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":236,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-18T04:57:17.364086+0000","last_change":"2024-09-18T04:57:08.325066+0000","last_active":"2024-09-18T04:57:17.364086+0000","last_peered":"2024-09-18T04:57:17.364086+0000","last_clean":"2024-09-18T04:57:17.364086+0000","last_became_active":"2024-09-18T04:57:08.324619+0000","last_became_peered":"2024-09-18T04:57:08.324619+0000","last_unstale":"2024-09-18T04:57:17.364086+0000","last_undegraded":"2024-09-18T04:57:17.364086+0000","last_fullsized":"2024-09-18T04:57:17.364086+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_clean_scrub_stamp":"2024-09-18T04:56:10.373785+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:18:36.104129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":32,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-18T04:57:17.411023+0000","last_change":"2024-09-18T04:57:05.305809+0000","last_active":"2024-09-18T04:57:17.411023+0000","last_peered":"2024-09-18T04:57:17.411023+0000","last_clean":"2024-09-18T04:57:17.411023+0000","last_became_active":"2024-09-18T04:57:05.303475+0000","last_became_peered":"2024-09-18T04:57:05.303475+0000","last_unstale":"2024-09-18T04:57:17.411023+0000","last_undegraded":"2024-09-18T04:57:17.411023+0000","last_fullsized":"2024-09-18T04:57:17.411023+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:19:38.330085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"70'296","reported_seq":8959,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:18.585793+0000","last_change":"2024-09-18T04:57:08.323505+0000","last_active":"2024-09-18T04:57:18.585793+0000","last_peered":"2024-09-18T04:57:18.585793+0000","last_clean":"2024-09-18T04:57:18.585793+0000","last_became_active":"2024-09-18T04:57:08.322649+0000","last_became_peered":"2024-09-18T04:57:08.322649+0000","last_unstale":"2024-09-18T04:57:18.585793+0000","last_undegraded":"2024-09-18T04:57:18.585793+0000","last_fullsized":"2024-09-18T04:57:18.585793+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:36:00.900988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010536390000000001,"stat_sum":{"num_bytes":53260436,"num_objects":33,"num_object_clones":7,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":7,"num_read":9611,"num_read_kb":19667,"num_write":324,"num_write_kb":4733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":1172339,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":73,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-18T04:57:17.364117+0000","last_change":"2024-09-18T04:56:12.398631+0000","last_active":"2024-09-18T04:57:17.364117+0000","last_peered":"2024-09-18T04:57:17.364117+0000","last_clean":"2024-09-18T04:57:17.364117+0000","last_became_active":"2024-09-18T04:56:10.412614+0000","last_became_peered":"2024-09-18T04:56:10.412614+0000","last_unstale":"2024-09-18T04:57:17.364117+0000","last_undegraded":"2024-09-18T04:57:17.364117+0000","last_fullsized":"2024-09-18T04:57:17.364117+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:44:13.401320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075126299999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"72'236","reported_seq":799,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.227013+0000","last_change":"2024-09-18T04:56:23.133345+0000","last_active":"2024-09-18T04:57:19.227013+0000","last_peered":"2024-09-18T04:57:19.227013+0000","last_clean":"2024-09-18T04:57:19.227013+0000","last_became_active":"2024-09-18T04:56:21.128793+0000","last_became_peered":"2024-09-18T04:56:21.128793+0000","last_unstale":"2024-09-18T04:57:19.227013+0000","last_undegraded":"2024-09-18T04:57:19.227013+0000","last_fullsized":"2024-09-18T04:57:19.227013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:10:36.658665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067161900000000004,"stat_sum":{"num_bytes":61146260,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":7,"num_read":583,"num_read_kb":15295,"num_write":247,"num_write_kb":10842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-18T04:57:17.359543+0000","last_change":"2024-09-18T04:56:12.390044+0000","last_active":"2024-09-18T04:57:17.359543+0000","last_peered":"2024-09-18T04:57:17.359543+0000","last_clean":"2024-09-18T04:57:17.359543+0000","last_became_active":"2024-09-18T04:56:10.413042+0000","last_became_peered":"2024-09-18T04:56:10.413042+0000","last_unstale":"2024-09-18T04:57:17.359543+0000","last_undegraded":"2024-09-18T04:57:17.359543+0000","last_fullsized":"2024-09-18T04:57:17.359543+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:46:11.074243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"72'308","reported_seq":3708,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.238818+0000","last_change":"2024-09-18T04:56:23.581937+0000","last_active":"2024-09-18T04:57:19.238818+0000","last_peered":"2024-09-18T04:57:19.238818+0000","last_clean":"2024-09-18T04:57:19.238818+0000","last_became_active":"2024-09-18T04:56:21.130465+0000","last_became_peered":"2024-09-18T04:56:21.130465+0000","last_unstale":"2024-09-18T04:57:19.238818+0000","last_undegraded":"2024-09-18T04:57:19.238818+0000","last_fullsized":"2024-09-18T04:57:19.238818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:38:38.231233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037357100000000001,"stat_sum":{"num_bytes":40107239,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":4,"num_read":4956,"num_read_kb":12702,"num_write":304,"num_write_kb":3780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-18T04:57:17.410982+0000","last_change":"2024-09-18T04:56:59.285326+0000","last_active":"2024-09-18T04:57:17.410982+0000","last_peered":"2024-09-18T04:57:17.410982+0000","last_clean":"2024-09-18T04:57:17.410982+0000","last_became_active":"2024-09-18T04:56:59.284974+0000","last_became_peered":"2024-09-18T04:56:59.284974+0000","last_unstale":"2024-09-18T04:57:17.410982+0000","last_undegraded":"2024-09-18T04:57:17.410982+0000","last_fullsized":"2024-09-18T04:57:17.410982+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:33:29.194155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"72'273","reported_seq":1963,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:18.833410+0000","last_change":"2024-09-18T04:57:08.326523+0000","last_active":"2024-09-18T04:57:18.833410+0000","last_peered":"2024-09-18T04:57:18.833410+0000","last_clean":"2024-09-18T04:57:18.833410+0000","last_became_active":"2024-09-18T04:57:08.325666+0000","last_became_peered":"2024-09-18T04:57:08.325666+0000","last_unstale":"2024-09-18T04:57:18.833410+0000","last_undegraded":"2024-09-18T04:57:18.833410+0000","last_fullsized":"2024-09-18T04:57:18.833410+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:02:48.526668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053349799999999998,"stat_sum":{"num_bytes":27659717,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":11,"num_read":2016,"num_read_kb":6995,"num_write":297,"num_write_kb":5631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":1439423,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"72'328","reported_seq":9849,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:18.833319+0000","last_change":"2024-09-18T04:56:23.133113+0000","last_active":"2024-09-18T04:57:18.833319+0000","last_peered":"2024-09-18T04:57:18.833319+0000","last_clean":"2024-09-18T04:57:18.833319+0000","last_became_active":"2024-09-18T04:56:21.128135+0000","last_became_peered":"2024-09-18T04:56:21.128135+0000","last_unstale":"2024-09-18T04:57:18.833319+0000","last_undegraded":"2024-09-18T04:57:18.833319+0000","last_fullsized":"2024-09-18T04:57:18.833319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:46:04.362749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000513702,"stat_sum":{"num_bytes":49056308,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":7,"num_read":10732,"num_read_kb":23707,"num_write":331,"num_write_kb":13735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"72'226","reported_seq":1071,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:18.585545+0000","last_change":"2024-09-18T04:56:23.641225+0000","last_active":"2024-09-18T04:57:18.585545+0000","last_peered":"2024-09-18T04:57:18.585545+0000","last_clean":"2024-09-18T04:57:18.585545+0000","last_became_active":"2024-09-18T04:56:21.127832+0000","last_became_peered":"2024-09-18T04:56:21.127832+0000","last_unstale":"2024-09-18T04:57:18.585545+0000","last_undegraded":"2024-09-18T04:57:18.585545+0000","last_fullsized":"2024-09-18T04:57:18.585545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:23:15.821734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029947799999999999,"stat_sum":{"num_bytes":37308659,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":5,"num_read":967,"num_read_kb":7012,"num_write":247,"num_write_kb":5343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"72'940","reported_seq":3515,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.050037+0000","last_change":"2024-09-18T04:56:23.582528+0000","last_active":"2024-09-18T04:57:19.050037+0000","last_peered":"2024-09-18T04:57:19.050037+0000","last_clean":"2024-09-18T04:57:19.050037+0000","last_became_active":"2024-09-18T04:56:21.127444+0000","last_became_peered":"2024-09-18T04:56:21.127444+0000","last_unstale":"2024-09-18T04:57:19.050037+0000","last_undegraded":"2024-09-18T04:57:19.050037+0000","last_fullsized":"2024-09-18T04:57:19.050037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":940,"log_dups_size":0,"ondisk_log_size":940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:15:41.251274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060817200000000003,"stat_sum":{"num_bytes":15175618,"num_objects":33,"num_object_clones":7,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":6,"num_read":6696,"num_read_kb":11993,"num_write":1496,"num_write_kb":5584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"72'483","reported_seq":2442,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:18.900774+0000","last_change":"2024-09-18T04:56:23.136791+0000","last_active":"2024-09-18T04:57:18.900774+0000","last_peered":"2024-09-18T04:57:18.900774+0000","last_clean":"2024-09-18T04:57:18.900774+0000","last_became_active":"2024-09-18T04:56:21.131676+0000","last_became_peered":"2024-09-18T04:56:21.131676+0000","last_unstale":"2024-09-18T04:57:18.900774+0000","last_undegraded":"2024-09-18T04:57:18.900774+0000","last_fullsized":"2024-09-18T04:57:18.900774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:34:36.563239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000302343,"stat_sum":{"num_bytes":29718572,"num_objects":35,"num_object_clones":10,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":8,"num_read":4850,"num_read_kb":15987,"num_write":519,"num_write_kb":5157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"72'303","reported_seq":5480,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.081747+0000","last_change":"2024-09-18T04:56:23.143279+0000","last_active":"2024-09-18T04:57:19.081747+0000","last_peered":"2024-09-18T04:57:19.081747+0000","last_clean":"2024-09-18T04:57:19.081747+0000","last_became_active":"2024-09-18T04:56:21.133686+0000","last_became_peered":"2024-09-18T04:56:21.133686+0000","last_unstale":"2024-09-18T04:57:19.081747+0000","last_undegraded":"2024-09-18T04:57:19.081747+0000","last_fullsized":"2024-09-18T04:57:19.081747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:50:51.011660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087309599999999996,"stat_sum":{"num_bytes":31434762,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":7,"num_read":7049,"num_read_kb":9923,"num_write":302,"num_write_kb":5218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"71'774","reported_seq":2034,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:18.586491+0000","last_change":"2024-09-18T04:56:23.143532+0000","last_active":"2024-09-18T04:57:18.586491+0000","last_peered":"2024-09-18T04:57:18.586491+0000","last_clean":"2024-09-18T04:57:18.586491+0000","last_became_active":"2024-09-18T04:56:21.132364+0000","last_became_peered":"2024-09-18T04:56:21.132364+0000","last_unstale":"2024-09-18T04:57:18.586491+0000","last_undegraded":"2024-09-18T04:57:18.586491+0000","last_fullsized":"2024-09-18T04:57:18.586491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:25:44.430438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010190329999999999,"stat_sum":{"num_bytes":24886814,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":5,"num_read":4739,"num_read_kb":11608,"num_write":1346,"num_write_kb":3512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"72'322","reported_seq":3352,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:18.784471+0000","last_change":"2024-09-18T04:56:23.143500+0000","last_active":"2024-09-18T04:57:18.784471+0000","last_peered":"2024-09-18T04:57:18.784471+0000","last_clean":"2024-09-18T04:57:18.784471+0000","last_became_active":"2024-09-18T04:56:21.130105+0000","last_became_peered":"2024-09-18T04:56:21.130105+0000","last_unstale":"2024-09-18T04:57:18.784471+0000","last_undegraded":"2024-09-18T04:57:18.784471+0000","last_fullsized":"2024-09-18T04:57:18.784471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:31:55.392378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010487090000000001,"stat_sum":{"num_bytes":43160339,"num_objects":38,"num_object_clones":11,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":9,"num_read":3873,"num_read_kb":25018,"num_write":380,"num_write_kb":6083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"72'546","reported_seq":1079,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.115550+0000","last_change":"2024-09-18T04:56:23.641201+0000","last_active":"2024-09-18T04:57:19.115550+0000","last_peered":"2024-09-18T04:57:19.115550+0000","last_clean":"2024-09-18T04:57:19.115550+0000","last_became_active":"2024-09-18T04:56:21.131202+0000","last_became_peered":"2024-09-18T04:56:21.131202+0000","last_unstale":"2024-09-18T04:57:19.115550+0000","last_undegraded":"2024-09-18T04:57:19.115550+0000","last_fullsized":"2024-09-18T04:57:19.115550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:17:35.438723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000304567,"stat_sum":{"num_bytes":55492530,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":5,"num_read":2492,"num_read_kb":6942,"num_write":760,"num_write_kb":7258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"72'400","reported_seq":5134,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:18.946656+0000","last_change":"2024-09-18T04:56:23.142612+0000","last_active":"2024-09-18T04:57:18.946656+0000","last_peered":"2024-09-18T04:57:18.946656+0000","last_clean":"2024-09-18T04:57:18.946656+0000","last_became_active":"2024-09-18T04:56:21.130594+0000","last_became_peered":"2024-09-18T04:56:21.130594+0000","last_unstale":"2024-09-18T04:57:18.946656+0000","last_undegraded":"2024-09-18T04:57:18.946656+0000","last_fullsized":"2024-09-18T04:57:18.946656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:46:34.324509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030763399999999999,"stat_sum":{"num_bytes":56997038,"num_objects":31,"num_object_clones":4,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":2,"num_read":6786,"num_read_kb":13534,"num_write":446,"num_write_kb":16805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"72'545","reported_seq":3878,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.115896+0000","last_change":"2024-09-18T04:56:23.129633+0000","last_active":"2024-09-18T04:57:19.115896+0000","last_peered":"2024-09-18T04:57:19.115896+0000","last_clean":"2024-09-18T04:57:19.115896+0000","last_became_active":"2024-09-18T04:56:21.129597+0000","last_became_peered":"2024-09-18T04:56:21.129597+0000","last_unstale":"2024-09-18T04:57:19.115896+0000","last_undegraded":"2024-09-18T04:57:19.115896+0000","last_fullsized":"2024-09-18T04:57:19.115896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:50.665769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059848499999999995,"stat_sum":{"num_bytes":43553327,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":10,"num_read":5680,"num_read_kb":30171,"num_write":840,"num_write_kb":11931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"72'363","reported_seq":921,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.120700+0000","last_change":"2024-09-18T04:56:55.263009+0000","last_active":"2024-09-18T04:57:19.120700+0000","last_peered":"2024-09-18T04:57:19.120700+0000","last_clean":"2024-09-18T04:57:19.120700+0000","last_became_active":"2024-09-18T04:56:55.261829+0000","last_became_peered":"2024-09-18T04:56:55.261829+0000","last_unstale":"2024-09-18T04:57:19.120700+0000","last_undegraded":"2024-09-18T04:57:19.120700+0000","last_fullsized":"2024-09-18T04:57:19.120700+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:01:08.639453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085320600000000002,"stat_sum":{"num_bytes":39720490,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":5,"num_read":2006,"num_read_kb":7933,"num_write":416,"num_write_kb":7952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":4916923,"num_keys_recovered":29,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1393698446,"num_objects":1152,"num_object_clones":252,"num_object_copies":2304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1152,"num_whiteouts":216,"num_read":147166,"num_read_kb":524264,"num_write":16285,"num_write_kb":306396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":416,"num_bytes_recovered":34229789,"num_keys_recovered":65,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":58839040,"data_stored":401369468,"data_compressed":16665885,"data_compressed_allocated":36253696,"data_compressed_original":382851700,"omap_allocated":0,"internal_metadata":0},"log_size":12349,"ondisk_log_size":12349,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":1771104,"data_compressed":27132,"data_compressed_allocated":110592,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54144,"kb_used_data":9076,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94317696,"statfs":{"total":96636764160,"available":96581320704,"internally_reserved":0,"allocated":9293824,"data_stored":53616456,"data_compressed":2095799,"data_compressed_allocated":4845568,"data_compressed_original":50508059,"omap_allocated":1589,"internal_metadata":46135755},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":2.3420000000000001,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.60899999999999999}]},{"osd":1,"last update":"Wed Sep 18 04:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.70099999999999996}]},{"osd":2,"last update":"Wed Sep 18 04:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.443}]},{"osd":3,"last update":"Wed Sep 18 04:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.65200000000000002}]},{"osd":4,"last update":"Wed Sep 18 04:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.629}]},{"osd":5,"last update":"Wed Sep 18 04:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":3.145,"5min":3.145,"15min":3.145},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.40400000000000003}]},{"osd":6,"last update":"Wed Sep 18 04:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.59499999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38468,"kb_used_data":3260,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333372,"statfs":{"total":96636764160,"available":96597372928,"internally_reserved":0,"allocated":3338240,"data_stored":16706896,"data_compressed":417698,"data_compressed_allocated":1241088,"data_compressed_original":15237923,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":6.1319999999999997,"5min":6.1319999999999997,"15min":6.1319999999999997},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":6.1429999999999998,"5min":6.1429999999999998,"15min":6.1429999999999998},"last":0.39000000000000001}]},{"osd":1,"last update":"Wed Sep 18 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":6.0410000000000004,"5min":6.0410000000000004,"15min":6.0410000000000004},"last":0.375},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":5.5380000000000003,"5min":5.5380000000000003,"15min":5.5380000000000003},"last":0.33000000000000002}]},{"osd":2,"last update":"Wed Sep 18 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.317},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.41099999999999998}]},{"osd":3,"last update":"Wed Sep 18 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.40100000000000002}]},{"osd":4,"last update":"Wed Sep 18 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.35399999999999998}]},{"osd":5,"last update":"Wed Sep 18 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.30399999999999999}]},{"osd":7,"last update":"Wed Sep 18 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.36399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48248,"kb_used_data":8940,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94323592,"statfs":{"total":96636764160,"available":96587358208,"internally_reserved":0,"allocated":9154560,"data_stored":52989378,"data_compressed":1945353,"data_compressed_allocated":4628480,"data_compressed_original":49505640,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":3.0179999999999998,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.55400000000000005}]},{"osd":1,"last update":"Wed Sep 18 04:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":3.1349999999999998,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.435}]},{"osd":2,"last update":"Wed Sep 18 04:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":3.1070000000000002,"5min":3.1070000000000002,"15min":3.1070000000000002},"last":0.45700000000000002}]},{"osd":3,"last update":"Wed Sep 18 04:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.63}]},{"osd":4,"last update":"Wed Sep 18 04:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":3.379,"5min":3.379,"15min":3.379},"last":0.36199999999999999}]},{"osd":6,"last update":"Wed Sep 18 04:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":3.1669999999999998,"5min":3.1669999999999998,"15min":3.1669999999999998},"last":0.41299999999999998}]},{"osd":7,"last update":"Wed Sep 18 04:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.56999999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41116,"kb_used_data":14088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94330724,"statfs":{"total":96636764160,"available":96594661376,"internally_reserved":0,"allocated":14426112,"data_stored":100417881,"data_compressed":4529541,"data_compressed_allocated":9211904,"data_compressed_original":96448344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.33900000000000002}]},{"osd":1,"last update":"Wed Sep 18 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.52700000000000002}]},{"osd":2,"last update":"Wed Sep 18 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.49399999999999999}]},{"osd":3,"last update":"Wed Sep 18 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.50800000000000001}]},{"osd":5,"last update":"Wed Sep 18 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.40999999999999998}]},{"osd":6,"last update":"Wed Sep 18 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.55400000000000005}]},{"osd":7,"last update":"Wed Sep 18 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.53800000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57220,"kb_used_data":8056,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94314620,"statfs":{"total":96636764160,"available":96578170880,"internally_reserved":0,"allocated":8249344,"data_stored":47192331,"data_compressed":1876369,"data_compressed_allocated":4186112,"data_compressed_original":44102108,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.49299999999999999}]},{"osd":1,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.35999999999999999}]},{"osd":2,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.46500000000000002}]},{"osd":4,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.63400000000000001}]},{"osd":5,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.40400000000000003}]},{"osd":6,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.72399999999999998}]},{"osd":7,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.753}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39164,"kb_used_data":3944,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94332676,"statfs":{"total":96636764160,"available":96596660224,"internally_reserved":0,"allocated":4038656,"data_stored":20932552,"data_compressed":972100,"data_compressed_allocated":1998848,"data_compressed_original":19561359,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.247}]},{"osd":1,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.28399999999999997}]},{"osd":3,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.32500000000000001}]},{"osd":4,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.36299999999999999}]},{"osd":5,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.312},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.377}]},{"osd":6,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.35199999999999998}]},{"osd":7,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.442},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.40000000000000002}]}]},{"osd":1,"up_from":0,"seq":55834574866,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55200,"kb_used_data":10132,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94316640,"statfs":{"total":96636764160,"available":96580239360,"internally_reserved":0,"allocated":10375168,"data_stored":72152169,"data_compressed":3335783,"data_compressed_allocated":6639616,"data_compressed_original":69454872,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.247},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.34399999999999997}]},{"osd":2,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.45600000000000002}]},{"osd":3,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.33100000000000002}]},{"osd":4,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":3.4710000000000001,"5min":3.4710000000000001,"15min":3.4710000000000001},"last":0.314},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.38200000000000001}]},{"osd":5,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.5619999999999998,"5min":3.5619999999999998,"15min":3.5619999999999998},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.434}]},{"osd":6,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":3.536,"5min":3.536,"15min":3.536},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.48899999999999999}]},{"osd":7,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":3.5089999999999999,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.44600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9734,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9768960,"data_stored":71422056,"data_compressed":3326116,"data_compressed_allocated":6606848,"data_compressed_original":68873240,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4075520,"data_stored":32778014,"data_compressed":1099505,"data_compressed_allocated":2736128,"data_compressed_original":31618951,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3215360,"data_stored":20613148,"data_compressed":972410,"data_compressed_allocated":2007040,"data_compressed_original":19622799,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7618560,"data_stored":47032445,"data_compressed":1876580,"data_compressed_allocated":4190208,"data_compressed_original":44147164,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":13975552,"data_stored":102425055,"data_compressed":4582318,"data_compressed_allocated":9392128,"data_compressed_original":98660184,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8736768,"data_stored":54813096,"data_compressed":2107703,"data_compressed_allocated":4902912,"data_compressed_original":51598750,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2523136,"data_stored":16342667,"data_compressed":417765,"data_compressed_allocated":1245184,"data_compressed_original":15246115,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8925184,"data_stored":55942987,"data_compressed":2283488,"data_compressed_allocated":5173248,"data_compressed_original":53084497,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T04:57:21.810 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T04:57:21.823 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:21.823 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:21.832 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:57:21.899 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:21.897+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:21.980 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:21.980 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:21.990 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:57:22.000 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:21.997+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:22.097 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:22.097 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:22.101 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:22.098+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:22.106 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:57:22.114 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:57:22.115 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-18T04:57:22.126 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-09-18T04:57:21.379145+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1394288833,"num_objects":1156,"num_object_clones":252,"num_object_copies":2312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1156,"num_whiteouts":216,"num_read":147282,"num_read_kb":524364,"num_write":16446,"num_write_kb":308814,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":419,"num_bytes_recovered":34820157,"num_keys_recovered":65,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12469,"ondisk_log_size":12469,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":333560,"kb_used_data":57496,"kb_used_omap":10,"kb_used_meta":275957,"kb_avail":660269320,"statfs":{"total":676457349120,"available":676115783680,"internally_reserved":0,"allocated":58875904,"data_stored":364007663,"data_compressed":15172643,"data_compressed_allocated":32751616,"data_compressed_original":344818305,"omap_allocated":11149,"internal_metadata":282580083},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":88843622,"num_objects":53,"num_object_clones":0,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":0,"num_read":8085,"num_read_kb":56825,"num_write":2075,"num_write_kb":29934,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.024921"},"pg_stats":[{"pgid":"3.1f","version":"72'445","reported_seq":3242,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:18.904055+0000","last_change":"2024-09-18T04:56:23.130295+0000","last_active":"2024-09-18T04:57:18.904055+0000","last_peered":"2024-09-18T04:57:18.904055+0000","last_clean":"2024-09-18T04:57:18.904055+0000","last_became_active":"2024-09-18T04:56:21.132342+0000","last_became_peered":"2024-09-18T04:56:21.132342+0000","last_unstale":"2024-09-18T04:57:18.904055+0000","last_undegraded":"2024-09-18T04:57:18.904055+0000","last_fullsized":"2024-09-18T04:57:18.904055+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:43:47.128460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011070279999999999,"stat_sum":{"num_bytes":49363979,"num_objects":39,"num_object_clones":6,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":4,"num_read":4656,"num_read_kb":16410,"num_write":555,"num_write_kb":8423,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"72'565","reported_seq":962,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.078717+0000","last_change":"2024-09-18T04:56:23.582447+0000","last_active":"2024-09-18T04:57:19.078717+0000","last_peered":"2024-09-18T04:57:19.078717+0000","last_clean":"2024-09-18T04:57:19.078717+0000","last_became_active":"2024-09-18T04:56:21.126431+0000","last_became_peered":"2024-09-18T04:56:21.126431+0000","last_unstale":"2024-09-18T04:57:19.078717+0000","last_undegraded":"2024-09-18T04:57:19.078717+0000","last_fullsized":"2024-09-18T04:57:19.078717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:59.671751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062644600000000001,"stat_sum":{"num_bytes":55973624,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":6,"num_read":2228,"num_read_kb":12318,"num_write":846,"num_write_kb":6341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"70'252","reported_seq":8375,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-18T04:57:17.359739+0000","last_change":"2024-09-18T04:56:23.137064+0000","last_active":"2024-09-18T04:57:17.359739+0000","last_peered":"2024-09-18T04:57:17.359739+0000","last_clean":"2024-09-18T04:57:17.359739+0000","last_became_active":"2024-09-18T04:56:21.131613+0000","last_became_peered":"2024-09-18T04:56:21.131613+0000","last_unstale":"2024-09-18T04:57:17.359739+0000","last_undegraded":"2024-09-18T04:57:17.359739+0000","last_fullsized":"2024-09-18T04:57:17.359739+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":252,"log_dups_size":0,"ondisk_log_size":252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:15:27.419409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000455248,"stat_sum":{"num_bytes":56110006,"num_objects":40,"num_object_clones":11,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":5,"num_read":8178,"num_read_kb":25621,"num_write":305,"num_write_kb":21539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"72'557","reported_seq":6022,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.233331+0000","last_change":"2024-09-18T04:56:23.137422+0000","last_active":"2024-09-18T04:57:19.233331+0000","last_peered":"2024-09-18T04:57:19.233331+0000","last_clean":"2024-09-18T04:57:19.233331+0000","last_became_active":"2024-09-18T04:56:21.129551+0000","last_became_peered":"2024-09-18T04:56:21.129551+0000","last_unstale":"2024-09-18T04:57:19.233331+0000","last_undegraded":"2024-09-18T04:57:19.233331+0000","last_fullsized":"2024-09-18T04:57:19.233331+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":557,"log_dups_size":0,"ondisk_log_size":557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:38:38.231233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081678499999999999,"stat_sum":{"num_bytes":10773715,"num_objects":24,"num_object_clones":4,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":5,"num_read":7443,"num_read_kb":7698,"num_write":821,"num_write_kb":4603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"72'200","reported_seq":554,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.157961+0000","last_change":"2024-09-18T04:56:23.130841+0000","last_active":"2024-09-18T04:57:19.157961+0000","last_peered":"2024-09-18T04:57:19.157961+0000","last_clean":"2024-09-18T04:57:19.157961+0000","last_became_active":"2024-09-18T04:56:21.129660+0000","last_became_peered":"2024-09-18T04:56:21.129660+0000","last_unstale":"2024-09-18T04:57:19.157961+0000","last_undegraded":"2024-09-18T04:57:19.157961+0000","last_fullsized":"2024-09-18T04:57:19.157961+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:42:43.007677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014550170000000001,"stat_sum":{"num_bytes":7144998,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":7,"num_read":379,"num_read_kb":6956,"num_write":221,"num_write_kb":4955,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"72'299","reported_seq":2133,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.173670+0000","last_change":"2024-09-18T04:57:08.324870+0000","last_active":"2024-09-18T04:57:19.173670+0000","last_peered":"2024-09-18T04:57:19.173670+0000","last_clean":"2024-09-18T04:57:19.173670+0000","last_became_active":"2024-09-18T04:57:08.323638+0000","last_became_peered":"2024-09-18T04:57:08.323638+0000","last_unstale":"2024-09-18T04:57:19.173670+0000","last_undegraded":"2024-09-18T04:57:19.173670+0000","last_fullsized":"2024-09-18T04:57:19.173670+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":299,"log_dups_size":0,"ondisk_log_size":299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:55:00.155452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087073300000000005,"stat_sum":{"num_bytes":34050594,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":9,"num_read":2065,"num_read_kb":9990,"num_write":355,"num_write_kb":17778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":2691477,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"72'226","reported_seq":2985,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.020124+0000","last_change":"2024-09-18T04:57:08.326645+0000","last_active":"2024-09-18T04:57:19.020124+0000","last_peered":"2024-09-18T04:57:19.020124+0000","last_clean":"2024-09-18T04:57:19.020124+0000","last_became_active":"2024-09-18T04:57:08.325917+0000","last_became_peered":"2024-09-18T04:57:08.325917+0000","last_unstale":"2024-09-18T04:57:19.020124+0000","last_undegraded":"2024-09-18T04:57:19.020124+0000","last_fullsized":"2024-09-18T04:57:19.020124+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:03:57.377160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012370669999999999,"stat_sum":{"num_bytes":39941352,"num_objects":47,"num_object_clones":13,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":8,"num_read":2922,"num_read_kb":23260,"num_write":265,"num_write_kb":11203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":8557650,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"72'281","reported_seq":2790,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.161602+0000","last_change":"2024-09-18T04:56:23.582074+0000","last_active":"2024-09-18T04:57:19.161602+0000","last_peered":"2024-09-18T04:57:19.161602+0000","last_clean":"2024-09-18T04:57:19.161602+0000","last_became_active":"2024-09-18T04:56:21.127959+0000","last_became_peered":"2024-09-18T04:56:21.127959+0000","last_unstale":"2024-09-18T04:57:19.161602+0000","last_undegraded":"2024-09-18T04:57:19.161602+0000","last_fullsized":"2024-09-18T04:57:19.161602+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":281,"log_dups_size":0,"ondisk_log_size":281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:17:32.019845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053040699999999999,"stat_sum":{"num_bytes":45914219,"num_objects":45,"num_object_clones":8,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":9,"num_read":2807,"num_read_kb":14765,"num_write":358,"num_write_kb":12257,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"72'199","reported_seq":2965,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.052385+0000","last_change":"2024-09-18T04:56:23.130832+0000","last_active":"2024-09-18T04:57:19.052385+0000","last_peered":"2024-09-18T04:57:19.052385+0000","last_clean":"2024-09-18T04:57:19.052385+0000","last_became_active":"2024-09-18T04:56:21.130316+0000","last_became_peered":"2024-09-18T04:56:21.130316+0000","last_unstale":"2024-09-18T04:57:19.052385+0000","last_undegraded":"2024-09-18T04:57:19.052385+0000","last_fullsized":"2024-09-18T04:57:19.052385+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":199,"log_dups_size":0,"ondisk_log_size":199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:12:05.411040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001612023,"stat_sum":{"num_bytes":33035339,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":5,"num_read":3381,"num_read_kb":10356,"num_write":238,"num_write_kb":5321,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"72'542","reported_seq":3300,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.036793+0000","last_change":"2024-09-18T04:56:23.136846+0000","last_active":"2024-09-18T04:57:19.036793+0000","last_peered":"2024-09-18T04:57:19.036793+0000","last_clean":"2024-09-18T04:57:19.036793+0000","last_became_active":"2024-09-18T04:56:21.131133+0000","last_became_peered":"2024-09-18T04:56:21.131133+0000","last_unstale":"2024-09-18T04:57:19.036793+0000","last_undegraded":"2024-09-18T04:57:19.036793+0000","last_fullsized":"2024-09-18T04:57:19.036793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":542,"log_dups_size":0,"ondisk_log_size":542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:01:57.036904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025847199999999999,"stat_sum":{"num_bytes":63625902,"num_objects":47,"num_object_clones":13,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":13,"num_read":4640,"num_read_kb":22037,"num_write":852,"num_write_kb":10939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"72'376","reported_seq":2877,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.115457+0000","last_change":"2024-09-18T04:56:23.142657+0000","last_active":"2024-09-18T04:57:19.115457+0000","last_peered":"2024-09-18T04:57:19.115457+0000","last_clean":"2024-09-18T04:57:19.115457+0000","last_became_active":"2024-09-18T04:56:21.131979+0000","last_became_peered":"2024-09-18T04:56:21.131979+0000","last_unstale":"2024-09-18T04:57:19.115457+0000","last_undegraded":"2024-09-18T04:57:19.115457+0000","last_fullsized":"2024-09-18T04:57:19.115457+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":376,"log_dups_size":0,"ondisk_log_size":376,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:16:43.303511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029355000000000001,"stat_sum":{"num_bytes":93323204,"num_objects":43,"num_object_clones":11,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":9,"num_read":3972,"num_read_kb":21850,"num_write":380,"num_write_kb":6078,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"72'295","reported_seq":1936,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.123815+0000","last_change":"2024-09-18T04:56:23.132876+0000","last_active":"2024-09-18T04:57:19.123815+0000","last_peered":"2024-09-18T04:57:19.123815+0000","last_clean":"2024-09-18T04:57:19.123815+0000","last_became_active":"2024-09-18T04:56:21.134979+0000","last_became_peered":"2024-09-18T04:56:21.134979+0000","last_unstale":"2024-09-18T04:57:19.123815+0000","last_undegraded":"2024-09-18T04:57:19.123815+0000","last_fullsized":"2024-09-18T04:57:19.123815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:36:24.881782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038004100000000002,"stat_sum":{"num_bytes":25704269,"num_objects":33,"num_object_clones":5,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":5,"num_read":2776,"num_read_kb":26480,"num_write":401,"num_write_kb":11068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":71,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-18T04:57:17.411081+0000","last_change":"2024-09-18T04:56:12.390222+0000","last_active":"2024-09-18T04:57:17.411081+0000","last_peered":"2024-09-18T04:57:17.411081+0000","last_clean":"2024-09-18T04:57:17.411081+0000","last_became_active":"2024-09-18T04:56:10.401272+0000","last_became_peered":"2024-09-18T04:56:10.401272+0000","last_unstale":"2024-09-18T04:57:17.411081+0000","last_undegraded":"2024-09-18T04:57:17.411081+0000","last_fullsized":"2024-09-18T04:57:17.411081+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:33:50.692096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063436799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"72'290","reported_seq":5867,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.115215+0000","last_change":"2024-09-18T04:56:23.137022+0000","last_active":"2024-09-18T04:57:19.115215+0000","last_peered":"2024-09-18T04:57:19.115215+0000","last_clean":"2024-09-18T04:57:19.115215+0000","last_became_active":"2024-09-18T04:56:21.129964+0000","last_became_peered":"2024-09-18T04:56:21.129964+0000","last_unstale":"2024-09-18T04:57:19.115215+0000","last_undegraded":"2024-09-18T04:57:19.115215+0000","last_fullsized":"2024-09-18T04:57:19.115215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:29:32.843883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043204099999999998,"stat_sum":{"num_bytes":56178906,"num_objects":41,"num_object_clones":7,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":7,"num_read":6281,"num_read_kb":20527,"num_write":409,"num_write_kb":14396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-18T04:57:17.411041+0000","last_change":"2024-09-18T04:56:12.389971+0000","last_active":"2024-09-18T04:57:17.411041+0000","last_peered":"2024-09-18T04:57:17.411041+0000","last_clean":"2024-09-18T04:57:17.411041+0000","last_became_active":"2024-09-18T04:56:10.412595+0000","last_became_peered":"2024-09-18T04:56:10.412595+0000","last_unstale":"2024-09-18T04:57:17.411041+0000","last_undegraded":"2024-09-18T04:57:17.411041+0000","last_fullsized":"2024-09-18T04:57:17.411041+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:45.638420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038576799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"72'547","reported_seq":5692,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.115425+0000","last_change":"2024-09-18T04:57:08.322165+0000","last_active":"2024-09-18T04:57:19.115425+0000","last_peered":"2024-09-18T04:57:19.115425+0000","last_clean":"2024-09-18T04:57:19.115425+0000","last_became_active":"2024-09-18T04:57:08.321093+0000","last_became_peered":"2024-09-18T04:57:08.321093+0000","last_unstale":"2024-09-18T04:57:19.115425+0000","last_undegraded":"2024-09-18T04:57:19.115425+0000","last_fullsized":"2024-09-18T04:57:19.115425+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:36:01.358492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001072197,"stat_sum":{"num_bytes":54552768,"num_objects":49,"num_object_clones":14,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":10,"num_read":6896,"num_read_kb":22852,"num_write":822,"num_write_kb":9476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":5575777,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-18T04:57:17.372102+0000","last_change":"2024-09-18T04:56:12.412298+0000","last_active":"2024-09-18T04:57:17.372102+0000","last_peered":"2024-09-18T04:57:17.372102+0000","last_clean":"2024-09-18T04:57:17.372102+0000","last_became_active":"2024-09-18T04:56:10.399718+0000","last_became_peered":"2024-09-18T04:56:10.399718+0000","last_unstale":"2024-09-18T04:57:17.372102+0000","last_undegraded":"2024-09-18T04:57:17.372102+0000","last_fullsized":"2024-09-18T04:57:17.372102+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:41:31.956685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024059099999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"72'529","reported_seq":2881,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:18.585012+0000","last_change":"2024-09-18T04:56:23.582182+0000","last_active":"2024-09-18T04:57:18.585012+0000","last_peered":"2024-09-18T04:57:18.585012+0000","last_clean":"2024-09-18T04:57:18.585012+0000","last_became_active":"2024-09-18T04:56:21.125785+0000","last_became_peered":"2024-09-18T04:56:21.125785+0000","last_unstale":"2024-09-18T04:57:18.585012+0000","last_undegraded":"2024-09-18T04:57:18.585012+0000","last_fullsized":"2024-09-18T04:57:18.585012+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":529,"log_dups_size":0,"ondisk_log_size":529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:35:40.684580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048263400000000001,"stat_sum":{"num_bytes":30875762,"num_objects":38,"num_object_clones":6,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":10,"num_read":4215,"num_read_kb":14675,"num_write":724,"num_write_kb":2366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":40,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-18T04:57:17.411022+0000","last_change":"2024-09-18T04:56:46.254241+0000","last_active":"2024-09-18T04:57:17.411022+0000","last_peered":"2024-09-18T04:57:17.411022+0000","last_clean":"2024-09-18T04:57:17.411022+0000","last_became_active":"2024-09-18T04:56:46.253894+0000","last_became_peered":"2024-09-18T04:56:46.253894+0000","last_unstale":"2024-09-18T04:57:17.411022+0000","last_undegraded":"2024-09-18T04:57:17.411022+0000","last_fullsized":"2024-09-18T04:57:17.411022+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:58:42.098099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045144100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"72'209","reported_seq":4047,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.116103+0000","last_change":"2024-09-18T04:57:08.325361+0000","last_active":"2024-09-18T04:57:19.116103+0000","last_peered":"2024-09-18T04:57:19.116103+0000","last_clean":"2024-09-18T04:57:19.116103+0000","last_became_active":"2024-09-18T04:57:08.324375+0000","last_became_peered":"2024-09-18T04:57:08.324375+0000","last_unstale":"2024-09-18T04:57:19.116103+0000","last_undegraded":"2024-09-18T04:57:19.116103+0000","last_fullsized":"2024-09-18T04:57:19.116103+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:40:54.994870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039168900000000002,"stat_sum":{"num_bytes":56794934,"num_objects":35,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":1,"num_read":3780,"num_read_kb":28001,"num_write":250,"num_write_kb":34486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":9876200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"71'194","reported_seq":7678,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:18.586717+0000","last_change":"2024-09-18T04:56:23.130445+0000","last_active":"2024-09-18T04:57:18.586717+0000","last_peered":"2024-09-18T04:57:18.586717+0000","last_clean":"2024-09-18T04:57:18.586717+0000","last_became_active":"2024-09-18T04:56:21.130514+0000","last_became_peered":"2024-09-18T04:56:21.130514+0000","last_unstale":"2024-09-18T04:57:18.586717+0000","last_undegraded":"2024-09-18T04:57:18.586717+0000","last_fullsized":"2024-09-18T04:57:18.586717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":194,"log_dups_size":0,"ondisk_log_size":194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:06:37.491009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001360497,"stat_sum":{"num_bytes":71656766,"num_objects":30,"num_object_clones":5,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":5,"num_read":7511,"num_read_kb":21981,"num_write":228,"num_write_kb":11603,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":236,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-18T04:57:17.364086+0000","last_change":"2024-09-18T04:57:08.325066+0000","last_active":"2024-09-18T04:57:17.364086+0000","last_peered":"2024-09-18T04:57:17.364086+0000","last_clean":"2024-09-18T04:57:17.364086+0000","last_became_active":"2024-09-18T04:57:08.324619+0000","last_became_peered":"2024-09-18T04:57:08.324619+0000","last_unstale":"2024-09-18T04:57:17.364086+0000","last_undegraded":"2024-09-18T04:57:17.364086+0000","last_fullsized":"2024-09-18T04:57:17.364086+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_clean_scrub_stamp":"2024-09-18T04:56:10.373785+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:18:36.104129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":32,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-18T04:57:17.411023+0000","last_change":"2024-09-18T04:57:05.305809+0000","last_active":"2024-09-18T04:57:17.411023+0000","last_peered":"2024-09-18T04:57:17.411023+0000","last_clean":"2024-09-18T04:57:17.411023+0000","last_became_active":"2024-09-18T04:57:05.303475+0000","last_became_peered":"2024-09-18T04:57:05.303475+0000","last_unstale":"2024-09-18T04:57:17.411023+0000","last_undegraded":"2024-09-18T04:57:17.411023+0000","last_fullsized":"2024-09-18T04:57:17.411023+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:19:38.330085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"70'296","reported_seq":8959,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:18.585793+0000","last_change":"2024-09-18T04:57:08.323505+0000","last_active":"2024-09-18T04:57:18.585793+0000","last_peered":"2024-09-18T04:57:18.585793+0000","last_clean":"2024-09-18T04:57:18.585793+0000","last_became_active":"2024-09-18T04:57:08.322649+0000","last_became_peered":"2024-09-18T04:57:08.322649+0000","last_unstale":"2024-09-18T04:57:18.585793+0000","last_undegraded":"2024-09-18T04:57:18.585793+0000","last_fullsized":"2024-09-18T04:57:18.585793+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":296,"log_dups_size":0,"ondisk_log_size":296,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:36:00.900988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010536390000000001,"stat_sum":{"num_bytes":53260436,"num_objects":33,"num_object_clones":7,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":7,"num_read":9611,"num_read_kb":19667,"num_write":324,"num_write_kb":4733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":1172339,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":73,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-18T04:57:17.364117+0000","last_change":"2024-09-18T04:56:12.398631+0000","last_active":"2024-09-18T04:57:17.364117+0000","last_peered":"2024-09-18T04:57:17.364117+0000","last_clean":"2024-09-18T04:57:17.364117+0000","last_became_active":"2024-09-18T04:56:10.412614+0000","last_became_peered":"2024-09-18T04:56:10.412614+0000","last_unstale":"2024-09-18T04:57:17.364117+0000","last_undegraded":"2024-09-18T04:57:17.364117+0000","last_fullsized":"2024-09-18T04:57:17.364117+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:44:13.401320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075126299999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"72'236","reported_seq":799,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.227013+0000","last_change":"2024-09-18T04:56:23.133345+0000","last_active":"2024-09-18T04:57:19.227013+0000","last_peered":"2024-09-18T04:57:19.227013+0000","last_clean":"2024-09-18T04:57:19.227013+0000","last_became_active":"2024-09-18T04:56:21.128793+0000","last_became_peered":"2024-09-18T04:56:21.128793+0000","last_unstale":"2024-09-18T04:57:19.227013+0000","last_undegraded":"2024-09-18T04:57:19.227013+0000","last_fullsized":"2024-09-18T04:57:19.227013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":236,"log_dups_size":0,"ondisk_log_size":236,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:10:36.658665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067161900000000004,"stat_sum":{"num_bytes":61146260,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":7,"num_read":583,"num_read_kb":15295,"num_write":247,"num_write_kb":10842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-18T04:57:17.359543+0000","last_change":"2024-09-18T04:56:12.390044+0000","last_active":"2024-09-18T04:57:17.359543+0000","last_peered":"2024-09-18T04:57:17.359543+0000","last_clean":"2024-09-18T04:57:17.359543+0000","last_became_active":"2024-09-18T04:56:10.413042+0000","last_became_peered":"2024-09-18T04:56:10.413042+0000","last_unstale":"2024-09-18T04:57:17.359543+0000","last_undegraded":"2024-09-18T04:57:17.359543+0000","last_fullsized":"2024-09-18T04:57:17.359543+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:46:11.074243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"72'308","reported_seq":3708,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.238818+0000","last_change":"2024-09-18T04:56:23.581937+0000","last_active":"2024-09-18T04:57:19.238818+0000","last_peered":"2024-09-18T04:57:19.238818+0000","last_clean":"2024-09-18T04:57:19.238818+0000","last_became_active":"2024-09-18T04:56:21.130465+0000","last_became_peered":"2024-09-18T04:56:21.130465+0000","last_unstale":"2024-09-18T04:57:19.238818+0000","last_undegraded":"2024-09-18T04:57:19.238818+0000","last_fullsized":"2024-09-18T04:57:19.238818+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":308,"log_dups_size":0,"ondisk_log_size":308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:38:38.231233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037357100000000001,"stat_sum":{"num_bytes":40107239,"num_objects":26,"num_object_clones":5,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":4,"num_read":4956,"num_read_kb":12702,"num_write":304,"num_write_kb":3780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":29,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-18T04:57:17.410982+0000","last_change":"2024-09-18T04:56:59.285326+0000","last_active":"2024-09-18T04:57:17.410982+0000","last_peered":"2024-09-18T04:57:17.410982+0000","last_clean":"2024-09-18T04:57:17.410982+0000","last_became_active":"2024-09-18T04:56:59.284974+0000","last_became_peered":"2024-09-18T04:56:59.284974+0000","last_unstale":"2024-09-18T04:57:17.410982+0000","last_undegraded":"2024-09-18T04:57:17.410982+0000","last_fullsized":"2024-09-18T04:57:17.410982+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:33:29.194155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.8","version":"72'273","reported_seq":1963,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:18.833410+0000","last_change":"2024-09-18T04:57:08.326523+0000","last_active":"2024-09-18T04:57:18.833410+0000","last_peered":"2024-09-18T04:57:18.833410+0000","last_clean":"2024-09-18T04:57:18.833410+0000","last_became_active":"2024-09-18T04:57:08.325666+0000","last_became_peered":"2024-09-18T04:57:08.325666+0000","last_unstale":"2024-09-18T04:57:18.833410+0000","last_undegraded":"2024-09-18T04:57:18.833410+0000","last_fullsized":"2024-09-18T04:57:18.833410+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:02:48.526668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053349799999999998,"stat_sum":{"num_bytes":27659717,"num_objects":37,"num_object_clones":10,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":11,"num_read":2016,"num_read_kb":6995,"num_write":297,"num_write_kb":5631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":71,"num_bytes_recovered":1439423,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"72'328","reported_seq":9849,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:18.833319+0000","last_change":"2024-09-18T04:56:23.133113+0000","last_active":"2024-09-18T04:57:18.833319+0000","last_peered":"2024-09-18T04:57:18.833319+0000","last_clean":"2024-09-18T04:57:18.833319+0000","last_became_active":"2024-09-18T04:56:21.128135+0000","last_became_peered":"2024-09-18T04:56:21.128135+0000","last_unstale":"2024-09-18T04:57:18.833319+0000","last_undegraded":"2024-09-18T04:57:18.833319+0000","last_fullsized":"2024-09-18T04:57:18.833319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:46:04.362749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000513702,"stat_sum":{"num_bytes":49056308,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":7,"num_read":10732,"num_read_kb":23707,"num_write":331,"num_write_kb":13735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"72'226","reported_seq":1071,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:18.585545+0000","last_change":"2024-09-18T04:56:23.641225+0000","last_active":"2024-09-18T04:57:18.585545+0000","last_peered":"2024-09-18T04:57:18.585545+0000","last_clean":"2024-09-18T04:57:18.585545+0000","last_became_active":"2024-09-18T04:56:21.127832+0000","last_became_peered":"2024-09-18T04:56:21.127832+0000","last_unstale":"2024-09-18T04:57:18.585545+0000","last_undegraded":"2024-09-18T04:57:18.585545+0000","last_fullsized":"2024-09-18T04:57:18.585545+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":226,"log_dups_size":0,"ondisk_log_size":226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:23:15.821734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029947799999999999,"stat_sum":{"num_bytes":37308659,"num_objects":34,"num_object_clones":7,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":5,"num_read":967,"num_read_kb":7012,"num_write":247,"num_write_kb":5343,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"72'940","reported_seq":3515,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.050037+0000","last_change":"2024-09-18T04:56:23.582528+0000","last_active":"2024-09-18T04:57:19.050037+0000","last_peered":"2024-09-18T04:57:19.050037+0000","last_clean":"2024-09-18T04:57:19.050037+0000","last_became_active":"2024-09-18T04:56:21.127444+0000","last_became_peered":"2024-09-18T04:56:21.127444+0000","last_unstale":"2024-09-18T04:57:19.050037+0000","last_undegraded":"2024-09-18T04:57:19.050037+0000","last_fullsized":"2024-09-18T04:57:19.050037+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":940,"log_dups_size":0,"ondisk_log_size":940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:15:41.251274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060817200000000003,"stat_sum":{"num_bytes":15175618,"num_objects":33,"num_object_clones":7,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":6,"num_read":6696,"num_read_kb":11993,"num_write":1496,"num_write_kb":5584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"72'483","reported_seq":2442,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:18.900774+0000","last_change":"2024-09-18T04:56:23.136791+0000","last_active":"2024-09-18T04:57:18.900774+0000","last_peered":"2024-09-18T04:57:18.900774+0000","last_clean":"2024-09-18T04:57:18.900774+0000","last_became_active":"2024-09-18T04:56:21.131676+0000","last_became_peered":"2024-09-18T04:56:21.131676+0000","last_unstale":"2024-09-18T04:57:18.900774+0000","last_undegraded":"2024-09-18T04:57:18.900774+0000","last_fullsized":"2024-09-18T04:57:18.900774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:34:36.563239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000302343,"stat_sum":{"num_bytes":29718572,"num_objects":35,"num_object_clones":10,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":8,"num_read":4850,"num_read_kb":15987,"num_write":519,"num_write_kb":5157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"72'303","reported_seq":5480,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.081747+0000","last_change":"2024-09-18T04:56:23.143279+0000","last_active":"2024-09-18T04:57:19.081747+0000","last_peered":"2024-09-18T04:57:19.081747+0000","last_clean":"2024-09-18T04:57:19.081747+0000","last_became_active":"2024-09-18T04:56:21.133686+0000","last_became_peered":"2024-09-18T04:56:21.133686+0000","last_unstale":"2024-09-18T04:57:19.081747+0000","last_undegraded":"2024-09-18T04:57:19.081747+0000","last_fullsized":"2024-09-18T04:57:19.081747+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":303,"log_dups_size":0,"ondisk_log_size":303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:50:51.011660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087309599999999996,"stat_sum":{"num_bytes":31434762,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":7,"num_read":7049,"num_read_kb":9923,"num_write":302,"num_write_kb":5218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"71'774","reported_seq":2034,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:18.586491+0000","last_change":"2024-09-18T04:56:23.143532+0000","last_active":"2024-09-18T04:57:18.586491+0000","last_peered":"2024-09-18T04:57:18.586491+0000","last_clean":"2024-09-18T04:57:18.586491+0000","last_became_active":"2024-09-18T04:56:21.132364+0000","last_became_peered":"2024-09-18T04:56:21.132364+0000","last_unstale":"2024-09-18T04:57:18.586491+0000","last_undegraded":"2024-09-18T04:57:18.586491+0000","last_fullsized":"2024-09-18T04:57:18.586491+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":774,"log_dups_size":0,"ondisk_log_size":774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:25:44.430438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010190329999999999,"stat_sum":{"num_bytes":24886814,"num_objects":25,"num_object_clones":4,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":5,"num_read":4739,"num_read_kb":11608,"num_write":1346,"num_write_kb":3512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"72'322","reported_seq":3352,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:18.784471+0000","last_change":"2024-09-18T04:56:23.143500+0000","last_active":"2024-09-18T04:57:18.784471+0000","last_peered":"2024-09-18T04:57:18.784471+0000","last_clean":"2024-09-18T04:57:18.784471+0000","last_became_active":"2024-09-18T04:56:21.130105+0000","last_became_peered":"2024-09-18T04:56:21.130105+0000","last_unstale":"2024-09-18T04:57:18.784471+0000","last_undegraded":"2024-09-18T04:57:18.784471+0000","last_fullsized":"2024-09-18T04:57:18.784471+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:31:55.392378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010487090000000001,"stat_sum":{"num_bytes":43160339,"num_objects":38,"num_object_clones":11,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":9,"num_read":3873,"num_read_kb":25018,"num_write":380,"num_write_kb":6083,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"72'546","reported_seq":1079,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.115550+0000","last_change":"2024-09-18T04:56:23.641201+0000","last_active":"2024-09-18T04:57:19.115550+0000","last_peered":"2024-09-18T04:57:19.115550+0000","last_clean":"2024-09-18T04:57:19.115550+0000","last_became_active":"2024-09-18T04:56:21.131202+0000","last_became_peered":"2024-09-18T04:56:21.131202+0000","last_unstale":"2024-09-18T04:57:19.115550+0000","last_undegraded":"2024-09-18T04:57:19.115550+0000","last_fullsized":"2024-09-18T04:57:19.115550+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:17:35.438723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000304567,"stat_sum":{"num_bytes":55492530,"num_objects":35,"num_object_clones":9,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":5,"num_read":2492,"num_read_kb":6942,"num_write":760,"num_write_kb":7258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"72'400","reported_seq":5134,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:18.946656+0000","last_change":"2024-09-18T04:56:23.142612+0000","last_active":"2024-09-18T04:57:18.946656+0000","last_peered":"2024-09-18T04:57:18.946656+0000","last_clean":"2024-09-18T04:57:18.946656+0000","last_became_active":"2024-09-18T04:56:21.130594+0000","last_became_peered":"2024-09-18T04:56:21.130594+0000","last_unstale":"2024-09-18T04:57:18.946656+0000","last_undegraded":"2024-09-18T04:57:18.946656+0000","last_fullsized":"2024-09-18T04:57:18.946656+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:46:34.324509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00030763399999999999,"stat_sum":{"num_bytes":56997038,"num_objects":31,"num_object_clones":4,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":2,"num_read":6786,"num_read_kb":13534,"num_write":446,"num_write_kb":16805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"72'545","reported_seq":3878,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.115896+0000","last_change":"2024-09-18T04:56:23.129633+0000","last_active":"2024-09-18T04:57:19.115896+0000","last_peered":"2024-09-18T04:57:19.115896+0000","last_clean":"2024-09-18T04:57:19.115896+0000","last_became_active":"2024-09-18T04:56:21.129597+0000","last_became_peered":"2024-09-18T04:56:21.129597+0000","last_unstale":"2024-09-18T04:57:19.115896+0000","last_undegraded":"2024-09-18T04:57:19.115896+0000","last_fullsized":"2024-09-18T04:57:19.115896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:50:50.665769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059848499999999995,"stat_sum":{"num_bytes":43553327,"num_objects":37,"num_object_clones":9,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":10,"num_read":5680,"num_read_kb":30171,"num_write":840,"num_write_kb":11931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"72'363","reported_seq":921,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-18T04:57:19.120700+0000","last_change":"2024-09-18T04:56:55.263009+0000","last_active":"2024-09-18T04:57:19.120700+0000","last_peered":"2024-09-18T04:57:19.120700+0000","last_clean":"2024-09-18T04:57:19.120700+0000","last_became_active":"2024-09-18T04:56:55.261829+0000","last_became_peered":"2024-09-18T04:56:55.261829+0000","last_unstale":"2024-09-18T04:57:19.120700+0000","last_undegraded":"2024-09-18T04:57:19.120700+0000","last_fullsized":"2024-09-18T04:57:19.120700+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:01:08.639453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085320600000000002,"stat_sum":{"num_bytes":39720490,"num_objects":32,"num_object_clones":6,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":5,"num_read":2006,"num_read_kb":7933,"num_write":416,"num_write_kb":7952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":4916923,"num_keys_recovered":29,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1393698446,"num_objects":1152,"num_object_clones":252,"num_object_copies":2304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1152,"num_whiteouts":216,"num_read":147166,"num_read_kb":524264,"num_write":16285,"num_write_kb":306396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":416,"num_bytes_recovered":34229789,"num_keys_recovered":65,"num_objects_omap":38,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":58839040,"data_stored":401369468,"data_compressed":16665885,"data_compressed_allocated":36253696,"data_compressed_original":382851700,"omap_allocated":0,"internal_metadata":0},"log_size":12349,"ondisk_log_size":12349,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":1771104,"data_compressed":27132,"data_compressed_allocated":110592,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54144,"kb_used_data":9076,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94317696,"statfs":{"total":96636764160,"available":96581320704,"internally_reserved":0,"allocated":9293824,"data_stored":53616456,"data_compressed":2095799,"data_compressed_allocated":4845568,"data_compressed_original":50508059,"omap_allocated":1589,"internal_metadata":46135755},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":2.3420000000000001,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.60899999999999999}]},{"osd":1,"last update":"Wed Sep 18 04:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.70099999999999996}]},{"osd":2,"last update":"Wed Sep 18 04:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.443}]},{"osd":3,"last update":"Wed Sep 18 04:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.554,"5min":1.554,"15min":1.554},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"last":0.65200000000000002}]},{"osd":4,"last update":"Wed Sep 18 04:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":1.954,"5min":1.954,"15min":1.954},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.478,"5min":1.478,"15min":1.478},"last":0.629}]},{"osd":5,"last update":"Wed Sep 18 04:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":3.145,"5min":3.145,"15min":3.145},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.40400000000000003}]},{"osd":6,"last update":"Wed Sep 18 04:57:11 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.4410000000000001,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.59499999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38468,"kb_used_data":3260,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94333372,"statfs":{"total":96636764160,"available":96597372928,"internally_reserved":0,"allocated":3338240,"data_stored":16706896,"data_compressed":417698,"data_compressed_allocated":1241088,"data_compressed_original":15237923,"omap_allocated":1593,"internal_metadata":36043207},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":6.1319999999999997,"5min":6.1319999999999997,"15min":6.1319999999999997},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":6.1429999999999998,"5min":6.1429999999999998,"15min":6.1429999999999998},"last":0.39000000000000001}]},{"osd":1,"last update":"Wed Sep 18 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":6.0410000000000004,"5min":6.0410000000000004,"15min":6.0410000000000004},"last":0.375},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":5.5380000000000003,"5min":5.5380000000000003,"15min":5.5380000000000003},"last":0.33000000000000002}]},{"osd":2,"last update":"Wed Sep 18 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.317},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.41099999999999998}]},{"osd":3,"last update":"Wed Sep 18 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.40100000000000002}]},{"osd":4,"last update":"Wed Sep 18 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.35399999999999998}]},{"osd":5,"last update":"Wed Sep 18 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.30399999999999999}]},{"osd":7,"last update":"Wed Sep 18 04:57:10 2024","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.36399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48248,"kb_used_data":8940,"kb_used_omap":1,"kb_used_meta":39294,"kb_avail":94323592,"statfs":{"total":96636764160,"available":96587358208,"internally_reserved":0,"allocated":9154560,"data_stored":52989378,"data_compressed":1945353,"data_compressed_allocated":4628480,"data_compressed_original":49505640,"omap_allocated":1594,"internal_metadata":40237510},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":3.0179999999999998,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.55400000000000005}]},{"osd":1,"last update":"Wed Sep 18 04:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":3.1349999999999998,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.1499999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.435}]},{"osd":2,"last update":"Wed Sep 18 04:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":3.1070000000000002,"5min":3.1070000000000002,"15min":3.1070000000000002},"last":0.45700000000000002}]},{"osd":3,"last update":"Wed Sep 18 04:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.63}]},{"osd":4,"last update":"Wed Sep 18 04:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.599,"5min":1.599,"15min":1.599},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":3.379,"5min":3.379,"15min":3.379},"last":0.36199999999999999}]},{"osd":6,"last update":"Wed Sep 18 04:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":3.1669999999999998,"5min":3.1669999999999998,"15min":3.1669999999999998},"last":0.41299999999999998}]},{"osd":7,"last update":"Wed Sep 18 04:57:12 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.56999999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41116,"kb_used_data":14088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94330724,"statfs":{"total":96636764160,"available":96594661376,"internally_reserved":0,"allocated":14426112,"data_stored":100417881,"data_compressed":4529541,"data_compressed_allocated":9211904,"data_compressed_original":96448344,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.33900000000000002}]},{"osd":1,"last update":"Wed Sep 18 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"last":0.52700000000000002}]},{"osd":2,"last update":"Wed Sep 18 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.49399999999999999}]},{"osd":3,"last update":"Wed Sep 18 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.50800000000000001}]},{"osd":5,"last update":"Wed Sep 18 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.40999999999999998}]},{"osd":6,"last update":"Wed Sep 18 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.1899999999999999,"5min":1.1899999999999999,"15min":1.1899999999999999},"last":0.55400000000000005}]},{"osd":7,"last update":"Wed Sep 18 04:57:08 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.53800000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":57220,"kb_used_data":8056,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94314620,"statfs":{"total":96636764160,"available":96578170880,"internally_reserved":0,"allocated":8249344,"data_stored":47192331,"data_compressed":1876369,"data_compressed_allocated":4186112,"data_compressed_original":44102108,"omap_allocated":1592,"internal_metadata":50330056},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.49299999999999999}]},{"osd":1,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.35999999999999999}]},{"osd":2,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.46500000000000002}]},{"osd":4,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.63400000000000001}]},{"osd":5,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.40400000000000003}]},{"osd":6,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.72399999999999998}]},{"osd":7,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.753}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39164,"kb_used_data":3944,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94332676,"statfs":{"total":96636764160,"available":96596660224,"internally_reserved":0,"allocated":4038656,"data_stored":20932552,"data_compressed":972100,"data_compressed_allocated":1998848,"data_compressed_original":19561359,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.20999999999999999},{"interface":"front","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.247}]},{"osd":1,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.28399999999999997}]},{"osd":3,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"last":0.32500000000000001}]},{"osd":4,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":1.107,"5min":1.107,"15min":1.107},"last":0.36299999999999999}]},{"osd":5,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.312},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.377}]},{"osd":6,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.35199999999999998}]},{"osd":7,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.442},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.40000000000000002}]}]},{"osd":1,"up_from":0,"seq":55834574866,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55200,"kb_used_data":10132,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94316640,"statfs":{"total":96636764160,"available":96580239360,"internally_reserved":0,"allocated":10375168,"data_stored":72152169,"data_compressed":3335783,"data_compressed_allocated":6639616,"data_compressed_original":69454872,"omap_allocated":1593,"internal_metadata":46135751},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.247},{"interface":"front","average":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.621,"5min":0.621,"15min":0.621},"last":0.34399999999999997}]},{"osd":2,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.45600000000000002}]},{"osd":3,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.33100000000000002}]},{"osd":4,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":3.4710000000000001,"5min":3.4710000000000001,"15min":3.4710000000000001},"last":0.314},{"interface":"front","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.38200000000000001}]},{"osd":5,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.5619999999999998,"5min":3.5619999999999998,"15min":3.5619999999999998},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.434}]},{"osd":6,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":3.536,"5min":3.536,"15min":3.536},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.48899999999999999}]},{"osd":7,"last update":"Wed Sep 18 04:57:09 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":3.5089999999999999,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.44600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9734,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":9768960,"data_stored":71422056,"data_compressed":3326116,"data_compressed_allocated":6606848,"data_compressed_original":68873240,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4075520,"data_stored":32778014,"data_compressed":1099505,"data_compressed_allocated":2736128,"data_compressed_original":31618951,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3215360,"data_stored":20613148,"data_compressed":972410,"data_compressed_allocated":2007040,"data_compressed_original":19622799,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7618560,"data_stored":47032445,"data_compressed":1876580,"data_compressed_allocated":4190208,"data_compressed_original":44147164,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":13975552,"data_stored":102425055,"data_compressed":4582318,"data_compressed_allocated":9392128,"data_compressed_original":98660184,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8736768,"data_stored":54813096,"data_compressed":2107703,"data_compressed_allocated":4902912,"data_compressed_original":51598750,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2523136,"data_stored":16342667,"data_compressed":417765,"data_compressed_allocated":1245184,"data_compressed_original":15246115,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8925184,"data_stored":55942987,"data_compressed":2283488,"data_compressed_allocated":5173248,"data_compressed_original":53084497,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T04:57:22.129 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-18T04:57:22.129 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T04:57:22.129 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-18T04:57:22.130 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 0.000000 2024-09-18T04:57:22.130 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 0 2024-09-18T04:57:22.203 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:22.200+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:22.221 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:22.221 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:22.230 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:57:22.304 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:22.301+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:22.356 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:22.356 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:22.365 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:57:22.401 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T04:57:22.446 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:22.443+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:22.475 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:22.475 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:22.484 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:57:22.528 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:22.526+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:22.588 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:22.588 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:22.597 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:57:22.601 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:57:22.620 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-09-18T04:57:22.621 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:22.618+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:22.626 INFO:teuthology.orchestra.run.smithi002.stdout:4970 read 0x5d59aff thru 0x5d5c306 (0x2808 bytes) 2024-09-18T04:57:22.627 INFO:teuthology.orchestra.run.smithi002.stdout:4971 writesame 0x6e70e4e thru 0x6e7d1b8 data_size 0x1(0xc36b bytes) 2024-09-18T04:57:22.658 INFO:teuthology.orchestra.run.smithi002.stdout:4972 read 0xaf15f8d thru 0xaf19843 (0x38b7 bytes) 2024-09-18T04:57:22.658 INFO:teuthology.orchestra.run.smithi002.stdout:4973 write 0x6f6bc2 thru 0x6f955e (0x299d bytes) 2024-09-18T04:57:22.663 INFO:teuthology.orchestra.run.smithi002.stdout:4974 read 0x5f05f62 thru 0x5f13c4c (0xdceb bytes) 2024-09-18T04:57:22.667 INFO:teuthology.orchestra.run.smithi002.stdout:4976 trunc from 0xd1a2a83 to 0xa072bc5 2024-09-18T04:57:22.690 INFO:teuthology.orchestra.run.smithi002.stdout:4979 write 0xa422566 thru 0xa42eead (0xc948 bytes) 2024-09-18T04:57:22.698 INFO:teuthology.orchestra.run.smithi002.stdout:4980 write 0xa437962 thru 0xa43c761 (0x4e00 bytes) 2024-09-18T04:57:22.703 INFO:teuthology.orchestra.run.smithi002.stdout:4981 punch from 0x86574a3 to 0x865c6de, (0x523b bytes) 2024-09-18T04:57:22.703 INFO:teuthology.orchestra.run.smithi002.stdout:4982 read 0x5e6bb4c thru 0x5e6d411 (0x18c6 bytes) 2024-09-18T04:57:22.703 INFO:teuthology.orchestra.run.smithi002.stdout:4983 read 0x22833f5 thru 0x229177a (0xe386 bytes) 2024-09-18T04:57:22.703 INFO:teuthology.orchestra.run.smithi002.stdout:4984 read 0x1e7287c thru 0x1e75eb4 (0x3639 bytes) 2024-09-18T04:57:22.703 INFO:teuthology.orchestra.run.smithi002.stdout:4985 trunc from 0xa43c762 to 0xa391d7a 2024-09-18T04:57:22.708 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:22.705+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:22.709 INFO:teuthology.orchestra.run.smithi002.stdout:4987 write 0x83e2e thru 0x8be82 (0x8055 bytes) 2024-09-18T04:57:22.717 INFO:teuthology.orchestra.run.smithi002.stdout:4988 read 0x5b87e68 thru 0x5b95e7c (0xe015 bytes) 2024-09-18T04:57:22.726 INFO:teuthology.orchestra.run.smithi002.stdout:4991 write 0xb5efd7f thru 0xb5f071a (0x99c bytes) 2024-09-18T04:57:22.730 INFO:teuthology.orchestra.run.smithi002.stdout:4992 read 0x531db5 thru 0x5400ab (0xe2f7 bytes) 2024-09-18T04:57:22.730 INFO:teuthology.orchestra.run.smithi002.stdout:4996 read 0x4b37de thru 0x4b3ea0 (0x6c3 bytes) 2024-09-18T04:57:22.739 INFO:teuthology.orchestra.run.smithi002.stdout:4997 write 0xecbfec3 thru 0xecc9abc (0x9bfa bytes) 2024-09-18T04:57:22.744 INFO:teuthology.orchestra.run.smithi002.stdout:5000 write 0x9a46977 thru 0x9a49dbb (0x3445 bytes) 2024-09-18T04:57:22.745 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:22.746 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:22.747 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:57:22.747 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:57:22.749 INFO:teuthology.orchestra.run.smithi002.stdout:5001 read 0xbfc16c9 thru 0xbfca3d6 (0x8d0e bytes) 2024-09-18T04:57:22.749 INFO:teuthology.orchestra.run.smithi002.stdout:5002 punch from 0xda7065a to 0xda76f49, (0x68ef bytes) 2024-09-18T04:57:22.749 INFO:teuthology.orchestra.run.smithi002.stdout:5003 read 0x114bffd thru 0x1158ee8 (0xceec bytes) 2024-09-18T04:57:22.749 INFO:teuthology.orchestra.run.smithi002.stdout:5004 write 0x10aa1a5 thru 0x10aad9c (0xbf8 bytes) 2024-09-18T04:57:22.752 INFO:teuthology.orchestra.run.smithi002.stdout:5005 read 0x76f1a08 thru 0x76f73dc (0x59d5 bytes) 2024-09-18T04:57:22.752 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:22.752 INFO:teuthology.orchestra.run.smithi002.stdout:5007 writesame 0xe5fc6ce thru 0xe6072c2 data_size 0x1(0xabf5 bytes) 2024-09-18T04:57:22.755 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:57:22.767 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:57:22.767 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:57:22.771 INFO:teuthology.orchestra.run.smithi002.stdout:5008 writesame 0x765c3b8 thru 0x76629fa data_size 0x1(0x6643 bytes) 2024-09-18T04:57:22.782 INFO:teuthology.orchestra.run.smithi002.stdout:5009 write 0xcbc4443 thru 0xcbc4530 (0xee bytes) 2024-09-18T04:57:22.783 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:57:22.784 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:57:22.785 INFO:teuthology.orchestra.run.smithi002.stdout:5010 write 0xff2177 thru 0xffd6d0 (0xb55a bytes) 2024-09-18T04:57:22.790 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:22.790 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:22.790 INFO:teuthology.orchestra.run.smithi002.stdout:5014 write 0xccf65dc thru 0xccfff7a (0x999f bytes) 2024-09-18T04:57:22.794 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:22.794 INFO:teuthology.orchestra.run.smithi002.stdout:5017 read 0x16f0462 thru 0x16fecb2 (0xe851 bytes) 2024-09-18T04:57:22.795 INFO:teuthology.orchestra.run.smithi002.stdout:5018 write 0x8790be6 thru 0x87994f8 (0x8913 bytes) 2024-09-18T04:57:22.799 INFO:teuthology.orchestra.run.smithi002.stdout:5021 punch from 0x5f08c9b to 0x5f17490, (0xe7f5 bytes) 2024-09-18T04:57:22.799 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:22.799 INFO:teuthology.orchestra.run.smithi002.stdout:5023 read 0x9f3366e thru 0x9f3e8ae (0xb241 bytes) 2024-09-18T04:57:22.799 INFO:teuthology.orchestra.run.smithi002.stdout:5025 write 0x49ac1a6 thru 0x49b1bb7 (0x5a12 bytes) 2024-09-18T04:57:22.803 INFO:teuthology.orchestra.run.smithi002.stdout:5026 read 0xd98c254 thru 0xd998c7b (0xca28 bytes) 2024-09-18T04:57:22.803 INFO:teuthology.orchestra.run.smithi002.stdout:5027 read 0x78c9c23 thru 0x78c9fcb (0x3a9 bytes) 2024-09-18T04:57:22.804 INFO:teuthology.orchestra.run.smithi002.stdout:5028 read 0x97e363d thru 0x97eea6a (0xb42e bytes) 2024-09-18T04:57:22.804 INFO:teuthology.orchestra.run.smithi002.stdout:5029 write 0x486ab7f thru 0x4875bb9 (0xb03b bytes) 2024-09-18T04:57:22.805 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:57:22.805 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:57:22.823 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:57:22.823 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:57:22.824 INFO:teuthology.orchestra.run.smithi002.stdout:5030 trunc from 0xecc9abd to 0x4cd06 2024-09-18T04:57:22.840 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:22.838+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:22.843 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:57:22.844 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:57:22.857 INFO:teuthology.orchestra.run.smithi002.stdout:5031 punch from 0x2367a to 0x2f449, (0xbdcf bytes) 2024-09-18T04:57:22.863 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:57:22.864 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:57:22.868 INFO:teuthology.orchestra.run.smithi002.stdout:5032 write 0x52fa797 thru 0x52fe08b (0x38f5 bytes) 2024-09-18T04:57:22.871 INFO:teuthology.orchestra.run.smithi002.stdout:5033 write 0x2f6ef2b thru 0x2f756a9 (0x677f bytes) 2024-09-18T04:57:22.876 INFO:teuthology.orchestra.run.smithi002.stdout:5034 write 0x2be3888 thru 0x2be923c (0x59b5 bytes) 2024-09-18T04:57:22.883 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:57:22.883 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:57:22.892 INFO:teuthology.orchestra.run.smithi002.stdout:5036 write 0x9feaa78 thru 0x9fec00c (0x1595 bytes) 2024-09-18T04:57:22.895 INFO:teuthology.orchestra.run.smithi002.stdout:5037 write 0x7ace14a thru 0x7ad26b0 (0x4567 bytes) 2024-09-18T04:57:22.899 INFO:teuthology.orchestra.run.smithi002.stdout:5038 read 0x33ff8aa thru 0x340f7d6 (0xff2d bytes) 2024-09-18T04:57:22.899 INFO:teuthology.orchestra.run.smithi002.stdout:5039 read 0x8d08792 thru 0x8d17146 (0xe9b5 bytes) 2024-09-18T04:57:22.899 INFO:teuthology.orchestra.run.smithi002.stdout:5040 read 0x4df697e thru 0x4e01c50 (0xb2d3 bytes) 2024-09-18T04:57:22.899 INFO:teuthology.orchestra.run.smithi002.stdout:5041 write 0x631de1c thru 0x632561e (0x7803 bytes) 2024-09-18T04:57:22.906 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:22.906 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:22.906 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:57:22.906 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:22.911 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:22.908+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:22.913 INFO:teuthology.orchestra.run.smithi002.stdout:5042 writesame 0xe3755c5 thru 0xe381736 data_size 0xb(0xc172 bytes) 2024-09-18T04:57:22.913 INFO:teuthology.orchestra.run.smithi002.stdout:5044 punch from 0xba42630 to 0xba432dc, (0xcac bytes) 2024-09-18T04:57:22.915 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:57:22.918 INFO:teuthology.orchestra.run.smithi002.stdout:5046 trunc from 0xe381737 to 0x9bd1a99 2024-09-18T04:57:22.928 INFO:teuthology.orchestra.run.smithi002.stdout:5047 write 0x5253204 thru 0x525b6b0 (0x84ad bytes) 2024-09-18T04:57:22.931 INFO:teuthology.orchestra.run.smithi002.stdout:5050 read 0x1155cf5 thru 0x115d4cf (0x77db bytes) 2024-09-18T04:57:22.940 INFO:teuthology.orchestra.run.smithi002.stdout:5051 write 0xb8e3a34 thru 0xb8ed756 (0x9d23 bytes) 2024-09-18T04:57:22.943 INFO:teuthology.orchestra.run.smithi002.stdout:5052 write 0x7fe5db3 thru 0x7fea83e (0x4a8c bytes) 2024-09-18T04:57:22.956 INFO:teuthology.orchestra.run.smithi002.stdout:5053 writesame 0xde510f4 thru 0xde5f088 data_size 0x1(0xdf95 bytes) 2024-09-18T04:57:22.967 INFO:teuthology.orchestra.run.smithi002.stdout:5054 trunc from 0xde5f089 to 0x3da9971 2024-09-18T04:57:22.980 INFO:teuthology.orchestra.run.smithi002.stdout:5055 read 0x3835826 thru 0x383a0db (0x48b6 bytes) 2024-09-18T04:57:22.980 INFO:teuthology.orchestra.run.smithi002.stdout:5056 writesame 0x3d2173c thru 0x3d28b5e data_size 0x1(0x7423 bytes) 2024-09-18T04:57:22.996 INFO:teuthology.orchestra.run.smithi002.stdout:5057 read 0x86d97b thru 0x86dbd4 (0x25a bytes) 2024-09-18T04:57:22.996 INFO:teuthology.orchestra.run.smithi002.stdout:5059 read 0x1ada552 thru 0x1ae80fc (0xdbab bytes) 2024-09-18T04:57:22.996 INFO:teuthology.orchestra.run.smithi002.stdout:5060 read 0x13a7dc1 thru 0x13ac26f (0x44af bytes) 2024-09-18T04:57:22.996 INFO:teuthology.orchestra.run.smithi002.stdout:5061 read 0x9424c2 thru 0x94bb9f (0x96de bytes) 2024-09-18T04:57:22.996 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:23.004 INFO:teuthology.orchestra.run.smithi002.stdout:5063 write 0x6ba2376 thru 0x6bacde4 (0xaa6f bytes) 2024-09-18T04:57:23.008 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:23.008 INFO:teuthology.orchestra.run.smithi002.stdout:5065 writesame 0x3b9f46f thru 0x3ba7a58 data_size 0x2(0x85ea bytes) 2024-09-18T04:57:23.011 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:23.009+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:23.022 INFO:teuthology.orchestra.run.smithi002.stdout:5066 write 0x78b66d6 thru 0x78bf5b0 (0x8edb bytes) 2024-09-18T04:57:23.026 INFO:teuthology.orchestra.run.smithi002.stdout:5067 read 0x4591508 thru 0x459d93e (0xc437 bytes) 2024-09-18T04:57:23.033 INFO:teuthology.orchestra.run.smithi002.stdout:5068 write 0x8756a00 thru 0x875f667 (0x8c68 bytes) 2024-09-18T04:57:23.037 INFO:teuthology.orchestra.run.smithi002.stdout:5071 writesame 0x7a21bd9 thru 0x7a2da74 data_size 0x16(0xbe9c bytes) 2024-09-18T04:57:23.040 INFO:teuthology.orchestra.run.smithi002.stdout:5072 read 0x7df3796 thru 0x7df4744 (0xfaf bytes) 2024-09-18T04:57:23.051 INFO:teuthology.orchestra.run.smithi002.stdout:5073 write 0xd5e1ffb thru 0xd5ec9db (0xa9e1 bytes) 2024-09-18T04:57:23.055 INFO:teuthology.orchestra.run.smithi002.stdout:5077 clone 39 order 19 su 2048 sc 12 2024-09-18T04:57:23.063 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:23.063 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:23.063 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:57:23.063 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:57:23.063 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:23.072 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:57:23.113 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:23.110+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:23.214 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:23.212+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:23.217 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:23.217 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:23.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:23.217 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:23.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.218 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60344 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1456128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:23.218 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:23.218 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.100155201, 2024-09-18T04:57:23.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044026324999999998, 2024-09-18T04:57:23.218 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60344 2024-09-18T04:57:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:23.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.220 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.220 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:23.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112224+0000", 2024-09-18T04:57:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1320000000000001e-06 2024-09-18T04:57:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112230+0000", 2024-09-18T04:57:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9619999999999998e-06 2024-09-18T04:57:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112234+0000", 2024-09-18T04:57:23.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7890000000000002e-06 2024-09-18T04:57:23.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156105+0000", 2024-09-18T04:57:23.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043870531999999997 2024-09-18T04:57:23.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156170+0000", 2024-09-18T04:57:23.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5036000000000005e-05 2024-09-18T04:57:23.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156248+0000", 2024-09-18T04:57:23.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8295999999999997e-05 2024-09-18T04:57:23.223 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.223 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.223 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.224 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60345 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1471488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:23.224 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:57:23.225 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.100138436000002, 2024-09-18T04:57:23.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044180090999999998, 2024-09-18T04:57:23.225 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.225 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.225 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.225 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.225 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.225 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60345 2024-09-18T04:57:23.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.225 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:57:23.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:57:23.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112238+0000", 2024-09-18T04:57:23.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112240+0000", 2024-09-18T04:57:23.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9910000000000001e-06 2024-09-18T04:57:23.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112246+0000", 2024-09-18T04:57:23.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0859999999999998e-06 2024-09-18T04:57:23.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112251+0000", 2024-09-18T04:57:23.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5020000000000003e-06 2024-09-18T04:57:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156274+0000", 2024-09-18T04:57:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044023063000000001 2024-09-18T04:57:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156337+0000", 2024-09-18T04:57:23.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3077000000000006e-05 2024-09-18T04:57:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156419+0000", 2024-09-18T04:57:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1699000000000005e-05 2024-09-18T04:57:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60346 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1486848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:23.230 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:57:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.100120654999998, 2024-09-18T04:57:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044336331999999999, 2024-09-18T04:57:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60346 2024-09-18T04:57:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:57:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:57:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112256+0000", 2024-09-18T04:57:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112258+0000", 2024-09-18T04:57:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1720000000000001e-06 2024-09-18T04:57:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112265+0000", 2024-09-18T04:57:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4280000000000001e-06 2024-09-18T04:57:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112269+0000", 2024-09-18T04:57:23.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.741e-06 2024-09-18T04:57:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156445+0000", 2024-09-18T04:57:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044175341 2024-09-18T04:57:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156512+0000", 2024-09-18T04:57:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6723000000000005e-05 2024-09-18T04:57:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156593+0000", 2024-09-18T04:57:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1284000000000003e-05 2024-09-18T04:57:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60347 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1502208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:57:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.100092244000002, 2024-09-18T04:57:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044482560999999997, 2024-09-18T04:57:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.235 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60347 2024-09-18T04:57:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:57:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:57:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112284+0000", 2024-09-18T04:57:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112286+0000", 2024-09-18T04:57:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9709999999999998e-06 2024-09-18T04:57:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112290+0000", 2024-09-18T04:57:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9199999999999997e-06 2024-09-18T04:57:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112293+0000", 2024-09-18T04:57:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870000000000001e-06 2024-09-18T04:57:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156617+0000", 2024-09-18T04:57:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044323960000000003 2024-09-18T04:57:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156687+0000", 2024-09-18T04:57:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0141000000000005e-05 2024-09-18T04:57:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156767+0000", 2024-09-18T04:57:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0140000000000002e-05 2024-09-18T04:57:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.239 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60348 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1517568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.100081240000002, 2024-09-18T04:57:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044645562, 2024-09-18T04:57:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.240 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60348 2024-09-18T04:57:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112297+0000", 2024-09-18T04:57:23.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.384e-06 2024-09-18T04:57:23.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112301+0000", 2024-09-18T04:57:23.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0570000000000003e-06 2024-09-18T04:57:23.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112304+0000", 2024-09-18T04:57:23.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6000000000000001e-06 2024-09-18T04:57:23.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156793+0000", 2024-09-18T04:57:23.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044489494999999997 2024-09-18T04:57:23.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156859+0000", 2024-09-18T04:57:23.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5483999999999998e-05 2024-09-18T04:57:23.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156941+0000", 2024-09-18T04:57:23.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2856999999999996e-05 2024-09-18T04:57:23.246 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.246 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.247 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60349 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1532928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:23.247 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:23.247 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.100071264, 2024-09-18T04:57:23.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044832167999999999, 2024-09-18T04:57:23.247 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.247 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.247 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.248 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60349 2024-09-18T04:57:23.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.248 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:23.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:23.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:23.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112307+0000", 2024-09-18T04:57:23.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.308e-06 2024-09-18T04:57:23.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112310+0000", 2024-09-18T04:57:23.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-18T04:57:23.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112312+0000", 2024-09-18T04:57:23.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4889999999999998e-06 2024-09-18T04:57:23.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156977+0000", 2024-09-18T04:57:23.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044664598999999999 2024-09-18T04:57:23.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157053+0000", 2024-09-18T04:57:23.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6014999999999996e-05 2024-09-18T04:57:23.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157138+0000", 2024-09-18T04:57:23.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5162999999999998e-05 2024-09-18T04:57:23.252 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.252 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.252 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.253 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60350 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1548288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:23.253 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:23.253 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.100062350000002, 2024-09-18T04:57:23.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045011858000000002, 2024-09-18T04:57:23.253 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.253 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.253 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.253 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.253 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.253 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60350 2024-09-18T04:57:23.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.254 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:23.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:23.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:23.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112316+0000", 2024-09-18T04:57:23.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7579999999999999e-06 2024-09-18T04:57:23.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112321+0000", 2024-09-18T04:57:23.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1649999999999997e-06 2024-09-18T04:57:23.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112325+0000", 2024-09-18T04:57:23.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.726e-06 2024-09-18T04:57:23.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157172+0000", 2024-09-18T04:57:23.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.04484718 2024-09-18T04:57:23.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157242+0000", 2024-09-18T04:57:23.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9675e-05 2024-09-18T04:57:23.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157327+0000", 2024-09-18T04:57:23.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4614e-05 2024-09-18T04:57:23.258 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.258 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.258 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.258 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60351 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1563648~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:23.259 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:23.259 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.100048934, 2024-09-18T04:57:23.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045132113000000001, 2024-09-18T04:57:23.259 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.259 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.259 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.259 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.259 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.259 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60351 2024-09-18T04:57:23.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.259 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:23.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:23.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:23.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112330+0000", 2024-09-18T04:57:23.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1660000000000001e-06 2024-09-18T04:57:23.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112336+0000", 2024-09-18T04:57:23.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5940000000000001e-06 2024-09-18T04:57:23.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112340+0000", 2024-09-18T04:57:23.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5589999999999999e-06 2024-09-18T04:57:23.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157365+0000", 2024-09-18T04:57:23.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045024675 2024-09-18T04:57:23.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157409+0000", 2024-09-18T04:57:23.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4356e-05 2024-09-18T04:57:23.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157460+0000", 2024-09-18T04:57:23.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1072000000000002e-05 2024-09-18T04:57:23.263 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.263 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.263 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.263 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60501 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1494016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:23.264 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:23.264 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.093988238000001, 2024-09-18T04:57:23.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044007728000000003, 2024-09-18T04:57:23.264 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.264 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.264 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.264 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.264 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.264 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60501 2024-09-18T04:57:23.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.264 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:23.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:23.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:23.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118391+0000", 2024-09-18T04:57:23.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:57:23.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118396+0000", 2024-09-18T04:57:23.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2680000000000002e-06 2024-09-18T04:57:23.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118400+0000", 2024-09-18T04:57:23.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2080000000000002e-06 2024-09-18T04:57:23.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162317+0000", 2024-09-18T04:57:23.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043916753000000003 2024-09-18T04:57:23.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162351+0000", 2024-09-18T04:57:23.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4372000000000002e-05 2024-09-18T04:57:23.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162397+0000", 2024-09-18T04:57:23.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5207999999999998e-05 2024-09-18T04:57:23.268 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.268 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.268 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.268 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60502 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1509376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:23.268 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:23.268 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.093972761000003, 2024-09-18T04:57:23.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044092438999999997, 2024-09-18T04:57:23.269 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.269 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.269 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.269 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.269 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.269 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60502 2024-09-18T04:57:23.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.269 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:23.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:23.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:23.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118406+0000", 2024-09-18T04:57:23.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:57:23.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118412+0000", 2024-09-18T04:57:23.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.524e-06 2024-09-18T04:57:23.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118416+0000", 2024-09-18T04:57:23.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:57:23.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162418+0000", 2024-09-18T04:57:23.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001508000000002 2024-09-18T04:57:23.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162451+0000", 2024-09-18T04:57:23.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3662999999999999e-05 2024-09-18T04:57:23.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162497+0000", 2024-09-18T04:57:23.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5550000000000003e-05 2024-09-18T04:57:23.274 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.274 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.274 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.274 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60503 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1524736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:23.274 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:57:23.274 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.093956603000002, 2024-09-18T04:57:23.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044178780000000001, 2024-09-18T04:57:23.275 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.275 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.275 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60503 2024-09-18T04:57:23.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.275 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:57:23.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:57:23.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118420+0000", 2024-09-18T04:57:23.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118422+0000", 2024-09-18T04:57:23.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.08e-06 2024-09-18T04:57:23.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118428+0000", 2024-09-18T04:57:23.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.964e-06 2024-09-18T04:57:23.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118433+0000", 2024-09-18T04:57:23.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.425e-06 2024-09-18T04:57:23.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162519+0000", 2024-09-18T04:57:23.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044086082999999998 2024-09-18T04:57:23.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162552+0000", 2024-09-18T04:57:23.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3719e-05 2024-09-18T04:57:23.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162599+0000", 2024-09-18T04:57:23.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6873999999999999e-05 2024-09-18T04:57:23.279 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.280 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.280 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.280 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60504 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1540096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:23.280 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:57:23.280 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.093940498000002, 2024-09-18T04:57:23.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044261921000000003, 2024-09-18T04:57:23.280 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.280 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.280 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.280 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.280 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.280 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60504 2024-09-18T04:57:23.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.281 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:57:23.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:57:23.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118436+0000", 2024-09-18T04:57:23.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118438+0000", 2024-09-18T04:57:23.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0250000000000001e-06 2024-09-18T04:57:23.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118444+0000", 2024-09-18T04:57:23.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4360000000000001e-06 2024-09-18T04:57:23.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118448+0000", 2024-09-18T04:57:23.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.357e-06 2024-09-18T04:57:23.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162620+0000", 2024-09-18T04:57:23.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044171821 2024-09-18T04:57:23.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162653+0000", 2024-09-18T04:57:23.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3479999999999998e-05 2024-09-18T04:57:23.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162699+0000", 2024-09-18T04:57:23.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5113e-05 2024-09-18T04:57:23.285 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.285 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.285 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.285 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60584 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1196032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:23.285 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:57:23.285 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.092147480999998, 2024-09-18T04:57:23.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044048727000000003, 2024-09-18T04:57:23.286 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.286 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.286 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60584 2024-09-18T04:57:23.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.286 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:57:23.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:57:23.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120229+0000", 2024-09-18T04:57:23.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120231+0000", 2024-09-18T04:57:23.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:57:23.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120237+0000", 2024-09-18T04:57:23.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8379999999999998e-06 2024-09-18T04:57:23.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120242+0000", 2024-09-18T04:57:23.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3889999999999998e-06 2024-09-18T04:57:23.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164187+0000", 2024-09-18T04:57:23.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043945045000000002 2024-09-18T04:57:23.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164227+0000", 2024-09-18T04:57:23.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0627999999999998e-05 2024-09-18T04:57:23.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164278+0000", 2024-09-18T04:57:23.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1146000000000001e-05 2024-09-18T04:57:23.290 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.290 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.290 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.291 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60585 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1211392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:23.291 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:57:23.291 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.092131373000001, 2024-09-18T04:57:23.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044389814, 2024-09-18T04:57:23.291 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.291 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.291 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.291 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.291 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.291 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60585 2024-09-18T04:57:23.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.292 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:57:23.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:57:23.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120245+0000", 2024-09-18T04:57:23.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120247+0000", 2024-09-18T04:57:23.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0150000000000002e-06 2024-09-18T04:57:23.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120255+0000", 2024-09-18T04:57:23.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.858e-06 2024-09-18T04:57:23.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120260+0000", 2024-09-18T04:57:23.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6530000000000002e-06 2024-09-18T04:57:23.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164489+0000", 2024-09-18T04:57:23.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044229483999999999 2024-09-18T04:57:23.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164555+0000", 2024-09-18T04:57:23.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.533e-05 2024-09-18T04:57:23.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164636+0000", 2024-09-18T04:57:23.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0819e-05 2024-09-18T04:57:23.295 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.295 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.295 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.295 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60586 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1226752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:23.295 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:23.296 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.092112921999998, 2024-09-18T04:57:23.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044578214999999997, 2024-09-18T04:57:23.296 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.296 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.296 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.296 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.296 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.296 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60586 2024-09-18T04:57:23.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.296 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:23.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:23.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:23.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120266+0000", 2024-09-18T04:57:23.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0660000000000002e-06 2024-09-18T04:57:23.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120272+0000", 2024-09-18T04:57:23.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-18T04:57:23.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120276+0000", 2024-09-18T04:57:23.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.476e-06 2024-09-18T04:57:23.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164673+0000", 2024-09-18T04:57:23.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044396672999999998 2024-09-18T04:57:23.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164749+0000", 2024-09-18T04:57:23.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5811999999999999e-05 2024-09-18T04:57:23.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164842+0000", 2024-09-18T04:57:23.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3560000000000005e-05 2024-09-18T04:57:23.302 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.302 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.302 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.303 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60587 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1242112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:23.303 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:57:23.303 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.092063822999997, 2024-09-18T04:57:23.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044634794999999998, 2024-09-18T04:57:23.303 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.303 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.303 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.303 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.303 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.303 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60587 2024-09-18T04:57:23.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.304 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:57:23.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:57:23.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120280+0000", 2024-09-18T04:57:23.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999671 2024-09-18T04:57:23.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120316+0000", 2024-09-18T04:57:23.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5728000000000001e-05 2024-09-18T04:57:23.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120329+0000", 2024-09-18T04:57:23.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2914e-05 2024-09-18T04:57:23.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120335+0000", 2024-09-18T04:57:23.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8289999999999996e-06 2024-09-18T04:57:23.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164858+0000", 2024-09-18T04:57:23.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044523417000000003 2024-09-18T04:57:23.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164896+0000", 2024-09-18T04:57:23.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7969999999999997e-05 2024-09-18T04:57:23.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164948+0000", 2024-09-18T04:57:23.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1996000000000002e-05 2024-09-18T04:57:23.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.308 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.309 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.309 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60595 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1364992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:23.309 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:57:23.309 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.090220606999999, 2024-09-18T04:57:23.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001360000000003, 2024-09-18T04:57:23.310 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.310 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.310 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.310 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.310 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.310 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60595 2024-09-18T04:57:23.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.310 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:57:23.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:57:23.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122156+0000", 2024-09-18T04:57:23.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122158+0000", 2024-09-18T04:57:23.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0040000000000002e-06 2024-09-18T04:57:23.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122164+0000", 2024-09-18T04:57:23.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8719999999999999e-06 2024-09-18T04:57:23.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122169+0000", 2024-09-18T04:57:23.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5090000000000004e-06 2024-09-18T04:57:23.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166040+0000", 2024-09-18T04:57:23.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043871226999999999 2024-09-18T04:57:23.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166096+0000", 2024-09-18T04:57:23.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6495e-05 2024-09-18T04:57:23.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166158+0000", 2024-09-18T04:57:23.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1626999999999998e-05 2024-09-18T04:57:23.315 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.315 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.315 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.315 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60596 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1380352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:23.315 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:23.315 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.090204286000002, 2024-09-18T04:57:23.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044120800000000002, 2024-09-18T04:57:23.316 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.316 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.316 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.316 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.316 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.316 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60596 2024-09-18T04:57:23.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.316 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:23.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:23.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:23.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122175+0000", 2024-09-18T04:57:23.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:57:23.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122182+0000", 2024-09-18T04:57:23.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3509999999999996e-06 2024-09-18T04:57:23.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122187+0000", 2024-09-18T04:57:23.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7940000000000002e-06 2024-09-18T04:57:23.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166184+0000", 2024-09-18T04:57:23.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043997236000000002 2024-09-18T04:57:23.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166227+0000", 2024-09-18T04:57:23.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2561000000000001e-05 2024-09-18T04:57:23.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166294+0000", 2024-09-18T04:57:23.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7068000000000002e-05 2024-09-18T04:57:23.321 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.321 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.321 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.321 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60597 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1395712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:23.321 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:23.321 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.090186258000003, 2024-09-18T04:57:23.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044242166999999999, 2024-09-18T04:57:23.321 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.322 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.322 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.322 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.322 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.322 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60597 2024-09-18T04:57:23.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.322 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:23.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:23.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:23.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122193+0000", 2024-09-18T04:57:23.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:57:23.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122199+0000", 2024-09-18T04:57:23.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6449999999999999e-06 2024-09-18T04:57:23.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122204+0000", 2024-09-18T04:57:23.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4819999999999996e-06 2024-09-18T04:57:23.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166329+0000", 2024-09-18T04:57:23.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044125059000000001 2024-09-18T04:57:23.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166370+0000", 2024-09-18T04:57:23.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1012e-05 2024-09-18T04:57:23.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166433+0000", 2024-09-18T04:57:23.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3178999999999999e-05 2024-09-18T04:57:23.326 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.326 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.326 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.326 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60608 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1564672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:23.326 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:57:23.326 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.088188396, 2024-09-18T04:57:23.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044082723999999997, 2024-09-18T04:57:23.326 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.326 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.326 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.326 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.327 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.327 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60608 2024-09-18T04:57:23.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.327 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:57:23.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:57:23.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124188+0000", 2024-09-18T04:57:23.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124190+0000", 2024-09-18T04:57:23.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0329999999999998e-06 2024-09-18T04:57:23.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124196+0000", 2024-09-18T04:57:23.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.575e-06 2024-09-18T04:57:23.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124201+0000", 2024-09-18T04:57:23.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5000000000000001e-06 2024-09-18T04:57:23.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168166+0000", 2024-09-18T04:57:23.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043965763999999997 2024-09-18T04:57:23.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168218+0000", 2024-09-18T04:57:23.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1785e-05 2024-09-18T04:57:23.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168271+0000", 2024-09-18T04:57:23.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3429e-05 2024-09-18T04:57:23.330 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.330 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.330 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.330 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.331 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.331 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:23.332 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:57:23.333 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:23.330+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:23.387 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-18T04:57:23.388 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.0 primary-affinity to 0 (802) 2024-09-18T04:57:23.446 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:23.444+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:23.476 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:23.477 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:23.477 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:57:23.477 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:23.485 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:57:23.534 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:23.531+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:23.587 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:23.588 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:23.588 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:57:23.588 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:57:23.588 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:23.596 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:57:23.635 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:23.632+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:23.735 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:23.733+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:23.740 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:23.740 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:23.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:23.740 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:23.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.741 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26189 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 237568~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:23.741 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:23.741 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.097360340000002, 2024-09-18T04:57:23.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022263431, 2024-09-18T04:57:23.741 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.741 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.741 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.741 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.741 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.741 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26189 2024-09-18T04:57:23.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.742 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:23.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:23.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:23.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639787+0000", 2024-09-18T04:57:23.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9759999999999998e-06 2024-09-18T04:57:23.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639792+0000", 2024-09-18T04:57:23.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5829999999999997e-06 2024-09-18T04:57:23.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639796+0000", 2024-09-18T04:57:23.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0690000000000003e-06 2024-09-18T04:57:23.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.661944+0000", 2024-09-18T04:57:23.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022147463999999999 2024-09-18T04:57:23.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.661985+0000", 2024-09-18T04:57:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1360999999999999e-05 2024-09-18T04:57:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662048+0000", 2024-09-18T04:57:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3298000000000003e-05 2024-09-18T04:57:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26199 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 4096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:23.745 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:23.746 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.097345054000002, 2024-09-18T04:57:23.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022348776000000001, 2024-09-18T04:57:23.746 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.746 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.746 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.746 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.746 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.746 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26199 2024-09-18T04:57:23.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.746 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:23.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639806+0000", 2024-09-18T04:57:23.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1539999999999999e-06 2024-09-18T04:57:23.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639812+0000", 2024-09-18T04:57:23.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5520000000000004e-06 2024-09-18T04:57:23.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639816+0000", 2024-09-18T04:57:23.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1840000000000001e-06 2024-09-18T04:57:23.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662063+0000", 2024-09-18T04:57:23.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022247125999999999 2024-09-18T04:57:23.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662098+0000", 2024-09-18T04:57:23.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4968000000000001e-05 2024-09-18T04:57:23.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662149+0000", 2024-09-18T04:57:23.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1122000000000003e-05 2024-09-18T04:57:23.750 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.750 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.750 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.750 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26200 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 32768~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:23.751 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:23.751 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.097325529000003, 2024-09-18T04:57:23.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022501118000000001, 2024-09-18T04:57:23.751 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.751 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.751 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.751 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.751 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.751 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26200 2024-09-18T04:57:23.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.751 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:23.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639822+0000", 2024-09-18T04:57:23.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.007e-06 2024-09-18T04:57:23.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639827+0000", 2024-09-18T04:57:23.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5709999999999997e-06 2024-09-18T04:57:23.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639831+0000", 2024-09-18T04:57:23.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2819999999999998e-06 2024-09-18T04:57:23.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662181+0000", 2024-09-18T04:57:23.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022349878 2024-09-18T04:57:23.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662252+0000", 2024-09-18T04:57:23.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0542000000000003e-05 2024-09-18T04:57:23.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662321+0000", 2024-09-18T04:57:23.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9167999999999999e-05 2024-09-18T04:57:23.755 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.755 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.755 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.756 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26201 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 61440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:23.756 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:23.756 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.097310237999999, 2024-09-18T04:57:23.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022569783, 2024-09-18T04:57:23.756 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.756 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.756 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.756 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.756 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.756 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26201 2024-09-18T04:57:23.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.757 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:23.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:23.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:23.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639837+0000", 2024-09-18T04:57:23.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.928e-06 2024-09-18T04:57:23.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639843+0000", 2024-09-18T04:57:23.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8130000000000001e-06 2024-09-18T04:57:23.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639847+0000", 2024-09-18T04:57:23.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0999999999999997e-06 2024-09-18T04:57:23.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662335+0000", 2024-09-18T04:57:23.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022488207999999999 2024-09-18T04:57:23.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662370+0000", 2024-09-18T04:57:23.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5292e-05 2024-09-18T04:57:23.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662405+0000", 2024-09-18T04:57:23.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4765999999999999e-05 2024-09-18T04:57:23.761 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.761 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.761 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.761 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26202 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 90112~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:23.761 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:57:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.097294820999998, 2024-09-18T04:57:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022655937000000001, 2024-09-18T04:57:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26202 2024-09-18T04:57:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:57:23.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:57:23.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639850+0000", 2024-09-18T04:57:23.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639852+0000", 2024-09-18T04:57:23.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9700000000000002e-06 2024-09-18T04:57:23.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639858+0000", 2024-09-18T04:57:23.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.434e-06 2024-09-18T04:57:23.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639862+0000", 2024-09-18T04:57:23.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3499999999999999e-06 2024-09-18T04:57:23.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662429+0000", 2024-09-18T04:57:23.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022567409 2024-09-18T04:57:23.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662469+0000", 2024-09-18T04:57:23.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9801000000000001e-05 2024-09-18T04:57:23.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662507+0000", 2024-09-18T04:57:23.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7311000000000002e-05 2024-09-18T04:57:23.767 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.767 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.767 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.767 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26203 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 118784~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:23.767 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:57:23.767 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.097279665999999, 2024-09-18T04:57:23.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022730358999999999, 2024-09-18T04:57:23.768 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.768 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.768 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26203 2024-09-18T04:57:23.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.768 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:57:23.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:57:23.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639865+0000", 2024-09-18T04:57:23.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639867+0000", 2024-09-18T04:57:23.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0889999999999998e-06 2024-09-18T04:57:23.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639873+0000", 2024-09-18T04:57:23.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4979999999999997e-06 2024-09-18T04:57:23.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639877+0000", 2024-09-18T04:57:23.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4510000000000002e-06 2024-09-18T04:57:23.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662528+0000", 2024-09-18T04:57:23.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022650791999999999 2024-09-18T04:57:23.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662563+0000", 2024-09-18T04:57:23.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4530000000000003e-05 2024-09-18T04:57:23.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662596+0000", 2024-09-18T04:57:23.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3463000000000001e-05 2024-09-18T04:57:23.772 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.773 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.773 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.773 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26204 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 147456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:23.773 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:23.773 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.097264273999997, 2024-09-18T04:57:23.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022803233999999999, 2024-09-18T04:57:23.773 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.773 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.773 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.773 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.773 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.773 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26204 2024-09-18T04:57:23.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.774 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:23.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:23.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:23.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639883+0000", 2024-09-18T04:57:23.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6520000000000002e-06 2024-09-18T04:57:23.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639888+0000", 2024-09-18T04:57:23.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2870000000000004e-06 2024-09-18T04:57:23.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639893+0000", 2024-09-18T04:57:23.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3839999999999999e-06 2024-09-18T04:57:23.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662617+0000", 2024-09-18T04:57:23.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022723876 2024-09-18T04:57:23.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662651+0000", 2024-09-18T04:57:23.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3797000000000002e-05 2024-09-18T04:57:23.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662684+0000", 2024-09-18T04:57:23.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3828999999999997e-05 2024-09-18T04:57:23.777 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.777 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.777 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.778 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26205 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 176128~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:23.778 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:23.778 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.097241312999998, 2024-09-18T04:57:23.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022919108000000001, 2024-09-18T04:57:23.778 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.778 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.778 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.778 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.778 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.778 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26205 2024-09-18T04:57:23.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.778 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:23.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:23.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:23.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639906+0000", 2024-09-18T04:57:23.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.531e-06 2024-09-18T04:57:23.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639913+0000", 2024-09-18T04:57:23.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.331e-06 2024-09-18T04:57:23.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639920+0000", 2024-09-18T04:57:23.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5399999999999998e-06 2024-09-18T04:57:23.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662706+0000", 2024-09-18T04:57:23.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.02278587 2024-09-18T04:57:23.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662764+0000", 2024-09-18T04:57:23.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7630000000000002e-05 2024-09-18T04:57:23.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662823+0000", 2024-09-18T04:57:23.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9648000000000002e-05 2024-09-18T04:57:23.782 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.782 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.782 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.783 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26206 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 204800~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:23.783 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:23.783 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.097220913999998, 2024-09-18T04:57:23.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023047517, 2024-09-18T04:57:23.783 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.783 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.783 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26206 2024-09-18T04:57:23.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.784 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:23.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:23.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:23.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639926+0000", 2024-09-18T04:57:23.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3120000000000001e-06 2024-09-18T04:57:23.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639932+0000", 2024-09-18T04:57:23.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4890000000000003e-06 2024-09-18T04:57:23.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639936+0000", 2024-09-18T04:57:23.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.07e-06 2024-09-18T04:57:23.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662850+0000", 2024-09-18T04:57:23.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022913579999999999 2024-09-18T04:57:23.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662903+0000", 2024-09-18T04:57:23.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3176e-05 2024-09-18T04:57:23.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662972+0000", 2024-09-18T04:57:23.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9270999999999995e-05 2024-09-18T04:57:23.788 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.788 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.788 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.788 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26207 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 233472~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:23.788 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:57:23.788 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.097205770000002, 2024-09-18T04:57:23.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023139541999999999, 2024-09-18T04:57:23.789 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.789 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.789 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.789 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.789 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.789 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26207 2024-09-18T04:57:23.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.789 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:57:23.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:57:23.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639939+0000", 2024-09-18T04:57:23.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639941+0000", 2024-09-18T04:57:23.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0080000000000001e-06 2024-09-18T04:57:23.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639947+0000", 2024-09-18T04:57:23.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.485e-06 2024-09-18T04:57:23.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639951+0000", 2024-09-18T04:57:23.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4549999999999997e-06 2024-09-18T04:57:23.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663005+0000", 2024-09-18T04:57:23.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023053569999999999 2024-09-18T04:57:23.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663042+0000", 2024-09-18T04:57:23.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7245e-05 2024-09-18T04:57:23.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663079+0000", 2024-09-18T04:57:23.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7119000000000002e-05 2024-09-18T04:57:23.794 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.794 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.794 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.794 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26208 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 524288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:23.794 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:23.794 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.097189022000002, 2024-09-18T04:57:23.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023283564999999999, 2024-09-18T04:57:23.795 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.795 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.795 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.795 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.795 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.795 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26208 2024-09-18T04:57:23.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.795 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639958+0000", 2024-09-18T04:57:23.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0899999999999999e-06 2024-09-18T04:57:23.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639963+0000", 2024-09-18T04:57:23.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2209999999999996e-06 2024-09-18T04:57:23.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639967+0000", 2024-09-18T04:57:23.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1169999999999997e-06 2024-09-18T04:57:23.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663114+0000", 2024-09-18T04:57:23.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023146102000000002 2024-09-18T04:57:23.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663183+0000", 2024-09-18T04:57:23.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9686999999999999e-05 2024-09-18T04:57:23.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663240+0000", 2024-09-18T04:57:23.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6728000000000003e-05 2024-09-18T04:57:23.799 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.799 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.799 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.800 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26209 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 552960~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:23.800 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:23.800 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.097171246000002, 2024-09-18T04:57:23.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023440966000000001, 2024-09-18T04:57:23.800 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.800 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.800 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.800 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.801 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.801 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26209 2024-09-18T04:57:23.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.801 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:23.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:23.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:23.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639976+0000", 2024-09-18T04:57:23.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4940000000000002e-06 2024-09-18T04:57:23.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639982+0000", 2024-09-18T04:57:23.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0560000000000001e-06 2024-09-18T04:57:23.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639987+0000", 2024-09-18T04:57:23.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-18T04:57:23.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663278+0000", 2024-09-18T04:57:23.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023290789999999999 2024-09-18T04:57:23.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663348+0000", 2024-09-18T04:57:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0837e-05 2024-09-18T04:57:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663415+0000", 2024-09-18T04:57:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6736000000000006e-05 2024-09-18T04:57:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26210 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 581632~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.097154238000002, 2024-09-18T04:57:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023616437000000001, 2024-09-18T04:57:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26210 2024-09-18T04:57:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:23.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639993+0000", 2024-09-18T04:57:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.419e-06 2024-09-18T04:57:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640006+0000", 2024-09-18T04:57:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2918000000000001e-05 2024-09-18T04:57:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640011+0000", 2024-09-18T04:57:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9320000000000004e-06 2024-09-18T04:57:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663454+0000", 2024-09-18T04:57:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023443035000000001 2024-09-18T04:57:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663532+0000", 2024-09-18T04:57:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7443999999999999e-05 2024-09-18T04:57:23.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663608+0000", 2024-09-18T04:57:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6093000000000005e-05 2024-09-18T04:57:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26211 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 610304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:57:23.809 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.097129686000002, 2024-09-18T04:57:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023774337999999999, 2024-09-18T04:57:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26211 2024-09-18T04:57:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:57:23.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:57:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640015+0000", 2024-09-18T04:57:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640023+0000", 2024-09-18T04:57:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3900000000000004e-06 2024-09-18T04:57:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640036+0000", 2024-09-18T04:57:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3091999999999999e-05 2024-09-18T04:57:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640041+0000", 2024-09-18T04:57:23.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6219999999999997e-06 2024-09-18T04:57:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663644+0000", 2024-09-18T04:57:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023602643999999999 2024-09-18T04:57:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663718+0000", 2024-09-18T04:57:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4481999999999996e-05 2024-09-18T04:57:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663790+0000", 2024-09-18T04:57:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1589999999999997e-05 2024-09-18T04:57:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26212 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 638976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:57:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.097096645999997, 2024-09-18T04:57:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023924748999999999, 2024-09-18T04:57:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.814 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26212 2024-09-18T04:57:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:57:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:57:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640048+0000", 2024-09-18T04:57:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640051+0000", 2024-09-18T04:57:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.095e-06 2024-09-18T04:57:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640060+0000", 2024-09-18T04:57:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8929999999999994e-06 2024-09-18T04:57:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640065+0000", 2024-09-18T04:57:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0270000000000003e-06 2024-09-18T04:57:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663824+0000", 2024-09-18T04:57:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023758873999999999 2024-09-18T04:57:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663904+0000", 2024-09-18T04:57:23.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9856000000000003e-05 2024-09-18T04:57:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663974+0000", 2024-09-18T04:57:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9450999999999996e-05 2024-09-18T04:57:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.819 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26213 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 667648~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.097075474999997, 2024-09-18T04:57:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024059812999999999, 2024-09-18T04:57:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26213 2024-09-18T04:57:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.820 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640072+0000", 2024-09-18T04:57:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4200000000000001e-06 2024-09-18T04:57:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640078+0000", 2024-09-18T04:57:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0179999999999998e-06 2024-09-18T04:57:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640083+0000", 2024-09-18T04:57:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:57:23.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664008+0000", 2024-09-18T04:57:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023925287 2024-09-18T04:57:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664059+0000", 2024-09-18T04:57:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0730999999999999e-05 2024-09-18T04:57:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664130+0000", 2024-09-18T04:57:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0801000000000002e-05 2024-09-18T04:57:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26214 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 696320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:23.825 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.097058505, 2024-09-18T04:57:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024223738000000002, 2024-09-18T04:57:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26214 2024-09-18T04:57:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640089+0000", 2024-09-18T04:57:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0849999999999999e-06 2024-09-18T04:57:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640095+0000", 2024-09-18T04:57:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9279999999999998e-06 2024-09-18T04:57:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640099+0000", 2024-09-18T04:57:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:57:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664166+0000", 2024-09-18T04:57:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024066513000000001 2024-09-18T04:57:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664239+0000", 2024-09-18T04:57:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3201999999999995e-05 2024-09-18T04:57:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664311+0000", 2024-09-18T04:57:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1434999999999996e-05 2024-09-18T04:57:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26215 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 724992~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:57:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.097041744000002, 2024-09-18T04:57:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024302284, 2024-09-18T04:57:23.831 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26215 2024-09-18T04:57:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:57:23.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:57:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640103+0000", 2024-09-18T04:57:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640106+0000", 2024-09-18T04:57:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2120000000000002e-06 2024-09-18T04:57:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640111+0000", 2024-09-18T04:57:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7880000000000003e-06 2024-09-18T04:57:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640116+0000", 2024-09-18T04:57:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6099999999999999e-06 2024-09-18T04:57:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664327+0000", 2024-09-18T04:57:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024210941999999999 2024-09-18T04:57:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664367+0000", 2024-09-18T04:57:23.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9666000000000003e-05 2024-09-18T04:57:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664406+0000", 2024-09-18T04:57:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9402e-05 2024-09-18T04:57:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26216 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 753664~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:23.836 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.097025250000002, 2024-09-18T04:57:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024452854999999999, 2024-09-18T04:57:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26216 2024-09-18T04:57:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640122+0000", 2024-09-18T04:57:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3599999999999999e-06 2024-09-18T04:57:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640128+0000", 2024-09-18T04:57:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6339999999999998e-06 2024-09-18T04:57:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640132+0000", 2024-09-18T04:57:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:57:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664436+0000", 2024-09-18T04:57:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024303585999999999 2024-09-18T04:57:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664507+0000", 2024-09-18T04:57:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1046999999999999e-05 2024-09-18T04:57:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664573+0000", 2024-09-18T04:57:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6217000000000007e-05 2024-09-18T04:57:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26217 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 782336~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.097009016000001, 2024-09-18T04:57:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024526287000000001, 2024-09-18T04:57:23.841 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26217 2024-09-18T04:57:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640140+0000", 2024-09-18T04:57:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2989999999999998e-06 2024-09-18T04:57:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640146+0000", 2024-09-18T04:57:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9569999999999999e-06 2024-09-18T04:57:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:23.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640151+0000", 2024-09-18T04:57:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:57:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664590+0000", 2024-09-18T04:57:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0244389 2024-09-18T04:57:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664628+0000", 2024-09-18T04:57:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7910999999999997e-05 2024-09-18T04:57:23.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664663+0000", 2024-09-18T04:57:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5024999999999998e-05 2024-09-18T04:57:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:23.846 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:23.846 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:23.848 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:57:23.849 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:23.846+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:23.949 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:23.946+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:23.991 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:23.991 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:23.991 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:57:23.991 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:23.999 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:57:24.049 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:24.047+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:24.141 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:24.141 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:24.141 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:57:24.141 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:57:24.141 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:24.149 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:57:24.150 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:24.148+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:24.251 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:24.249+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:24.253 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:24.253 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:57:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:57:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 72.857133906000001, 2024-09-18T04:57:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037718511000000003, 2024-09-18T04:57:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:24.254 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:57:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:57:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-18T04:57:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:57:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:57:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393125+0000", 2024-09-18T04:57:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:57:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393133+0000", 2024-09-18T04:57:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.225e-06 2024-09-18T04:57:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:24.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393144+0000", 2024-09-18T04:57:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1279e-05 2024-09-18T04:57:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393153+0000", 2024-09-18T04:57:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4540000000000007e-06 2024-09-18T04:57:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393203+0000", 2024-09-18T04:57:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9963000000000003e-05 2024-09-18T04:57:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:57:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393210+0000", 2024-09-18T04:57:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5689999999999999e-06 2024-09-18T04:57:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427399+0000", 2024-09-18T04:57:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034188784999999999 2024-09-18T04:57:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427532+0000", 2024-09-18T04:57:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013264700000000001 2024-09-18T04:57:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:57:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427685+0000", 2024-09-18T04:57:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015300599999999999 2024-09-18T04:57:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.428520+0000", 2024-09-18T04:57:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00083502499999999998 2024-09-18T04:57:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:57:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430772+0000", 2024-09-18T04:57:24.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022518880000000001 2024-09-18T04:57:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:57:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430817+0000", 2024-09-18T04:57:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5077000000000002e-05 2024-09-18T04:57:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430845+0000", 2024-09-18T04:57:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8036999999999999e-05 2024-09-18T04:57:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:24.261 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54066 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3469312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221875+0000", 2024-09-18T04:57:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.028385370000002, 2024-09-18T04:57:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029331864999999999, 2024-09-18T04:57:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54066 2024-09-18T04:57:24.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221875+0000", 2024-09-18T04:57:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221875+0000", 2024-09-18T04:57:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221874+0000", 2024-09-18T04:57:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221877+0000", 2024-09-18T04:57:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1280000000000002e-06 2024-09-18T04:57:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221883+0000", 2024-09-18T04:57:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7109999999999998e-06 2024-09-18T04:57:24.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221888+0000", 2024-09-18T04:57:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.668e-06 2024-09-18T04:57:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251030+0000", 2024-09-18T04:57:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029142208999999999 2024-09-18T04:57:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:24.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251138+0000", 2024-09-18T04:57:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000107518 2024-09-18T04:57:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251207+0000", 2024-09-18T04:57:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8978000000000003e-05 2024-09-18T04:57:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54067 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3484672~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221893+0000", 2024-09-18T04:57:24.266 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.028367224999997, 2024-09-18T04:57:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029508817999999999, 2024-09-18T04:57:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54067 2024-09-18T04:57:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:24.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221893+0000", 2024-09-18T04:57:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221893+0000", 2024-09-18T04:57:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:24.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221893+0000", 2024-09-18T04:57:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221895+0000", 2024-09-18T04:57:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9709999999999998e-06 2024-09-18T04:57:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221902+0000", 2024-09-18T04:57:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4379999999999998e-06 2024-09-18T04:57:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221907+0000", 2024-09-18T04:57:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0019999999999997e-06 2024-09-18T04:57:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251260+0000", 2024-09-18T04:57:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029353338999999999 2024-09-18T04:57:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251336+0000", 2024-09-18T04:57:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5591000000000002e-05 2024-09-18T04:57:24.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251402+0000", 2024-09-18T04:57:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5752000000000003e-05 2024-09-18T04:57:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54068 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3500032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221912+0000", 2024-09-18T04:57:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.028348536000003, 2024-09-18T04:57:24.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029654553, 2024-09-18T04:57:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54068 2024-09-18T04:57:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221912+0000", 2024-09-18T04:57:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221912+0000", 2024-09-18T04:57:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221911+0000", 2024-09-18T04:57:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221916+0000", 2024-09-18T04:57:24.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0130000000000001e-06 2024-09-18T04:57:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221922+0000", 2024-09-18T04:57:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.105e-06 2024-09-18T04:57:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221928+0000", 2024-09-18T04:57:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0430000000000004e-06 2024-09-18T04:57:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251427+0000", 2024-09-18T04:57:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029498586 2024-09-18T04:57:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251495+0000", 2024-09-18T04:57:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8125000000000003e-05 2024-09-18T04:57:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251566+0000", 2024-09-18T04:57:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1145999999999999e-05 2024-09-18T04:57:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.275 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54069 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3515392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221945+0000", 2024-09-18T04:57:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.028314926999997, 2024-09-18T04:57:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029786185999999999, 2024-09-18T04:57:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54069 2024-09-18T04:57:24.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221945+0000", 2024-09-18T04:57:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221945+0000", 2024-09-18T04:57:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221945+0000", 2024-09-18T04:57:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221947+0000", 2024-09-18T04:57:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6189999999999998e-06 2024-09-18T04:57:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221955+0000", 2024-09-18T04:57:24.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6249999999999998e-06 2024-09-18T04:57:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221968+0000", 2024-09-18T04:57:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2846e-05 2024-09-18T04:57:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251595+0000", 2024-09-18T04:57:24.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029627158000000001 2024-09-18T04:57:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251666+0000", 2024-09-18T04:57:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1437000000000001e-05 2024-09-18T04:57:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251731+0000", 2024-09-18T04:57:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5071000000000001e-05 2024-09-18T04:57:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54070 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221973+0000", 2024-09-18T04:57:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.028286944000001, 2024-09-18T04:57:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029891219, 2024-09-18T04:57:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54070 2024-09-18T04:57:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:24.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221973+0000", 2024-09-18T04:57:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221973+0000", 2024-09-18T04:57:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221973+0000", 2024-09-18T04:57:24.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221985+0000", 2024-09-18T04:57:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1888999999999999e-05 2024-09-18T04:57:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221992+0000", 2024-09-18T04:57:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2830000000000004e-06 2024-09-18T04:57:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222008+0000", 2024-09-18T04:57:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5608999999999999e-05 2024-09-18T04:57:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251757+0000", 2024-09-18T04:57:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029749652000000001 2024-09-18T04:57:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251826+0000", 2024-09-18T04:57:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9103000000000006e-05 2024-09-18T04:57:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251864+0000", 2024-09-18T04:57:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8179000000000002e-05 2024-09-18T04:57:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54071 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3546112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222014+0000", 2024-09-18T04:57:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.028246512999999, 2024-09-18T04:57:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03002749, 2024-09-18T04:57:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:24.285 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54071 2024-09-18T04:57:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222014+0000", 2024-09-18T04:57:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:24.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222014+0000", 2024-09-18T04:57:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222013+0000", 2024-09-18T04:57:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222016+0000", 2024-09-18T04:57:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6639999999999998e-06 2024-09-18T04:57:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222023+0000", 2024-09-18T04:57:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.849e-06 2024-09-18T04:57:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222028+0000", 2024-09-18T04:57:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0030000000000002e-06 2024-09-18T04:57:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251895+0000", 2024-09-18T04:57:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029867482000000001 2024-09-18T04:57:24.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251963+0000", 2024-09-18T04:57:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8038000000000001e-05 2024-09-18T04:57:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252041+0000", 2024-09-18T04:57:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8028999999999994e-05 2024-09-18T04:57:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.289 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54072 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3561472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222034+0000", 2024-09-18T04:57:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.028226443000001, 2024-09-18T04:57:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030190214, 2024-09-18T04:57:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54072 2024-09-18T04:57:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:24.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222034+0000", 2024-09-18T04:57:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222034+0000", 2024-09-18T04:57:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222033+0000", 2024-09-18T04:57:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222035+0000", 2024-09-18T04:57:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0490000000000002e-06 2024-09-18T04:57:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222051+0000", 2024-09-18T04:57:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5914e-05 2024-09-18T04:57:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222057+0000", 2024-09-18T04:57:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4269999999999999e-06 2024-09-18T04:57:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252079+0000", 2024-09-18T04:57:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030022338999999999 2024-09-18T04:57:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252157+0000", 2024-09-18T04:57:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7459999999999994e-05 2024-09-18T04:57:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252224+0000", 2024-09-18T04:57:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7415000000000004e-05 2024-09-18T04:57:24.293 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54073 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3576832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222062+0000", 2024-09-18T04:57:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.028198666000002, 2024-09-18T04:57:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030280892, 2024-09-18T04:57:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54073 2024-09-18T04:57:24.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222062+0000", 2024-09-18T04:57:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222062+0000", 2024-09-18T04:57:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:24.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222061+0000", 2024-09-18T04:57:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222063+0000", 2024-09-18T04:57:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0729999999999999e-06 2024-09-18T04:57:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222070+0000", 2024-09-18T04:57:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3199999999999996e-06 2024-09-18T04:57:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222074+0000", 2024-09-18T04:57:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8659999999999997e-06 2024-09-18T04:57:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252255+0000", 2024-09-18T04:57:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030180350000000002 2024-09-18T04:57:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252305+0000", 2024-09-18T04:57:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0086000000000001e-05 2024-09-18T04:57:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252342+0000", 2024-09-18T04:57:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7594e-05 2024-09-18T04:57:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54074 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3592192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222079+0000", 2024-09-18T04:57:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.028181275000001, 2024-09-18T04:57:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030370844000000001, 2024-09-18T04:57:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:24.298 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54074 2024-09-18T04:57:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222079+0000", 2024-09-18T04:57:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222079+0000", 2024-09-18T04:57:24.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222079+0000", 2024-09-18T04:57:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222081+0000", 2024-09-18T04:57:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0820000000000001e-06 2024-09-18T04:57:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222087+0000", 2024-09-18T04:57:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0789999999999997e-06 2024-09-18T04:57:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222097+0000", 2024-09-18T04:57:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0446e-05 2024-09-18T04:57:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252365+0000", 2024-09-18T04:57:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030267544 2024-09-18T04:57:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252402+0000", 2024-09-18T04:57:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7117999999999999e-05 2024-09-18T04:57:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252450+0000", 2024-09-18T04:57:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7930999999999999e-05 2024-09-18T04:57:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.302 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54075 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3607552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222103+0000", 2024-09-18T04:57:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.028157223000001, 2024-09-18T04:57:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030531175000000001, 2024-09-18T04:57:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54075 2024-09-18T04:57:24.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222103+0000", 2024-09-18T04:57:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222103+0000", 2024-09-18T04:57:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:24.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222101+0000", 2024-09-18T04:57:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:57:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222105+0000", 2024-09-18T04:57:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4089999999999999e-06 2024-09-18T04:57:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222111+0000", 2024-09-18T04:57:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3720000000000001e-06 2024-09-18T04:57:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222116+0000", 2024-09-18T04:57:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.544e-06 2024-09-18T04:57:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252475+0000", 2024-09-18T04:57:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030358975999999999 2024-09-18T04:57:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252562+0000", 2024-09-18T04:57:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7257999999999997e-05 2024-09-18T04:57:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252634+0000", 2024-09-18T04:57:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2144999999999995e-05 2024-09-18T04:57:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54076 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3622912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222121+0000", 2024-09-18T04:57:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.028139332000002, 2024-09-18T04:57:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030699046000000001, 2024-09-18T04:57:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:24.307 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54076 2024-09-18T04:57:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222121+0000", 2024-09-18T04:57:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:24.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222121+0000", 2024-09-18T04:57:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222120+0000", 2024-09-18T04:57:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222123+0000", 2024-09-18T04:57:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2199999999999999e-06 2024-09-18T04:57:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:24.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222129+0000", 2024-09-18T04:57:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5250000000000002e-06 2024-09-18T04:57:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222134+0000", 2024-09-18T04:57:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8180000000000003e-06 2024-09-18T04:57:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252677+0000", 2024-09-18T04:57:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030542969999999999 2024-09-18T04:57:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252750+0000", 2024-09-18T04:57:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2601000000000005e-05 2024-09-18T04:57:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252820+0000", 2024-09-18T04:57:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0409999999999998e-05 2024-09-18T04:57:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54077 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3638272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:24.311 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222138+0000", 2024-09-18T04:57:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.028121831999997, 2024-09-18T04:57:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031008865, 2024-09-18T04:57:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54077 2024-09-18T04:57:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222138+0000", 2024-09-18T04:57:24.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222138+0000", 2024-09-18T04:57:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222138+0000", 2024-09-18T04:57:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:24.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222140+0000", 2024-09-18T04:57:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9700000000000002e-06 2024-09-18T04:57:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222146+0000", 2024-09-18T04:57:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1460000000000001e-06 2024-09-18T04:57:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222151+0000", 2024-09-18T04:57:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.763e-06 2024-09-18T04:57:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252989+0000", 2024-09-18T04:57:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030838580000000001 2024-09-18T04:57:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253072+0000", 2024-09-18T04:57:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2204000000000001e-05 2024-09-18T04:57:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253147+0000", 2024-09-18T04:57:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5661000000000006e-05 2024-09-18T04:57:24.315 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54078 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222155+0000", 2024-09-18T04:57:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.028105246000003, 2024-09-18T04:57:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031175118000000002, 2024-09-18T04:57:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.316 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54078 2024-09-18T04:57:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222155+0000", 2024-09-18T04:57:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222155+0000", 2024-09-18T04:57:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222155+0000", 2024-09-18T04:57:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222157+0000", 2024-09-18T04:57:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0660000000000002e-06 2024-09-18T04:57:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222163+0000", 2024-09-18T04:57:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2820000000000002e-06 2024-09-18T04:57:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222168+0000", 2024-09-18T04:57:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7979999999999996e-06 2024-09-18T04:57:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253174+0000", 2024-09-18T04:57:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031005721999999999 2024-09-18T04:57:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253251+0000", 2024-09-18T04:57:24.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7717000000000001e-05 2024-09-18T04:57:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253330+0000", 2024-09-18T04:57:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8858999999999998e-05 2024-09-18T04:57:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54079 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3668992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222172+0000", 2024-09-18T04:57:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.028088095999998, 2024-09-18T04:57:24.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031327661, 2024-09-18T04:57:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54079 2024-09-18T04:57:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222172+0000", 2024-09-18T04:57:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222172+0000", 2024-09-18T04:57:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222172+0000", 2024-09-18T04:57:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222174+0000", 2024-09-18T04:57:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0159999999999998e-06 2024-09-18T04:57:24.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222180+0000", 2024-09-18T04:57:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9950000000000002e-06 2024-09-18T04:57:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222185+0000", 2024-09-18T04:57:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7679999999999999e-06 2024-09-18T04:57:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253360+0000", 2024-09-18T04:57:24.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031175423000000001 2024-09-18T04:57:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253430+0000", 2024-09-18T04:57:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9929000000000001e-05 2024-09-18T04:57:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253500+0000", 2024-09-18T04:57:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9896999999999998e-05 2024-09-18T04:57:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54332 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3623936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.229403+0000", 2024-09-18T04:57:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.020857413000002, 2024-09-18T04:57:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029249332999999999, 2024-09-18T04:57:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54332 2024-09-18T04:57:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:24.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229403+0000", 2024-09-18T04:57:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229403+0000", 2024-09-18T04:57:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229402+0000", 2024-09-18T04:57:24.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229405+0000", 2024-09-18T04:57:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0150000000000002e-06 2024-09-18T04:57:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229412+0000", 2024-09-18T04:57:24.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6260000000000003e-06 2024-09-18T04:57:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229417+0000", 2024-09-18T04:57:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6199999999999998e-06 2024-09-18T04:57:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258523+0000", 2024-09-18T04:57:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029106396999999999 2024-09-18T04:57:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:24.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258589+0000", 2024-09-18T04:57:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6054000000000001e-05 2024-09-18T04:57:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258652+0000", 2024-09-18T04:57:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2917999999999996e-05 2024-09-18T04:57:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54333 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3639296~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:24.331 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.229446+0000", 2024-09-18T04:57:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.020814414999997, 2024-09-18T04:57:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029358565, 2024-09-18T04:57:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54333 2024-09-18T04:57:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:24.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229446+0000", 2024-09-18T04:57:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229446+0000", 2024-09-18T04:57:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229445+0000", 2024-09-18T04:57:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229448+0000", 2024-09-18T04:57:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8200000000000001e-06 2024-09-18T04:57:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229456+0000", 2024-09-18T04:57:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.371e-06 2024-09-18T04:57:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229473+0000", 2024-09-18T04:57:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6595e-05 2024-09-18T04:57:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258677+0000", 2024-09-18T04:57:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029203863 2024-09-18T04:57:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258742+0000", 2024-09-18T04:57:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5273999999999999e-05 2024-09-18T04:57:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258804+0000", 2024-09-18T04:57:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2176999999999998e-05 2024-09-18T04:57:24.335 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.336 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:24.336 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.336 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54334 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3654656~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:24.336 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.229481+0000", 2024-09-18T04:57:24.336 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 34.020779134000001, 2024-09-18T04:57:24.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029474845999999999, 2024-09-18T04:57:24.336 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:24.336 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:24.336 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:24.336 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:24.337 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.337 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54334 2024-09-18T04:57:24.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.337 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:24.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:24.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229481+0000", 2024-09-18T04:57:24.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:24.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229481+0000", 2024-09-18T04:57:24.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:24.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229481+0000", 2024-09-18T04:57:24.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:24.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229483+0000", 2024-09-18T04:57:24.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7580000000000001e-06 2024-09-18T04:57:24.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:24.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229490+0000", 2024-09-18T04:57:24.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7020000000000004e-06 2024-09-18T04:57:24.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.229495+0000", 2024-09-18T04:57:24.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7770000000000002e-06 2024-09-18T04:57:24.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:24.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258830+0000", 2024-09-18T04:57:24.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029334828 2024-09-18T04:57:24.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:24.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258896+0000", 2024-09-18T04:57:24.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6384999999999995e-05 2024-09-18T04:57:24.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:24.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.258956+0000", 2024-09-18T04:57:24.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9889999999999997e-05 2024-09-18T04:57:24.340 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.340 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:24.340 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.340 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:62520 3.13 3.84cdb893 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T04:57:24.341 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:24.341 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.952824022999998, 2024-09-18T04:57:24.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94452134799999998, 2024-09-18T04:57:24.341 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:24.341 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:57:24.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:24.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:24.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.341 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 62520 2024-09-18T04:57:24.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.341 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:24.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:24.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:24.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:24.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:24.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:24.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:24.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:24.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297441+0000", 2024-09-18T04:57:24.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9840000000000001e-06 2024-09-18T04:57:24.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:24.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297449+0000", 2024-09-18T04:57:24.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3999999999999992e-06 2024-09-18T04:57:24.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297455+0000", 2024-09-18T04:57:24.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3350000000000003e-06 2024-09-18T04:57:24.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:24.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297552+0000", 2024-09-18T04:57:24.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6602999999999996e-05 2024-09-18T04:57:24.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:57:24.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297559+0000", 2024-09-18T04:57:24.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.52e-06 2024-09-18T04:57:24.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:24.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241943+0000", 2024-09-18T04:57:24.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94438357100000003 2024-09-18T04:57:24.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:24.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241958+0000", 2024-09-18T04:57:24.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4598e-05 2024-09-18T04:57:24.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.345 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:24.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.345 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:113179 3.13 3:cf138017:::rbd_data.11ac5f86a3f9.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T04:57:24.345 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:57:24.346 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.5630336119999999, 2024-09-18T04:57:24.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036508304999999998, 2024-09-18T04:57:24.346 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:24.346 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:24.346 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:24.346 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:24.346 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.346 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 113179 2024-09-18T04:57:24.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.346 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:24.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:24.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:57:24.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:57:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687226+0000", 2024-09-18T04:57:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:57:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688169+0000", 2024-09-18T04:57:24.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00094328200000000004 2024-09-18T04:57:24.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:24.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688182+0000", 2024-09-18T04:57:24.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2756e-05 2024-09-18T04:57:24.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688188+0000", 2024-09-18T04:57:24.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0290000000000002e-06 2024-09-18T04:57:24.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:24.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688225+0000", 2024-09-18T04:57:24.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7301000000000001e-05 2024-09-18T04:57:24.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:24.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688324+0000", 2024-09-18T04:57:24.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9419999999999993e-05 2024-09-18T04:57:24.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:57:24.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.700483+0000", 2024-09-18T04:57:24.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012158771000000001 2024-09-18T04:57:24.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:57:24.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723586+0000", 2024-09-18T04:57:24.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023102952 2024-09-18T04:57:24.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:24.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723634+0000", 2024-09-18T04:57:24.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7468e-05 2024-09-18T04:57:24.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:57:24.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723693+0000", 2024-09-18T04:57:24.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9539e-05 2024-09-18T04:57:24.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:24.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:24.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:24.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723735+0000", 2024-09-18T04:57:24.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1829000000000002e-05 2024-09-18T04:57:24.351 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.351 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:24.351 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.351 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:24.351 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:24.351 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:24.353 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:24.350+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:24.353 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:57:24.453 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:24.451+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:24.458 INFO:teuthology.orchestra.run.smithi002.stdout:flattening image image_client.0-clone38 2024-09-18T04:57:24.480 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:24.481 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:24.481 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:24.481 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:24.491 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:57:24.554 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:24.552+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:24.606 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-09-18T04:57:24.645 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:24.645 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:24.645 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:24.645 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:24.645 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:24.655 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:24.652+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:24.659 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:57:24.756 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:24.754+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:24.775 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:24.776 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:24.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:24.776 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:24.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.776 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9316 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:24.776 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:24.776 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 49.005133033, 2024-09-18T04:57:24.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044027208999999998, 2024-09-18T04:57:24.776 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:24.776 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:24.776 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:24.776 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:24.777 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.777 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9316 2024-09-18T04:57:24.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.777 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:24.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:24.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:24.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:24.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:24.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:24.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:24.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:24.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767144+0000", 2024-09-18T04:57:24.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3070000000000001e-06 2024-09-18T04:57:24.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:24.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767150+0000", 2024-09-18T04:57:24.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1700000000000002e-06 2024-09-18T04:57:24.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767163+0000", 2024-09-18T04:57:24.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2670999999999999e-05 2024-09-18T04:57:24.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:24.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810995+0000", 2024-09-18T04:57:24.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043832310999999999 2024-09-18T04:57:24.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:24.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811087+0000", 2024-09-18T04:57:24.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1345e-05 2024-09-18T04:57:24.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:24.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811169+0000", 2024-09-18T04:57:24.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2848000000000004e-05 2024-09-18T04:57:24.781 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.781 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:24.781 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.782 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9317 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1728512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:24.782 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:57:24.782 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 49.005107398, 2024-09-18T04:57:24.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044190170000000001, 2024-09-18T04:57:24.782 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:24.782 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:24.782 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:24.782 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:24.782 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.782 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9317 2024-09-18T04:57:24.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.783 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:24.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:24.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:57:24.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:24.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:57:24.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:24.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767167+0000", 2024-09-18T04:57:24.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:24.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767170+0000", 2024-09-18T04:57:24.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7259999999999998e-06 2024-09-18T04:57:24.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:24.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767176+0000", 2024-09-18T04:57:24.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1460000000000001e-06 2024-09-18T04:57:24.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.787 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767182+0000", 2024-09-18T04:57:24.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1020000000000002e-06 2024-09-18T04:57:24.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:24.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811204+0000", 2024-09-18T04:57:24.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044022039999999998 2024-09-18T04:57:24.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:24.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811286+0000", 2024-09-18T04:57:24.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1623000000000001e-05 2024-09-18T04:57:24.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:24.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811358+0000", 2024-09-18T04:57:24.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2125000000000005e-05 2024-09-18T04:57:24.789 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.789 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:24.789 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.790 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9318 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1777664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:24.790 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:24.790 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 49.005088852999997, 2024-09-18T04:57:24.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044380181999999997, 2024-09-18T04:57:24.790 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:24.790 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:24.790 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:24.790 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:24.790 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.790 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9318 2024-09-18T04:57:24.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.791 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:24.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:24.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:24.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:24.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:24.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:24.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:24.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:24.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767189+0000", 2024-09-18T04:57:24.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5909999999999998e-06 2024-09-18T04:57:24.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:24.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767194+0000", 2024-09-18T04:57:24.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7690000000000002e-06 2024-09-18T04:57:24.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767199+0000", 2024-09-18T04:57:24.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189999999999999e-06 2024-09-18T04:57:24.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:24.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811394+0000", 2024-09-18T04:57:24.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044195350000000001 2024-09-18T04:57:24.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:24.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811483+0000", 2024-09-18T04:57:24.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8282999999999995e-05 2024-09-18T04:57:24.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:24.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811567+0000", 2024-09-18T04:57:24.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3855000000000003e-05 2024-09-18T04:57:24.795 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.795 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:24.795 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.795 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9325 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:24.795 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:24.795 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 49.003358315, 2024-09-18T04:57:24.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044063989999999997, 2024-09-18T04:57:24.796 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:24.796 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:24.796 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:24.797 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:24.797 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.797 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9325 2024-09-18T04:57:24.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.797 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:24.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:24.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:24.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:24.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:24.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:24.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:24.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:24.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768919+0000", 2024-09-18T04:57:24.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:57:24.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:24.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768925+0000", 2024-09-18T04:57:24.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2310000000000001e-06 2024-09-18T04:57:24.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768929+0000", 2024-09-18T04:57:24.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4469999999999999e-06 2024-09-18T04:57:24.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:24.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812823+0000", 2024-09-18T04:57:24.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043893975000000002 2024-09-18T04:57:24.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:24.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812908+0000", 2024-09-18T04:57:24.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4862000000000002e-05 2024-09-18T04:57:24.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:24.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812981+0000", 2024-09-18T04:57:24.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2892000000000006e-05 2024-09-18T04:57:24.800 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.800 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:24.800 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.801 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9334 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:24.801 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:57:24.801 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 49.001339528999999, 2024-09-18T04:57:24.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044042853999999999, 2024-09-18T04:57:24.801 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:24.801 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:24.801 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:24.801 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:24.801 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.801 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9334 2024-09-18T04:57:24.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.801 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:24.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:24.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:57:24.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:24.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:57:24.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:24.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770935+0000", 2024-09-18T04:57:24.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:24.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770937+0000", 2024-09-18T04:57:24.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1489999999999999e-06 2024-09-18T04:57:24.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:24.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770940+0000", 2024-09-18T04:57:24.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9569999999999998e-06 2024-09-18T04:57:24.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770945+0000", 2024-09-18T04:57:24.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:57:24.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:24.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814838+0000", 2024-09-18T04:57:24.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043892635999999999 2024-09-18T04:57:24.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:24.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814920+0000", 2024-09-18T04:57:24.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2175000000000005e-05 2024-09-18T04:57:24.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:24.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814979+0000", 2024-09-18T04:57:24.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8372000000000003e-05 2024-09-18T04:57:24.805 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.805 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:24.805 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.805 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9352 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1159168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:24.805 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:57:24.805 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 48.999118351999996, 2024-09-18T04:57:24.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044011269999999998, 2024-09-18T04:57:24.805 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:24.805 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:24.805 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:24.806 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:24.806 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.806 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9352 2024-09-18T04:57:24.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.806 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:24.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:24.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:57:24.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:24.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:57:24.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:24.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773156+0000", 2024-09-18T04:57:24.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:24.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773159+0000", 2024-09-18T04:57:24.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2919999999999998e-06 2024-09-18T04:57:24.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:24.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773165+0000", 2024-09-18T04:57:24.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0580000000000003e-06 2024-09-18T04:57:24.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773177+0000", 2024-09-18T04:57:24.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.204e-05 2024-09-18T04:57:24.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:24.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817082+0000", 2024-09-18T04:57:24.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043905528999999999 2024-09-18T04:57:24.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:24.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817122+0000", 2024-09-18T04:57:24.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9258999999999998e-05 2024-09-18T04:57:24.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:24.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817168+0000", 2024-09-18T04:57:24.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6545000000000002e-05 2024-09-18T04:57:24.809 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.809 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:24.809 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.809 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9353 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1208320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:24.809 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:24.809 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 48.999094053999997, 2024-09-18T04:57:24.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044105537, 2024-09-18T04:57:24.809 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:24.810 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:24.810 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:24.810 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:24.810 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.810 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9353 2024-09-18T04:57:24.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.810 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:24.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:24.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:24.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:24.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:24.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:24.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:24.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:24.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773183+0000", 2024-09-18T04:57:24.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2929999999999999e-06 2024-09-18T04:57:24.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:24.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773190+0000", 2024-09-18T04:57:24.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4330000000000001e-06 2024-09-18T04:57:24.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773194+0000", 2024-09-18T04:57:24.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.865e-06 2024-09-18T04:57:24.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:24.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817196+0000", 2024-09-18T04:57:24.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044001708 2024-09-18T04:57:24.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:24.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817236+0000", 2024-09-18T04:57:24.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0043999999999998e-05 2024-09-18T04:57:24.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:24.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817287+0000", 2024-09-18T04:57:24.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0605000000000001e-05 2024-09-18T04:57:24.813 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.813 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:24.813 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.814 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9354 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:24.814 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:57:24.814 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 48.999076543000001, 2024-09-18T04:57:24.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044199796, 2024-09-18T04:57:24.814 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:24.814 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:24.814 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:24.814 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:24.814 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.814 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9354 2024-09-18T04:57:24.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.815 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:24.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:24.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:57:24.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:24.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:57:24.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:24.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773198+0000", 2024-09-18T04:57:24.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:24.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773200+0000", 2024-09-18T04:57:24.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4159999999999998e-06 2024-09-18T04:57:24.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:24.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773206+0000", 2024-09-18T04:57:24.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.789e-06 2024-09-18T04:57:24.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773210+0000", 2024-09-18T04:57:24.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2679999999999996e-06 2024-09-18T04:57:24.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:24.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817314+0000", 2024-09-18T04:57:24.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044103187000000002 2024-09-18T04:57:24.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:24.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817353+0000", 2024-09-18T04:57:24.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8973000000000001e-05 2024-09-18T04:57:24.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:24.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817399+0000", 2024-09-18T04:57:24.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5877999999999997e-05 2024-09-18T04:57:24.818 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.818 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:24.818 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.819 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46591 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2097152~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:24.819 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:24.819 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.074220126, 2024-09-18T04:57:24.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043987310000000002, 2024-09-18T04:57:24.819 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:24.819 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:24.819 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:24.819 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:24.820 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.820 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46591 2024-09-18T04:57:24.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.820 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:24.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:24.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:24.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:24.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:24.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:24.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:24.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:24.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698057+0000", 2024-09-18T04:57:24.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.531e-06 2024-09-18T04:57:24.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:24.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698064+0000", 2024-09-18T04:57:24.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-18T04:57:24.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698069+0000", 2024-09-18T04:57:24.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.091e-06 2024-09-18T04:57:24.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:24.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741843+0000", 2024-09-18T04:57:24.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043773470000000002 2024-09-18T04:57:24.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:24.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741928+0000", 2024-09-18T04:57:24.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5113999999999999e-05 2024-09-18T04:57:24.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:24.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742042+0000", 2024-09-18T04:57:24.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114527 2024-09-18T04:57:24.825 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.825 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:24.825 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.825 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46592 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2121728~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:24.825 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:57:24.825 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.074201699, 2024-09-18T04:57:24.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044077948999999998, 2024-09-18T04:57:24.825 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:24.826 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:24.826 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:24.826 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:24.826 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.826 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46592 2024-09-18T04:57:24.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.826 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:24.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:24.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:57:24.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:24.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:57:24.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:24.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698073+0000", 2024-09-18T04:57:24.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:24.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698076+0000", 2024-09-18T04:57:24.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.481e-06 2024-09-18T04:57:24.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:24.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698082+0000", 2024-09-18T04:57:24.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8730000000000004e-06 2024-09-18T04:57:24.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698086+0000", 2024-09-18T04:57:24.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:57:24.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:24.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742055+0000", 2024-09-18T04:57:24.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043969170000000002 2024-09-18T04:57:24.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:24.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742093+0000", 2024-09-18T04:57:24.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7707000000000003e-05 2024-09-18T04:57:24.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:24.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742152+0000", 2024-09-18T04:57:24.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8323999999999999e-05 2024-09-18T04:57:24.830 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.830 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:24.830 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.830 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46593 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2211840~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:24.830 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:24.830 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.074184795999997, 2024-09-18T04:57:24.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044259917000000003, 2024-09-18T04:57:24.831 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:24.831 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:24.831 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:24.831 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:24.831 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.831 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46593 2024-09-18T04:57:24.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.831 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:24.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:24.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:24.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:24.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:24.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:24.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:24.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:24.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698092+0000", 2024-09-18T04:57:24.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2249999999999999e-06 2024-09-18T04:57:24.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:24.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698098+0000", 2024-09-18T04:57:24.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:57:24.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698103+0000", 2024-09-18T04:57:24.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:57:24.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:24.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742182+0000", 2024-09-18T04:57:24.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044079508000000003 2024-09-18T04:57:24.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:24.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742257+0000", 2024-09-18T04:57:24.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4678000000000006e-05 2024-09-18T04:57:24.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:24.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742350+0000", 2024-09-18T04:57:24.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3531999999999998e-05 2024-09-18T04:57:24.835 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.835 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:24.835 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.835 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46594 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2236416~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:24.835 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:24.835 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.074169099000002, 2024-09-18T04:57:24.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044378984000000003, 2024-09-18T04:57:24.835 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:24.835 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:24.835 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:24.835 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:24.836 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.836 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46594 2024-09-18T04:57:24.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.836 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:24.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:24.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:24.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:24.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:24.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:24.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:24.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:24.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698108+0000", 2024-09-18T04:57:24.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0880000000000002e-06 2024-09-18T04:57:24.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:24.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698114+0000", 2024-09-18T04:57:24.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6269999999999997e-06 2024-09-18T04:57:24.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698118+0000", 2024-09-18T04:57:24.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4050000000000002e-06 2024-09-18T04:57:24.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:24.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742365+0000", 2024-09-18T04:57:24.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044246799000000003 2024-09-18T04:57:24.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:24.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742407+0000", 2024-09-18T04:57:24.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2008000000000002e-05 2024-09-18T04:57:24.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:24.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742485+0000", 2024-09-18T04:57:24.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8405999999999994e-05 2024-09-18T04:57:24.840 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.840 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:24.840 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.840 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46595 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:24.840 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:24.840 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.074153025999998, 2024-09-18T04:57:24.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.04440463, 2024-09-18T04:57:24.840 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:24.840 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:24.840 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:24.841 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:24.841 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.841 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46595 2024-09-18T04:57:24.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.841 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:24.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:24.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:24.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:24.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:24.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:24.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:24.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:24.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698124+0000", 2024-09-18T04:57:24.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1840000000000002e-06 2024-09-18T04:57:24.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:24.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698130+0000", 2024-09-18T04:57:24.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.947e-06 2024-09-18T04:57:24.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698136+0000", 2024-09-18T04:57:24.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6520000000000003e-06 2024-09-18T04:57:24.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:24.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742521+0000", 2024-09-18T04:57:24.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044385446000000002 2024-09-18T04:57:24.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:24.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742527+0000", 2024-09-18T04:57:24.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:57:24.845 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.845 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:24.845 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.845 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46596 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:24.845 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:24.845 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.074134782000002, 2024-09-18T04:57:24.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044418066999999999, 2024-09-18T04:57:24.845 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:24.845 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:24.846 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:24.846 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:24.846 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.846 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46596 2024-09-18T04:57:24.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.846 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:24.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:24.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:24.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:24.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:24.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:24.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:24.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:24.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698146+0000", 2024-09-18T04:57:24.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2779999999999999e-06 2024-09-18T04:57:24.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:24.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698153+0000", 2024-09-18T04:57:24.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3849999999999999e-06 2024-09-18T04:57:24.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698164+0000", 2024-09-18T04:57:24.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1375999999999999e-05 2024-09-18T04:57:24.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:24.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742554+0000", 2024-09-18T04:57:24.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044389482000000001 2024-09-18T04:57:24.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:24.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742559+0000", 2024-09-18T04:57:24.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9540000000000003e-06 2024-09-18T04:57:24.850 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.850 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:24.850 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.850 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46597 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:24.850 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:57:24.850 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.074106417000003, 2024-09-18T04:57:24.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044416513999999997, 2024-09-18T04:57:24.851 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:24.851 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:24.851 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:24.851 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:24.851 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.851 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46597 2024-09-18T04:57:24.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.851 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:24.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:24.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:57:24.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:24.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:57:24.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:24.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698168+0000", 2024-09-18T04:57:24.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:24.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698175+0000", 2024-09-18T04:57:24.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4520000000000002e-06 2024-09-18T04:57:24.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:24.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698181+0000", 2024-09-18T04:57:24.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4150000000000004e-06 2024-09-18T04:57:24.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698186+0000", 2024-09-18T04:57:24.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7489999999999998e-06 2024-09-18T04:57:24.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:24.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742580+0000", 2024-09-18T04:57:24.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044394325999999998 2024-09-18T04:57:24.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:24.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742585+0000", 2024-09-18T04:57:24.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9849999999999997e-06 2024-09-18T04:57:24.856 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.856 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:24.856 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.856 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46598 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:24.856 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:57:24.856 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 35.073998940999999, 2024-09-18T04:57:24.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044335316999999999, 2024-09-18T04:57:24.856 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:24.857 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:24.857 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:24.857 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:24.857 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.857 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46598 2024-09-18T04:57:24.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.857 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:24.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:24.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:57:24.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:24.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:57:24.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:24.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698190+0000", 2024-09-18T04:57:24.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999137 2024-09-18T04:57:24.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:24.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698280+0000", 2024-09-18T04:57:24.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0409999999999997e-05 2024-09-18T04:57:24.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:24.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698289+0000", 2024-09-18T04:57:24.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9490000000000002e-06 2024-09-18T04:57:24.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698296+0000", 2024-09-18T04:57:24.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2670000000000003e-06 2024-09-18T04:57:24.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:24.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742607+0000", 2024-09-18T04:57:24.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044311072999999999 2024-09-18T04:57:24.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:24.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742612+0000", 2024-09-18T04:57:24.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9400000000000001e-06 2024-09-18T04:57:24.860 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.860 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:24.860 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.861 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57986 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3086336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:24.861 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:24.861 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.087153217999997, 2024-09-18T04:57:24.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044047731999999999, 2024-09-18T04:57:24.861 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:24.861 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:24.861 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:24.861 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:24.861 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.861 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57986 2024-09-18T04:57:24.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.862 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:24.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:24.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:24.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:24.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:24.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:24.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:24.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:24.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685124+0000", 2024-09-18T04:57:24.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2859999999999998e-06 2024-09-18T04:57:24.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:24.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685130+0000", 2024-09-18T04:57:24.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5400000000000001e-06 2024-09-18T04:57:24.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685136+0000", 2024-09-18T04:57:24.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.152e-06 2024-09-18T04:57:24.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:24.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.728976+0000", 2024-09-18T04:57:24.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043840037999999998 2024-09-18T04:57:24.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:24.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729064+0000", 2024-09-18T04:57:24.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8709999999999996e-05 2024-09-18T04:57:24.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:24.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729170+0000", 2024-09-18T04:57:24.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000105386 2024-09-18T04:57:24.865 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.865 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:24.865 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.865 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57987 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3101696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:24.865 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:57:24.866 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.087104242999999, 2024-09-18T04:57:24.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044238123999999997, 2024-09-18T04:57:24.866 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:24.866 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:24.866 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:24.866 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:24.866 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.866 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57987 2024-09-18T04:57:24.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.866 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:24.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:24.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:57:24.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:24.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:57:24.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:24.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685170+0000", 2024-09-18T04:57:24.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:24.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:24.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685175+0000", 2024-09-18T04:57:24.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9069999999999998e-06 2024-09-18T04:57:24.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:24.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685187+0000", 2024-09-18T04:57:24.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.189e-05 2024-09-18T04:57:24.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685195+0000", 2024-09-18T04:57:24.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9459999999999998e-06 2024-09-18T04:57:24.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:24.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729212+0000", 2024-09-18T04:57:24.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044017239 2024-09-18T04:57:24.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:24.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729301+0000", 2024-09-18T04:57:24.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8887999999999993e-05 2024-09-18T04:57:24.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:24.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729409+0000", 2024-09-18T04:57:24.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108119 2024-09-18T04:57:24.870 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.870 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:24.870 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.871 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57988 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3117056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:24.871 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:57:24.871 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.087062123999999, 2024-09-18T04:57:24.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044429835000000001, 2024-09-18T04:57:24.871 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:24.871 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:24.871 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:24.871 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:24.871 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.872 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57988 2024-09-18T04:57:24.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.872 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:24.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:24.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:57:24.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:24.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:57:24.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:24.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685212+0000", 2024-09-18T04:57:24.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:24.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685216+0000", 2024-09-18T04:57:24.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7390000000000001e-06 2024-09-18T04:57:24.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:24.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685224+0000", 2024-09-18T04:57:24.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2260000000000005e-06 2024-09-18T04:57:24.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685230+0000", 2024-09-18T04:57:24.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0299999999999999e-06 2024-09-18T04:57:24.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:24.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729454+0000", 2024-09-18T04:57:24.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044223248999999999 2024-09-18T04:57:24.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:24.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729539+0000", 2024-09-18T04:57:24.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5029000000000002e-05 2024-09-18T04:57:24.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:24.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729643+0000", 2024-09-18T04:57:24.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104267 2024-09-18T04:57:24.876 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.876 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:24.876 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.876 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57989 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3132416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:24.876 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:57:24.876 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.087039372, 2024-09-18T04:57:24.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044642881000000002, 2024-09-18T04:57:24.877 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:24.877 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:24.877 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:24.877 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:24.877 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:24.877 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57989 2024-09-18T04:57:24.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.877 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:24.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:24.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:57:24.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:24.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:57:24.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:24.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:24.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685235+0000", 2024-09-18T04:57:24.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:24.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:24.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685238+0000", 2024-09-18T04:57:24.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.667e-06 2024-09-18T04:57:24.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:24.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685245+0000", 2024-09-18T04:57:24.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7909999999999999e-06 2024-09-18T04:57:24.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:24.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685250+0000", 2024-09-18T04:57:24.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3730000000000001e-06 2024-09-18T04:57:24.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:24.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729685+0000", 2024-09-18T04:57:24.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044434552000000002 2024-09-18T04:57:24.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:24.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729775+0000", 2024-09-18T04:57:24.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0395000000000004e-05 2024-09-18T04:57:24.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:24.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:24.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:24.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729879+0000", 2024-09-18T04:57:24.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010361099999999999 2024-09-18T04:57:24.881 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.881 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:24.881 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.881 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:24.881 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:24.881 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:24.883 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:57:24.884 INFO:teuthology.orchestra.run.smithi002.stdout:5080 write 0x23c647b thru 0x23ce564 (0x80ea bytes) 2024-09-18T04:57:24.884 INFO:teuthology.orchestra.run.smithi002.stdout:5081 write 0xe7518a5 thru 0xe75269a (0xdf6 bytes) 2024-09-18T04:57:24.884 INFO:teuthology.orchestra.run.smithi002.stdout:5084 write 0x3cc95c8 thru 0x3cd2b02 (0x953b bytes) 2024-09-18T04:57:24.884 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:24.880+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:24.897 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:57:24.933 INFO:teuthology.orchestra.run.smithi002.stdout:5085 write 0x28a185c thru 0x28a4502 (0x2ca7 bytes) 2024-09-18T04:57:24.941 INFO:teuthology.orchestra.run.smithi002.stdout:5086 write 0x5ad8a5a thru 0x5adae89 (0x2430 bytes) 2024-09-18T04:57:24.954 INFO:teuthology.orchestra.run.smithi002.stdout:5087 writesame 0xe74eeef thru 0xe75c0d4 data_size 0x43(0xd1e6 bytes) 2024-09-18T04:57:24.971 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:24.972 INFO:teuthology.orchestra.run.smithi002.stdout:5090 read 0x558c175 thru 0x559282f (0x66bb bytes) 2024-09-18T04:57:24.972 INFO:teuthology.orchestra.run.smithi002.stdout:5093 writesame 0xb75855b thru 0xb75aa1b data_size 0x1(0x24c1 bytes) 2024-09-18T04:57:24.981 INFO:teuthology.orchestra.run.smithi002.stdout:5094 write 0xa3e5791 thru 0xa3e5808 (0x78 bytes) 2024-09-18T04:57:24.983 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:24.981+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:24.985 INFO:teuthology.orchestra.run.smithi002.stdout:5095 write 0xc16a38 thru 0xc1eaa4 (0x806d bytes) 2024-09-18T04:57:24.998 INFO:teuthology.orchestra.run.smithi002.stdout:5096 read 0x5439207 thru 0x5448243 (0xf03d bytes) 2024-09-18T04:57:24.998 INFO:teuthology.orchestra.run.smithi002.stdout:5097 writesame 0xd5f0009 thru 0xd5f5839 data_size 0x1(0x5831 bytes) 2024-09-18T04:57:25.028 INFO:teuthology.orchestra.run.smithi002.stdout:5098 writesame 0x2241b1e thru 0x2244323 data_size 0x2f(0x2806 bytes) 2024-09-18T04:57:25.031 INFO:teuthology.orchestra.run.smithi002.stdout:5099 write 0x49daa4 thru 0x4a7ded (0xa34a bytes) 2024-09-18T04:57:25.066 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:25.066 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:25.066 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:25.066 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:25.078 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:57:25.082 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:57:25.082 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:57:25.085 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:25.082+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:25.103 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:57:25.103 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:57:25.116 INFO:teuthology.orchestra.run.smithi002.stdout:5103 write 0x88cfb0 thru 0x898684 (0xb6d5 bytes) 2024-09-18T04:57:25.122 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:57:25.122 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:57:25.138 INFO:teuthology.orchestra.run.smithi002.stdout:5104 write 0x2b05a1c thru 0x2b07223 (0x1808 bytes) 2024-09-18T04:57:25.143 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:57:25.144 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:57:25.164 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:57:25.164 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:57:25.182 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:57:25.182 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:57:25.184 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:25.182+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:25.197 INFO:teuthology.orchestra.run.smithi002.stdout:5105 writesame 0x7e95d17 thru 0x7e97b03 data_size 0x1(0x1ded bytes) 2024-09-18T04:57:25.202 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:57:25.203 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:57:25.206 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:25.207 INFO:teuthology.orchestra.run.smithi002.stdout:5107 write 0xa591044 thru 0xa597c6d (0x6c2a bytes) 2024-09-18T04:57:25.218 INFO:teuthology.orchestra.run.smithi002.stdout:5108 write 0xe2e7d60 thru 0xe2e9b3f (0x1de0 bytes) 2024-09-18T04:57:25.218 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:25.218 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:25.224 INFO:teuthology.orchestra.run.smithi002.stdout:5109 punch from 0x8e2b429 to 0x8e2d9a5, (0x257c bytes) 2024-09-18T04:57:25.224 INFO:teuthology.orchestra.run.smithi002.stdout:5110 read 0x2fba50a thru 0x2fbe3a7 (0x3e9e bytes) 2024-09-18T04:57:25.225 INFO:teuthology.orchestra.run.smithi002.stdout:5111 read 0x882f659 thru 0x883c533 (0xcedb bytes) 2024-09-18T04:57:25.225 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:57:25.225 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:57:25.225 INFO:teuthology.orchestra.run.smithi002.stdout:5113 write 0x5d73c9e thru 0x5d796d3 (0x5a36 bytes) 2024-09-18T04:57:25.231 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:57:25.239 INFO:teuthology.orchestra.run.smithi002.stdout:5114 read 0xa1b659 thru 0xa260c2 (0xaa6a bytes) 2024-09-18T04:57:25.239 INFO:teuthology.orchestra.run.smithi002.stdout:5116 read 0x721fbf0 thru 0x7227f6c (0x837d bytes) 2024-09-18T04:57:25.239 INFO:teuthology.orchestra.run.smithi002.stdout:5117 punch from 0x1452c5 to 0x1529fa, (0xd735 bytes) 2024-09-18T04:57:25.239 INFO:teuthology.orchestra.run.smithi002.stdout:5118 punch from 0x8f15895 to 0x8f20e00, (0xb56b bytes) 2024-09-18T04:57:25.240 INFO:teuthology.orchestra.run.smithi002.stdout:5119 clone 40 order 18 su 1024 sc 15 2024-09-18T04:57:25.285 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:25.283+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:25.345 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:25.345 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:25.353 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:57:25.394 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:25.392+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:25.405 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T04:57:25.452 INFO:teuthology.orchestra.run.smithi002.stdout:flattening image image_client.0-clone39 2024-09-18T04:57:25.487 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:25.484+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:25.539 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:25.540 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:25.553 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:57:25.588 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:25.586+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:25.690 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:25.687+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:25.721 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:25.722 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:25.731 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:57:25.791 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:25.788+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:25.844 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:25.844 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:25.853 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:57:25.869 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-09-18T04:57:25.892 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:25.890+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:25.974 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:25.974 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:25.984 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:57:25.993 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:25.990+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:26.094 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:26.091+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:26.127 INFO:teuthology.orchestra.run.smithi002.stdout:5120 trunc from 0xe75c0d5 to 0xb14202 2024-09-18T04:57:26.142 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:26.142 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:26.156 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:57:26.197 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:26.192+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:26.282 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:26.282 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:26.292 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:57:26.336 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:26.334+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:26.396 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:26.394+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:26.399 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:26.399 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:26.407 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-18T04:57:26.409 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:57:26.420 INFO:teuthology.orchestra.run.smithi002.stdout:5121 write 0x99d4ed5 thru 0x99d95d3 (0x46ff bytes) 2024-09-18T04:57:26.437 INFO:teuthology.orchestra.run.smithi002.stdout:5122 writesame 0x8164cea thru 0x8174639 data_size 0x8(0xf950 bytes) 2024-09-18T04:57:26.471 INFO:teuthology.orchestra.run.smithi002.stdout:5123 write 0xceccdd1 thru 0xced6715 (0x9945 bytes) 2024-09-18T04:57:26.497 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:26.494+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:26.498 INFO:teuthology.orchestra.run.smithi002.stdout:5124 trunc from 0xced6716 to 0x258775b 2024-09-18T04:57:26.525 INFO:teuthology.orchestra.run.smithi002.stdout:5126 write 0x343db3c thru 0x3444ea5 (0x736a bytes) 2024-09-18T04:57:26.542 INFO:teuthology.orchestra.run.smithi002.stdout:5127 read 0xc93077 thru 0xc9f96a (0xc8f4 bytes) 2024-09-18T04:57:26.545 INFO:teuthology.orchestra.run.smithi002.stdout:5128 trunc from 0x3444ea6 to 0xbb1feb 2024-09-18T04:57:26.564 INFO:teuthology.orchestra.run.smithi002.stdout:5129 trunc from 0xbb1feb to 0x4606e1b 2024-09-18T04:57:26.569 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:26.569 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:26.578 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:57:26.584 INFO:teuthology.orchestra.run.smithi002.stdout:5130 write 0xb54e63b thru 0xb55973f (0xb105 bytes) 2024-09-18T04:57:26.598 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:26.595+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:26.602 INFO:teuthology.orchestra.run.smithi002.stdout:5131 read 0x36ca1a0 thru 0x36d6a0a (0xc86b bytes) 2024-09-18T04:57:26.602 INFO:teuthology.orchestra.run.smithi002.stdout:5132 read 0x711b8ba thru 0x711ece1 (0x3428 bytes) 2024-09-18T04:57:26.602 INFO:teuthology.orchestra.run.smithi002.stdout:5133 read 0xb47718c thru 0xb478177 (0xfec bytes) 2024-09-18T04:57:26.603 INFO:teuthology.orchestra.run.smithi002.stdout:5135 read 0x2182354 thru 0x2186ef7 (0x4ba4 bytes) 2024-09-18T04:57:26.603 INFO:teuthology.orchestra.run.smithi002.stdout:5136 write 0x27a081e thru 0x27ae94e (0xe131 bytes) 2024-09-18T04:57:26.623 INFO:teuthology.orchestra.run.smithi002.stdout:5139 trunc from 0xb559740 to 0x7b26db2 2024-09-18T04:57:26.649 INFO:teuthology.orchestra.run.smithi002.stdout:5140 writesame 0xe4d8421 thru 0xe4dc04a data_size 0x22(0x3c2a bytes) 2024-09-18T04:57:26.649 INFO:teuthology.orchestra.run.smithi002.stdout:5141 trunc from 0xe4dc04b to 0xe98a801 2024-09-18T04:57:26.670 INFO:teuthology.orchestra.run.smithi002.stdout:5143 trunc from 0xe98a801 to 0x945e97 2024-09-18T04:57:26.699 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:26.697+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:26.705 INFO:teuthology.orchestra.run.smithi002.stdout:5144 write 0x88d07cc thru 0x88dd5a1 (0xcdd6 bytes) 2024-09-18T04:57:26.721 INFO:teuthology.orchestra.run.smithi002.stdout:5145 write 0x80c5c60 thru 0x80c8a64 (0x2e05 bytes) 2024-09-18T04:57:26.735 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:26.735 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:26.742 INFO:teuthology.orchestra.run.smithi002.stdout:5146 writesame 0xdd2d1f thru 0xdd342b data_size 0x5(0x70d bytes) 2024-09-18T04:57:26.745 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:57:26.747 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:26.749 INFO:teuthology.orchestra.run.smithi002.stdout:5149 trunc from 0x88dd5a2 to 0x787e9e6 2024-09-18T04:57:26.773 INFO:teuthology.orchestra.run.smithi002.stdout:5150 writesame 0x2a65b91 thru 0x2a72eaf data_size 0x7(0xd31f bytes) 2024-09-18T04:57:26.789 INFO:teuthology.orchestra.run.smithi002.stdout:5153 write 0x3a1a33e thru 0x3a1ca32 (0x26f5 bytes) 2024-09-18T04:57:26.800 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:26.797+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:26.805 INFO:teuthology.orchestra.run.smithi002.stdout:5154 punch from 0x5e43dc1 to 0x5e4d150, (0x938f bytes) 2024-09-18T04:57:26.806 INFO:teuthology.orchestra.run.smithi002.stdout:5155 punch from 0x4e7677f to 0x4e78b4c, (0x23cd bytes) 2024-09-18T04:57:26.806 INFO:teuthology.orchestra.run.smithi002.stdout:5156 writesame 0x76f3633 thru 0x76fac88 data_size 0x99(0x7656 bytes) 2024-09-18T04:57:26.821 INFO:teuthology.orchestra.run.smithi002.stdout:5157 write 0x2964c43 thru 0x297442b (0xf7e9 bytes) 2024-09-18T04:57:26.825 INFO:teuthology.orchestra.run.smithi002.stdout:5158 write 0x50f9206 thru 0x51064bf (0xd2ba bytes) 2024-09-18T04:57:26.847 INFO:teuthology.orchestra.run.smithi002.stdout:5159 trunc from 0x787e9e6 to 0x312a3f7 2024-09-18T04:57:26.872 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:26.872 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:26.875 INFO:teuthology.orchestra.run.smithi002.stdout:5161 trunc from 0x312a3f7 to 0xa364318 2024-09-18T04:57:26.881 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:26.881 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:26.881 INFO:teuthology.orchestra.run.smithi002.stdout:5164 writesame 0xa319803 thru 0xa31e638 data_size 0x6(0x4e36 bytes) 2024-09-18T04:57:26.882 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:57:26.900 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:26.898+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:27.001 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:26.999+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:27.025 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:27.025 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:27.034 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:57:27.097 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:27.103 INFO:teuthology.orchestra.run.smithi002.stdout:5167 trunc from 0xa364318 to 0x5d71ac4 2024-09-18T04:57:27.115 INFO:teuthology.orchestra.run.smithi002.stdout:5169 write 0x1617982 thru 0x1626cd2 (0xf351 bytes) 2024-09-18T04:57:27.118 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:27.116+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:27.131 INFO:teuthology.orchestra.run.smithi002.stdout:5170 writesame 0x310fbc7 thru 0x3116a94 data_size 0xd(0x6ece bytes) 2024-09-18T04:57:27.157 INFO:teuthology.orchestra.run.smithi002.stdout:5171 trunc from 0x5d71ac4 to 0xe5df02b 2024-09-18T04:57:27.163 INFO:teuthology.orchestra.run.smithi002.stdout:5172 writesame 0xbf49351 thru 0xbf52fba data_size 0x1(0x9c6a bytes) 2024-09-18T04:57:27.185 INFO:teuthology.orchestra.run.smithi002.stdout:5174 writesame 0x8d3c0c3 thru 0x8d47dfd data_size 0x1(0xbd3b bytes) 2024-09-18T04:57:27.202 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:27.200+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:27.203 INFO:teuthology.orchestra.run.smithi002.stdout:5175 read 0x8976229 thru 0x89860e3 (0xfebb bytes) 2024-09-18T04:57:27.205 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:27.205 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:27.214 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:57:27.217 INFO:teuthology.orchestra.run.smithi002.stdout:5176 trunc from 0xe5df02b to 0xe05c11 2024-09-18T04:57:27.240 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:57:27.250 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:27.261 INFO:teuthology.orchestra.run.smithi002.stdout:5178 trunc from 0xe05c11 to 0xab8e7d4 2024-09-18T04:57:27.268 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:27.274 INFO:teuthology.orchestra.run.smithi002.stdout:5181 trunc from 0xab8e7d4 to 0x509b2e3 2024-09-18T04:57:27.286 INFO:teuthology.orchestra.run.smithi002.stdout:5182 write 0x51a9a60 thru 0x51b684a (0xcdeb bytes) 2024-09-18T04:57:27.301 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:27.304 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:27.301+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:27.314 INFO:teuthology.orchestra.run.smithi002.stdout:5184 write 0xb1eb4fd thru 0xb1edaa3 (0x25a7 bytes) 2024-09-18T04:57:27.326 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:27.327 INFO:teuthology.orchestra.run.smithi002.stdout:5186 trunc from 0xb1edaa4 to 0xb8d29ac 2024-09-18T04:57:27.332 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:27.333 INFO:teuthology.orchestra.run.smithi002.stdout:5189 read 0x52f176e thru 0x52f5171 (0x3a04 bytes) 2024-09-18T04:57:27.340 INFO:teuthology.orchestra.run.smithi002.stdout:5190 writesame 0xd215804 thru 0xd2206b2 data_size 0x1(0xaeaf bytes) 2024-09-18T04:57:27.340 INFO:teuthology.orchestra.run.smithi002.stdout:5191 punch from 0x1ab0b93 to 0x1ab6e04, (0x6271 bytes) 2024-09-18T04:57:27.340 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:27.340 INFO:teuthology.orchestra.run.smithi002.stdout:5193 read 0xce1dbee thru 0xce21cda (0x40ed bytes) 2024-09-18T04:57:27.340 INFO:teuthology.orchestra.run.smithi002.stdout:5195 read 0x15b3bd1 thru 0x15c2022 (0xe452 bytes) 2024-09-18T04:57:27.341 INFO:teuthology.orchestra.run.smithi002.stdout:5196 read 0xb59a813 thru 0xb59b434 (0xc22 bytes) 2024-09-18T04:57:27.341 INFO:teuthology.orchestra.run.smithi002.stdout:5199 writesame 0x4c2a750 thru 0x4c3857b data_size 0x4(0xde2c bytes) 2024-09-18T04:57:27.359 INFO:teuthology.orchestra.run.smithi002.stdout:5200 trunc from 0xd2206b3 to 0xe3aaabf 2024-09-18T04:57:27.365 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:27.365 INFO:teuthology.orchestra.run.smithi002.stdout:5204 writesame 0x856d4aa thru 0x8577a27 data_size 0x45(0xa57e bytes) 2024-09-18T04:57:27.373 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:27.374 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:27.388 INFO:teuthology.orchestra.run.smithi002.stdout:5205 write 0x8411949 thru 0x841beae (0xa566 bytes) 2024-09-18T04:57:27.388 INFO:teuthology.orchestra.run.smithi002.stdout:5206 punch from 0x2b39831 to 0x2b45fc0, (0xc78f bytes) 2024-09-18T04:57:27.388 INFO:teuthology.orchestra.run.smithi002.stdout:5208 punch from 0xb81396c to 0xb81a17e, (0x6812 bytes) 2024-09-18T04:57:27.388 INFO:teuthology.orchestra.run.smithi002.stdout:5211 punch from 0x4f22af6 to 0x4f3055e, (0xda68 bytes) 2024-09-18T04:57:27.389 INFO:teuthology.orchestra.run.smithi002.stdout:5213 writesame 0x46cc35b thru 0x46d8088 data_size 0x3a(0xbd2e bytes) 2024-09-18T04:57:27.389 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:57:27.390 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:57:27.390 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:57:27.410 INFO:teuthology.orchestra.run.smithi002.stdout:5214 trunc from 0xe3aaabf to 0xb5e6792 2024-09-18T04:57:27.411 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:57:27.411 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:57:27.416 INFO:teuthology.orchestra.run.smithi002.stdout:5215 read 0x8962e11 thru 0x896ff8d (0xd17d bytes) 2024-09-18T04:57:27.416 INFO:teuthology.orchestra.run.smithi002.stdout:5216 write 0x1cc9896 thru 0x1cd5a4d (0xc1b8 bytes) 2024-09-18T04:57:27.431 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:57:27.431 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:57:27.432 INFO:teuthology.orchestra.run.smithi002.stdout:5217 read 0x1a1e77f thru 0x1a29521 (0xada3 bytes) 2024-09-18T04:57:27.432 INFO:teuthology.orchestra.run.smithi002.stdout:5218 punch from 0x57fdd2 to 0x5846fc, (0x492a bytes) 2024-09-18T04:57:27.433 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:27.431+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:27.440 INFO:teuthology.orchestra.run.smithi002.stdout:5220 trunc from 0xb5e6792 to 0x53e1a60 2024-09-18T04:57:27.451 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:57:27.451 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:57:27.454 INFO:teuthology.orchestra.run.smithi002.stdout:5223 punch from 0x9fd8e5 to 0x9febea, (0x1305 bytes) 2024-09-18T04:57:27.455 INFO:teuthology.orchestra.run.smithi002.stdout:5225 punch from 0x2663ffc to 0x2664bca, (0xbce bytes) 2024-09-18T04:57:27.463 INFO:teuthology.orchestra.run.smithi002.stdout:5226 write 0x855203b thru 0x8553519 (0x14df bytes) 2024-09-18T04:57:27.469 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:57:27.469 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:57:27.471 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:27.471 INFO:teuthology.orchestra.run.smithi002.stdout:5229 read 0x714cba thru 0x72116a (0xc4b1 bytes) 2024-09-18T04:57:27.473 INFO:teuthology.orchestra.run.smithi002.stdout:5230 punch from 0x9c2c58 to 0x9cd879, (0xac21 bytes) 2024-09-18T04:57:27.479 INFO:teuthology.orchestra.run.smithi002.stdout:5231 write 0x891a96d thru 0x8925dbc (0xb450 bytes) 2024-09-18T04:57:27.488 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:57:27.488 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:57:27.505 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:27.503+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:27.506 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:27.506 INFO:teuthology.orchestra.run.smithi002.stdout:5235 read 0x25a365f thru 0x25ad393 (0x9d35 bytes) 2024-09-18T04:57:27.507 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:57:27.507 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:57:27.513 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:27.513 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:27.513 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:57:27.513 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:57:27.513 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:27.514 INFO:teuthology.orchestra.run.smithi002.stdout:5236 writesame 0xaa6074d thru 0xaa68736 data_size 0x7(0x7fea bytes) 2024-09-18T04:57:27.514 INFO:teuthology.orchestra.run.smithi002.stdout:5237 punch from 0xa60e6ea to 0xa61b7da, (0xd0f0 bytes) 2024-09-18T04:57:27.522 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:57:27.524 INFO:teuthology.orchestra.run.smithi002.stdout:5238 write 0xe33a71d thru 0xe3423ba (0x7c9e bytes) 2024-09-18T04:57:27.528 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:57:27.528 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:57:27.545 INFO:teuthology.orchestra.run.smithi002.stdout:5240 read 0xd0f6daa thru 0xd104b6d (0xddc4 bytes) 2024-09-18T04:57:27.628 INFO:teuthology.orchestra.run.smithi002.stdout:5241 write 0x6f19bc thru 0x6f7c8c (0x62d1 bytes) 2024-09-18T04:57:27.630 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:27.627+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:27.675 INFO:teuthology.orchestra.run.smithi002.stdout:5242 read 0x8251a78 thru 0x825d82f (0xbdb8 bytes) 2024-09-18T04:57:27.675 INFO:teuthology.orchestra.run.smithi002.stdout:5244 punch from 0xde1da10 to 0xde21975, (0x3f65 bytes) 2024-09-18T04:57:27.675 INFO:teuthology.orchestra.run.smithi002.stdout:5245 write 0x7290e2 thru 0x733f7d (0xae9c bytes) 2024-09-18T04:57:27.685 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:27.692 INFO:teuthology.orchestra.run.smithi002.stdout:5247 writesame 0xe909d93 thru 0xe912126 data_size 0x2a(0x8394 bytes) 2024-09-18T04:57:27.692 INFO:teuthology.orchestra.run.smithi002.stdout:5248 read 0x5fbbe20 thru 0x5fca1c7 (0xe3a8 bytes) 2024-09-18T04:57:27.698 INFO:teuthology.orchestra.run.smithi002.stdout:5249 trunc from 0xe912127 to 0x8b53127 2024-09-18T04:57:27.713 INFO:teuthology.orchestra.run.smithi002.stdout:5250 read 0x76403a9 thru 0x764db7f (0xd7d7 bytes) 2024-09-18T04:57:27.714 INFO:teuthology.orchestra.run.smithi002.stdout:5252 punch from 0x74fc20c to 0x7502781, (0x6575 bytes) 2024-09-18T04:57:27.719 INFO:teuthology.orchestra.run.smithi002.stdout:5253 writesame 0x8f61532 thru 0x8f6ebcf data_size 0x3(0xd69e bytes) 2024-09-18T04:57:27.719 INFO:teuthology.orchestra.run.smithi002.stdout:5254 read 0x1105fae thru 0x110b23e (0x5291 bytes) 2024-09-18T04:57:27.719 INFO:teuthology.orchestra.run.smithi002.stdout:5255 punch from 0x7268460 to 0x726bf2c, (0x3acc bytes) 2024-09-18T04:57:27.719 INFO:teuthology.orchestra.run.smithi002.stdout:5257 writesame 0x7bf5f1e thru 0x7bf99d1 data_size 0x4(0x3ab4 bytes) 2024-09-18T04:57:27.730 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:27.728+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:27.744 INFO:teuthology.orchestra.run.smithi002.stdout:5258 read 0x41f4182 thru 0x41f46e3 (0x562 bytes) 2024-09-18T04:57:27.744 INFO:teuthology.orchestra.run.smithi002.stdout:5259 punch from 0x4253bd3 to 0x4262c99, (0xf0c6 bytes) 2024-09-18T04:57:27.744 INFO:teuthology.orchestra.run.smithi002.stdout:5260 punch from 0x8d761d8 to 0x8d7a4e2, (0x430a bytes) 2024-09-18T04:57:27.744 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:27.744 INFO:teuthology.orchestra.run.smithi002.stdout:5262 punch from 0x1b16354 to 0x1b220df, (0xbd8b bytes) 2024-09-18T04:57:27.753 INFO:teuthology.orchestra.run.smithi002.stdout:5263 trunc from 0x8f6ebd0 to 0x63041d 2024-09-18T04:57:27.762 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:27.762 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:27.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:27.762 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:27.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.762 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26189 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 237568~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:27.762 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:27.762 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.118562097000002, 2024-09-18T04:57:27.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022263431, 2024-09-18T04:57:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26189 2024-09-18T04:57:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:27.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:27.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:27.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:27.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:27.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:27.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639787+0000", 2024-09-18T04:57:27.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9759999999999998e-06 2024-09-18T04:57:27.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:27.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639792+0000", 2024-09-18T04:57:27.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5829999999999997e-06 2024-09-18T04:57:27.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:27.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639796+0000", 2024-09-18T04:57:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0690000000000003e-06 2024-09-18T04:57:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.661944+0000", 2024-09-18T04:57:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022147463999999999 2024-09-18T04:57:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.661985+0000", 2024-09-18T04:57:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1360999999999999e-05 2024-09-18T04:57:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:27.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662048+0000", 2024-09-18T04:57:27.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3298000000000003e-05 2024-09-18T04:57:27.768 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.768 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:27.768 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.768 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26199 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 4096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:27.768 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:27.768 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.118546811000002, 2024-09-18T04:57:27.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022348776000000001, 2024-09-18T04:57:27.768 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:27.769 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:27.769 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:27.769 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:27.769 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:27.769 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26199 2024-09-18T04:57:27.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.769 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:27.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:27.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:27.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:27.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:27.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:27.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:27.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:27.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639806+0000", 2024-09-18T04:57:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1539999999999999e-06 2024-09-18T04:57:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639812+0000", 2024-09-18T04:57:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5520000000000004e-06 2024-09-18T04:57:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639816+0000", 2024-09-18T04:57:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1840000000000001e-06 2024-09-18T04:57:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:27.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662063+0000", 2024-09-18T04:57:27.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022247125999999999 2024-09-18T04:57:27.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:27.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662098+0000", 2024-09-18T04:57:27.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4968000000000001e-05 2024-09-18T04:57:27.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:27.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662149+0000", 2024-09-18T04:57:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1122000000000003e-05 2024-09-18T04:57:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26200 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 32768~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.118527286000003, 2024-09-18T04:57:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022501118000000001, 2024-09-18T04:57:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:27.774 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:27.774 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26200 2024-09-18T04:57:27.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.774 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:27.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:27.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:27.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:27.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:27.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:27.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:27.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:27.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:27.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639822+0000", 2024-09-18T04:57:27.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.007e-06 2024-09-18T04:57:27.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:27.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639827+0000", 2024-09-18T04:57:27.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5709999999999997e-06 2024-09-18T04:57:27.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:27.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639831+0000", 2024-09-18T04:57:27.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2819999999999998e-06 2024-09-18T04:57:27.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:27.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662181+0000", 2024-09-18T04:57:27.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022349878 2024-09-18T04:57:27.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662252+0000", 2024-09-18T04:57:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0542000000000003e-05 2024-09-18T04:57:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662321+0000", 2024-09-18T04:57:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9167999999999999e-05 2024-09-18T04:57:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.778 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26201 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 61440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:27.778 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:27.778 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.118511994999999, 2024-09-18T04:57:27.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022569783, 2024-09-18T04:57:27.778 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:27.778 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:27.778 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:27.778 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:27.778 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:27.778 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26201 2024-09-18T04:57:27.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.778 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:27.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:27.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:27.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639837+0000", 2024-09-18T04:57:27.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.928e-06 2024-09-18T04:57:27.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:27.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639843+0000", 2024-09-18T04:57:27.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8130000000000001e-06 2024-09-18T04:57:27.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:27.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639847+0000", 2024-09-18T04:57:27.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0999999999999997e-06 2024-09-18T04:57:27.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:27.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662335+0000", 2024-09-18T04:57:27.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022488207999999999 2024-09-18T04:57:27.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:27.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662370+0000", 2024-09-18T04:57:27.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5292e-05 2024-09-18T04:57:27.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:27.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662405+0000", 2024-09-18T04:57:27.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4765999999999999e-05 2024-09-18T04:57:27.782 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.782 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:27.782 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.782 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26202 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 90112~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:27.782 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:57:27.782 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.118496577999998, 2024-09-18T04:57:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022655937000000001, 2024-09-18T04:57:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26202 2024-09-18T04:57:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:57:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:27.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:57:27.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:27.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639850+0000", 2024-09-18T04:57:27.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:27.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:27.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639852+0000", 2024-09-18T04:57:27.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9700000000000002e-06 2024-09-18T04:57:27.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:27.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639858+0000", 2024-09-18T04:57:27.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.434e-06 2024-09-18T04:57:27.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:27.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639862+0000", 2024-09-18T04:57:27.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3499999999999999e-06 2024-09-18T04:57:27.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662429+0000", 2024-09-18T04:57:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022567409 2024-09-18T04:57:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662469+0000", 2024-09-18T04:57:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9801000000000001e-05 2024-09-18T04:57:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662507+0000", 2024-09-18T04:57:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7311000000000002e-05 2024-09-18T04:57:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26203 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 118784~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:57:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.118481422999999, 2024-09-18T04:57:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022730358999999999, 2024-09-18T04:57:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:27.788 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26203 2024-09-18T04:57:27.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.788 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:27.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:27.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:57:27.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:27.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:57:27.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:27.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639865+0000", 2024-09-18T04:57:27.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:27.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:27.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639867+0000", 2024-09-18T04:57:27.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0889999999999998e-06 2024-09-18T04:57:27.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:27.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639873+0000", 2024-09-18T04:57:27.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4979999999999997e-06 2024-09-18T04:57:27.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:27.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639877+0000", 2024-09-18T04:57:27.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4510000000000002e-06 2024-09-18T04:57:27.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:27.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662528+0000", 2024-09-18T04:57:27.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022650791999999999 2024-09-18T04:57:27.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:27.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662563+0000", 2024-09-18T04:57:27.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4530000000000003e-05 2024-09-18T04:57:27.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:27.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662596+0000", 2024-09-18T04:57:27.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3463000000000001e-05 2024-09-18T04:57:27.792 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.792 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:27.792 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.792 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26204 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 147456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:27.792 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:27.792 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.118466030999997, 2024-09-18T04:57:27.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022803233999999999, 2024-09-18T04:57:27.793 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:27.793 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:27.793 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:27.793 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:27.793 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:27.793 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26204 2024-09-18T04:57:27.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.793 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:27.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:27.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:27.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639883+0000", 2024-09-18T04:57:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6520000000000002e-06 2024-09-18T04:57:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639888+0000", 2024-09-18T04:57:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2870000000000004e-06 2024-09-18T04:57:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639893+0000", 2024-09-18T04:57:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3839999999999999e-06 2024-09-18T04:57:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:27.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662617+0000", 2024-09-18T04:57:27.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022723876 2024-09-18T04:57:27.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:27.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662651+0000", 2024-09-18T04:57:27.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3797000000000002e-05 2024-09-18T04:57:27.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:27.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662684+0000", 2024-09-18T04:57:27.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3828999999999997e-05 2024-09-18T04:57:27.797 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.797 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:27.797 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.797 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26205 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 176128~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:27.797 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:27.797 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.118443069999998, 2024-09-18T04:57:27.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022919108000000001, 2024-09-18T04:57:27.797 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:27.797 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:27.797 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:27.797 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:27.798 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:27.798 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26205 2024-09-18T04:57:27.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.798 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:27.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:27.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:27.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:27.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:27.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639906+0000", 2024-09-18T04:57:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.531e-06 2024-09-18T04:57:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639913+0000", 2024-09-18T04:57:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.331e-06 2024-09-18T04:57:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639920+0000", 2024-09-18T04:57:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5399999999999998e-06 2024-09-18T04:57:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662706+0000", 2024-09-18T04:57:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.02278587 2024-09-18T04:57:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662764+0000", 2024-09-18T04:57:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7630000000000002e-05 2024-09-18T04:57:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662823+0000", 2024-09-18T04:57:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9648000000000002e-05 2024-09-18T04:57:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26206 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 204800~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.118422670999998, 2024-09-18T04:57:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023047517, 2024-09-18T04:57:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26206 2024-09-18T04:57:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639926+0000", 2024-09-18T04:57:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3120000000000001e-06 2024-09-18T04:57:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639932+0000", 2024-09-18T04:57:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4890000000000003e-06 2024-09-18T04:57:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639936+0000", 2024-09-18T04:57:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.07e-06 2024-09-18T04:57:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662850+0000", 2024-09-18T04:57:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022913579999999999 2024-09-18T04:57:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662903+0000", 2024-09-18T04:57:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3176e-05 2024-09-18T04:57:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662972+0000", 2024-09-18T04:57:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9270999999999995e-05 2024-09-18T04:57:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26207 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 233472~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:57:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.118407527000002, 2024-09-18T04:57:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023139541999999999, 2024-09-18T04:57:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26207 2024-09-18T04:57:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:57:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:57:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639939+0000", 2024-09-18T04:57:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639941+0000", 2024-09-18T04:57:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0080000000000001e-06 2024-09-18T04:57:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639947+0000", 2024-09-18T04:57:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.485e-06 2024-09-18T04:57:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639951+0000", 2024-09-18T04:57:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4549999999999997e-06 2024-09-18T04:57:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663005+0000", 2024-09-18T04:57:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023053569999999999 2024-09-18T04:57:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663042+0000", 2024-09-18T04:57:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7245e-05 2024-09-18T04:57:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663079+0000", 2024-09-18T04:57:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7119000000000002e-05 2024-09-18T04:57:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26208 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 524288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.118390779000002, 2024-09-18T04:57:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023283564999999999, 2024-09-18T04:57:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26208 2024-09-18T04:57:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639958+0000", 2024-09-18T04:57:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0899999999999999e-06 2024-09-18T04:57:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639963+0000", 2024-09-18T04:57:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2209999999999996e-06 2024-09-18T04:57:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639967+0000", 2024-09-18T04:57:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1169999999999997e-06 2024-09-18T04:57:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663114+0000", 2024-09-18T04:57:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023146102000000002 2024-09-18T04:57:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663183+0000", 2024-09-18T04:57:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9686999999999999e-05 2024-09-18T04:57:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663240+0000", 2024-09-18T04:57:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6728000000000003e-05 2024-09-18T04:57:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26209 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 552960~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.118373003000002, 2024-09-18T04:57:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023440966000000001, 2024-09-18T04:57:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26209 2024-09-18T04:57:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639976+0000", 2024-09-18T04:57:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4940000000000002e-06 2024-09-18T04:57:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639982+0000", 2024-09-18T04:57:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0560000000000001e-06 2024-09-18T04:57:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639987+0000", 2024-09-18T04:57:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-18T04:57:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663278+0000", 2024-09-18T04:57:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023290789999999999 2024-09-18T04:57:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663348+0000", 2024-09-18T04:57:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0837e-05 2024-09-18T04:57:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663415+0000", 2024-09-18T04:57:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6736000000000006e-05 2024-09-18T04:57:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26210 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 581632~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.118355995000002, 2024-09-18T04:57:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023616437000000001, 2024-09-18T04:57:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26210 2024-09-18T04:57:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639993+0000", 2024-09-18T04:57:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.419e-06 2024-09-18T04:57:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640006+0000", 2024-09-18T04:57:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2918000000000001e-05 2024-09-18T04:57:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640011+0000", 2024-09-18T04:57:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9320000000000004e-06 2024-09-18T04:57:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663454+0000", 2024-09-18T04:57:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023443035000000001 2024-09-18T04:57:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663532+0000", 2024-09-18T04:57:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7443999999999999e-05 2024-09-18T04:57:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663608+0000", 2024-09-18T04:57:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6093000000000005e-05 2024-09-18T04:57:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26211 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 610304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:57:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.118331443000002, 2024-09-18T04:57:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023774337999999999, 2024-09-18T04:57:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26211 2024-09-18T04:57:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:57:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:57:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640015+0000", 2024-09-18T04:57:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640023+0000", 2024-09-18T04:57:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3900000000000004e-06 2024-09-18T04:57:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640036+0000", 2024-09-18T04:57:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3091999999999999e-05 2024-09-18T04:57:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640041+0000", 2024-09-18T04:57:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6219999999999997e-06 2024-09-18T04:57:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663644+0000", 2024-09-18T04:57:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023602643999999999 2024-09-18T04:57:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663718+0000", 2024-09-18T04:57:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4481999999999996e-05 2024-09-18T04:57:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663790+0000", 2024-09-18T04:57:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1589999999999997e-05 2024-09-18T04:57:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26212 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 638976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:57:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.118298402999997, 2024-09-18T04:57:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023924748999999999, 2024-09-18T04:57:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:27.831 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26212 2024-09-18T04:57:27.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.831 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:27.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:27.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:57:27.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:57:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640048+0000", 2024-09-18T04:57:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640051+0000", 2024-09-18T04:57:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.095e-06 2024-09-18T04:57:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640060+0000", 2024-09-18T04:57:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8929999999999994e-06 2024-09-18T04:57:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640065+0000", 2024-09-18T04:57:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0270000000000003e-06 2024-09-18T04:57:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663824+0000", 2024-09-18T04:57:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023758873999999999 2024-09-18T04:57:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663904+0000", 2024-09-18T04:57:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9856000000000003e-05 2024-09-18T04:57:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663974+0000", 2024-09-18T04:57:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9450999999999996e-05 2024-09-18T04:57:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26213 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 667648~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.118277231999997, 2024-09-18T04:57:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024059812999999999, 2024-09-18T04:57:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26213 2024-09-18T04:57:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640072+0000", 2024-09-18T04:57:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4200000000000001e-06 2024-09-18T04:57:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640078+0000", 2024-09-18T04:57:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0179999999999998e-06 2024-09-18T04:57:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640083+0000", 2024-09-18T04:57:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:57:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664008+0000", 2024-09-18T04:57:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023925287 2024-09-18T04:57:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664059+0000", 2024-09-18T04:57:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0730999999999999e-05 2024-09-18T04:57:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664130+0000", 2024-09-18T04:57:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0801000000000002e-05 2024-09-18T04:57:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26214 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 696320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.118260262, 2024-09-18T04:57:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024223738000000002, 2024-09-18T04:57:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26214 2024-09-18T04:57:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640089+0000", 2024-09-18T04:57:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0849999999999999e-06 2024-09-18T04:57:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640095+0000", 2024-09-18T04:57:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9279999999999998e-06 2024-09-18T04:57:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640099+0000", 2024-09-18T04:57:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:57:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664166+0000", 2024-09-18T04:57:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024066513000000001 2024-09-18T04:57:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:27.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664239+0000", 2024-09-18T04:57:27.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3201999999999995e-05 2024-09-18T04:57:27.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:27.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664311+0000", 2024-09-18T04:57:27.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1434999999999996e-05 2024-09-18T04:57:27.848 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.848 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:27.848 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.849 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26215 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 724992~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:27.849 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:57:27.849 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.118243501000002, 2024-09-18T04:57:27.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024302284, 2024-09-18T04:57:27.849 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:27.849 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:27.849 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:27.849 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:27.849 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:27.849 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26215 2024-09-18T04:57:27.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.850 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:27.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:27.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:57:27.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:27.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:57:27.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640103+0000", 2024-09-18T04:57:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640106+0000", 2024-09-18T04:57:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2120000000000002e-06 2024-09-18T04:57:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:27.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640111+0000", 2024-09-18T04:57:27.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7880000000000003e-06 2024-09-18T04:57:27.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:27.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640116+0000", 2024-09-18T04:57:27.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6099999999999999e-06 2024-09-18T04:57:27.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:27.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664327+0000", 2024-09-18T04:57:27.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024210941999999999 2024-09-18T04:57:27.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:27.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664367+0000", 2024-09-18T04:57:27.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9666000000000003e-05 2024-09-18T04:57:27.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:27.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664406+0000", 2024-09-18T04:57:27.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9402e-05 2024-09-18T04:57:27.853 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.853 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:27.853 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.854 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26216 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 753664~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:27.854 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:27.854 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.118227007000002, 2024-09-18T04:57:27.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024452854999999999, 2024-09-18T04:57:27.854 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:27.854 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:27.854 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:27.854 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:27.854 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:27.854 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26216 2024-09-18T04:57:27.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.855 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:27.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:27.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:27.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:27.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:27.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:27.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:27.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:27.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:27.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640122+0000", 2024-09-18T04:57:27.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3599999999999999e-06 2024-09-18T04:57:27.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:27.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640128+0000", 2024-09-18T04:57:27.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6339999999999998e-06 2024-09-18T04:57:27.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:27.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640132+0000", 2024-09-18T04:57:27.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:57:27.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664436+0000", 2024-09-18T04:57:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024303585999999999 2024-09-18T04:57:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664507+0000", 2024-09-18T04:57:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1046999999999999e-05 2024-09-18T04:57:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664573+0000", 2024-09-18T04:57:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6217000000000007e-05 2024-09-18T04:57:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.858 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:27.859 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.859 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26217 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 782336~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:27.859 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:27.859 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.118210773000001, 2024-09-18T04:57:27.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024526287000000001, 2024-09-18T04:57:27.859 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:27.859 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:27.859 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:27.859 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:27.859 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:27.859 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26217 2024-09-18T04:57:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:27.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:27.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:27.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:27.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640140+0000", 2024-09-18T04:57:27.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2989999999999998e-06 2024-09-18T04:57:27.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:27.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640146+0000", 2024-09-18T04:57:27.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9569999999999999e-06 2024-09-18T04:57:27.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640151+0000", 2024-09-18T04:57:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:57:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664590+0000", 2024-09-18T04:57:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0244389 2024-09-18T04:57:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664628+0000", 2024-09-18T04:57:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7910999999999997e-05 2024-09-18T04:57:27.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:27.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:27.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:27.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664663+0000", 2024-09-18T04:57:27.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5024999999999998e-05 2024-09-18T04:57:27.863 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.863 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:27.863 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.863 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:27.863 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:27.863 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:27.864 INFO:teuthology.orchestra.run.smithi002.stdout:5264 writesame 0x28fa0a thru 0x29916b data_size 0x9(0x9762 bytes) 2024-09-18T04:57:27.865 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:57:27.866 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:27.864+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:27.965 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:27.963+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:27.980 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:27.980 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:27.980 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:57:27.980 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:27.988 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:57:28.066 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:28.063+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:28.095 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:28.096 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:28.096 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:57:28.096 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:57:28.096 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:28.109 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:57:28.120 INFO:teuthology.orchestra.run.smithi002.stdout:5265 trunc from 0x63041d to 0x91d0c29 2024-09-18T04:57:28.126 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:28.126 INFO:teuthology.orchestra.run.smithi002.stdout:5270 read 0x8b4c39 thru 0x8ba3e8 (0x57b0 bytes) 2024-09-18T04:57:28.126 INFO:teuthology.orchestra.run.smithi002.stdout:5273 writesame 0x1a13bf9 thru 0x1a23b86 data_size 0x7(0xff8e bytes) 2024-09-18T04:57:28.144 INFO:teuthology.orchestra.run.smithi002.stdout:5274 writesame 0x37a49f5 thru 0x37b067c data_size 0xc(0xbc88 bytes) 2024-09-18T04:57:28.167 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:28.164+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:28.168 INFO:teuthology.orchestra.run.smithi002.stdout:5275 punch from 0x1ec8ef8 to 0x1ed0636, (0x773e bytes) 2024-09-18T04:57:28.168 INFO:teuthology.orchestra.run.smithi002.stdout:5276 punch from 0x5308fd2 to 0x5312a00, (0x9a2e bytes) 2024-09-18T04:57:28.168 INFO:teuthology.orchestra.run.smithi002.stdout:5279 punch from 0x8b89479 to 0x8b94bdf, (0xb766 bytes) 2024-09-18T04:57:28.168 INFO:teuthology.orchestra.run.smithi002.stdout:5280 writesame 0x6d9fd61 thru 0x6dadc83 data_size 0x21(0xdf23 bytes) 2024-09-18T04:57:28.185 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:28.186 INFO:teuthology.orchestra.run.smithi002.stdout:5282 write 0x4fe141d thru 0x4fe6081 (0x4c65 bytes) 2024-09-18T04:57:28.204 INFO:teuthology.orchestra.run.smithi002.stdout:5284 write 0x2fe0cfc thru 0x2feef80 (0xe285 bytes) 2024-09-18T04:57:28.226 INFO:teuthology.orchestra.run.smithi002.stdout:5285 write 0x440cf96 thru 0x440e5bf (0x162a bytes) 2024-09-18T04:57:28.235 INFO:teuthology.orchestra.run.smithi002.stdout:5286 read 0x2d73412 thru 0x2d81d14 (0xe903 bytes) 2024-09-18T04:57:28.239 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:28.240 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:28.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:28.240 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:28.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.240 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:57:28.240 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:57:28.240 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 76.843542249999999, 2024-09-18T04:57:28.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037718511000000003, 2024-09-18T04:57:28.240 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:28.240 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:28.240 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:28.240 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:57:28.240 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:57:28.241 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-18T04:57:28.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.241 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:28.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:28.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:57:28.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:28.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:57:28.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:28.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393125+0000", 2024-09-18T04:57:28.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:57:28.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:28.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393133+0000", 2024-09-18T04:57:28.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.225e-06 2024-09-18T04:57:28.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:28.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393144+0000", 2024-09-18T04:57:28.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1279e-05 2024-09-18T04:57:28.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393153+0000", 2024-09-18T04:57:28.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4540000000000007e-06 2024-09-18T04:57:28.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:28.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393203+0000", 2024-09-18T04:57:28.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9963000000000003e-05 2024-09-18T04:57:28.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:57:28.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393210+0000", 2024-09-18T04:57:28.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5689999999999999e-06 2024-09-18T04:57:28.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:28.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427399+0000", 2024-09-18T04:57:28.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034188784999999999 2024-09-18T04:57:28.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:28.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427532+0000", 2024-09-18T04:57:28.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013264700000000001 2024-09-18T04:57:28.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:57:28.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427685+0000", 2024-09-18T04:57:28.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015300599999999999 2024-09-18T04:57:28.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:28.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.428520+0000", 2024-09-18T04:57:28.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00083502499999999998 2024-09-18T04:57:28.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:57:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430772+0000", 2024-09-18T04:57:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022518880000000001 2024-09-18T04:57:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:57:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430817+0000", 2024-09-18T04:57:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5077000000000002e-05 2024-09-18T04:57:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430845+0000", 2024-09-18T04:57:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8036999999999999e-05 2024-09-18T04:57:28.246 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.247 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:28.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.247 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54068 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3500032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:28.247 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221912+0000", 2024-09-18T04:57:28.247 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.01475688, 2024-09-18T04:57:28.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029654553, 2024-09-18T04:57:28.247 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:28.247 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:28.247 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54068 2024-09-18T04:57:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221912+0000", 2024-09-18T04:57:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:28.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221912+0000", 2024-09-18T04:57:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221911+0000", 2024-09-18T04:57:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221916+0000", 2024-09-18T04:57:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0130000000000001e-06 2024-09-18T04:57:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221922+0000", 2024-09-18T04:57:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.105e-06 2024-09-18T04:57:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221928+0000", 2024-09-18T04:57:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0430000000000004e-06 2024-09-18T04:57:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251427+0000", 2024-09-18T04:57:28.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029498586 2024-09-18T04:57:28.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:28.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251495+0000", 2024-09-18T04:57:28.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8125000000000003e-05 2024-09-18T04:57:28.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:28.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251566+0000", 2024-09-18T04:57:28.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1145999999999999e-05 2024-09-18T04:57:28.253 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.253 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:28.253 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.253 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54069 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3515392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:28.253 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221945+0000", 2024-09-18T04:57:28.253 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.014723271000001, 2024-09-18T04:57:28.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029786185999999999, 2024-09-18T04:57:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54069 2024-09-18T04:57:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221945+0000", 2024-09-18T04:57:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:28.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221945+0000", 2024-09-18T04:57:28.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:28.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221945+0000", 2024-09-18T04:57:28.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:28.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221947+0000", 2024-09-18T04:57:28.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6189999999999998e-06 2024-09-18T04:57:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221955+0000", 2024-09-18T04:57:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6249999999999998e-06 2024-09-18T04:57:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221968+0000", 2024-09-18T04:57:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2846e-05 2024-09-18T04:57:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251595+0000", 2024-09-18T04:57:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029627158000000001 2024-09-18T04:57:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251666+0000", 2024-09-18T04:57:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1437000000000001e-05 2024-09-18T04:57:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251731+0000", 2024-09-18T04:57:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5071000000000001e-05 2024-09-18T04:57:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.257 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:28.258 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.258 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54070 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:28.258 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221973+0000", 2024-09-18T04:57:28.258 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.014695287999999, 2024-09-18T04:57:28.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029891219, 2024-09-18T04:57:28.258 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:28.258 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:28.258 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:28.258 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:28.258 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.258 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54070 2024-09-18T04:57:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221973+0000", 2024-09-18T04:57:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221973+0000", 2024-09-18T04:57:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221973+0000", 2024-09-18T04:57:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221985+0000", 2024-09-18T04:57:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1888999999999999e-05 2024-09-18T04:57:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221992+0000", 2024-09-18T04:57:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2830000000000004e-06 2024-09-18T04:57:28.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222008+0000", 2024-09-18T04:57:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5608999999999999e-05 2024-09-18T04:57:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251757+0000", 2024-09-18T04:57:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029749652000000001 2024-09-18T04:57:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:28.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251826+0000", 2024-09-18T04:57:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9103000000000006e-05 2024-09-18T04:57:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251864+0000", 2024-09-18T04:57:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8179000000000002e-05 2024-09-18T04:57:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54071 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3546112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222014+0000", 2024-09-18T04:57:28.262 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.014654856999996, 2024-09-18T04:57:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03002749, 2024-09-18T04:57:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54071 2024-09-18T04:57:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222014+0000", 2024-09-18T04:57:28.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222014+0000", 2024-09-18T04:57:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222013+0000", 2024-09-18T04:57:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222016+0000", 2024-09-18T04:57:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6639999999999998e-06 2024-09-18T04:57:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222023+0000", 2024-09-18T04:57:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.849e-06 2024-09-18T04:57:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222028+0000", 2024-09-18T04:57:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0030000000000002e-06 2024-09-18T04:57:28.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251895+0000", 2024-09-18T04:57:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029867482000000001 2024-09-18T04:57:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251963+0000", 2024-09-18T04:57:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8038000000000001e-05 2024-09-18T04:57:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:28.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252041+0000", 2024-09-18T04:57:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8028999999999994e-05 2024-09-18T04:57:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54072 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3561472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222034+0000", 2024-09-18T04:57:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.014634786999999, 2024-09-18T04:57:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030190214, 2024-09-18T04:57:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:28.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54072 2024-09-18T04:57:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222034+0000", 2024-09-18T04:57:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222034+0000", 2024-09-18T04:57:28.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222033+0000", 2024-09-18T04:57:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222035+0000", 2024-09-18T04:57:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0490000000000002e-06 2024-09-18T04:57:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:28.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222051+0000", 2024-09-18T04:57:28.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5914e-05 2024-09-18T04:57:28.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222057+0000", 2024-09-18T04:57:28.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4269999999999999e-06 2024-09-18T04:57:28.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:28.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252079+0000", 2024-09-18T04:57:28.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030022338999999999 2024-09-18T04:57:28.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252157+0000", 2024-09-18T04:57:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7459999999999994e-05 2024-09-18T04:57:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252224+0000", 2024-09-18T04:57:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7415000000000004e-05 2024-09-18T04:57:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.272 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54073 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3576832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:28.272 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222062+0000", 2024-09-18T04:57:28.272 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.014607009999999, 2024-09-18T04:57:28.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030280892, 2024-09-18T04:57:28.272 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:28.272 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:28.272 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:28.272 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:28.272 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.272 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54073 2024-09-18T04:57:28.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.272 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222062+0000", 2024-09-18T04:57:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222062+0000", 2024-09-18T04:57:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222061+0000", 2024-09-18T04:57:28.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222063+0000", 2024-09-18T04:57:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0729999999999999e-06 2024-09-18T04:57:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222070+0000", 2024-09-18T04:57:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3199999999999996e-06 2024-09-18T04:57:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222074+0000", 2024-09-18T04:57:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8659999999999997e-06 2024-09-18T04:57:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252255+0000", 2024-09-18T04:57:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030180350000000002 2024-09-18T04:57:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252305+0000", 2024-09-18T04:57:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0086000000000001e-05 2024-09-18T04:57:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252342+0000", 2024-09-18T04:57:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7594e-05 2024-09-18T04:57:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54074 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3592192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222079+0000", 2024-09-18T04:57:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.014589618999999, 2024-09-18T04:57:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030370844000000001, 2024-09-18T04:57:28.276 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54074 2024-09-18T04:57:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222079+0000", 2024-09-18T04:57:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222079+0000", 2024-09-18T04:57:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222079+0000", 2024-09-18T04:57:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222081+0000", 2024-09-18T04:57:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0820000000000001e-06 2024-09-18T04:57:28.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222087+0000", 2024-09-18T04:57:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0789999999999997e-06 2024-09-18T04:57:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222097+0000", 2024-09-18T04:57:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0446e-05 2024-09-18T04:57:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252365+0000", 2024-09-18T04:57:28.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030267544 2024-09-18T04:57:28.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:28.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252402+0000", 2024-09-18T04:57:28.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7117999999999999e-05 2024-09-18T04:57:28.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:28.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252450+0000", 2024-09-18T04:57:28.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7930999999999999e-05 2024-09-18T04:57:28.280 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.280 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:28.280 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54075 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3607552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222103+0000", 2024-09-18T04:57:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.014565566999998, 2024-09-18T04:57:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030531175000000001, 2024-09-18T04:57:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54075 2024-09-18T04:57:28.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222103+0000", 2024-09-18T04:57:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222103+0000", 2024-09-18T04:57:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222101+0000", 2024-09-18T04:57:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:57:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222105+0000", 2024-09-18T04:57:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4089999999999999e-06 2024-09-18T04:57:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222111+0000", 2024-09-18T04:57:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3720000000000001e-06 2024-09-18T04:57:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222116+0000", 2024-09-18T04:57:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.544e-06 2024-09-18T04:57:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252475+0000", 2024-09-18T04:57:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030358975999999999 2024-09-18T04:57:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252562+0000", 2024-09-18T04:57:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7257999999999997e-05 2024-09-18T04:57:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252634+0000", 2024-09-18T04:57:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2144999999999995e-05 2024-09-18T04:57:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54076 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3622912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222121+0000", 2024-09-18T04:57:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.014547675999999, 2024-09-18T04:57:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030699046000000001, 2024-09-18T04:57:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54076 2024-09-18T04:57:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222121+0000", 2024-09-18T04:57:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222121+0000", 2024-09-18T04:57:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222120+0000", 2024-09-18T04:57:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222123+0000", 2024-09-18T04:57:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2199999999999999e-06 2024-09-18T04:57:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222129+0000", 2024-09-18T04:57:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5250000000000002e-06 2024-09-18T04:57:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222134+0000", 2024-09-18T04:57:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8180000000000003e-06 2024-09-18T04:57:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252677+0000", 2024-09-18T04:57:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030542969999999999 2024-09-18T04:57:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252750+0000", 2024-09-18T04:57:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2601000000000005e-05 2024-09-18T04:57:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252820+0000", 2024-09-18T04:57:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0409999999999998e-05 2024-09-18T04:57:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54077 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3638272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222138+0000", 2024-09-18T04:57:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.014530176000001, 2024-09-18T04:57:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031008865, 2024-09-18T04:57:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54077 2024-09-18T04:57:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222138+0000", 2024-09-18T04:57:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222138+0000", 2024-09-18T04:57:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222138+0000", 2024-09-18T04:57:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222140+0000", 2024-09-18T04:57:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9700000000000002e-06 2024-09-18T04:57:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222146+0000", 2024-09-18T04:57:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1460000000000001e-06 2024-09-18T04:57:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222151+0000", 2024-09-18T04:57:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.763e-06 2024-09-18T04:57:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252989+0000", 2024-09-18T04:57:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030838580000000001 2024-09-18T04:57:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253072+0000", 2024-09-18T04:57:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2204000000000001e-05 2024-09-18T04:57:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253147+0000", 2024-09-18T04:57:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5661000000000006e-05 2024-09-18T04:57:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54078 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222155+0000", 2024-09-18T04:57:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.01451359, 2024-09-18T04:57:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031175118000000002, 2024-09-18T04:57:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54078 2024-09-18T04:57:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222155+0000", 2024-09-18T04:57:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222155+0000", 2024-09-18T04:57:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222155+0000", 2024-09-18T04:57:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222157+0000", 2024-09-18T04:57:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0660000000000002e-06 2024-09-18T04:57:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222163+0000", 2024-09-18T04:57:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2820000000000002e-06 2024-09-18T04:57:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222168+0000", 2024-09-18T04:57:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7979999999999996e-06 2024-09-18T04:57:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253174+0000", 2024-09-18T04:57:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031005721999999999 2024-09-18T04:57:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253251+0000", 2024-09-18T04:57:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7717000000000001e-05 2024-09-18T04:57:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253330+0000", 2024-09-18T04:57:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8858999999999998e-05 2024-09-18T04:57:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54079 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3668992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222172+0000", 2024-09-18T04:57:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 38.014496440000002, 2024-09-18T04:57:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031327661, 2024-09-18T04:57:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54079 2024-09-18T04:57:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222172+0000", 2024-09-18T04:57:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222172+0000", 2024-09-18T04:57:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222172+0000", 2024-09-18T04:57:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222174+0000", 2024-09-18T04:57:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0159999999999998e-06 2024-09-18T04:57:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222180+0000", 2024-09-18T04:57:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9950000000000002e-06 2024-09-18T04:57:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222185+0000", 2024-09-18T04:57:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7679999999999999e-06 2024-09-18T04:57:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253360+0000", 2024-09-18T04:57:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031175423000000001 2024-09-18T04:57:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253430+0000", 2024-09-18T04:57:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9929000000000001e-05 2024-09-18T04:57:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253500+0000", 2024-09-18T04:57:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9896999999999998e-05 2024-09-18T04:57:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:62520 3.13 3.84cdb893 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T04:57:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 36.939232367000002, 2024-09-18T04:57:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94452134799999998, 2024-09-18T04:57:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:57:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 62520 2024-09-18T04:57:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297441+0000", 2024-09-18T04:57:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9840000000000001e-06 2024-09-18T04:57:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297449+0000", 2024-09-18T04:57:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3999999999999992e-06 2024-09-18T04:57:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297455+0000", 2024-09-18T04:57:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3350000000000003e-06 2024-09-18T04:57:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297552+0000", 2024-09-18T04:57:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6602999999999996e-05 2024-09-18T04:57:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:57:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297559+0000", 2024-09-18T04:57:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.52e-06 2024-09-18T04:57:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241943+0000", 2024-09-18T04:57:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94438357100000003 2024-09-18T04:57:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241958+0000", 2024-09-18T04:57:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4598e-05 2024-09-18T04:57:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:113179 3.13 3:cf138017:::rbd_data.11ac5f86a3f9.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T04:57:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:57:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.5494419560000008, 2024-09-18T04:57:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036508304999999998, 2024-09-18T04:57:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 113179 2024-09-18T04:57:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:57:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:57:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687226+0000", 2024-09-18T04:57:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:57:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688169+0000", 2024-09-18T04:57:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00094328200000000004 2024-09-18T04:57:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688182+0000", 2024-09-18T04:57:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2756e-05 2024-09-18T04:57:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688188+0000", 2024-09-18T04:57:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0290000000000002e-06 2024-09-18T04:57:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688225+0000", 2024-09-18T04:57:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7301000000000001e-05 2024-09-18T04:57:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688324+0000", 2024-09-18T04:57:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9419999999999993e-05 2024-09-18T04:57:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:57:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.700483+0000", 2024-09-18T04:57:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012158771000000001 2024-09-18T04:57:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:57:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723586+0000", 2024-09-18T04:57:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023102952 2024-09-18T04:57:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723634+0000", 2024-09-18T04:57:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7468e-05 2024-09-18T04:57:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:57:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723693+0000", 2024-09-18T04:57:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9539e-05 2024-09-18T04:57:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723735+0000", 2024-09-18T04:57:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1829000000000002e-05 2024-09-18T04:57:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:120568 3.f 3:f36f9030:::rbd_data.11acae5d5365.0000000000000001:2a [read 407552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T04:57:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:25.044527+0000", 2024-09-18T04:57:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.192141517, 2024-09-18T04:57:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029750146000000002, 2024-09-18T04:57:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 120568 2024-09-18T04:57:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044527+0000", 2024-09-18T04:57:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044527+0000", 2024-09-18T04:57:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044527+0000", 2024-09-18T04:57:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044533+0000", 2024-09-18T04:57:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3029999999999996e-06 2024-09-18T04:57:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044550+0000", 2024-09-18T04:57:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6702999999999999e-05 2024-09-18T04:57:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044555+0000", 2024-09-18T04:57:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.361e-06 2024-09-18T04:57:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074098+0000", 2024-09-18T04:57:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029543506000000001 2024-09-18T04:57:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074193+0000", 2024-09-18T04:57:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4297999999999997e-05 2024-09-18T04:57:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074277+0000", 2024-09-18T04:57:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4389999999999997e-05 2024-09-18T04:57:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:120569 3.f 3:f36f9030:::rbd_data.11acae5d5365.0000000000000001:2a [read 448512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T04:57:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:25.044573+0000", 2024-09-18T04:57:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.1920952639999998, 2024-09-18T04:57:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029916971000000001, 2024-09-18T04:57:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 120569 2024-09-18T04:57:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044573+0000", 2024-09-18T04:57:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044573+0000", 2024-09-18T04:57:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044573+0000", 2024-09-18T04:57:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044578+0000", 2024-09-18T04:57:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4219999999999999e-06 2024-09-18T04:57:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044586+0000", 2024-09-18T04:57:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0870000000000007e-06 2024-09-18T04:57:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044590+0000", 2024-09-18T04:57:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.518e-06 2024-09-18T04:57:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074332+0000", 2024-09-18T04:57:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029742721 2024-09-18T04:57:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074411+0000", 2024-09-18T04:57:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8592999999999998e-05 2024-09-18T04:57:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074490+0000", 2024-09-18T04:57:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9271999999999996e-05 2024-09-18T04:57:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:120570 3.f 3:f36f9030:::rbd_data.11acae5d5365.0000000000000001:2a [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T04:57:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:25.044593+0000", 2024-09-18T04:57:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.192075676, 2024-09-18T04:57:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030073685999999999, 2024-09-18T04:57:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 120570 2024-09-18T04:57:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044593+0000", 2024-09-18T04:57:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044593+0000", 2024-09-18T04:57:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044593+0000", 2024-09-18T04:57:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044594+0000", 2024-09-18T04:57:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5850000000000001e-06 2024-09-18T04:57:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044597+0000", 2024-09-18T04:57:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.281e-06 2024-09-18T04:57:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044600+0000", 2024-09-18T04:57:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.475e-06 2024-09-18T04:57:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074519+0000", 2024-09-18T04:57:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029918962 2024-09-18T04:57:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074594+0000", 2024-09-18T04:57:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5174999999999998e-05 2024-09-18T04:57:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074667+0000", 2024-09-18T04:57:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2543e-05 2024-09-18T04:57:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:120571 3.f 3:f36f9030:::rbd_data.11acae5d5365.0000000000000001:2a [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T04:57:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:25.044602+0000", 2024-09-18T04:57:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.192066525, 2024-09-18T04:57:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030242781, 2024-09-18T04:57:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 120571 2024-09-18T04:57:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044602+0000", 2024-09-18T04:57:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044602+0000", 2024-09-18T04:57:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044602+0000", 2024-09-18T04:57:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:57:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044603+0000", 2024-09-18T04:57:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.252e-06 2024-09-18T04:57:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044606+0000", 2024-09-18T04:57:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9780000000000001e-06 2024-09-18T04:57:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044608+0000", 2024-09-18T04:57:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4090000000000001e-06 2024-09-18T04:57:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074699+0000", 2024-09-18T04:57:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030090781 2024-09-18T04:57:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074774+0000", 2024-09-18T04:57:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5258000000000003e-05 2024-09-18T04:57:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:28.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074845+0000", 2024-09-18T04:57:28.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0335999999999999e-05 2024-09-18T04:57:28.333 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.333 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:28.333 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.333 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:120572 3.f 3:f36f9030:::rbd_data.11acae5d5365.0000000000000001:2a [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T04:57:28.333 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:25.044610+0000", 2024-09-18T04:57:28.333 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 3.192058098, 2024-09-18T04:57:28.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03043088, 2024-09-18T04:57:28.333 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 120572 2024-09-18T04:57:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044610+0000", 2024-09-18T04:57:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044610+0000", 2024-09-18T04:57:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044610+0000", 2024-09-18T04:57:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:57:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044611+0000", 2024-09-18T04:57:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2500000000000001e-06 2024-09-18T04:57:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044614+0000", 2024-09-18T04:57:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.971e-06 2024-09-18T04:57:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044617+0000", 2024-09-18T04:57:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4399999999999999e-06 2024-09-18T04:57:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074884+0000", 2024-09-18T04:57:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030266972 2024-09-18T04:57:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074966+0000", 2024-09-18T04:57:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2404999999999995e-05 2024-09-18T04:57:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.075041+0000", 2024-09-18T04:57:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5069999999999998e-05 2024-09-18T04:57:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.338 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:28.338 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:28.338 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:28.339 INFO:teuthology.orchestra.run.smithi002.stdout:5287 trunc from 0x91d0c29 to 0xd2676a 2024-09-18T04:57:28.339 INFO:teuthology.orchestra.run.smithi002.stdout:5288 write 0xa098340 thru 0xa09ec8e (0x694f bytes) 2024-09-18T04:57:28.339 INFO:teuthology.orchestra.run.smithi002.stdout:5289 write 0x66ce52b thru 0x66d908a (0xab60 bytes) 2024-09-18T04:57:28.339 INFO:teuthology.orchestra.run.smithi002.stdout:5290 writesame 0xc9e3be thru 0xca63e3 data_size 0x2f(0x8026 bytes) 2024-09-18T04:57:28.340 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:28.336+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:28.340 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:57:28.345 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:28.346 INFO:teuthology.orchestra.run.smithi002.stdout:5295 writesame 0x531e3b8 thru 0x53236a6 data_size 0x15(0x52ef bytes) 2024-09-18T04:57:28.364 INFO:teuthology.orchestra.run.smithi002.stdout:5296 read 0x63e2ee6 thru 0x63ee6c3 (0xb7de bytes) 2024-09-18T04:57:28.364 INFO:teuthology.orchestra.run.smithi002.stdout:5297 read 0x73395e3 thru 0x7341aa1 (0x84bf bytes) 2024-09-18T04:57:28.365 INFO:teuthology.orchestra.run.smithi002.stdout:5298 writesame 0x2de584d thru 0x2df49a3 data_size 0x1(0xf157 bytes) 2024-09-18T04:57:28.386 INFO:teuthology.orchestra.run.smithi002.stdout:5299 read 0x15a9cfc thru 0x15a9ea7 (0x1ac bytes) 2024-09-18T04:57:28.396 INFO:teuthology.orchestra.run.smithi002.stdout:5300 write 0xe57b12a thru 0xe57b163 (0x3a bytes) 2024-09-18T04:57:28.399 INFO:teuthology.orchestra.run.smithi002.stdout:5301 read 0x96727e8 thru 0x967fe07 (0xd620 bytes) 2024-09-18T04:57:28.399 INFO:teuthology.orchestra.run.smithi002.stdout:5302 read 0x50dd3df thru 0x50e4b31 (0x7753 bytes) 2024-09-18T04:57:28.405 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T04:57:28.405 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T04:57:28.405 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-18T04:57:28.406 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T04:57:28.406 INFO:teuthology.orchestra.run.smithi002.stdout:5303 write 0xe9f2ece thru 0xe9f73c3 (0x44f6 bytes) 2024-09-18T04:57:28.421 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T04:57:28.424 INFO:teuthology.orchestra.run.smithi002.stdout:5304 punch from 0xc2b38e0 to 0xc2bc9b2, (0x90d2 bytes) 2024-09-18T04:57:28.424 INFO:teuthology.orchestra.run.smithi002.stdout:5305 writesame 0xe6b7b8c thru 0xe6bd7bd data_size 0x1(0x5c32 bytes) 2024-09-18T04:57:28.440 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:28.437+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:28.446 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:28.446 INFO:teuthology.orchestra.run.smithi002.stdout:5308 read 0x435c7b8 thru 0x4364ee3 (0x872c bytes) 2024-09-18T04:57:28.446 INFO:teuthology.orchestra.run.smithi002.stdout:5309 read 0x3c1ef28 thru 0x3c26b77 (0x7c50 bytes) 2024-09-18T04:57:28.449 INFO:teuthology.orchestra.run.smithi002.stdout:5310 trunc from 0xe9f73c4 to 0xb81bdcf 2024-09-18T04:57:28.467 INFO:teuthology.orchestra.run.smithi002.stdout:5311 write 0xb0dbdd7 thru 0xb0e1e8f (0x60b9 bytes) 2024-09-18T04:57:28.471 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:28.471 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:28.471 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:28.471 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:28.484 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:57:28.488 INFO:teuthology.orchestra.run.smithi002.stdout:5312 read 0x200de21 thru 0x2016cda (0x8eba bytes) 2024-09-18T04:57:28.498 INFO:teuthology.orchestra.run.smithi002.stdout:5313 write 0xe21e20d thru 0xe21ea14 (0x808 bytes) 2024-09-18T04:57:28.513 INFO:teuthology.orchestra.run.smithi002.stdout:5314 write 0xe711101 thru 0xe715d75 (0x4c75 bytes) 2024-09-18T04:57:28.531 INFO:teuthology.orchestra.run.smithi002.stdout:5315 write 0x14175c1 thru 0x1425dd1 (0xe811 bytes) 2024-09-18T04:57:28.541 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:28.539+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:28.549 INFO:teuthology.orchestra.run.smithi002.stdout:5318 punch from 0x2129030 to 0x212ac36, (0x1c06 bytes) 2024-09-18T04:57:28.550 INFO:teuthology.orchestra.run.smithi002.stdout:5319 punch from 0xa121fd0 to 0xa12b48f, (0x94bf bytes) 2024-09-18T04:57:28.550 INFO:teuthology.orchestra.run.smithi002.stdout:5320 read 0x25a3025 thru 0x25ac571 (0x954d bytes) 2024-09-18T04:57:28.550 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:28.550 INFO:teuthology.orchestra.run.smithi002.stdout:5322 write 0x83014a2 thru 0x8310095 (0xebf4 bytes) 2024-09-18T04:57:28.578 INFO:teuthology.orchestra.run.smithi002.stdout:5323 trunc from 0xe715d76 to 0xac3e79e 2024-09-18T04:57:28.613 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:28.613 INFO:teuthology.orchestra.run.smithi002.stdout:5325 read 0x80d9555 thru 0x80e78d1 (0xe37d bytes) 2024-09-18T04:57:28.613 INFO:teuthology.orchestra.run.smithi002.stdout:5326 writesame 0xa796272 thru 0xa799d16 data_size 0x1(0x3aa5 bytes) 2024-09-18T04:57:28.626 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:28.626 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:28.626 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:28.626 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:28.626 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:28.635 INFO:teuthology.orchestra.run.smithi002.stdout:5327 punch from 0x1ae5df1 to 0x1ae94ab, (0x36ba bytes) 2024-09-18T04:57:28.635 INFO:teuthology.orchestra.run.smithi002.stdout:5330 read 0x5b3c0c2 thru 0x5b3da01 (0x1940 bytes) 2024-09-18T04:57:28.635 INFO:teuthology.orchestra.run.smithi002.stdout:5331 read 0xa582685 thru 0xa58b309 (0x8c85 bytes) 2024-09-18T04:57:28.635 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:28.637 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:57:28.648 INFO:teuthology.orchestra.run.smithi002.stdout:5335 trunc from 0xac3e79e to 0x18d8fda 2024-09-18T04:57:28.649 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:28.646+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:28.698 INFO:teuthology.orchestra.run.smithi002.stdout:5336 trunc from 0x18d8fda to 0x5d044d1 2024-09-18T04:57:28.704 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:28.704 INFO:teuthology.orchestra.run.smithi002.stdout:5339 read 0x47117d4 thru 0x471cf6f (0xb79c bytes) 2024-09-18T04:57:28.704 INFO:teuthology.orchestra.run.smithi002.stdout:5341 read 0x53e3401 thru 0x53e6bc7 (0x37c7 bytes) 2024-09-18T04:57:28.704 INFO:teuthology.orchestra.run.smithi002.stdout:5345 read 0x2612c08 thru 0x261530d (0x2706 bytes) 2024-09-18T04:57:28.714 INFO:teuthology.orchestra.run.smithi002.stdout:5346 write 0x8ac6aa3 thru 0x8ac9999 (0x2ef7 bytes) 2024-09-18T04:57:28.735 INFO:teuthology.orchestra.run.smithi002.stdout:5347 read 0x9d08a8 thru 0x9d6fdd (0x6736 bytes) 2024-09-18T04:57:28.736 INFO:teuthology.orchestra.run.smithi002.stdout:5349 read 0x5328314 thru 0x5338227 (0xff14 bytes) 2024-09-18T04:57:28.736 INFO:teuthology.orchestra.run.smithi002.stdout:5350 writesame 0x87b5cee thru 0x87ba925 data_size 0x12(0x4c38 bytes) 2024-09-18T04:57:28.743 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:28.741+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:28.755 INFO:teuthology.orchestra.run.smithi002.stdout:5351 read 0x25a2af6 thru 0x25adc77 (0xb182 bytes) 2024-09-18T04:57:28.755 INFO:teuthology.orchestra.run.smithi002.stdout:5353 read 0x9bcf48 thru 0x9be1ba (0x1273 bytes) 2024-09-18T04:57:28.765 INFO:teuthology.orchestra.run.smithi002.stdout:5354 trunc from 0x8ac999a to 0x119b0db 2024-09-18T04:57:28.787 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:57:28.787 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:28.787 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 80, 2024-09-18T04:57:28.787 INFO:teuthology.orchestra.run.smithi002.stdout: "fsid": "c0761d0c-69d5-4b64-b8b9-c7d098b2d06d", 2024-09-18T04:57:28.788 INFO:teuthology.orchestra.run.smithi002.stdout: "created": "2024-09-18T04:55:58.216538+0000", 2024-09-18T04:57:28.788 INFO:teuthology.orchestra.run.smithi002.stdout: "modified": "2024-09-18T04:57:26.397235+0000", 2024-09-18T04:57:28.788 INFO:teuthology.orchestra.run.smithi002.stdout: "last_up_change": "2024-09-18T04:56:08.369924+0000", 2024-09-18T04:57:28.788 INFO:teuthology.orchestra.run.smithi002.stdout: "last_in_change": "2024-09-18T04:57:04.291057+0000", 2024-09-18T04:57:28.788 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T04:57:28.788 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_num": 5804032, 2024-09-18T04:57:28.788 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_set": [ 2024-09-18T04:57:28.788 INFO:teuthology.orchestra.run.smithi002.stdout: "nodeep-scrub", 2024-09-18T04:57:28.788 INFO:teuthology.orchestra.run.smithi002.stdout: "pglog_hardlimit", 2024-09-18T04:57:28.789 INFO:teuthology.orchestra.run.smithi002.stdout: "purged_snapdirs", 2024-09-18T04:57:28.789 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_deletes", 2024-09-18T04:57:28.789 INFO:teuthology.orchestra.run.smithi002.stdout: "sortbitwise" 2024-09-18T04:57:28.789 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:57:28.789 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_version": 4, 2024-09-18T04:57:28.789 INFO:teuthology.orchestra.run.smithi002.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T04:57:28.789 INFO:teuthology.orchestra.run.smithi002.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T04:57:28.789 INFO:teuthology.orchestra.run.smithi002.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T04:57:28.789 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_snapshot": "", 2024-09-18T04:57:28.789 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_max": 3, 2024-09-18T04:57:28.789 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd": 8, 2024-09-18T04:57:28.790 INFO:teuthology.orchestra.run.smithi002.stdout: "require_min_compat_client": "luminous", 2024-09-18T04:57:28.790 INFO:teuthology.orchestra.run.smithi002.stdout: "min_compat_client": "luminous", 2024-09-18T04:57:28.790 INFO:teuthology.orchestra.run.smithi002.stdout: "require_osd_release": "reef", 2024-09-18T04:57:28.790 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_crimson": false, 2024-09-18T04:57:28.790 INFO:teuthology.orchestra.run.smithi002.stdout: "pools": [ 2024-09-18T04:57:28.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.790 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 1, 2024-09-18T04:57:28.790 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "rbd", 2024-09-18T04:57:28.790 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-18T04:56:08.983344+0000", 2024-09-18T04:57:28.790 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-18T04:57:28.790 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T04:57:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-18T04:57:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-18T04:57:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-18T04:57:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-18T04:57:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-18T04:57:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-18T04:57:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T04:57:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T04:57:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-18T04:57:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-18T04:57:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 8, 2024-09-18T04:57:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 8, 2024-09-18T04:57:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 8, 2024-09-18T04:57:28.791 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 8, 2024-09-18T04:57:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 8, 2024-09-18T04:57:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-18T04:57:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-18T04:57:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-18T04:57:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-18T04:57:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-18T04:57:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-18T04:57:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-18T04:57:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "17", 2024-09-18T04:57:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-18T04:57:28.792 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T04:57:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T04:57:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-18T04:57:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-18T04:57:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 3, 2024-09-18T04:57:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 17, 2024-09-18T04:57:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-18T04:57:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-18T04:57:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-18T04:57:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-18T04:57:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-18T04:57:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-18T04:57:28.793 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-18T04:57:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-18T04:57:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-18T04:57:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-18T04:57:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-18T04:57:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T04:57:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T04:57:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T04:57:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-18T04:57:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-18T04:57:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-18T04:57:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-18T04:57:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-18T04:57:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.794 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-18T04:57:28.795 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-18T04:57:28.795 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-18T04:57:28.795 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-18T04:57:28.795 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-18T04:57:28.795 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T04:57:28.795 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-18T04:57:28.795 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-18T04:57:28.795 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-18T04:57:28.795 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-18T04:57:28.795 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-18T04:57:28.795 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-18T04:57:28.796 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-18T04:57:28.796 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-18T04:57:28.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.796 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-18T04:57:28.796 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 4.380000114440918, 2024-09-18T04:57:28.797 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 4.380000114440918, 2024-09-18T04:57:28.797 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 1, 2024-09-18T04:57:28.797 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 4.380000114440918, 2024-09-18T04:57:28.797 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 4.380000114440918, 2024-09-18T04:57:28.797 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 0.875, 2024-09-18T04:57:28.797 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-18T04:57:28.797 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-18T04:57:28.797 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.798 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 2, 2024-09-18T04:57:28.798 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": ".mgr", 2024-09-18T04:57:28.798 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-18T04:56:10.313988+0000", 2024-09-18T04:57:28.798 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-18T04:57:28.798 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-18T04:57:28.798 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-18T04:57:28.799 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-18T04:57:28.799 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-18T04:57:28.799 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-18T04:57:28.799 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-18T04:57:28.799 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-18T04:57:28.799 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T04:57:28.800 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T04:57:28.800 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-18T04:57:28.800 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-18T04:57:28.800 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 1, 2024-09-18T04:57:28.800 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 1, 2024-09-18T04:57:28.800 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 1, 2024-09-18T04:57:28.800 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 1, 2024-09-18T04:57:28.800 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 1, 2024-09-18T04:57:28.800 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-18T04:57:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-18T04:57:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-18T04:57:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-18T04:57:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-18T04:57:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-18T04:57:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-18T04:57:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "17", 2024-09-18T04:57:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-18T04:57:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T04:57:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T04:57:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-18T04:57:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-18T04:57:28.801 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-18T04:57:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-18T04:57:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-18T04:57:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-18T04:57:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-18T04:57:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-18T04:57:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-18T04:57:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-18T04:57:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-18T04:57:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-18T04:57:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-18T04:57:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-18T04:57:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-18T04:57:28.802 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T04:57:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T04:57:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T04:57:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-18T04:57:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-18T04:57:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-18T04:57:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-18T04:57:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-18T04:57:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-18T04:57:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-18T04:57:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-18T04:57:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-18T04:57:28.803 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-18T04:57:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T04:57:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-18T04:57:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-18T04:57:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-18T04:57:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-18T04:57:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-18T04:57:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-18T04:57:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_max": 32, 2024-09-18T04:57:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 1 2024-09-18T04:57:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-18T04:57:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": {} 2024-09-18T04:57:28.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.805 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-18T04:57:28.805 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 6.9000000953674316, 2024-09-18T04:57:28.805 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 6.9000000953674316, 2024-09-18T04:57:28.805 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.28999999165534973, 2024-09-18T04:57:28.805 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 2, 2024-09-18T04:57:28.805 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 2, 2024-09-18T04:57:28.805 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 0.875, 2024-09-18T04:57:28.805 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-18T04:57:28.805 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-18T04:57:28.805 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-18T04:57:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "pool_client.0", 2024-09-18T04:57:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-18T04:56:19.562579+0000", 2024-09-18T04:57:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-18T04:57:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T04:57:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-18T04:57:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-18T04:57:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-18T04:57:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-18T04:57:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-18T04:57:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-18T04:57:28.806 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T04:57:28.807 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T04:57:28.807 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-18T04:57:28.807 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-18T04:57:28.807 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 32, 2024-09-18T04:57:28.807 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 32, 2024-09-18T04:57:28.807 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 32, 2024-09-18T04:57:28.807 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 32, 2024-09-18T04:57:28.807 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 32, 2024-09-18T04:57:28.807 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-18T04:57:28.807 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-18T04:57:28.808 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-18T04:57:28.808 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-18T04:57:28.808 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-18T04:57:28.808 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-18T04:57:28.808 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-18T04:57:28.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.808 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "79", 2024-09-18T04:57:28.808 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-18T04:57:28.808 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T04:57:28.809 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T04:57:28.809 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-18T04:57:28.809 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-18T04:57:28.809 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 43, 2024-09-18T04:57:28.809 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 79, 2024-09-18T04:57:28.809 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-18T04:57:28.809 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-18T04:57:28.809 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-18T04:57:28.809 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-18T04:57:28.809 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-18T04:57:28.809 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-18T04:57:28.809 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-18T04:57:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-18T04:57:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-18T04:57:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-18T04:57:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-18T04:57:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T04:57:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T04:57:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T04:57:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-18T04:57:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-18T04:57:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-18T04:57:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-18T04:57:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-18T04:57:28.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.811 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-18T04:57:28.811 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-18T04:57:28.811 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-18T04:57:28.811 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-18T04:57:28.811 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-18T04:57:28.811 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T04:57:28.811 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-18T04:57:28.811 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-18T04:57:28.811 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-18T04:57:28.811 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-18T04:57:28.811 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-18T04:57:28.811 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-18T04:57:28.811 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-18T04:57:28.812 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-18T04:57:28.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.812 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-18T04:57:28.812 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 1.5299999713897705, 2024-09-18T04:57:28.812 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 1.5299999713897705, 2024-09-18T04:57:28.812 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 1, 2024-09-18T04:57:28.812 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-18T04:57:28.812 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-18T04:57:28.813 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 0.875, 2024-09-18T04:57:28.813 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-18T04:57:28.813 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-18T04:57:28.813 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.813 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.813 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:57:28.813 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-18T04:57:28.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.813 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-18T04:57:28.813 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "c43f8024-3a3e-424e-aa54-398310ceb70d", 2024-09-18T04:57:28.813 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:57:28.813 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:57:28.814 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:57:28.814 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 0, 2024-09-18T04:57:28.814 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:57:28.814 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:57:28.814 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:57:28.814 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 61, 2024-09-18T04:57:28.814 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:57:28.816 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:57:28.817 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:57:28.817 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.817 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.817 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6824", 2024-09-18T04:57:28.817 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:57:28.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.817 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.817 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6825", 2024-09-18T04:57:28.818 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:57:28.818 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.818 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.818 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:57:28.818 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.818 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.818 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6826", 2024-09-18T04:57:28.818 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:57:28.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.819 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.819 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6827", 2024-09-18T04:57:28.819 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:57:28.819 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.819 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.819 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:57:28.820 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.820 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.820 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6830", 2024-09-18T04:57:28.820 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:57:28.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.820 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.820 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6831", 2024-09-18T04:57:28.821 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:57:28.821 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.821 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.821 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:57:28.821 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.821 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.821 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6828", 2024-09-18T04:57:28.821 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:57:28.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.822 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.822 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6829", 2024-09-18T04:57:28.822 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:57:28.822 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.822 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.822 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6825/4106704527", 2024-09-18T04:57:28.822 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6827/4106704527", 2024-09-18T04:57:28.822 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6831/4106704527", 2024-09-18T04:57:28.822 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6829/4106704527", 2024-09-18T04:57:28.822 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:57:28.822 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:57:28.822 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:57:28.823 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.823 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-18T04:57:28.823 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "c664b970-145c-4b6f-8852-2f6dcf1fa30d", 2024-09-18T04:57:28.823 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:57:28.823 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 0, 2024-09-18T04:57:28.823 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 0, 2024-09-18T04:57:28.823 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:57:28.823 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:57:28.823 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:57:28.823 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:57:28.824 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 19, 2024-09-18T04:57:28.824 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:57:28.824 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:57:28.824 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:57:28.824 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.824 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.824 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6808", 2024-09-18T04:57:28.824 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:57:28.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.824 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.825 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6809", 2024-09-18T04:57:28.825 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:57:28.825 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.825 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.825 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:57:28.825 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.825 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.825 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6810", 2024-09-18T04:57:28.825 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:57:28.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.826 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.826 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6811", 2024-09-18T04:57:28.826 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:57:28.826 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.826 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.826 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:57:28.826 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.826 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.826 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6814", 2024-09-18T04:57:28.826 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:57:28.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.827 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.827 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6815", 2024-09-18T04:57:28.827 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:57:28.827 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.827 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.827 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:57:28.827 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.827 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.827 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6812", 2024-09-18T04:57:28.827 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:57:28.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.828 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.828 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6813", 2024-09-18T04:57:28.828 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:57:28.828 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.828 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.828 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6809/1781959259", 2024-09-18T04:57:28.828 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6811/1781959259", 2024-09-18T04:57:28.828 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6815/1781959259", 2024-09-18T04:57:28.828 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6813/1781959259", 2024-09-18T04:57:28.828 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:57:28.828 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:57:28.828 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:57:28.829 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.829 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-18T04:57:28.829 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "0609ea8c-d975-490e-8844-04a3e4512efd", 2024-09-18T04:57:28.829 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:57:28.829 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:57:28.829 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:57:28.829 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:57:28.829 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:57:28.829 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:57:28.829 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:57:28.829 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 61, 2024-09-18T04:57:28.830 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:57:28.830 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:57:28.830 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:57:28.830 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.830 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.830 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6800", 2024-09-18T04:57:28.830 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:57:28.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.831 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.831 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6801", 2024-09-18T04:57:28.831 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:57:28.831 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.831 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.831 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:57:28.831 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.831 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.831 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6802", 2024-09-18T04:57:28.831 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:57:28.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.832 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.832 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6803", 2024-09-18T04:57:28.832 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:57:28.832 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.832 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.833 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:57:28.833 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.833 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.833 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6806", 2024-09-18T04:57:28.833 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:57:28.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.834 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.834 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6807", 2024-09-18T04:57:28.834 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:57:28.834 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.834 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.835 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:57:28.835 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.835 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.835 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6804", 2024-09-18T04:57:28.836 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:57:28.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.836 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.836 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6805", 2024-09-18T04:57:28.836 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:57:28.837 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.837 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.837 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6801/1153570802", 2024-09-18T04:57:28.837 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6803/1153570802", 2024-09-18T04:57:28.838 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6807/1153570802", 2024-09-18T04:57:28.838 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6805/1153570802", 2024-09-18T04:57:28.838 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:57:28.838 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:57:28.838 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:57:28.838 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.839 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-18T04:57:28.839 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "c6bd5ac8-ab89-4fa2-b843-3b166d1b238b", 2024-09-18T04:57:28.839 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:57:28.839 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:57:28.839 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:57:28.840 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:57:28.840 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:57:28.840 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:57:28.840 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:57:28.840 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 61, 2024-09-18T04:57:28.840 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:57:28.841 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:57:28.841 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:57:28.841 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.841 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.842 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6816", 2024-09-18T04:57:28.842 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:57:28.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.842 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.842 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6817", 2024-09-18T04:57:28.842 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:57:28.843 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.843 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.843 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:57:28.843 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.844 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.844 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6818", 2024-09-18T04:57:28.844 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:57:28.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.844 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.845 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6819", 2024-09-18T04:57:28.845 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:57:28.845 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.845 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.846 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:57:28.846 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.846 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.846 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6822", 2024-09-18T04:57:28.846 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:57:28.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.847 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.847 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6823", 2024-09-18T04:57:28.847 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:57:28.847 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.848 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.848 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:57:28.848 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.849 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.849 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6820", 2024-09-18T04:57:28.849 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:57:28.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.850 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.850 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6821", 2024-09-18T04:57:28.850 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:57:28.850 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.850 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.851 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6817/2008321375", 2024-09-18T04:57:28.851 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6819/2008321375", 2024-09-18T04:57:28.851 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6823/2008321375", 2024-09-18T04:57:28.851 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6821/2008321375", 2024-09-18T04:57:28.851 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:57:28.851 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:57:28.852 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:57:28.852 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.852 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-18T04:57:28.853 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "89d6a371-4f4e-465e-a023-d5083cce6e6c", 2024-09-18T04:57:28.853 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:57:28.853 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:57:28.853 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:57:28.853 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:57:28.853 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:57:28.854 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:57:28.854 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:57:28.854 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 77, 2024-09-18T04:57:28.854 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:57:28.854 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:57:28.854 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:57:28.855 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.855 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.855 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6824", 2024-09-18T04:57:28.855 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:57:28.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.856 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.856 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6825", 2024-09-18T04:57:28.856 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:57:28.856 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.857 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.857 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:57:28.857 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.857 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.858 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6826", 2024-09-18T04:57:28.858 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:57:28.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.858 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.858 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6827", 2024-09-18T04:57:28.859 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:57:28.859 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.859 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.859 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:57:28.859 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.860 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.860 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6830", 2024-09-18T04:57:28.860 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:57:28.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.861 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.861 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6831", 2024-09-18T04:57:28.861 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:57:28.861 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.861 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.862 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:57:28.862 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.862 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.862 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6828", 2024-09-18T04:57:28.862 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:57:28.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.863 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.863 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6829", 2024-09-18T04:57:28.863 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:57:28.863 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.864 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.864 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6825/2939896598", 2024-09-18T04:57:28.864 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6827/2939896598", 2024-09-18T04:57:28.864 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6831/2939896598", 2024-09-18T04:57:28.864 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6829/2939896598", 2024-09-18T04:57:28.865 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:57:28.865 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:57:28.865 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:57:28.865 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.866 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-18T04:57:28.866 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "da99686b-d979-4753-b4aa-a3b26e9a0497", 2024-09-18T04:57:28.866 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:57:28.866 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:57:28.866 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:57:28.866 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:57:28.866 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:57:28.867 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:57:28.867 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:57:28.867 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 77, 2024-09-18T04:57:28.867 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:57:28.867 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:57:28.867 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:57:28.867 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.867 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.867 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6816", 2024-09-18T04:57:28.867 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:57:28.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.868 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.868 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6817", 2024-09-18T04:57:28.868 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:57:28.868 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.868 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.868 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:57:28.868 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.869 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.869 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6818", 2024-09-18T04:57:28.869 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:57:28.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.869 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.869 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6819", 2024-09-18T04:57:28.869 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:57:28.869 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.869 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.870 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:57:28.870 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.870 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.870 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6822", 2024-09-18T04:57:28.870 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:57:28.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.870 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.870 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6823", 2024-09-18T04:57:28.870 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:57:28.870 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.871 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.871 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:57:28.871 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.871 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.871 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6820", 2024-09-18T04:57:28.871 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:57:28.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.872 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.872 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6821", 2024-09-18T04:57:28.872 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:57:28.872 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.872 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.872 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6817/3440879618", 2024-09-18T04:57:28.872 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6819/3440879618", 2024-09-18T04:57:28.872 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6823/3440879618", 2024-09-18T04:57:28.872 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6821/3440879618", 2024-09-18T04:57:28.872 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:57:28.873 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:57:28.873 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:57:28.873 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.873 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-18T04:57:28.873 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "ceec6aa6-603d-4706-bfea-3c07dbdaed3a", 2024-09-18T04:57:28.873 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:57:28.873 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:57:28.873 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:57:28.873 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:57:28.873 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:57:28.874 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:57:28.874 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:57:28.874 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 58, 2024-09-18T04:57:28.874 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:57:28.874 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:57:28.874 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:57:28.874 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.874 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.874 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6800", 2024-09-18T04:57:28.874 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:57:28.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.875 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.875 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6801", 2024-09-18T04:57:28.875 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:57:28.875 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.875 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.875 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:57:28.875 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.876 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.876 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6802", 2024-09-18T04:57:28.876 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:57:28.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.876 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.876 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6803", 2024-09-18T04:57:28.876 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:57:28.876 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.876 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.877 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:57:28.877 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.877 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.877 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6806", 2024-09-18T04:57:28.877 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:57:28.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.877 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.878 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6807", 2024-09-18T04:57:28.878 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:57:28.878 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.878 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.878 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:57:28.878 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.878 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.878 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6804", 2024-09-18T04:57:28.878 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:57:28.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.879 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.879 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6805", 2024-09-18T04:57:28.879 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:57:28.879 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.879 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.879 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6801/2493830504", 2024-09-18T04:57:28.879 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6803/2493830504", 2024-09-18T04:57:28.879 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6807/2493830504", 2024-09-18T04:57:28.879 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6805/2493830504", 2024-09-18T04:57:28.880 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:57:28.880 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:57:28.880 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:57:28.880 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.880 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-18T04:57:28.880 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "6779dfad-f60b-41f4-b666-a99db6dc1814", 2024-09-18T04:57:28.880 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:57:28.880 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:57:28.880 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:57:28.880 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:57:28.881 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:57:28.881 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:57:28.881 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:57:28.881 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 77, 2024-09-18T04:57:28.881 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:57:28.881 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:57:28.881 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:57:28.881 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.881 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.881 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6808", 2024-09-18T04:57:28.881 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:57:28.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.882 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.882 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6809", 2024-09-18T04:57:28.882 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:57:28.882 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.882 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.882 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:57:28.882 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.882 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.882 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6810", 2024-09-18T04:57:28.882 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:57:28.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.883 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.883 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6811", 2024-09-18T04:57:28.883 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:57:28.883 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.883 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.883 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:57:28.883 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.884 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.884 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6814", 2024-09-18T04:57:28.884 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:57:28.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.884 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.884 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6815", 2024-09-18T04:57:28.884 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:57:28.884 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.884 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.884 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:57:28.884 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:28.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.885 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:28.885 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6812", 2024-09-18T04:57:28.885 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:57:28.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.885 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:28.885 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6813", 2024-09-18T04:57:28.885 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:57:28.885 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.885 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.886 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6809/91038858", 2024-09-18T04:57:28.886 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6811/91038858", 2024-09-18T04:57:28.886 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6815/91038858", 2024-09-18T04:57:28.886 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6813/91038858", 2024-09-18T04:57:28.886 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:57:28.886 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:57:28.886 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:57:28.886 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.886 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.886 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:57:28.886 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_xinfo": [ 2024-09-18T04:57:28.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.886 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-18T04:57:28.886 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:57:28.887 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:57:28.887 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:57:28.887 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:57:28.887 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:57:28.887 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.468213+0000", 2024-09-18T04:57:28.887 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:57:28.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.887 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-18T04:57:28.887 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:57:28.888 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:57:28.888 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:57:28.888 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:57:28.888 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 65536, 2024-09-18T04:57:28.888 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.538610+0000", 2024-09-18T04:57:28.888 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:57:28.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.888 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-18T04:57:28.888 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:57:28.888 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:57:28.888 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:57:28.889 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:57:28.889 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:57:28.889 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.500881+0000", 2024-09-18T04:57:28.889 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:57:28.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.889 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-18T04:57:28.889 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:57:28.889 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:57:28.889 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:57:28.889 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:57:28.889 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:57:28.890 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.474179+0000", 2024-09-18T04:57:28.890 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:57:28.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.890 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-18T04:57:28.890 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:57:28.890 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:57:28.890 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:57:28.890 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:57:28.890 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:57:28.890 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.483737+0000", 2024-09-18T04:57:28.890 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:57:28.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.891 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-18T04:57:28.891 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:57:28.891 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:57:28.891 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:57:28.891 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:57:28.891 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:57:28.891 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.460693+0000", 2024-09-18T04:57:28.891 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:57:28.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.892 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-18T04:57:28.892 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:57:28.892 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:57:28.892 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:57:28.892 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:57:28.892 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:57:28.892 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.475970+0000", 2024-09-18T04:57:28.893 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:57:28.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.893 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-18T04:57:28.893 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:57:28.893 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:57:28.893 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:57:28.893 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:57:28.893 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:57:28.894 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.519287+0000", 2024-09-18T04:57:28.894 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:57:28.894 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.894 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:57:28.894 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap": [ 2024-09-18T04:57:28.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.894 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "1.0", 2024-09-18T04:57:28.894 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-18T04:57:28.894 INFO:teuthology.orchestra.run.smithi002.stdout: 6, 2024-09-18T04:57:28.894 INFO:teuthology.orchestra.run.smithi002.stdout: 4 2024-09-18T04:57:28.894 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.894 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "1.4", 2024-09-18T04:57:28.895 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-18T04:57:28.895 INFO:teuthology.orchestra.run.smithi002.stdout: 6, 2024-09-18T04:57:28.895 INFO:teuthology.orchestra.run.smithi002.stdout: 7 2024-09-18T04:57:28.895 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.895 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.895 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:57:28.895 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_items": [], 2024-09-18T04:57:28.895 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_primaries": [], 2024-09-18T04:57:28.895 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_temp": [], 2024-09-18T04:57:28.895 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_temp": [], 2024-09-18T04:57:28.895 INFO:teuthology.orchestra.run.smithi002.stdout: "blocklist": {}, 2024-09-18T04:57:28.895 INFO:teuthology.orchestra.run.smithi002.stdout: "range_blocklist": {}, 2024-09-18T04:57:28.895 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profiles": { 2024-09-18T04:57:28.896 INFO:teuthology.orchestra.run.smithi002.stdout: "default": { 2024-09-18T04:57:28.896 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-failure-domain": "osd", 2024-09-18T04:57:28.896 INFO:teuthology.orchestra.run.smithi002.stdout: "k": "2", 2024-09-18T04:57:28.896 INFO:teuthology.orchestra.run.smithi002.stdout: "m": "1", 2024-09-18T04:57:28.896 INFO:teuthology.orchestra.run.smithi002.stdout: "plugin": "jerasure", 2024-09-18T04:57:28.896 INFO:teuthology.orchestra.run.smithi002.stdout: "technique": "reed_sol_van" 2024-09-18T04:57:28.896 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.896 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps_queue": [], 2024-09-18T04:57:28.896 INFO:teuthology.orchestra.run.smithi002.stdout: "new_removed_snaps": [], 2024-09-18T04:57:28.897 INFO:teuthology.orchestra.run.smithi002.stdout: "new_purged_snaps": [], 2024-09-18T04:57:28.897 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_node_flags": {}, 2024-09-18T04:57:28.897 INFO:teuthology.orchestra.run.smithi002.stdout: "device_class_flags": {}, 2024-09-18T04:57:28.897 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode": { 2024-09-18T04:57:28.897 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_enabled": false, 2024-09-18T04:57:28.897 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_bucket_count": 0, 2024-09-18T04:57:28.897 INFO:teuthology.orchestra.run.smithi002.stdout: "degraded_stretch_mode": 0, 2024-09-18T04:57:28.897 INFO:teuthology.orchestra.run.smithi002.stdout: "recovering_stretch_mode": 0, 2024-09-18T04:57:28.897 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_bucket": 0 2024-09-18T04:57:28.897 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.897 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:28.899 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:28.896+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:28.899 INFO:teuthology.orchestra.run.smithi002.stdout:5355 punch from 0x9ce80f to 0x9cf2c6, (0xab7 bytes) 2024-09-18T04:57:28.899 INFO:teuthology.orchestra.run.smithi002.stdout:5358 punch from 0xcbfc4 to 0xd89bb, (0xc9f7 bytes) 2024-09-18T04:57:28.900 INFO:teuthology.orchestra.run.smithi002.stdout:5359 trunc from 0x119b0db to 0x6e6d1d9 2024-09-18T04:57:28.901 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:28.901 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:28.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:28.901 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:28.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.902 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9316 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:28.902 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:28.902 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.027055963999999, 2024-09-18T04:57:28.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044027208999999998, 2024-09-18T04:57:28.902 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:28.902 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:28.903 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:28.903 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:28.903 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.903 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9316 2024-09-18T04:57:28.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.903 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:28.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:28.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:28.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:28.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:28.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:28.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:28.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:28.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767144+0000", 2024-09-18T04:57:28.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3070000000000001e-06 2024-09-18T04:57:28.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:28.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767150+0000", 2024-09-18T04:57:28.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1700000000000002e-06 2024-09-18T04:57:28.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767163+0000", 2024-09-18T04:57:28.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2670999999999999e-05 2024-09-18T04:57:28.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:28.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810995+0000", 2024-09-18T04:57:28.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043832310999999999 2024-09-18T04:57:28.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:28.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811087+0000", 2024-09-18T04:57:28.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1345e-05 2024-09-18T04:57:28.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:28.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811169+0000", 2024-09-18T04:57:28.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2848000000000004e-05 2024-09-18T04:57:28.910 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.910 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.910 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.910 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9317 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1728512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:28.911 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:57:28.911 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.027030328999999, 2024-09-18T04:57:28.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044190170000000001, 2024-09-18T04:57:28.911 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:28.911 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:28.911 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:28.911 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:28.911 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.911 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9317 2024-09-18T04:57:28.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.912 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:28.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:28.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:57:28.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:28.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:57:28.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:28.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767167+0000", 2024-09-18T04:57:28.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:28.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767170+0000", 2024-09-18T04:57:28.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7259999999999998e-06 2024-09-18T04:57:28.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:28.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767176+0000", 2024-09-18T04:57:28.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1460000000000001e-06 2024-09-18T04:57:28.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767182+0000", 2024-09-18T04:57:28.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1020000000000002e-06 2024-09-18T04:57:28.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:28.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811204+0000", 2024-09-18T04:57:28.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044022039999999998 2024-09-18T04:57:28.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:28.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811286+0000", 2024-09-18T04:57:28.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1623000000000001e-05 2024-09-18T04:57:28.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:28.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811358+0000", 2024-09-18T04:57:28.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2125000000000005e-05 2024-09-18T04:57:28.915 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.915 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.915 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.916 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9318 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1777664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:28.916 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:28.916 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.027011784000003, 2024-09-18T04:57:28.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044380181999999997, 2024-09-18T04:57:28.916 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:28.916 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:28.916 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:28.916 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:28.916 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.916 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9318 2024-09-18T04:57:28.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.916 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:28.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:28.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:28.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:28.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:28.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:28.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:28.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:28.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767189+0000", 2024-09-18T04:57:28.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5909999999999998e-06 2024-09-18T04:57:28.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:28.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767194+0000", 2024-09-18T04:57:28.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7690000000000002e-06 2024-09-18T04:57:28.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767199+0000", 2024-09-18T04:57:28.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189999999999999e-06 2024-09-18T04:57:28.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:28.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811394+0000", 2024-09-18T04:57:28.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044195350000000001 2024-09-18T04:57:28.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:28.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811483+0000", 2024-09-18T04:57:28.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8282999999999995e-05 2024-09-18T04:57:28.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:28.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811567+0000", 2024-09-18T04:57:28.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3855000000000003e-05 2024-09-18T04:57:28.921 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.921 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.921 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.921 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9325 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:28.921 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:28.921 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.025281245999999, 2024-09-18T04:57:28.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044063989999999997, 2024-09-18T04:57:28.922 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:28.922 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:28.922 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:28.922 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:28.922 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.922 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9325 2024-09-18T04:57:28.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.922 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:28.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:28.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:28.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:28.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:28.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:28.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:28.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:28.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768919+0000", 2024-09-18T04:57:28.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:57:28.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:28.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768925+0000", 2024-09-18T04:57:28.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2310000000000001e-06 2024-09-18T04:57:28.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768929+0000", 2024-09-18T04:57:28.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4469999999999999e-06 2024-09-18T04:57:28.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:28.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812823+0000", 2024-09-18T04:57:28.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043893975000000002 2024-09-18T04:57:28.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:28.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812908+0000", 2024-09-18T04:57:28.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4862000000000002e-05 2024-09-18T04:57:28.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:28.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812981+0000", 2024-09-18T04:57:28.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2892000000000006e-05 2024-09-18T04:57:28.926 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.926 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.926 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.926 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9334 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:28.926 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:57:28.926 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.023262459999998, 2024-09-18T04:57:28.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044042853999999999, 2024-09-18T04:57:28.926 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:28.926 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:28.927 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:28.927 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:28.927 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.927 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9334 2024-09-18T04:57:28.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.927 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:28.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:28.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:57:28.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.927 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:28.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:57:28.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:28.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770935+0000", 2024-09-18T04:57:28.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:28.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770937+0000", 2024-09-18T04:57:28.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1489999999999999e-06 2024-09-18T04:57:28.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:28.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770940+0000", 2024-09-18T04:57:28.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9569999999999998e-06 2024-09-18T04:57:28.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770945+0000", 2024-09-18T04:57:28.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:57:28.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:28.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814838+0000", 2024-09-18T04:57:28.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043892635999999999 2024-09-18T04:57:28.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:28.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814920+0000", 2024-09-18T04:57:28.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2175000000000005e-05 2024-09-18T04:57:28.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:28.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814979+0000", 2024-09-18T04:57:28.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8372000000000003e-05 2024-09-18T04:57:28.930 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.930 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.930 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.931 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9352 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1159168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:28.931 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:57:28.931 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.021041283000002, 2024-09-18T04:57:28.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044011269999999998, 2024-09-18T04:57:28.931 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:28.932 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:28.932 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:28.932 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:28.932 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.932 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9352 2024-09-18T04:57:28.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.933 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:28.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:28.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:57:28.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:28.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:57:28.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:28.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773156+0000", 2024-09-18T04:57:28.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:28.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773159+0000", 2024-09-18T04:57:28.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2919999999999998e-06 2024-09-18T04:57:28.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:28.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773165+0000", 2024-09-18T04:57:28.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0580000000000003e-06 2024-09-18T04:57:28.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773177+0000", 2024-09-18T04:57:28.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.204e-05 2024-09-18T04:57:28.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:28.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817082+0000", 2024-09-18T04:57:28.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043905528999999999 2024-09-18T04:57:28.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:28.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817122+0000", 2024-09-18T04:57:28.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9258999999999998e-05 2024-09-18T04:57:28.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:28.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817168+0000", 2024-09-18T04:57:28.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6545000000000002e-05 2024-09-18T04:57:28.937 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.937 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.937 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.937 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9353 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1208320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:28.937 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:28.937 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.021016985000003, 2024-09-18T04:57:28.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044105537, 2024-09-18T04:57:28.938 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:28.938 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:28.938 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:28.938 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:28.938 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.938 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9353 2024-09-18T04:57:28.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.938 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:28.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:28.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:28.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:28.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:28.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:28.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:28.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:28.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773183+0000", 2024-09-18T04:57:28.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2929999999999999e-06 2024-09-18T04:57:28.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:28.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773190+0000", 2024-09-18T04:57:28.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4330000000000001e-06 2024-09-18T04:57:28.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773194+0000", 2024-09-18T04:57:28.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.865e-06 2024-09-18T04:57:28.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:28.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817196+0000", 2024-09-18T04:57:28.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044001708 2024-09-18T04:57:28.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:28.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817236+0000", 2024-09-18T04:57:28.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0043999999999998e-05 2024-09-18T04:57:28.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:28.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817287+0000", 2024-09-18T04:57:28.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0605000000000001e-05 2024-09-18T04:57:28.942 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.942 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.942 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.942 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9354 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:28.942 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:57:28.942 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.020999474, 2024-09-18T04:57:28.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044199796, 2024-09-18T04:57:28.942 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:28.942 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:28.942 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:28.943 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:28.943 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.943 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9354 2024-09-18T04:57:28.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.943 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:28.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:28.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:57:28.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:28.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:57:28.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:28.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773198+0000", 2024-09-18T04:57:28.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:28.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773200+0000", 2024-09-18T04:57:28.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4159999999999998e-06 2024-09-18T04:57:28.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:28.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773206+0000", 2024-09-18T04:57:28.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.789e-06 2024-09-18T04:57:28.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773210+0000", 2024-09-18T04:57:28.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2679999999999996e-06 2024-09-18T04:57:28.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:28.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817314+0000", 2024-09-18T04:57:28.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044103187000000002 2024-09-18T04:57:28.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:28.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817353+0000", 2024-09-18T04:57:28.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8973000000000001e-05 2024-09-18T04:57:28.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:28.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817399+0000", 2024-09-18T04:57:28.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5877999999999997e-05 2024-09-18T04:57:28.946 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.946 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.946 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.946 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46591 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2097152~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:28.947 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:28.947 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 39.096143056999999, 2024-09-18T04:57:28.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043987310000000002, 2024-09-18T04:57:28.947 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:28.947 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:28.947 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:28.947 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:28.947 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.947 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46591 2024-09-18T04:57:28.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.947 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:28.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:28.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:28.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:28.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:28.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:28.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:28.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:28.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698057+0000", 2024-09-18T04:57:28.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.531e-06 2024-09-18T04:57:28.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:28.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698064+0000", 2024-09-18T04:57:28.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-18T04:57:28.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698069+0000", 2024-09-18T04:57:28.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.091e-06 2024-09-18T04:57:28.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:28.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741843+0000", 2024-09-18T04:57:28.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043773470000000002 2024-09-18T04:57:28.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:28.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741928+0000", 2024-09-18T04:57:28.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5113999999999999e-05 2024-09-18T04:57:28.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:28.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742042+0000", 2024-09-18T04:57:28.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114527 2024-09-18T04:57:28.951 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.951 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.951 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.951 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46592 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2121728~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:28.951 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:57:28.951 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 39.096124629999998, 2024-09-18T04:57:28.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044077948999999998, 2024-09-18T04:57:28.951 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:28.951 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:28.951 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:28.951 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:28.951 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.952 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46592 2024-09-18T04:57:28.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.952 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:28.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:28.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:57:28.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:28.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:57:28.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:28.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698073+0000", 2024-09-18T04:57:28.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:28.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698076+0000", 2024-09-18T04:57:28.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.481e-06 2024-09-18T04:57:28.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:28.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698082+0000", 2024-09-18T04:57:28.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8730000000000004e-06 2024-09-18T04:57:28.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698086+0000", 2024-09-18T04:57:28.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:57:28.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:28.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742055+0000", 2024-09-18T04:57:28.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043969170000000002 2024-09-18T04:57:28.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:28.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742093+0000", 2024-09-18T04:57:28.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7707000000000003e-05 2024-09-18T04:57:28.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:28.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742152+0000", 2024-09-18T04:57:28.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8323999999999999e-05 2024-09-18T04:57:28.956 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.956 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.956 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.956 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46593 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2211840~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:28.956 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:28.956 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 39.096107727000003, 2024-09-18T04:57:28.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044259917000000003, 2024-09-18T04:57:28.956 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:28.956 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:28.956 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:28.957 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:28.957 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.957 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46593 2024-09-18T04:57:28.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.957 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:28.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:28.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:28.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:28.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:28.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:28.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:28.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:28.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698092+0000", 2024-09-18T04:57:28.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2249999999999999e-06 2024-09-18T04:57:28.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:28.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698098+0000", 2024-09-18T04:57:28.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:57:28.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698103+0000", 2024-09-18T04:57:28.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:57:28.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:28.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742182+0000", 2024-09-18T04:57:28.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044079508000000003 2024-09-18T04:57:28.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:28.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742257+0000", 2024-09-18T04:57:28.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4678000000000006e-05 2024-09-18T04:57:28.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:28.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742350+0000", 2024-09-18T04:57:28.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3531999999999998e-05 2024-09-18T04:57:28.960 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.960 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.960 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.961 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46594 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2236416~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:28.961 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:28.961 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 39.096092030000001, 2024-09-18T04:57:28.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044378984000000003, 2024-09-18T04:57:28.961 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:28.961 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:28.961 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:28.961 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:28.961 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.961 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46594 2024-09-18T04:57:28.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.961 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:28.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:28.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:28.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:28.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:28.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:28.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:28.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:28.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698108+0000", 2024-09-18T04:57:28.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0880000000000002e-06 2024-09-18T04:57:28.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:28.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698114+0000", 2024-09-18T04:57:28.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6269999999999997e-06 2024-09-18T04:57:28.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698118+0000", 2024-09-18T04:57:28.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4050000000000002e-06 2024-09-18T04:57:28.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:28.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742365+0000", 2024-09-18T04:57:28.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044246799000000003 2024-09-18T04:57:28.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:28.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742407+0000", 2024-09-18T04:57:28.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2008000000000002e-05 2024-09-18T04:57:28.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:28.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742485+0000", 2024-09-18T04:57:28.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8405999999999994e-05 2024-09-18T04:57:28.965 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.965 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.965 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.965 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46595 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:28.966 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:28.966 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 39.096075956999996, 2024-09-18T04:57:28.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.04440463, 2024-09-18T04:57:28.966 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:28.966 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:28.966 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:28.966 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:28.966 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.966 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46595 2024-09-18T04:57:28.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.966 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:28.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:28.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:28.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:28.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:28.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:28.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:28.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:28.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698124+0000", 2024-09-18T04:57:28.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1840000000000002e-06 2024-09-18T04:57:28.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:28.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698130+0000", 2024-09-18T04:57:28.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.947e-06 2024-09-18T04:57:28.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698136+0000", 2024-09-18T04:57:28.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6520000000000003e-06 2024-09-18T04:57:28.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:28.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742521+0000", 2024-09-18T04:57:28.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044385446000000002 2024-09-18T04:57:28.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:28.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742527+0000", 2024-09-18T04:57:28.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:57:28.970 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.970 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.970 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.970 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46596 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:28.970 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:28.970 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 39.096057713, 2024-09-18T04:57:28.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044418066999999999, 2024-09-18T04:57:28.971 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:28.971 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:28.971 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:28.971 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:28.971 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.971 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46596 2024-09-18T04:57:28.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.971 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:28.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:28.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:28.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:28.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:28.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:28.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:28.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:28.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698146+0000", 2024-09-18T04:57:28.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2779999999999999e-06 2024-09-18T04:57:28.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:28.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698153+0000", 2024-09-18T04:57:28.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3849999999999999e-06 2024-09-18T04:57:28.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698164+0000", 2024-09-18T04:57:28.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1375999999999999e-05 2024-09-18T04:57:28.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:28.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742554+0000", 2024-09-18T04:57:28.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044389482000000001 2024-09-18T04:57:28.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:28.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742559+0000", 2024-09-18T04:57:28.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9540000000000003e-06 2024-09-18T04:57:28.974 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.974 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.974 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.975 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46597 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:28.975 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:57:28.975 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 39.096029348000002, 2024-09-18T04:57:28.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044416513999999997, 2024-09-18T04:57:28.975 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:28.975 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:28.975 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:28.975 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:28.975 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.975 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46597 2024-09-18T04:57:28.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.976 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:28.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:28.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:57:28.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:28.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:57:28.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:28.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698168+0000", 2024-09-18T04:57:28.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:28.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698175+0000", 2024-09-18T04:57:28.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4520000000000002e-06 2024-09-18T04:57:28.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:28.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698181+0000", 2024-09-18T04:57:28.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4150000000000004e-06 2024-09-18T04:57:28.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698186+0000", 2024-09-18T04:57:28.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7489999999999998e-06 2024-09-18T04:57:28.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:28.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742580+0000", 2024-09-18T04:57:28.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044394325999999998 2024-09-18T04:57:28.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:28.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742585+0000", 2024-09-18T04:57:28.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9849999999999997e-06 2024-09-18T04:57:28.980 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.980 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.981 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.981 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46598 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:28.981 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:57:28.981 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 39.095921871999998, 2024-09-18T04:57:28.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044335316999999999, 2024-09-18T04:57:28.981 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:28.981 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:28.981 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:28.981 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:28.981 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.981 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46598 2024-09-18T04:57:28.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.982 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:28.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:28.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:57:28.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:28.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:57:28.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:28.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698190+0000", 2024-09-18T04:57:28.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999137 2024-09-18T04:57:28.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:28.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698280+0000", 2024-09-18T04:57:28.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0409999999999997e-05 2024-09-18T04:57:28.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:28.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698289+0000", 2024-09-18T04:57:28.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9490000000000002e-06 2024-09-18T04:57:28.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698296+0000", 2024-09-18T04:57:28.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2670000000000003e-06 2024-09-18T04:57:28.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:28.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742607+0000", 2024-09-18T04:57:28.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044311072999999999 2024-09-18T04:57:28.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:28.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742612+0000", 2024-09-18T04:57:28.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9400000000000001e-06 2024-09-18T04:57:28.985 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.985 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.985 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.985 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57986 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3086336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:28.985 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:28.985 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.109076149000003, 2024-09-18T04:57:28.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044047731999999999, 2024-09-18T04:57:28.985 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:28.986 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:28.986 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:28.986 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:28.986 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.986 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57986 2024-09-18T04:57:28.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.986 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:28.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:28.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:28.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:28.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:28.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:28.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:28.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:28.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685124+0000", 2024-09-18T04:57:28.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2859999999999998e-06 2024-09-18T04:57:28.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:28.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685130+0000", 2024-09-18T04:57:28.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5400000000000001e-06 2024-09-18T04:57:28.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685136+0000", 2024-09-18T04:57:28.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.152e-06 2024-09-18T04:57:28.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:28.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.728976+0000", 2024-09-18T04:57:28.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043840037999999998 2024-09-18T04:57:28.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:28.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729064+0000", 2024-09-18T04:57:28.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8709999999999996e-05 2024-09-18T04:57:28.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:28.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729170+0000", 2024-09-18T04:57:28.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000105386 2024-09-18T04:57:28.989 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.989 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.989 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.990 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57987 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3101696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:28.990 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:57:28.990 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.109027173999998, 2024-09-18T04:57:28.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044238123999999997, 2024-09-18T04:57:28.990 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:28.990 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:28.990 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:28.990 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:28.990 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.990 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57987 2024-09-18T04:57:28.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.990 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:28.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:28.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:57:28.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:28.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:57:28.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:28.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685170+0000", 2024-09-18T04:57:28.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:28.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:28.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685175+0000", 2024-09-18T04:57:28.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9069999999999998e-06 2024-09-18T04:57:28.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:28.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685187+0000", 2024-09-18T04:57:28.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.189e-05 2024-09-18T04:57:28.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685195+0000", 2024-09-18T04:57:28.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9459999999999998e-06 2024-09-18T04:57:28.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:28.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729212+0000", 2024-09-18T04:57:28.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044017239 2024-09-18T04:57:28.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:28.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729301+0000", 2024-09-18T04:57:28.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8887999999999993e-05 2024-09-18T04:57:28.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:28.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729409+0000", 2024-09-18T04:57:28.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108119 2024-09-18T04:57:28.995 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.995 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:28.995 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:28.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.995 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57988 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3117056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:28.995 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:57:28.995 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.108985054999998, 2024-09-18T04:57:28.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044429835000000001, 2024-09-18T04:57:28.996 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:28.996 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:28.996 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:28.996 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:28.996 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:28.996 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57988 2024-09-18T04:57:28.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.996 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:28.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:28.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:57:28.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:28.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:57:28.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:28.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:28.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685212+0000", 2024-09-18T04:57:28.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:28.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:28.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685216+0000", 2024-09-18T04:57:28.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7390000000000001e-06 2024-09-18T04:57:28.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:28.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685224+0000", 2024-09-18T04:57:28.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2260000000000005e-06 2024-09-18T04:57:28.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:28.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685230+0000", 2024-09-18T04:57:28.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0299999999999999e-06 2024-09-18T04:57:28.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:28.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:28.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:28.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729454+0000", 2024-09-18T04:57:28.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044223248999999999 2024-09-18T04:57:28.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:29.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729539+0000", 2024-09-18T04:57:29.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5029000000000002e-05 2024-09-18T04:57:29.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:29.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729643+0000", 2024-09-18T04:57:29.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104267 2024-09-18T04:57:29.000 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.000 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:29.000 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.001 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57989 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3132416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:29.001 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:57:29.001 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.108962302999998, 2024-09-18T04:57:29.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044642881000000002, 2024-09-18T04:57:29.001 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:29.001 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:29.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:29.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:29.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:29.001 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57989 2024-09-18T04:57:29.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.002 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:29.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:29.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:57:29.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:29.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:57:29.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:29.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685235+0000", 2024-09-18T04:57:29.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:29.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:29.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685238+0000", 2024-09-18T04:57:29.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.667e-06 2024-09-18T04:57:29.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:29.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685245+0000", 2024-09-18T04:57:29.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7909999999999999e-06 2024-09-18T04:57:29.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:29.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685250+0000", 2024-09-18T04:57:29.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3730000000000001e-06 2024-09-18T04:57:29.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:29.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729685+0000", 2024-09-18T04:57:29.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044434552000000002 2024-09-18T04:57:29.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:29.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729775+0000", 2024-09-18T04:57:29.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0395000000000004e-05 2024-09-18T04:57:29.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:29.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729879+0000", 2024-09-18T04:57:29.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010361099999999999 2024-09-18T04:57:29.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.005 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:29.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.005 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:29.005 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:29.006 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:29.006 INFO:teuthology.orchestra.run.smithi002.stdout:5361 read 0x348798b thru 0x3497331 (0xf9a7 bytes) 2024-09-18T04:57:29.006 INFO:teuthology.orchestra.run.smithi002.stdout:5362 read 0x387c0c8 thru 0x38826fc (0x6635 bytes) 2024-09-18T04:57:29.007 INFO:tasks.thrashosds.thrasher:j is {'epoch': 80, 'fsid': 'c0761d0c-69d5-4b64-b8b9-c7d098b2d06d', 'created': '2024-09-18T04:55:58.216538+0000', 'modified': '2024-09-18T04:57:26.397235+0000', 'last_up_change': '2024-09-18T04:56:08.369924+0000', 'last_in_change': '2024-09-18T04:57:04.291057+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-18T04:56:08.983344+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 4.380000114440918, 'score_stable': 4.380000114440918, 'optimal_score': 1, 'raw_score_acting': 4.380000114440918, 'raw_score_stable': 4.380000114440918, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-18T04:56:10.313988+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T04:56:19.562579+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 43, 'snap_epoch': 79, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.5299999713897705, 'score_stable': 1.5299999713897705, 'optimal_score': 1, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'c43f8024-3a3e-424e-aa54-398310ceb70d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 61, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6824', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6825', 'nonce': 4106704527}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6826', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6827', 'nonce': 4106704527}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6830', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6831', 'nonce': 4106704527}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6828', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6829', 'nonce': 4106704527}]}, 'public_addr': '172.21.15.2:6825/4106704527', 'cluster_addr': '172.21.15.2:6827/4106704527', 'heartbeat_back_addr': '172.21.15.2:6831/4106704527', 'heartbeat_front_addr': '172.21.15.2:6829/4106704527', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c664b970-145c-4b6f-8852-2f6dcf1fa30d', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6808', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6809', 'nonce': 1781959259}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6810', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6811', 'nonce': 1781959259}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6814', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6815', 'nonce': 1781959259}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6812', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6813', 'nonce': 1781959259}]}, 'public_addr': '172.21.15.2:6809/1781959259', 'cluster_addr': '172.21.15.2:6811/1781959259', 'heartbeat_back_addr': '172.21.15.2:6815/1781959259', 'heartbeat_front_addr': '172.21.15.2:6813/1781959259', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0609ea8c-d975-490e-8844-04a3e4512efd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 61, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6800', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6801', 'nonce': 1153570802}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6802', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6803', 'nonce': 1153570802}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6806', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6807', 'nonce': 1153570802}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6804', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6805', 'nonce': 1153570802}]}, 'public_addr': '172.21.15.2:6801/1153570802', 'cluster_addr': '172.21.15.2:6803/1153570802', 'heartbeat_back_addr': '172.21.15.2:6807/1153570802', 'heartbeat_front_addr': '172.21.15.2:6805/1153570802', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'c6bd5ac8-ab89-4fa2-b843-3b166d1b238b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 61, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6816', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6817', 'nonce': 2008321375}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6818', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6819', 'nonce': 2008321375}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6822', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6823', 'nonce': 2008321375}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6820', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6821', 'nonce': 2008321375}]}, 'public_addr': '172.21.15.2:6817/2008321375', 'cluster_addr': '172.21.15.2:6819/2008321375', 'heartbeat_back_addr': '172.21.15.2:6823/2008321375', 'heartbeat_front_addr': '172.21.15.2:6821/2008321375', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '89d6a371-4f4e-465e-a023-d5083cce6e6c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 77, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6824', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6825', 'nonce': 2939896598}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6826', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6827', 'nonce': 2939896598}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6830', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6831', 'nonce': 2939896598}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6828', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6829', 'nonce': 2939896598}]}, 'public_addr': '172.21.15.99:6825/2939896598', 'cluster_addr': '172.21.15.99:6827/2939896598', 'heartbeat_back_addr': '172.21.15.99:6831/2939896598', 'heartbeat_front_addr': '172.21.15.99:6829/2939896598', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'da99686b-d979-4753-b4aa-a3b26e9a0497', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 77, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6816', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6817', 'nonce': 3440879618}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6818', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6819', 'nonce': 3440879618}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6822', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6823', 'nonce': 3440879618}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6820', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6821', 'nonce': 3440879618}]}, 'public_addr': '172.21.15.99:6817/3440879618', 'cluster_addr': '172.21.15.99:6819/3440879618', 'heartbeat_back_addr': '172.21.15.99:6823/3440879618', 'heartbeat_front_addr': '172.21.15.99:6821/3440879618', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ceec6aa6-603d-4706-bfea-3c07dbdaed3a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 58, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6800', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6801', 'nonce': 2493830504}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6802', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6803', 'nonce': 2493830504}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6806', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6807', 'nonce': 2493830504}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6804', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6805', 'nonce': 2493830504}]}, 'public_addr': '172.21.15.99:6801/2493830504', 'cluster_addr': '172.21.15.99:6803/2493830504', 'heartbeat_back_addr': '172.21.15.99:6807/2493830504', 'heartbeat_front_addr': '172.21.15.99:6805/2493830504', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '6779dfad-f60b-41f4-b666-a99db6dc1814', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 77, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6808', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6809', 'nonce': 91038858}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6810', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6811', 'nonce': 91038858}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6814', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6815', 'nonce': 91038858}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6812', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6813', 'nonce': 91038858}]}, 'public_addr': '172.21.15.99:6809/91038858', 'cluster_addr': '172.21.15.99:6811/91038858', 'heartbeat_back_addr': '172.21.15.99:6815/91038858', 'heartbeat_front_addr': '172.21.15.99:6813/91038858', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.468213+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.538610+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.500881+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.474179+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.483737+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.460693+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.475970+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.519287+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '1.0', 'osds': [6, 4]}, {'pgid': '1.4', 'osds': [6, 7]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T04:57:29.007 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-18T04:57:29.009 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:57:29.012 INFO:teuthology.orchestra.run.smithi002.stdout:5363 writesame 0xac00e8c thru 0xac019d0 data_size 0x5(0xb45 bytes) 2024-09-18T04:57:29.012 INFO:teuthology.orchestra.run.smithi002.stdout:5364 read 0x516d201 thru 0x517851a (0xb31a bytes) 2024-09-18T04:57:29.013 INFO:teuthology.orchestra.run.smithi002.stdout:5365 punch from 0x53ece35 to 0x53ef871, (0x2a3c bytes) 2024-09-18T04:57:29.013 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:29.013 INFO:teuthology.orchestra.run.smithi002.stdout:5367 trunc from 0xac019d1 to 0x1b908f 2024-09-18T04:57:29.013 INFO:teuthology.orchestra.run.smithi002.stdout:5368 writesame 0xaaedfa7 thru 0xaaf1118 data_size 0x1(0x3172 bytes) 2024-09-18T04:57:29.013 INFO:teuthology.orchestra.run.smithi002.stdout:5369 write 0x6520854 thru 0x65271fe (0x69ab bytes) 2024-09-18T04:57:29.013 INFO:teuthology.orchestra.run.smithi002.stdout:5370 punch from 0xa828b87 to 0xa83860f, (0xfa88 bytes) 2024-09-18T04:57:29.013 INFO:teuthology.orchestra.run.smithi002.stdout:5371 writesame 0x681189c thru 0x6814db3 data_size 0x4(0x3518 bytes) 2024-09-18T04:57:29.013 INFO:teuthology.orchestra.run.smithi002.stdout:5373 read 0x6ae4b6a thru 0x6ae5ffa (0x1491 bytes) 2024-09-18T04:57:29.013 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:29.013 INFO:teuthology.orchestra.run.smithi002.stdout:5375 punch from 0x3b28fc to 0x3bce19, (0xa51d bytes) 2024-09-18T04:57:29.014 INFO:teuthology.orchestra.run.smithi002.stdout:5376 punch from 0x56f2fd2 to 0x56fb0ed, (0x811b bytes) 2024-09-18T04:57:29.014 INFO:teuthology.orchestra.run.smithi002.stdout:5377 read 0x52fb207 thru 0x53089bd (0xd7b7 bytes) 2024-09-18T04:57:29.014 INFO:teuthology.orchestra.run.smithi002.stdout:5378 write 0xcdc2212 thru 0xcdd0e90 (0xec7f bytes) 2024-09-18T04:57:29.014 INFO:teuthology.orchestra.run.smithi002.stdout:5379 read 0x14d76c9 thru 0x14e23b0 (0xace8 bytes) 2024-09-18T04:57:29.014 INFO:teuthology.orchestra.run.smithi002.stdout:5380 trunc from 0xcdd0e91 to 0x826a916 2024-09-18T04:57:29.014 INFO:teuthology.orchestra.run.smithi002.stdout:5381 read 0x6da4504 thru 0x6dae8dc (0xa3d9 bytes) 2024-09-18T04:57:29.014 INFO:teuthology.orchestra.run.smithi002.stdout:5383 trunc from 0x826a916 to 0x6a38160 2024-09-18T04:57:29.014 INFO:teuthology.orchestra.run.smithi002.stdout:5384 write 0x34593eb thru 0x34615fe (0x8214 bytes) 2024-09-18T04:57:29.014 INFO:teuthology.orchestra.run.smithi002.stdout:5385 write 0xb288b5d thru 0xb297b8b (0xf02f bytes) 2024-09-18T04:57:29.015 INFO:teuthology.orchestra.run.smithi002.stdout:5387 write 0x53f07c3 thru 0x53fb170 (0xa9ae bytes) 2024-09-18T04:57:29.015 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:29.007+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:29.035 INFO:teuthology.orchestra.run.smithi002.stdout:5388 punch from 0x220bf05 to 0x221be3c, (0xff37 bytes) 2024-09-18T04:57:29.036 INFO:teuthology.orchestra.run.smithi002.stdout:5389 read 0x58f5124 thru 0x5902235 (0xd112 bytes) 2024-09-18T04:57:29.036 INFO:teuthology.orchestra.run.smithi002.stdout:5390 write 0x3b0bcba thru 0x3b1625e (0xa5a5 bytes) 2024-09-18T04:57:29.055 INFO:teuthology.orchestra.run.smithi002.stdout:5391 read 0x39bb0dd thru 0x39bcf6d (0x1e91 bytes) 2024-09-18T04:57:29.056 INFO:teuthology.orchestra.run.smithi002.stdout:5392 read 0x9eb59e1 thru 0x9ebb74d (0x5d6d bytes) 2024-09-18T04:57:29.056 INFO:teuthology.orchestra.run.smithi002.stdout:5395 write 0x8763de2 thru 0x876d2d0 (0x94ef bytes) 2024-09-18T04:57:29.075 INFO:teuthology.orchestra.run.smithi002.stdout:5397 write 0x2488996 thru 0x24977d1 (0xee3c bytes) 2024-09-18T04:57:29.092 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:29.092 INFO:teuthology.orchestra.run.smithi002.stdout:5399 write 0x60e72d0 thru 0x60f1c38 (0xa969 bytes) 2024-09-18T04:57:29.109 INFO:teuthology.orchestra.run.smithi002.stdout:5400 read 0xaa7bc0c thru 0xaa85d20 (0xa115 bytes) 2024-09-18T04:57:29.109 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:29.109 INFO:teuthology.orchestra.run.smithi002.stdout:5409 write 0xa3f5f7a thru 0xa401ad0 (0xbb57 bytes) 2024-09-18T04:57:29.110 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:29.108+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:29.128 INFO:teuthology.orchestra.run.smithi002.stdout:5410 writesame 0x93dbe30 thru 0x93e0380 data_size 0x41(0x4551 bytes) 2024-09-18T04:57:29.139 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:29.139 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:29.139 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:29.139 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:29.147 INFO:teuthology.orchestra.run.smithi002.stdout:5411 write 0x91e1776 thru 0x91ec401 (0xac8c bytes) 2024-09-18T04:57:29.150 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:57:29.173 INFO:teuthology.orchestra.run.smithi002.stdout:5412 writesame 0xe8e2e0d thru 0xe8e947c data_size 0x2c(0x6670 bytes) 2024-09-18T04:57:29.173 INFO:teuthology.orchestra.run.smithi002.stdout:5414 write 0xcefb29e thru 0xcf0a0c9 (0xee2c bytes) 2024-09-18T04:57:29.189 INFO:teuthology.orchestra.run.smithi002.stdout:5415 read 0x2d9763e thru 0x2da4cbb (0xd67e bytes) 2024-09-18T04:57:29.196 INFO:teuthology.orchestra.run.smithi002.stdout:5417 trunc from 0xe8e947d to 0x86d4e24 2024-09-18T04:57:29.212 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:29.209+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:29.232 INFO:teuthology.orchestra.run.smithi002.stdout:5418 punch from 0x55990f6 to 0x55a0df5, (0x7cff bytes) 2024-09-18T04:57:29.232 INFO:teuthology.orchestra.run.smithi002.stdout:5419 punch from 0x57d583a to 0x57e1946, (0xc10c bytes) 2024-09-18T04:57:29.233 INFO:teuthology.orchestra.run.smithi002.stdout:5423 writesame 0x6c5622b thru 0x6c5a2cc data_size 0x1(0x40a2 bytes) 2024-09-18T04:57:29.254 INFO:teuthology.orchestra.run.smithi002.stdout:5425 trunc from 0x86d4e24 to 0x45e0b21 2024-09-18T04:57:29.288 INFO:teuthology.orchestra.run.smithi002.stdout:5428 write 0x22020ab thru 0x2202397 (0x2ed bytes) 2024-09-18T04:57:29.301 INFO:teuthology.orchestra.run.smithi002.stdout:5429 write 0x983c8c1 thru 0x9849845 (0xcf85 bytes) 2024-09-18T04:57:29.303 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:29.304 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:29.304 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:29.304 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:29.304 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:29.312 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:29.310+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:29.315 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:57:29.319 INFO:teuthology.orchestra.run.smithi002.stdout:5432 punch from 0x743fef1 to 0x744b401, (0xb510 bytes) 2024-09-18T04:57:29.319 INFO:teuthology.orchestra.run.smithi002.stdout:5434 read 0x47073ad thru 0x470a8a4 (0x34f8 bytes) 2024-09-18T04:57:29.320 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:29.320 INFO:teuthology.orchestra.run.smithi002.stdout:5437 punch from 0x35c31a8 to 0x35c70bd, (0x3f15 bytes) 2024-09-18T04:57:29.320 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:29.331 INFO:teuthology.orchestra.run.smithi002.stdout:5439 writesame 0xe4ba196 thru 0xe4c1c6f data_size 0x25(0x7ada bytes) 2024-09-18T04:57:29.333 INFO:teuthology.orchestra.run.smithi002.stdout:5440 trunc from 0xe4c1c70 to 0xc927598 2024-09-18T04:57:29.339 INFO:teuthology.orchestra.run.smithi002.stdout:5442 writesame 0x8eb7b77 thru 0x8ec08f9 data_size 0x1(0x8d83 bytes) 2024-09-18T04:57:29.365 INFO:teuthology.orchestra.run.smithi002.stdout:5443 write 0xdb2dc71 thru 0xdb3bcb8 (0xe048 bytes) 2024-09-18T04:57:29.384 INFO:teuthology.orchestra.run.smithi002.stdout:5444 read 0x1e0ff5d thru 0x1e17c8f (0x7d33 bytes) 2024-09-18T04:57:29.384 INFO:teuthology.orchestra.run.smithi002.stdout:5447 writesame 0x9c668e6 thru 0x9c759d1 data_size 0x22(0xf0ec bytes) 2024-09-18T04:57:29.401 INFO:teuthology.orchestra.run.smithi002.stdout:5448 write 0x2a7782b thru 0x2a78d87 (0x155d bytes) 2024-09-18T04:57:29.414 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:29.411+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:29.422 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-18T04:57:29.430 INFO:teuthology.orchestra.run.smithi002.stdout:5449 trunc from 0xdb3bcb9 to 0x2c59ab0 2024-09-18T04:57:29.454 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:29.454 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:29.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:29.454 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:29.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.454 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14125 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:29.455 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:57:29.455 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.323165191000001, 2024-09-18T04:57:29.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052956470999999998, 2024-09-18T04:57:29.455 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:29.455 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:29.455 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:29.455 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:29.455 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:29.456 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14125 2024-09-18T04:57:29.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.456 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:29.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:29.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:57:29.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:29.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:57:29.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:29.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126849+0000", 2024-09-18T04:57:29.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:29.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:29.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126852+0000", 2024-09-18T04:57:29.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3659999999999999e-06 2024-09-18T04:57:29.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:29.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126858+0000", 2024-09-18T04:57:29.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4540000000000004e-06 2024-09-18T04:57:29.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:29.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126863+0000", 2024-09-18T04:57:29.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1780000000000002e-06 2024-09-18T04:57:29.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:29.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179608+0000", 2024-09-18T04:57:29.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052745159999999999 2024-09-18T04:57:29.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:29.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179703+0000", 2024-09-18T04:57:29.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4773999999999998e-05 2024-09-18T04:57:29.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:29.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179806+0000", 2024-09-18T04:57:29.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010296799999999999 2024-09-18T04:57:29.461 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.461 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:29.461 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.461 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14136 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1732608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:29.461 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:29.462 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.321004639999998, 2024-09-18T04:57:29.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052973412999999997, 2024-09-18T04:57:29.462 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:29.462 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:29.462 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:29.462 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:29.462 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:29.462 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14136 2024-09-18T04:57:29.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.462 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:29.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:29.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:29.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:29.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:29.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:29.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:29.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:29.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:29.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129013+0000", 2024-09-18T04:57:29.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.96e-06 2024-09-18T04:57:29.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:29.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129020+0000", 2024-09-18T04:57:29.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6809999999999998e-06 2024-09-18T04:57:29.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:29.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129026+0000", 2024-09-18T04:57:29.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:57:29.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:29.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181873+0000", 2024-09-18T04:57:29.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052847238999999997 2024-09-18T04:57:29.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:29.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181932+0000", 2024-09-18T04:57:29.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9339000000000002e-05 2024-09-18T04:57:29.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:29.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181984+0000", 2024-09-18T04:57:29.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1348999999999999e-05 2024-09-18T04:57:29.467 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.467 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:29.467 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.467 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14137 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1781760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:29.467 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:57:29.467 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.320984234999997, 2024-09-18T04:57:29.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053128914999999999, 2024-09-18T04:57:29.467 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:29.467 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:29.468 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:29.468 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:29.468 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:29.468 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14137 2024-09-18T04:57:29.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.468 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:29.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:29.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:57:29.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:29.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:57:29.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:29.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129030+0000", 2024-09-18T04:57:29.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:29.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:29.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129033+0000", 2024-09-18T04:57:29.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7439999999999999e-06 2024-09-18T04:57:29.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:29.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129039+0000", 2024-09-18T04:57:29.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.314e-06 2024-09-18T04:57:29.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:29.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129044+0000", 2024-09-18T04:57:29.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9919999999999998e-06 2024-09-18T04:57:29.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:29.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182015+0000", 2024-09-18T04:57:29.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052970875000000001 2024-09-18T04:57:29.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:29.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182058+0000", 2024-09-18T04:57:29.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2997000000000002e-05 2024-09-18T04:57:29.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:29.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182160+0000", 2024-09-18T04:57:29.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101437 2024-09-18T04:57:29.473 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.473 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:29.473 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.473 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14138 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1830912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:29.473 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:57:29.474 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.320966200999997, 2024-09-18T04:57:29.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053256435999999997, 2024-09-18T04:57:29.474 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:29.474 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:29.474 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:29.474 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:29.474 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:29.474 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14138 2024-09-18T04:57:29.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.474 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:29.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:29.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:57:29.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:29.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:57:29.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:29.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129048+0000", 2024-09-18T04:57:29.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:29.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:29.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129056+0000", 2024-09-18T04:57:29.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.413e-06 2024-09-18T04:57:29.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:29.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129062+0000", 2024-09-18T04:57:29.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6810000000000001e-06 2024-09-18T04:57:29.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:29.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129068+0000", 2024-09-18T04:57:29.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0459999999999996e-06 2024-09-18T04:57:29.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:29.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182191+0000", 2024-09-18T04:57:29.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053123210999999997 2024-09-18T04:57:29.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:29.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182238+0000", 2024-09-18T04:57:29.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7404999999999998e-05 2024-09-18T04:57:29.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:29.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182305+0000", 2024-09-18T04:57:29.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7028999999999998e-05 2024-09-18T04:57:29.480 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.480 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:29.480 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.480 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15910 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2236416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:29.480 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:29.480 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.221878668999999, 2024-09-18T04:57:29.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053241054000000003, 2024-09-18T04:57:29.481 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:29.481 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:29.481 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:29.481 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:29.481 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:29.481 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15910 2024-09-18T04:57:29.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.481 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:29.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:29.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:29.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:29.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:29.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:29.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:29.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:29.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:29.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228138+0000", 2024-09-18T04:57:29.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:57:29.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:29.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228144+0000", 2024-09-18T04:57:29.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5369999999999997e-06 2024-09-18T04:57:29.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:29.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228149+0000", 2024-09-18T04:57:29.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1209999999999998e-06 2024-09-18T04:57:29.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:29.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280822+0000", 2024-09-18T04:57:29.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052673194999999999 2024-09-18T04:57:29.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:29.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280903+0000", 2024-09-18T04:57:29.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1179000000000003e-05 2024-09-18T04:57:29.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:29.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281377+0000", 2024-09-18T04:57:29.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047417899999999997 2024-09-18T04:57:29.486 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.486 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:29.486 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.487 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15911 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2285568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:29.487 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:29.487 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.221861937, 2024-09-18T04:57:29.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053415151000000001, 2024-09-18T04:57:29.487 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:29.487 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:29.487 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:29.487 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:29.487 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:29.487 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15911 2024-09-18T04:57:29.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.488 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:29.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:29.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:29.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:29.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:29.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:29.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:29.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:29.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:29.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228155+0000", 2024-09-18T04:57:29.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:57:29.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:29.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228161+0000", 2024-09-18T04:57:29.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:57:29.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:29.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228166+0000", 2024-09-18T04:57:29.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2320000000000001e-06 2024-09-18T04:57:29.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:29.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281405+0000", 2024-09-18T04:57:29.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053239336999999998 2024-09-18T04:57:29.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:29.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281482+0000", 2024-09-18T04:57:29.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6569999999999994e-05 2024-09-18T04:57:29.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:29.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281568+0000", 2024-09-18T04:57:29.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6328000000000004e-05 2024-09-18T04:57:29.493 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.493 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:29.493 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.493 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15918 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2662400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:29.493 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:29.494 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.220092956000002, 2024-09-18T04:57:29.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053091065, 2024-09-18T04:57:29.494 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:29.494 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:29.494 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:29.494 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:29.494 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:29.494 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15918 2024-09-18T04:57:29.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.494 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:29.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:29.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:29.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:29.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:29.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:29.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:29.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:29.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:29.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229924+0000", 2024-09-18T04:57:29.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:57:29.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:29.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229930+0000", 2024-09-18T04:57:29.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.066e-06 2024-09-18T04:57:29.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:29.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229935+0000", 2024-09-18T04:57:29.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9119999999999997e-06 2024-09-18T04:57:29.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:29.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282840+0000", 2024-09-18T04:57:29.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052905119 2024-09-18T04:57:29.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:29.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282922+0000", 2024-09-18T04:57:29.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2081000000000002e-05 2024-09-18T04:57:29.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:29.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283013+0000", 2024-09-18T04:57:29.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0828000000000005e-05 2024-09-18T04:57:29.499 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.499 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:29.499 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.499 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15919 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2711552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:29.499 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:29.499 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.220075678000001, 2024-09-18T04:57:29.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053293021000000003, 2024-09-18T04:57:29.499 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:29.499 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:29.500 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:29.500 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:29.500 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:29.500 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15919 2024-09-18T04:57:29.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.500 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:29.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:29.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:29.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:29.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:29.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:29.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:29.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:29.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:29.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229941+0000", 2024-09-18T04:57:29.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2639999999999998e-06 2024-09-18T04:57:29.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:29.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229947+0000", 2024-09-18T04:57:29.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9750000000000004e-06 2024-09-18T04:57:29.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:29.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229952+0000", 2024-09-18T04:57:29.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7929999999999997e-06 2024-09-18T04:57:29.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:29.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283062+0000", 2024-09-18T04:57:29.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05310952 2024-09-18T04:57:29.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:29.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283162+0000", 2024-09-18T04:57:29.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100098 2024-09-18T04:57:29.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:29.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283232+0000", 2024-09-18T04:57:29.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0693999999999998e-05 2024-09-18T04:57:29.504 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.504 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:29.504 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.505 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15920 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2760704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:29.505 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:29.505 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.220058917000003, 2024-09-18T04:57:29.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053502367000000002, 2024-09-18T04:57:29.505 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:29.505 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:29.505 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:29.505 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:29.505 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:29.506 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15920 2024-09-18T04:57:29.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.506 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:29.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:29.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:29.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:29.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:29.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:29.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:29.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:29.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:29.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229958+0000", 2024-09-18T04:57:29.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1770000000000001e-06 2024-09-18T04:57:29.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:29.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229964+0000", 2024-09-18T04:57:29.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8470000000000001e-06 2024-09-18T04:57:29.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:29.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229968+0000", 2024-09-18T04:57:29.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5680000000000001e-06 2024-09-18T04:57:29.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:29.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283269+0000", 2024-09-18T04:57:29.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053300620999999999 2024-09-18T04:57:29.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:29.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283359+0000", 2024-09-18T04:57:29.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9971999999999998e-05 2024-09-18T04:57:29.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:29.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283458+0000", 2024-09-18T04:57:29.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9483999999999998e-05 2024-09-18T04:57:29.510 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.510 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:29.510 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.511 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15921 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2809856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:29.511 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:57:29.511 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.220042366999998, 2024-09-18T04:57:29.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053800977, 2024-09-18T04:57:29.511 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:29.511 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:29.511 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:29.512 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:29.512 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:29.512 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15921 2024-09-18T04:57:29.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.512 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:29.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:29.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:57:29.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:29.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:57:29.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:29.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229972+0000", 2024-09-18T04:57:29.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:29.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:29.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229974+0000", 2024-09-18T04:57:29.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2910000000000002e-06 2024-09-18T04:57:29.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:29.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229982+0000", 2024-09-18T04:57:29.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7249999999999997e-06 2024-09-18T04:57:29.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:29.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229989+0000", 2024-09-18T04:57:29.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6429999999999998e-06 2024-09-18T04:57:29.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:29.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283601+0000", 2024-09-18T04:57:29.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053612211 2024-09-18T04:57:29.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:29.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283681+0000", 2024-09-18T04:57:29.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0125999999999998e-05 2024-09-18T04:57:29.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:29.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283774+0000", 2024-09-18T04:57:29.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2362999999999996e-05 2024-09-18T04:57:29.518 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.518 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:29.518 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.518 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15922 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2859008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:29.518 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:57:29.519 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.219997427999999, 2024-09-18T04:57:29.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053882270000000003, 2024-09-18T04:57:29.519 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:29.519 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:29.519 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:29.519 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:29.519 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:29.519 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15922 2024-09-18T04:57:29.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.519 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:29.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:29.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:57:29.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:29.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:57:29.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:29.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230017+0000", 2024-09-18T04:57:29.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:29.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:29.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230019+0000", 2024-09-18T04:57:29.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4210000000000002e-06 2024-09-18T04:57:29.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:29.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230027+0000", 2024-09-18T04:57:29.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000002e-06 2024-09-18T04:57:29.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:29.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230032+0000", 2024-09-18T04:57:29.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0089999999999998e-06 2024-09-18T04:57:29.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:29.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283800+0000", 2024-09-18T04:57:29.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053768418999999998 2024-09-18T04:57:29.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:29.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283849+0000", 2024-09-18T04:57:29.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9339000000000003e-05 2024-09-18T04:57:29.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:29.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283900+0000", 2024-09-18T04:57:29.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0421999999999999e-05 2024-09-18T04:57:29.523 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.523 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:29.523 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.524 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15923 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2924544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:29.524 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:29.524 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.219978558999998, 2024-09-18T04:57:29.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054073016000000002, 2024-09-18T04:57:29.524 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:29.524 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:29.524 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:29.524 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:29.524 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:29.525 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15923 2024-09-18T04:57:29.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.525 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:29.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:29.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:29.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:29.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:29.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:29.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:29.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:29.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:29.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230038+0000", 2024-09-18T04:57:29.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.463e-06 2024-09-18T04:57:29.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:29.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230045+0000", 2024-09-18T04:57:29.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3570000000000003e-06 2024-09-18T04:57:29.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:29.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230050+0000", 2024-09-18T04:57:29.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7299999999999996e-06 2024-09-18T04:57:29.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:29.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283928+0000", 2024-09-18T04:57:29.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053878218999999998 2024-09-18T04:57:29.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:29.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284010+0000", 2024-09-18T04:57:29.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2577000000000006e-05 2024-09-18T04:57:29.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:29.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284109+0000", 2024-09-18T04:57:29.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9102000000000001e-05 2024-09-18T04:57:29.530 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.530 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:29.530 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.530 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15924 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:29.531 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:29.531 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.219960088999997, 2024-09-18T04:57:29.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054108470999999998, 2024-09-18T04:57:29.531 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:29.531 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:29.531 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:29.531 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:29.531 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:29.531 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15924 2024-09-18T04:57:29.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.532 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:29.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:29.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:29.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:29.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:29.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:29.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:29.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:29.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:29.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230061+0000", 2024-09-18T04:57:29.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8240000000000002e-06 2024-09-18T04:57:29.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:29.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230068+0000", 2024-09-18T04:57:29.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4320000000000004e-06 2024-09-18T04:57:29.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:29.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230073+0000", 2024-09-18T04:57:29.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.921e-06 2024-09-18T04:57:29.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:29.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284153+0000", 2024-09-18T04:57:29.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054080793000000002 2024-09-18T04:57:29.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:29.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284163+0000", 2024-09-18T04:57:29.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.859e-06 2024-09-18T04:57:29.536 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.536 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:29.536 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.536 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15925 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:29.536 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:29.536 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.219935974000002, 2024-09-18T04:57:29.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054134632000000002, 2024-09-18T04:57:29.536 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:29.536 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:29.536 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:29.537 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:29.537 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:29.537 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15925 2024-09-18T04:57:29.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.537 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:29.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:29.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:29.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:29.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:29.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:29.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:29.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:29.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:29.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230081+0000", 2024-09-18T04:57:29.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4109999999999999e-06 2024-09-18T04:57:29.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:29.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230087+0000", 2024-09-18T04:57:29.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.161e-06 2024-09-18T04:57:29.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:29.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230092+0000", 2024-09-18T04:57:29.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9259999999999999e-06 2024-09-18T04:57:29.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:29.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284204+0000", 2024-09-18T04:57:29.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054111546000000003 2024-09-18T04:57:29.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:29.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284214+0000", 2024-09-18T04:57:29.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9639999999999998e-06 2024-09-18T04:57:29.541 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.541 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:29.541 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.542 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15926 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:29.542 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:29.542 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.219918649999997, 2024-09-18T04:57:29.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054143193999999999, 2024-09-18T04:57:29.542 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:29.542 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:29.542 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:29.542 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:29.543 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:29.543 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15926 2024-09-18T04:57:29.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.543 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:29.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:29.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:29.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:29.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:29.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:29.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:29.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:29.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:29.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230098+0000", 2024-09-18T04:57:29.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1399999999999998e-06 2024-09-18T04:57:29.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:29.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230104+0000", 2024-09-18T04:57:29.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-18T04:57:29.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:29.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230109+0000", 2024-09-18T04:57:29.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9250000000000003e-06 2024-09-18T04:57:29.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:29.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284231+0000", 2024-09-18T04:57:29.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054122297999999999 2024-09-18T04:57:29.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:29.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284239+0000", 2024-09-18T04:57:29.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1689999999999992e-06 2024-09-18T04:57:29.547 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.547 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:29.548 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.548 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15927 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:29.548 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:29.548 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.219901602999997, 2024-09-18T04:57:29.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054168435000000001, 2024-09-18T04:57:29.548 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:29.549 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:29.549 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:29.549 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:29.549 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:29.549 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15927 2024-09-18T04:57:29.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.549 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:29.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:29.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:29.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:29.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:29.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:29.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:29.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:29.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:29.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230116+0000", 2024-09-18T04:57:29.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5150000000000001e-06 2024-09-18T04:57:29.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:29.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230122+0000", 2024-09-18T04:57:29.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1190000000000002e-06 2024-09-18T04:57:29.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:29.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230126+0000", 2024-09-18T04:57:29.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.792e-06 2024-09-18T04:57:29.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:29.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284273+0000", 2024-09-18T04:57:29.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054146086000000003 2024-09-18T04:57:29.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:29.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284282+0000", 2024-09-18T04:57:29.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2590000000000006e-06 2024-09-18T04:57:29.553 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.553 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:29.553 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.553 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15928 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:29.554 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:57:29.554 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.219884219999997, 2024-09-18T04:57:29.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054189739000000001, 2024-09-18T04:57:29.554 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:29.554 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:29.554 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:29.554 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:29.554 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:29.554 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15928 2024-09-18T04:57:29.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.554 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:29.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:29.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:57:29.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:29.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:57:29.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:29.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230130+0000", 2024-09-18T04:57:29.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:29.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:29.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230133+0000", 2024-09-18T04:57:29.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.244e-06 2024-09-18T04:57:29.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:29.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230139+0000", 2024-09-18T04:57:29.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9830000000000001e-06 2024-09-18T04:57:29.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:29.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230143+0000", 2024-09-18T04:57:29.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7530000000000001e-06 2024-09-18T04:57:29.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:29.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284313+0000", 2024-09-18T04:57:29.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054169332000000001 2024-09-18T04:57:29.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:29.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284320+0000", 2024-09-18T04:57:29.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8699999999999992e-06 2024-09-18T04:57:29.559 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.559 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:29.559 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.560 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15929 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:29.560 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:29.560 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.219867780000001, 2024-09-18T04:57:29.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054218771999999998, 2024-09-18T04:57:29.560 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:29.560 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:29.560 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:29.560 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:29.560 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:29.560 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15929 2024-09-18T04:57:29.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.561 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:29.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:29.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:29.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:29.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:29.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:29.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:29.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:29.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:29.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230149+0000", 2024-09-18T04:57:29.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-18T04:57:29.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:29.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230155+0000", 2024-09-18T04:57:29.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2469999999999997e-06 2024-09-18T04:57:29.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:29.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230160+0000", 2024-09-18T04:57:29.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7129999999999996e-06 2024-09-18T04:57:29.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:29.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284355+0000", 2024-09-18T04:57:29.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054194900999999997 2024-09-18T04:57:29.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:29.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284366+0000", 2024-09-18T04:57:29.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0993e-05 2024-09-18T04:57:29.566 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.566 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:29.566 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.566 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15930 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:29.566 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:29.566 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.219850780000002, 2024-09-18T04:57:29.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054244713999999999, 2024-09-18T04:57:29.567 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:29.567 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:29.567 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:29.567 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:29.567 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:29.567 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15930 2024-09-18T04:57:29.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.567 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:29.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:29.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:29.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:29.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:29.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:29.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:29.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:29.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:29.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230166+0000", 2024-09-18T04:57:29.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1009999999999999e-06 2024-09-18T04:57:29.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:29.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230172+0000", 2024-09-18T04:57:29.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.981e-06 2024-09-18T04:57:29.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:29.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230184+0000", 2024-09-18T04:57:29.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2449000000000001e-05 2024-09-18T04:57:29.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:29.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284400+0000", 2024-09-18T04:57:29.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054215452999999997 2024-09-18T04:57:29.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:29.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284409+0000", 2024-09-18T04:57:29.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0839999999999998e-06 2024-09-18T04:57:29.571 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.571 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:29.571 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.571 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15931 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:29.572 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:57:29.572 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 53.219824991000003, 2024-09-18T04:57:29.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054261928000000001, 2024-09-18T04:57:29.572 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:29.572 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:29.572 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:29.572 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:29.572 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:29.572 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15931 2024-09-18T04:57:29.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.573 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:29.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:29.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:57:29.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:29.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:57:29.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:29.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:29.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230189+0000", 2024-09-18T04:57:29.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:29.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:29.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230192+0000", 2024-09-18T04:57:29.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6599999999999999e-06 2024-09-18T04:57:29.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:29.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230199+0000", 2024-09-18T04:57:29.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5570000000000001e-06 2024-09-18T04:57:29.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.575 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:29.575 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230204+0000", 2024-09-18T04:57:29.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2270000000000001e-06 2024-09-18T04:57:29.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:29.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284442+0000", 2024-09-18T04:57:29.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054238383000000001 2024-09-18T04:57:29.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:29.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:29.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:29.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284452+0000", 2024-09-18T04:57:29.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6069999999999997e-06 2024-09-18T04:57:29.577 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.577 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:29.577 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.577 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:29.577 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:29.577 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:29.578 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:57:29.579 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:29.576+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:29.580 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:57:29.582 INFO:teuthology.orchestra.run.smithi002.stdout:5450 write 0xc70214 thru 0xc7ba40 (0xb82d bytes) 2024-09-18T04:57:29.582 INFO:teuthology.orchestra.run.smithi002.stdout:5453 read 0xf53343 thru 0xf538a1 (0x55f bytes) 2024-09-18T04:57:29.582 INFO:teuthology.orchestra.run.smithi002.stdout:5454 writesame 0x8589238 thru 0x85903a3 data_size 0x11(0x716c bytes) 2024-09-18T04:57:29.582 INFO:teuthology.orchestra.run.smithi002.stdout:5456 read 0x24ed477 thru 0x24fc718 (0xf2a2 bytes) 2024-09-18T04:57:29.582 INFO:teuthology.orchestra.run.smithi002.stdout:5458 punch from 0xd9964 to 0xdcde3, (0x347f bytes) 2024-09-18T04:57:29.582 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:29.582 INFO:teuthology.orchestra.run.smithi002.stdout:5460 write 0x5e321a6 thru 0x5e3c41b (0xa276 bytes) 2024-09-18T04:57:29.583 INFO:teuthology.orchestra.run.smithi002.stdout:5461 punch from 0x833b1ad to 0x833efc2, (0x3e15 bytes) 2024-09-18T04:57:29.583 INFO:teuthology.orchestra.run.smithi002.stdout:5462 writesame 0xedfaa74 thru 0xee05049 data_size 0x1(0xa5d6 bytes) 2024-09-18T04:57:29.583 INFO:teuthology.orchestra.run.smithi002.stdout:5463 writesame 0x199a97d thru 0x19a561a data_size 0x2d(0xac9e bytes) 2024-09-18T04:57:29.583 INFO:teuthology.orchestra.run.smithi002.stdout:5464 write 0x3b6c85d thru 0x3b7527d (0x8a21 bytes) 2024-09-18T04:57:29.583 INFO:teuthology.orchestra.run.smithi002.stdout:5466 trunc from 0xee0504a to 0xea9e1ad 2024-09-18T04:57:29.586 INFO:teuthology.orchestra.run.smithi002.stdout:5468 write 0xac6fc3e thru 0xac70e07 (0x11ca bytes) 2024-09-18T04:57:29.599 INFO:teuthology.orchestra.run.smithi002.stdout:5470 write 0x6bacb83 thru 0x6bb88ca (0xbd48 bytes) 2024-09-18T04:57:29.616 INFO:teuthology.orchestra.run.smithi002.stdout:5471 write 0xe153e90 thru 0xe15d32f (0x94a0 bytes) 2024-09-18T04:57:29.642 INFO:teuthology.orchestra.run.smithi002.stdout:5472 trunc from 0xea9e1ad to 0x8e2ff10 2024-09-18T04:57:29.662 INFO:teuthology.orchestra.run.smithi002.stdout:5473 punch from 0x1290dd5 to 0x129dd0b, (0xcf36 bytes) 2024-09-18T04:57:29.663 INFO:teuthology.orchestra.run.smithi002.stdout:5475 punch from 0xfe3836 to 0xfe4c02, (0x13cc bytes) 2024-09-18T04:57:29.663 INFO:teuthology.orchestra.run.smithi002.stdout:5477 write 0x6624a79 thru 0x66294eb (0x4a73 bytes) 2024-09-18T04:57:29.679 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:29.677+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:29.683 INFO:teuthology.orchestra.run.smithi002.stdout:5479 punch from 0x3d22d14 to 0x3d2b85d, (0x8b49 bytes) 2024-09-18T04:57:29.686 INFO:teuthology.orchestra.run.smithi002.stdout:5480 trunc from 0x8e2ff10 to 0xba332ec 2024-09-18T04:57:29.695 INFO:teuthology.orchestra.run.smithi002.stdout:5481 trunc from 0xba332ec to 0xe6c8444 2024-09-18T04:57:29.701 INFO:teuthology.orchestra.run.smithi002.stdout:5483 writesame 0xcaad8a4 thru 0xcab0981 data_size 0x9(0x30de bytes) 2024-09-18T04:57:29.714 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:29.714 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:29.714 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:29.714 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:29.720 INFO:teuthology.orchestra.run.smithi002.stdout:5484 punch from 0xa814c92 to 0xa8164db, (0x1849 bytes) 2024-09-18T04:57:29.724 INFO:teuthology.orchestra.run.smithi002.stdout:5485 trunc from 0xe6c8444 to 0xb7ad8d1 2024-09-18T04:57:29.725 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:57:29.734 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:57:29.734 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:57:29.737 INFO:teuthology.orchestra.run.smithi002.stdout:5486 read 0x9bd03d thru 0x9c2409 (0x53cd bytes) 2024-09-18T04:57:29.737 INFO:teuthology.orchestra.run.smithi002.stdout:5488 write 0x63b3ec8 thru 0x63b6ec9 (0x3002 bytes) 2024-09-18T04:57:29.753 INFO:teuthology.orchestra.run.smithi002.stdout:5490 punch from 0xa494aa6 to 0xa49e4ac, (0x9a06 bytes) 2024-09-18T04:57:29.753 INFO:teuthology.orchestra.run.smithi002.stdout:5492 read 0x5500330 thru 0x550483f (0x4510 bytes) 2024-09-18T04:57:29.756 INFO:teuthology.orchestra.run.smithi002.stdout:5493 trunc from 0xb7ad8d1 to 0x8926116 2024-09-18T04:57:29.757 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:57:29.757 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:57:29.762 INFO:teuthology.orchestra.run.smithi002.stdout:5494 write 0x4e5f171 thru 0x4e62ae9 (0x3979 bytes) 2024-09-18T04:57:29.778 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:57:29.779 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:57:29.780 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:29.777+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:29.780 INFO:teuthology.orchestra.run.smithi002.stdout:5495 writesame 0xf4ac9b thru 0xf4b061 data_size 0x1(0x3c7 bytes) 2024-09-18T04:57:29.784 INFO:teuthology.orchestra.run.smithi002.stdout:5496 read 0x7958737 thru 0x79613c1 (0x8c8b bytes) 2024-09-18T04:57:29.795 INFO:teuthology.orchestra.run.smithi002.stdout:5497 writesame 0xcd4fc56 thru 0xcd502a1 data_size 0x1(0x64c bytes) 2024-09-18T04:57:29.796 INFO:teuthology.orchestra.run.smithi002.stdout:5498 writesame 0x6b8e52d thru 0x6b9dac4 data_size 0x74(0xf598 bytes) 2024-09-18T04:57:29.800 INFO:teuthology.orchestra.run.smithi002.stdout:5499 read 0x98b6446 thru 0x98bfca2 (0x985d bytes) 2024-09-18T04:57:29.800 INFO:teuthology.orchestra.run.smithi002.stdout:5500 read 0x3fa7a34 thru 0x3fb09d1 (0x8f9e bytes) 2024-09-18T04:57:29.803 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:57:29.803 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:57:29.807 INFO:teuthology.orchestra.run.smithi002.stdout:5502 write 0xd3811c8 thru 0xd38b317 (0xa150 bytes) 2024-09-18T04:57:29.824 INFO:teuthology.orchestra.run.smithi002.stdout:5503 read 0x2edf44b thru 0x2ee9278 (0x9e2e bytes) 2024-09-18T04:57:29.824 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:29.824 INFO:teuthology.orchestra.run.smithi002.stdout:5505 write 0x639a229 thru 0x639d480 (0x3258 bytes) 2024-09-18T04:57:29.825 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:57:29.825 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:57:29.837 INFO:teuthology.orchestra.run.smithi002.stdout:5506 write 0x7e78aa9 thru 0x7e7acb9 (0x2211 bytes) 2024-09-18T04:57:29.847 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:57:29.848 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:57:29.852 INFO:teuthology.orchestra.run.smithi002.stdout:5507 punch from 0x1b4adb8 to 0x1b4ddb3, (0x2ffb bytes) 2024-09-18T04:57:29.852 INFO:teuthology.orchestra.run.smithi002.stdout:5508 writesame 0x2814ec0 thru 0x281db46 data_size 0x5(0x8c87 bytes) 2024-09-18T04:57:29.863 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:29.863 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:29.869 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:57:29.870 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:57:29.871 INFO:teuthology.orchestra.run.smithi002.stdout:5509 writesame 0x63b85d9 thru 0x63c457b data_size 0x4f(0xbfa3 bytes) 2024-09-18T04:57:29.874 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:57:29.878 INFO:teuthology.orchestra.run.smithi002.stdout:5510 writesame 0xa8323df thru 0xa838b5e data_size 0x60(0x6780 bytes) 2024-09-18T04:57:29.889 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:29.886+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:29.890 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:57:29.891 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:57:29.901 INFO:teuthology.orchestra.run.smithi002.stdout:5511 writesame 0x3ee7a5f thru 0x3eef0bf data_size 0x5(0x7661 bytes) 2024-09-18T04:57:29.925 INFO:teuthology.orchestra.run.smithi002.stdout:5512 write 0x58506d thru 0x58c670 (0x7604 bytes) 2024-09-18T04:57:29.960 INFO:teuthology.orchestra.run.smithi002.stdout:5513 trunc from 0xd38b318 to 0x228718d 2024-09-18T04:57:29.981 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:29.979+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:30.018 INFO:teuthology.orchestra.run.smithi002.stdout:5514 read 0x1c3f26b thru 0x1c4c58c (0xd322 bytes) 2024-09-18T04:57:30.022 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:30.023 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:30.028 INFO:teuthology.orchestra.run.smithi002.stdout:5515 writesame 0x692799b thru 0x692a743 data_size 0x1(0x2da9 bytes) 2024-09-18T04:57:30.032 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:57:30.032 INFO:teuthology.orchestra.run.smithi002.stdout:5517 trunc from 0x692a744 to 0x3e97eb2 2024-09-18T04:57:30.041 INFO:teuthology.orchestra.run.smithi002.stdout:5520 read 0x26096d thru 0x261d8f (0x1423 bytes) 2024-09-18T04:57:30.045 INFO:teuthology.orchestra.run.smithi002.stdout:5521 trunc from 0x3e97eb2 to 0x905037a 2024-09-18T04:57:30.049 INFO:teuthology.orchestra.run.smithi002.stdout:5522 writesame 0x82e5ac0 thru 0x82f4aad data_size 0x57(0xefee bytes) 2024-09-18T04:57:30.069 INFO:teuthology.orchestra.run.smithi002.stdout:5523 writesame 0x26e290e thru 0x26f006a data_size 0x1(0xd75d bytes) 2024-09-18T04:57:30.082 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:30.080+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:30.093 INFO:teuthology.orchestra.run.smithi002.stdout:5525 punch from 0x45759ae to 0x4581709, (0xbd5b bytes) 2024-09-18T04:57:30.099 INFO:teuthology.orchestra.run.smithi002.stdout:5526 writesame 0xa4c3397 thru 0xa4c670c data_size 0x7(0x3376 bytes) 2024-09-18T04:57:30.110 INFO:teuthology.orchestra.run.smithi002.stdout:5528 trunc from 0xa4c670d to 0x1937f39 2024-09-18T04:57:30.140 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:30.155 INFO:teuthology.orchestra.run.smithi002.stdout:5532 writesame 0x9f1c60a thru 0x9f1f5b8 data_size 0xd(0x2faf bytes) 2024-09-18T04:57:30.155 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:30.155 INFO:teuthology.orchestra.run.smithi002.stdout:5535 read 0x6916e8b thru 0x6923166 (0xc2dc bytes) 2024-09-18T04:57:30.161 INFO:teuthology.orchestra.run.smithi002.stdout:5536 write 0xa8624ce thru 0xa87176f (0xf2a2 bytes) 2024-09-18T04:57:30.168 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:30.168 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:30.178 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:57:30.182 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:30.182 INFO:teuthology.orchestra.run.smithi002.stdout:5538 read 0x90bfe3d thru 0x90c4947 (0x4b0b bytes) 2024-09-18T04:57:30.182 INFO:teuthology.orchestra.run.smithi002.stdout:5539 punch from 0x7dc57e3 to 0x7dca241, (0x4a5e bytes) 2024-09-18T04:57:30.182 INFO:teuthology.orchestra.run.smithi002.stdout:5540 read 0x42ecca1 thru 0x42ef729 (0x2a89 bytes) 2024-09-18T04:57:30.183 INFO:teuthology.orchestra.run.smithi002.stdout:5541 read 0x47f2ba thru 0x486c3b (0x7982 bytes) 2024-09-18T04:57:30.194 INFO:teuthology.orchestra.run.smithi002.stdout:5542 write 0xee148bc thru 0xee1e0f6 (0x983b bytes) 2024-09-18T04:57:30.195 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:30.192+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:30.218 INFO:teuthology.orchestra.run.smithi002.stdout:5545 read 0x3fa0a34 thru 0x3fa74b0 (0x6a7d bytes) 2024-09-18T04:57:30.227 INFO:teuthology.orchestra.run.smithi002.stdout:5547 trunc from 0xee1e0f7 to 0x7e27ec0 2024-09-18T04:57:30.259 INFO:teuthology.orchestra.run.smithi002.stdout:5550 write 0xd0427c4 thru 0xd042e06 (0x643 bytes) 2024-09-18T04:57:30.264 INFO:teuthology.orchestra.run.smithi002.stdout:5551 punch from 0x2bfb682 to 0x2c05687, (0xa005 bytes) 2024-09-18T04:57:30.265 INFO:teuthology.orchestra.run.smithi002.stdout:5552 write 0xbcbe3f1 thru 0xbcca867 (0xc477 bytes) 2024-09-18T04:57:30.285 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:30.283+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:30.285 INFO:teuthology.orchestra.run.smithi002.stdout:5553 writesame 0x713d150 thru 0x71498fa data_size 0x1(0xc7ab bytes) 2024-09-18T04:57:30.306 INFO:teuthology.orchestra.run.smithi002.stdout:5554 write 0xc800521 thru 0xc8032b6 (0x2d96 bytes) 2024-09-18T04:57:30.327 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:30.328 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:30.331 INFO:teuthology.orchestra.run.smithi002.stdout:5555 write 0xe789592 thru 0xe794bca (0xb639 bytes) 2024-09-18T04:57:30.338 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:57:30.355 INFO:teuthology.orchestra.run.smithi002.stdout:5556 read 0x9bb491f thru 0x9bb8854 (0x3f36 bytes) 2024-09-18T04:57:30.368 INFO:teuthology.orchestra.run.smithi002.stdout:5557 write 0x564bdd7 thru 0x564e53b (0x2765 bytes) 2024-09-18T04:57:30.385 INFO:teuthology.orchestra.run.smithi002.stdout:5559 trunc from 0xe794bcb to 0x626ecfb 2024-09-18T04:57:30.386 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:30.384+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:30.434 INFO:teuthology.orchestra.run.smithi002.stdout:5560 writesame 0xcfc8157 thru 0xcfd4926 data_size 0x5c(0xc7d0 bytes) 2024-09-18T04:57:30.434 INFO:teuthology.orchestra.run.smithi002.stdout:5561 write 0x9bc4e6a thru 0x9bca9a1 (0x5b38 bytes) 2024-09-18T04:57:30.462 INFO:teuthology.orchestra.run.smithi002.stdout:5562 trunc from 0xcfd4927 to 0x4778c2a 2024-09-18T04:57:30.478 INFO:teuthology.orchestra.run.smithi002.stdout:5563 read 0x57acf8 thru 0x587986 (0xcc8f bytes) 2024-09-18T04:57:30.479 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:30.487 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:30.485+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:30.489 INFO:teuthology.orchestra.run.smithi002.stdout:5565 writesame 0x86432a5 thru 0x864a6ed data_size 0x1(0x7449 bytes) 2024-09-18T04:57:30.490 INFO:teuthology.orchestra.run.smithi002.stdout:5566 read 0x64d02a6 thru 0x64d1030 (0xd8b bytes) 2024-09-18T04:57:30.490 INFO:teuthology.orchestra.run.smithi002.stdout:5568 punch from 0x69213f8 to 0x692302b, (0x1c33 bytes) 2024-09-18T04:57:30.490 INFO:teuthology.orchestra.run.smithi002.stdout:5569 read 0xc26019 thru 0xc2c3fb (0x63e3 bytes) 2024-09-18T04:57:30.494 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:30.494 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:30.505 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:57:30.506 INFO:teuthology.orchestra.run.smithi002.stdout:5572 write 0xe5405b9 thru 0xe5413f3 (0xe3b bytes) 2024-09-18T04:57:30.518 INFO:teuthology.orchestra.run.smithi002.stdout:5573 writesame 0x15dc047 thru 0x15e018c data_size 0xf(0x4146 bytes) 2024-09-18T04:57:30.540 INFO:teuthology.orchestra.run.smithi002.stdout:5574 read 0xc6e5702 thru 0xc6f11a5 (0xbaa4 bytes) 2024-09-18T04:57:30.540 INFO:teuthology.orchestra.run.smithi002.stdout:5578 punch from 0x3e19c0f to 0x3e1f038, (0x5429 bytes) 2024-09-18T04:57:30.540 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:30.554 INFO:teuthology.orchestra.run.smithi002.stdout:5584 trunc from 0xe5413f4 to 0x1384f80 2024-09-18T04:57:30.583 INFO:teuthology.orchestra.run.smithi002.stdout:5585 read 0x1302801 thru 0x130b380 (0x8b80 bytes) 2024-09-18T04:57:30.583 INFO:teuthology.orchestra.run.smithi002.stdout:5586 punch from 0x907657 to 0x916a61, (0xf40a bytes) 2024-09-18T04:57:30.584 INFO:teuthology.orchestra.run.smithi002.stdout:5587 punch from 0x10abfdb to 0x10b032f, (0x4354 bytes) 2024-09-18T04:57:30.588 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:30.586+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:30.592 INFO:teuthology.orchestra.run.smithi002.stdout:5590 write 0x32d76ff thru 0x32e675c (0xf05e bytes) 2024-09-18T04:57:30.624 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:30.625 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:30.630 INFO:teuthology.orchestra.run.smithi002.stdout:5591 write 0xb8af9d8 thru 0xb8b12b7 (0x18e0 bytes) 2024-09-18T04:57:30.634 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:57:30.647 INFO:teuthology.orchestra.run.smithi002.stdout:5592 write 0x5cc16f0 thru 0x5ccaad3 (0x93e4 bytes) 2024-09-18T04:57:30.671 INFO:teuthology.orchestra.run.smithi002.stdout:5593 write 0x5419ef thru 0x54c10c (0xa71e bytes) 2024-09-18T04:57:30.674 INFO:teuthology.orchestra.run.smithi002.stdout:5594 writesame 0xaee94aa thru 0xaef1216 data_size 0x4d(0x7d6d bytes) 2024-09-18T04:57:30.689 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:30.687+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:30.697 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:30.697 INFO:teuthology.orchestra.run.smithi002.stdout:5596 write 0xac978c5 thru 0xaca0c80 (0x93bc bytes) 2024-09-18T04:57:30.706 INFO:teuthology.orchestra.run.smithi002.stdout:5597 read 0x173c3d2 thru 0x1749f00 (0xdb2f bytes) 2024-09-18T04:57:30.706 INFO:teuthology.orchestra.run.smithi002.stdout:5598 write 0x16f4 thru 0x653b (0x4e48 bytes) 2024-09-18T04:57:30.716 INFO:teuthology.orchestra.run.smithi002.stdout:5600 trunc from 0xb8b12b8 to 0xc2f79ef 2024-09-18T04:57:30.722 INFO:teuthology.orchestra.run.smithi002.stdout:5601 read 0x6fb9c3 thru 0x70aae1 (0xf11f bytes) 2024-09-18T04:57:30.723 INFO:teuthology.orchestra.run.smithi002.stdout:5602 read 0x4356694 thru 0x43575f7 (0xf64 bytes) 2024-09-18T04:57:30.723 INFO:teuthology.orchestra.run.smithi002.stdout:5603 write 0x974e3e5 thru 0x9757561 (0x917d bytes) 2024-09-18T04:57:30.747 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:30.757 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:30.758 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:30.760 INFO:teuthology.orchestra.run.smithi002.stdout:5605 trunc from 0xc2f79ef to 0xc98b2 2024-09-18T04:57:30.767 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:57:30.813 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:30.811+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:30.832 INFO:teuthology.orchestra.run.smithi002.stdout:5607 read 0xadef7 thru 0xbb904 (0xda0e bytes) 2024-09-18T04:57:30.838 INFO:teuthology.orchestra.run.smithi002.stdout:5608 trunc from 0xc98b2 to 0x40a32ea 2024-09-18T04:57:30.845 INFO:teuthology.orchestra.run.smithi002.stdout:5609 read 0x16b445c thru 0x16b60b1 (0x1c56 bytes) 2024-09-18T04:57:30.850 INFO:teuthology.orchestra.run.smithi002.stdout:5610 trunc from 0x40a32ea to 0x83d7d68 2024-09-18T04:57:30.856 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:30.857 INFO:teuthology.orchestra.run.smithi002.stdout:5612 read 0x301eaf8 thru 0x3027f62 (0x946b bytes) 2024-09-18T04:57:30.857 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:30.857 INFO:teuthology.orchestra.run.smithi002.stdout:5614 read 0x77377ef thru 0x773e9ab (0x71bd bytes) 2024-09-18T04:57:30.857 INFO:teuthology.orchestra.run.smithi002.stdout:5615 read 0x18a1ff8 thru 0x18adc90 (0xbc99 bytes) 2024-09-18T04:57:30.862 INFO:teuthology.orchestra.run.smithi002.stdout:5616 trunc from 0x83d7d68 to 0xda5649b 2024-09-18T04:57:30.869 INFO:teuthology.orchestra.run.smithi002.stdout:5617 write 0x5215cf thru 0x528ec3 (0x78f5 bytes) 2024-09-18T04:57:30.890 INFO:teuthology.orchestra.run.smithi002.stdout:5618 read 0x98adbf6 thru 0x98afd3f (0x214a bytes) 2024-09-18T04:57:30.891 INFO:teuthology.orchestra.run.smithi002.stdout:5620 write 0x30c8e7b thru 0x30cb0ca (0x2250 bytes) 2024-09-18T04:57:30.892 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:30.888+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:30.892 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:30.892 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:30.903 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:57:30.907 INFO:teuthology.orchestra.run.smithi002.stdout:5621 trunc from 0xda5649b to 0xbca93e0 2024-09-18T04:57:30.915 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:30.915 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:30.915 INFO:teuthology.orchestra.run.smithi002.stdout:5626 write 0x283052c thru 0x2838aed (0x85c2 bytes) 2024-09-18T04:57:30.934 INFO:teuthology.orchestra.run.smithi002.stdout:5627 read 0x8b1385e thru 0x8b19a9d (0x6240 bytes) 2024-09-18T04:57:30.934 INFO:teuthology.orchestra.run.smithi002.stdout:5629 punch from 0x7d27f03 to 0x7d2d854, (0x5951 bytes) 2024-09-18T04:57:30.937 INFO:teuthology.orchestra.run.smithi002.stdout:5631 trunc from 0xbca93e0 to 0x88617f2 2024-09-18T04:57:30.955 INFO:teuthology.orchestra.run.smithi002.stdout:5632 writesame 0xc60908f thru 0xc60dfca data_size 0x16(0x4f3c bytes) 2024-09-18T04:57:30.960 INFO:teuthology.orchestra.run.smithi002.stdout:5633 trunc from 0xc60dfcb to 0x7eda7f0 2024-09-18T04:57:30.969 INFO:teuthology.orchestra.run.smithi002.stdout:5634 read 0x44e2c50 thru 0x44e73d2 (0x4783 bytes) 2024-09-18T04:57:30.969 INFO:teuthology.orchestra.run.smithi002.stdout:5635 punch from 0x425f44a to 0x4269a97, (0xa64d bytes) 2024-09-18T04:57:30.978 INFO:teuthology.orchestra.run.smithi002.stdout:5636 writesame 0xa360070 thru 0xa36f61a data_size 0x1(0xf5ab bytes) 2024-09-18T04:57:30.979 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:30.979 INFO:teuthology.orchestra.run.smithi002.stdout:5638 writesame 0x7fbbbc2 thru 0x7fca49c data_size 0x1(0xe8db bytes) 2024-09-18T04:57:30.992 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:30.990+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:31.009 INFO:teuthology.orchestra.run.smithi002.stdout:5640 read 0x3756b94 thru 0x375d72b (0x6b98 bytes) 2024-09-18T04:57:31.009 INFO:teuthology.orchestra.run.smithi002.stdout:5641 punch from 0x525be00 to 0x5268b12, (0xcd12 bytes) 2024-09-18T04:57:31.009 INFO:teuthology.orchestra.run.smithi002.stdout:5642 punch from 0x21c8939 to 0x21d14ad, (0x8b74 bytes) 2024-09-18T04:57:31.009 INFO:teuthology.orchestra.run.smithi002.stdout:5643 punch from 0x285c1e7 to 0x285ed08, (0x2b21 bytes) 2024-09-18T04:57:31.009 INFO:teuthology.orchestra.run.smithi002.stdout:5644 punch from 0x9022a4d to 0x9032242, (0xf7f5 bytes) 2024-09-18T04:57:31.009 INFO:teuthology.orchestra.run.smithi002.stdout:5645 read 0x26ffa7e thru 0x270dfcb (0xe54e bytes) 2024-09-18T04:57:31.009 INFO:teuthology.orchestra.run.smithi002.stdout:5646 clone 41 order 24 su 2097152 sc 6 2024-09-18T04:57:31.057 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:31.057 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:31.066 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:57:31.093 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:31.090+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:31.168 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:31.168 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:31.177 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:57:31.222 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:31.219+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:31.280 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:31.280 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:31.289 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:57:31.295 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:31.292+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:31.395 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:31.393+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:31.439 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:31.439 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:31.442 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T04:57:31.448 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:57:31.490 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone40 from 0xa36f61b (overlap 0xc98b2) to 0x509e0 2024-09-18T04:57:31.535 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:31.532+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:31.584 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-09-18T04:57:31.597 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:31.594+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:31.608 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:31.608 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:31.618 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:57:31.698 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:31.695+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:31.761 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:31.762 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:31.770 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:57:31.816 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:31.813+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:31.868 INFO:teuthology.orchestra.run.smithi002.stdout:5647 write 0x63717a8 thru 0x637aff8 (0x9851 bytes) 2024-09-18T04:57:31.874 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:31.874 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:31.881 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:31.882 INFO:teuthology.orchestra.run.smithi002.stdout:5649 write 0x8d000e0 thru 0x8d099fc (0x991d bytes) 2024-09-18T04:57:31.883 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:57:31.894 INFO:teuthology.orchestra.run.smithi002.stdout:5651 write 0xbb17081 thru 0xbb17b56 (0xad6 bytes) 2024-09-18T04:57:31.902 INFO:teuthology.orchestra.run.smithi002.stdout:5652 read 0x6f030e4 thru 0x6f11143 (0xe060 bytes) 2024-09-18T04:57:31.902 INFO:teuthology.orchestra.run.smithi002.stdout:5653 read 0xb28035c thru 0xb281767 (0x140c bytes) 2024-09-18T04:57:31.902 INFO:teuthology.orchestra.run.smithi002.stdout:5654 write 0x27ab6fe thru 0x27b8004 (0xc907 bytes) 2024-09-18T04:57:31.903 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:57:31.911 INFO:teuthology.orchestra.run.smithi002.stdout:5655 punch from 0x4ef4a6f to 0x4ef503b, (0x5cc bytes) 2024-09-18T04:57:31.912 INFO:teuthology.orchestra.run.smithi002.stdout:5656 write 0x3dbc2c3 thru 0x3dc5f52 (0x9c90 bytes) 2024-09-18T04:57:31.925 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:31.922+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:31.930 INFO:teuthology.orchestra.run.smithi002.stdout:5658 writesame 0xba30e66 thru 0xba36a91 data_size 0x22(0x5c2c bytes) 2024-09-18T04:57:31.932 INFO:teuthology.orchestra.run.smithi002.stdout:5659 read 0x14fb3fd thru 0x1501f6f (0x6b73 bytes) 2024-09-18T04:57:31.932 INFO:teuthology.orchestra.run.smithi002.stdout:5660 read 0x908c326 thru 0x909560d (0x92e8 bytes) 2024-09-18T04:57:31.932 INFO:teuthology.orchestra.run.smithi002.stdout:5662 write 0x38880c3 thru 0x38955d1 (0xd50f bytes) 2024-09-18T04:57:31.940 INFO:teuthology.orchestra.run.smithi002.stdout:5665 punch from 0x6926dc8 to 0x692e61c, (0x7854 bytes) 2024-09-18T04:57:31.940 INFO:teuthology.orchestra.run.smithi002.stdout:5666 read 0x15d1860 thru 0x15de4da (0xcc7b bytes) 2024-09-18T04:57:31.940 INFO:teuthology.orchestra.run.smithi002.stdout:5667 write 0x4a3f224 thru 0x4a42932 (0x370f bytes) 2024-09-18T04:57:31.942 INFO:teuthology.orchestra.run.smithi002.stdout:5668 punch from 0x5aa497b to 0x5aab1b9, (0x683e bytes) 2024-09-18T04:57:31.942 INFO:teuthology.orchestra.run.smithi002.stdout:5669 read 0x491e21e thru 0x492e16e (0xff51 bytes) 2024-09-18T04:57:31.942 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:31.942 INFO:teuthology.orchestra.run.smithi002.stdout:5671 read 0x567899b thru 0x56790c0 (0x726 bytes) 2024-09-18T04:57:31.943 INFO:teuthology.orchestra.run.smithi002.stdout:5672 read 0x2bfa9a5 thru 0x2c003a3 (0x59ff bytes) 2024-09-18T04:57:31.943 INFO:teuthology.orchestra.run.smithi002.stdout:5673 trunc from 0xbb17b57 to 0xbf22898 2024-09-18T04:57:31.948 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:31.948 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:31.948 INFO:teuthology.orchestra.run.smithi002.stdout:5677 read 0x6cee968 thru 0x6cf53eb (0x6a84 bytes) 2024-09-18T04:57:31.949 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:31.949 INFO:teuthology.orchestra.run.smithi002.stdout:5681 punch from 0x6479c09 to 0x6479e7f, (0x276 bytes) 2024-09-18T04:57:31.949 INFO:teuthology.orchestra.run.smithi002.stdout:5683 punch from 0x5f1a06a to 0x5f1a2cb, (0x261 bytes) 2024-09-18T04:57:31.949 INFO:teuthology.orchestra.run.smithi002.stdout:5685 read 0x5745396 thru 0x5751f9c (0xcc07 bytes) 2024-09-18T04:57:31.949 INFO:teuthology.orchestra.run.smithi002.stdout:5686 read 0xc126ca thru 0xc14eef (0x2826 bytes) 2024-09-18T04:57:31.949 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:31.956 INFO:teuthology.orchestra.run.smithi002.stdout:5689 writesame 0xcdd610c thru 0xcdd91c2 data_size 0x1(0x30b7 bytes) 2024-09-18T04:57:31.956 INFO:teuthology.orchestra.run.smithi002.stdout:5690 punch from 0x592f9f9 to 0x5933e28, (0x442f bytes) 2024-09-18T04:57:31.956 INFO:teuthology.orchestra.run.smithi002.stdout:5692 read 0x9bcf3f0 thru 0x9bd3e7d (0x4a8e bytes) 2024-09-18T04:57:31.956 INFO:teuthology.orchestra.run.smithi002.stdout:5693 writesame 0x78116c5 thru 0x7814ab8 data_size 0x19(0x33f4 bytes) 2024-09-18T04:57:31.964 INFO:teuthology.orchestra.run.smithi002.stdout:5694 punch from 0xcc700c2 to 0xcc712df, (0x121d bytes) 2024-09-18T04:57:31.964 INFO:teuthology.orchestra.run.smithi002.stdout:5695 read 0x8d6d42f thru 0x8d773c2 (0x9f94 bytes) 2024-09-18T04:57:31.966 INFO:teuthology.orchestra.run.smithi002.stdout:5696 trunc from 0xcdd91c3 to 0xa1f0efc 2024-09-18T04:57:31.982 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:31.982 INFO:teuthology.orchestra.run.smithi002.stdout:5698 read 0x2a23804 thru 0x2a2ca7c (0x9279 bytes) 2024-09-18T04:57:31.982 INFO:teuthology.orchestra.run.smithi002.stdout:5700 punch from 0x478e37e to 0x479a5dd, (0xc25f bytes) 2024-09-18T04:57:31.982 INFO:teuthology.orchestra.run.smithi002.stdout:5701 write 0x5175e83 thru 0x5184749 (0xe8c7 bytes) 2024-09-18T04:57:31.984 INFO:teuthology.orchestra.run.smithi002.stdout:5702 punch from 0xfade1 to 0x102ef0, (0x810f bytes) 2024-09-18T04:57:31.984 INFO:teuthology.orchestra.run.smithi002.stdout:5703 writesame 0x367229c thru 0x36723c6 data_size 0x1(0x12b bytes) 2024-09-18T04:57:31.991 INFO:teuthology.orchestra.run.smithi002.stdout:5704 write 0x34e242f thru 0x34e7197 (0x4d69 bytes) 2024-09-18T04:57:32.000 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:31.997+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:32.001 INFO:teuthology.orchestra.run.smithi002.stdout:5705 trunc from 0xa1f0efc to 0xb123a26 2024-09-18T04:57:32.013 INFO:teuthology.orchestra.run.smithi002.stdout:5708 write 0xb7c0e3b thru 0xb7c1597 (0x75d bytes) 2024-09-18T04:57:32.014 INFO:teuthology.orchestra.run.smithi002.stdout:5709 write 0xec4329 thru 0xec4cef (0x9c7 bytes) 2024-09-18T04:57:32.016 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:32.016 INFO:teuthology.orchestra.run.smithi002.stdout:5711 read 0x74283f6 thru 0x742edd1 (0x69dc bytes) 2024-09-18T04:57:32.017 INFO:teuthology.orchestra.run.smithi002.stdout:5712 read 0x71dc3f thru 0x728709 (0xaacb bytes) 2024-09-18T04:57:32.017 INFO:teuthology.orchestra.run.smithi002.stdout:5713 punch from 0x48cf6b1 to 0x48da73a, (0xb089 bytes) 2024-09-18T04:57:32.017 INFO:teuthology.orchestra.run.smithi002.stdout:5714 punch from 0x3cdc48a to 0x3ce6848, (0xa3be bytes) 2024-09-18T04:57:32.017 INFO:teuthology.orchestra.run.smithi002.stdout:5715 writesame 0x1b20bc6 thru 0x1b2d0d1 data_size 0x2(0xc50c bytes) 2024-09-18T04:57:32.030 INFO:teuthology.orchestra.run.smithi002.stdout:5716 write 0x7e5f7b5 thru 0x7e692b5 (0x9b01 bytes) 2024-09-18T04:57:32.032 INFO:teuthology.orchestra.run.smithi002.stdout:5717 write 0x3de259 thru 0x3dec6d (0xa15 bytes) 2024-09-18T04:57:32.034 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:32.035 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:32.037 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:57:32.038 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:57:32.038 INFO:teuthology.orchestra.run.smithi002.stdout:5718 writesame 0xb7c0496 thru 0xb7c4850 data_size 0x1(0x43bb bytes) 2024-09-18T04:57:32.044 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:57:32.046 INFO:teuthology.orchestra.run.smithi002.stdout:5719 punch from 0xa0439aa to 0xa045782, (0x1dd8 bytes) 2024-09-18T04:57:32.055 INFO:teuthology.orchestra.run.smithi002.stdout:5720 trunc from 0xb7c4851 to 0x47dbac4 2024-09-18T04:57:32.061 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:57:32.061 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:57:32.081 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:57:32.081 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:57:32.085 INFO:teuthology.orchestra.run.smithi002.stdout:5721 write 0x7806dbe thru 0x7810af7 (0x9d3a bytes) 2024-09-18T04:57:32.103 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:57:32.104 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:57:32.104 INFO:teuthology.orchestra.run.smithi002.stdout:5723 write 0xeafb2d6 thru 0xeaffde9 (0x4b14 bytes) 2024-09-18T04:57:32.109 INFO:teuthology.orchestra.run.smithi002.stdout:5724 write 0xba9ad4c thru 0xbaa2ada (0x7d8f bytes) 2024-09-18T04:57:32.111 INFO:teuthology.orchestra.run.smithi002.stdout:5725 read 0x11d5687 thru 0x11d8f9f (0x3919 bytes) 2024-09-18T04:57:32.112 INFO:teuthology.orchestra.run.smithi002.stdout:5727 read 0xd3676dd thru 0xd36954e (0x1e72 bytes) 2024-09-18T04:57:32.112 INFO:teuthology.orchestra.run.smithi002.stdout:5728 writesame 0x8b2ae94 thru 0x8b33e51 data_size 0x3(0x8fbe bytes) 2024-09-18T04:57:32.118 INFO:teuthology.orchestra.run.smithi002.stdout:5730 writesame 0x24fe90e thru 0x2504a5f data_size 0x1(0x6152 bytes) 2024-09-18T04:57:32.121 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:57:32.121 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:57:32.130 INFO:teuthology.orchestra.run.smithi002.stdout:5731 write 0x56b74bc thru 0x56c6247 (0xed8c bytes) 2024-09-18T04:57:32.131 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:32.128+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:32.132 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:32.132 INFO:teuthology.orchestra.run.smithi002.stdout:5733 read 0xa3e2b01 thru 0xa3e9765 (0x6c65 bytes) 2024-09-18T04:57:32.133 INFO:teuthology.orchestra.run.smithi002.stdout:5734 read 0x441b485 thru 0x4425f7b (0xaaf7 bytes) 2024-09-18T04:57:32.133 INFO:teuthology.orchestra.run.smithi002.stdout:5735 clone 42 order 19 su 8192 sc 6 2024-09-18T04:57:32.138 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:57:32.139 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:57:32.155 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:57:32.156 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:57:32.172 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:57:32.172 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:57:32.192 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:32.192 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:32.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:32.192 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:32.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.192 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:57:32.193 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:57:32.193 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 80.795130068000006, 2024-09-18T04:57:32.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037718511000000003, 2024-09-18T04:57:32.193 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:32.193 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:32.193 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:32.193 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:57:32.193 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:57:32.193 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-18T04:57:32.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.193 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:32.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:32.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:57:32.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:32.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:57:32.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:32.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393125+0000", 2024-09-18T04:57:32.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:57:32.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:32.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393133+0000", 2024-09-18T04:57:32.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.225e-06 2024-09-18T04:57:32.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:32.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393144+0000", 2024-09-18T04:57:32.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1279e-05 2024-09-18T04:57:32.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393153+0000", 2024-09-18T04:57:32.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4540000000000007e-06 2024-09-18T04:57:32.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.196 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:32.196 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393203+0000", 2024-09-18T04:57:32.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9963000000000003e-05 2024-09-18T04:57:32.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.196 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:57:32.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393210+0000", 2024-09-18T04:57:32.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5689999999999999e-06 2024-09-18T04:57:32.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:32.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427399+0000", 2024-09-18T04:57:32.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034188784999999999 2024-09-18T04:57:32.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:32.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427532+0000", 2024-09-18T04:57:32.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013264700000000001 2024-09-18T04:57:32.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:57:32.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427685+0000", 2024-09-18T04:57:32.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015300599999999999 2024-09-18T04:57:32.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:32.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.428520+0000", 2024-09-18T04:57:32.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00083502499999999998 2024-09-18T04:57:32.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:57:32.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430772+0000", 2024-09-18T04:57:32.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022518880000000001 2024-09-18T04:57:32.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:57:32.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430817+0000", 2024-09-18T04:57:32.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5077000000000002e-05 2024-09-18T04:57:32.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:32.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430845+0000", 2024-09-18T04:57:32.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8036999999999999e-05 2024-09-18T04:57:32.199 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.199 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:32.199 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.200 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54068 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3500032~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:32.200 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221912+0000", 2024-09-18T04:57:32.200 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.966344698, 2024-09-18T04:57:32.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029654553, 2024-09-18T04:57:32.200 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:32.200 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:32.200 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:32.200 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:32.200 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.200 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54068 2024-09-18T04:57:32.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.200 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:32.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:32.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221912+0000", 2024-09-18T04:57:32.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:32.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221912+0000", 2024-09-18T04:57:32.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:32.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221911+0000", 2024-09-18T04:57:32.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:32.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221916+0000", 2024-09-18T04:57:32.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0130000000000001e-06 2024-09-18T04:57:32.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:32.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221922+0000", 2024-09-18T04:57:32.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.105e-06 2024-09-18T04:57:32.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.203 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221928+0000", 2024-09-18T04:57:32.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0430000000000004e-06 2024-09-18T04:57:32.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:32.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251427+0000", 2024-09-18T04:57:32.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029498586 2024-09-18T04:57:32.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:32.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251495+0000", 2024-09-18T04:57:32.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8125000000000003e-05 2024-09-18T04:57:32.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:32.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251566+0000", 2024-09-18T04:57:32.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1145999999999999e-05 2024-09-18T04:57:32.204 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.205 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:32.205 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.205 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54069 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3515392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:32.205 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221945+0000", 2024-09-18T04:57:32.205 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.966311089000001, 2024-09-18T04:57:32.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029786185999999999, 2024-09-18T04:57:32.205 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:32.205 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:32.205 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:32.205 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:32.205 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.205 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54069 2024-09-18T04:57:32.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.206 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:32.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:32.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221945+0000", 2024-09-18T04:57:32.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:32.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221945+0000", 2024-09-18T04:57:32.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:32.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221945+0000", 2024-09-18T04:57:32.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:32.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221947+0000", 2024-09-18T04:57:32.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6189999999999998e-06 2024-09-18T04:57:32.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:32.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221955+0000", 2024-09-18T04:57:32.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6249999999999998e-06 2024-09-18T04:57:32.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221968+0000", 2024-09-18T04:57:32.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2846e-05 2024-09-18T04:57:32.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:32.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251595+0000", 2024-09-18T04:57:32.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029627158000000001 2024-09-18T04:57:32.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:32.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251666+0000", 2024-09-18T04:57:32.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1437000000000001e-05 2024-09-18T04:57:32.209 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.209 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.209 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:32.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251731+0000", 2024-09-18T04:57:32.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5071000000000001e-05 2024-09-18T04:57:32.210 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.210 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:32.210 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.210 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.210 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.210 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54070 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3530752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:32.210 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.221973+0000", 2024-09-18T04:57:32.210 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.966283105999999, 2024-09-18T04:57:32.210 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029891219, 2024-09-18T04:57:32.210 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:32.210 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:32.210 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:32.210 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:32.211 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.211 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54070 2024-09-18T04:57:32.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.211 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:32.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.211 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:32.211 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221973+0000", 2024-09-18T04:57:32.211 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.211 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.211 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:32.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221973+0000", 2024-09-18T04:57:32.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:32.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221973+0000", 2024-09-18T04:57:32.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.212 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.212 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.212 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:32.212 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221985+0000", 2024-09-18T04:57:32.212 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1888999999999999e-05 2024-09-18T04:57:32.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:32.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.221992+0000", 2024-09-18T04:57:32.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2830000000000004e-06 2024-09-18T04:57:32.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.213 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222008+0000", 2024-09-18T04:57:32.213 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5608999999999999e-05 2024-09-18T04:57:32.213 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.213 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.213 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:32.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251757+0000", 2024-09-18T04:57:32.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029749652000000001 2024-09-18T04:57:32.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:32.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251826+0000", 2024-09-18T04:57:32.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9103000000000006e-05 2024-09-18T04:57:32.214 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.214 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.214 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:32.214 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251864+0000", 2024-09-18T04:57:32.214 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8179000000000002e-05 2024-09-18T04:57:32.215 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.215 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:32.215 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.215 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.215 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54071 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3546112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:32.215 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222014+0000", 2024-09-18T04:57:32.215 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.966242674999997, 2024-09-18T04:57:32.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03002749, 2024-09-18T04:57:32.215 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:32.215 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:32.215 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:32.215 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:32.216 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.216 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54071 2024-09-18T04:57:32.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.216 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:32.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:32.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222014+0000", 2024-09-18T04:57:32.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:32.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222014+0000", 2024-09-18T04:57:32.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:32.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222013+0000", 2024-09-18T04:57:32.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:32.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222016+0000", 2024-09-18T04:57:32.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6639999999999998e-06 2024-09-18T04:57:32.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:32.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222023+0000", 2024-09-18T04:57:32.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.849e-06 2024-09-18T04:57:32.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222028+0000", 2024-09-18T04:57:32.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0030000000000002e-06 2024-09-18T04:57:32.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:32.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251895+0000", 2024-09-18T04:57:32.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029867482000000001 2024-09-18T04:57:32.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:32.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.251963+0000", 2024-09-18T04:57:32.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8038000000000001e-05 2024-09-18T04:57:32.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:32.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252041+0000", 2024-09-18T04:57:32.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8028999999999994e-05 2024-09-18T04:57:32.219 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.219 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:32.219 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.220 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54072 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3561472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:32.220 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222034+0000", 2024-09-18T04:57:32.220 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.966222604999999, 2024-09-18T04:57:32.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030190214, 2024-09-18T04:57:32.220 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:32.220 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:32.220 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:32.220 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:32.220 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.220 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54072 2024-09-18T04:57:32.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.220 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:32.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:32.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222034+0000", 2024-09-18T04:57:32.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:32.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222034+0000", 2024-09-18T04:57:32.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:32.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222033+0000", 2024-09-18T04:57:32.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:32.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222035+0000", 2024-09-18T04:57:32.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0490000000000002e-06 2024-09-18T04:57:32.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:32.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222051+0000", 2024-09-18T04:57:32.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5914e-05 2024-09-18T04:57:32.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222057+0000", 2024-09-18T04:57:32.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4269999999999999e-06 2024-09-18T04:57:32.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:32.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252079+0000", 2024-09-18T04:57:32.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030022338999999999 2024-09-18T04:57:32.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:32.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252157+0000", 2024-09-18T04:57:32.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7459999999999994e-05 2024-09-18T04:57:32.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:32.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252224+0000", 2024-09-18T04:57:32.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7415000000000004e-05 2024-09-18T04:57:32.224 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.225 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:32.225 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.225 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54073 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3576832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:32.225 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222062+0000", 2024-09-18T04:57:32.225 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.966194827999999, 2024-09-18T04:57:32.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030280892, 2024-09-18T04:57:32.225 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:32.225 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:32.225 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:32.225 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:32.225 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.225 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54073 2024-09-18T04:57:32.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.226 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:32.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:32.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222062+0000", 2024-09-18T04:57:32.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:32.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222062+0000", 2024-09-18T04:57:32.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:32.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222061+0000", 2024-09-18T04:57:32.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:32.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222063+0000", 2024-09-18T04:57:32.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0729999999999999e-06 2024-09-18T04:57:32.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:32.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222070+0000", 2024-09-18T04:57:32.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3199999999999996e-06 2024-09-18T04:57:32.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222074+0000", 2024-09-18T04:57:32.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8659999999999997e-06 2024-09-18T04:57:32.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:32.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252255+0000", 2024-09-18T04:57:32.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030180350000000002 2024-09-18T04:57:32.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:32.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252305+0000", 2024-09-18T04:57:32.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0086000000000001e-05 2024-09-18T04:57:32.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:32.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252342+0000", 2024-09-18T04:57:32.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7594e-05 2024-09-18T04:57:32.231 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.231 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:32.231 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.231 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54074 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3592192~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:32.232 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222079+0000", 2024-09-18T04:57:32.232 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.966177436999999, 2024-09-18T04:57:32.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030370844000000001, 2024-09-18T04:57:32.232 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:32.232 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:32.232 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:32.232 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:32.232 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.232 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54074 2024-09-18T04:57:32.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.232 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:32.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:32.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222079+0000", 2024-09-18T04:57:32.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:32.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222079+0000", 2024-09-18T04:57:32.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:32.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222079+0000", 2024-09-18T04:57:32.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:32.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222081+0000", 2024-09-18T04:57:32.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0820000000000001e-06 2024-09-18T04:57:32.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:32.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222087+0000", 2024-09-18T04:57:32.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0789999999999997e-06 2024-09-18T04:57:32.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222097+0000", 2024-09-18T04:57:32.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0446e-05 2024-09-18T04:57:32.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:32.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252365+0000", 2024-09-18T04:57:32.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030267544 2024-09-18T04:57:32.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:32.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252402+0000", 2024-09-18T04:57:32.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7117999999999999e-05 2024-09-18T04:57:32.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:32.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252450+0000", 2024-09-18T04:57:32.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7930999999999999e-05 2024-09-18T04:57:32.236 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.236 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:32.236 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.236 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54075 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3607552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:32.236 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222103+0000", 2024-09-18T04:57:32.236 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.966153384999998, 2024-09-18T04:57:32.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030531175000000001, 2024-09-18T04:57:32.236 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:32.236 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:32.236 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:32.237 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:32.237 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.237 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54075 2024-09-18T04:57:32.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.237 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:32.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:32.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222103+0000", 2024-09-18T04:57:32.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:32.237 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222103+0000", 2024-09-18T04:57:32.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:32.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222101+0000", 2024-09-18T04:57:32.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:57:32.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:32.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222105+0000", 2024-09-18T04:57:32.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4089999999999999e-06 2024-09-18T04:57:32.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:32.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222111+0000", 2024-09-18T04:57:32.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3720000000000001e-06 2024-09-18T04:57:32.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222116+0000", 2024-09-18T04:57:32.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.544e-06 2024-09-18T04:57:32.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:32.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252475+0000", 2024-09-18T04:57:32.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030358975999999999 2024-09-18T04:57:32.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:32.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252562+0000", 2024-09-18T04:57:32.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7257999999999997e-05 2024-09-18T04:57:32.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:32.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252634+0000", 2024-09-18T04:57:32.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2144999999999995e-05 2024-09-18T04:57:32.241 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.241 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:32.241 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.241 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54076 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3622912~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:32.241 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222121+0000", 2024-09-18T04:57:32.241 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.966135494, 2024-09-18T04:57:32.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030699046000000001, 2024-09-18T04:57:32.241 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:32.242 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:32.242 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:32.242 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:32.243 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.243 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54076 2024-09-18T04:57:32.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.243 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:32.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:32.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222121+0000", 2024-09-18T04:57:32.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:32.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222121+0000", 2024-09-18T04:57:32.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:32.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222120+0000", 2024-09-18T04:57:32.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:32.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222123+0000", 2024-09-18T04:57:32.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2199999999999999e-06 2024-09-18T04:57:32.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:32.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222129+0000", 2024-09-18T04:57:32.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5250000000000002e-06 2024-09-18T04:57:32.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222134+0000", 2024-09-18T04:57:32.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8180000000000003e-06 2024-09-18T04:57:32.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:32.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252677+0000", 2024-09-18T04:57:32.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030542969999999999 2024-09-18T04:57:32.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:32.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252750+0000", 2024-09-18T04:57:32.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2601000000000005e-05 2024-09-18T04:57:32.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:32.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252820+0000", 2024-09-18T04:57:32.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0409999999999998e-05 2024-09-18T04:57:32.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.247 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:32.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.247 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54077 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3638272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:32.247 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222138+0000", 2024-09-18T04:57:32.247 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.966117994000001, 2024-09-18T04:57:32.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031008865, 2024-09-18T04:57:32.247 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:32.247 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:32.247 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:32.247 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:32.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.248 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54077 2024-09-18T04:57:32.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.248 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:32.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:32.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222138+0000", 2024-09-18T04:57:32.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:32.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222138+0000", 2024-09-18T04:57:32.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:32.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222138+0000", 2024-09-18T04:57:32.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:32.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222140+0000", 2024-09-18T04:57:32.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9700000000000002e-06 2024-09-18T04:57:32.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:32.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222146+0000", 2024-09-18T04:57:32.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1460000000000001e-06 2024-09-18T04:57:32.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222151+0000", 2024-09-18T04:57:32.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.763e-06 2024-09-18T04:57:32.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:32.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.252989+0000", 2024-09-18T04:57:32.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030838580000000001 2024-09-18T04:57:32.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253072+0000", 2024-09-18T04:57:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2204000000000001e-05 2024-09-18T04:57:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253147+0000", 2024-09-18T04:57:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5661000000000006e-05 2024-09-18T04:57:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.251 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54078 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3653632~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:32.252 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222155+0000", 2024-09-18T04:57:32.252 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.966101408, 2024-09-18T04:57:32.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031175118000000002, 2024-09-18T04:57:32.252 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:32.252 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:32.252 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:32.252 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:32.252 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.252 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54078 2024-09-18T04:57:32.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.252 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:32.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:32.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222155+0000", 2024-09-18T04:57:32.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:32.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222155+0000", 2024-09-18T04:57:32.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:32.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222155+0000", 2024-09-18T04:57:32.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:32.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222157+0000", 2024-09-18T04:57:32.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0660000000000002e-06 2024-09-18T04:57:32.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:32.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222163+0000", 2024-09-18T04:57:32.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2820000000000002e-06 2024-09-18T04:57:32.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222168+0000", 2024-09-18T04:57:32.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7979999999999996e-06 2024-09-18T04:57:32.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253174+0000", 2024-09-18T04:57:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031005721999999999 2024-09-18T04:57:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253251+0000", 2024-09-18T04:57:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7717000000000001e-05 2024-09-18T04:57:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:32.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253330+0000", 2024-09-18T04:57:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8858999999999998e-05 2024-09-18T04:57:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:54079 3.11 3:8aff44d3:::rbd_data.11acf2b3fa50.0000000000000005:17 [read 3668992~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:50.222172+0000", 2024-09-18T04:57:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.966084258000002, 2024-09-18T04:57:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031327661, 2024-09-18T04:57:32.262 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 54079 2024-09-18T04:57:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222172+0000", 2024-09-18T04:57:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222172+0000", 2024-09-18T04:57:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222172+0000", 2024-09-18T04:57:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:32.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222174+0000", 2024-09-18T04:57:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0159999999999998e-06 2024-09-18T04:57:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222180+0000", 2024-09-18T04:57:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9950000000000002e-06 2024-09-18T04:57:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.222185+0000", 2024-09-18T04:57:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7679999999999999e-06 2024-09-18T04:57:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253360+0000", 2024-09-18T04:57:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.031175423000000001 2024-09-18T04:57:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253430+0000", 2024-09-18T04:57:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9929000000000001e-05 2024-09-18T04:57:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:50.253500+0000", 2024-09-18T04:57:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9896999999999998e-05 2024-09-18T04:57:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.266 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:62520 3.13 3.84cdb893 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T04:57:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 40.890820185000003, 2024-09-18T04:57:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94452134799999998, 2024-09-18T04:57:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:57:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.267 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 62520 2024-09-18T04:57:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297441+0000", 2024-09-18T04:57:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9840000000000001e-06 2024-09-18T04:57:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297449+0000", 2024-09-18T04:57:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3999999999999992e-06 2024-09-18T04:57:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297455+0000", 2024-09-18T04:57:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3350000000000003e-06 2024-09-18T04:57:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297552+0000", 2024-09-18T04:57:32.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6602999999999996e-05 2024-09-18T04:57:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:57:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297559+0000", 2024-09-18T04:57:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.52e-06 2024-09-18T04:57:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241943+0000", 2024-09-18T04:57:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94438357100000003 2024-09-18T04:57:32.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241958+0000", 2024-09-18T04:57:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4598e-05 2024-09-18T04:57:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:113179 3.13 3:cf138017:::rbd_data.11ac5f86a3f9.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T04:57:32.272 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:57:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.501029773999999, 2024-09-18T04:57:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036508304999999998, 2024-09-18T04:57:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 113179 2024-09-18T04:57:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:32.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:57:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:57:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687226+0000", 2024-09-18T04:57:32.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:57:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688169+0000", 2024-09-18T04:57:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00094328200000000004 2024-09-18T04:57:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688182+0000", 2024-09-18T04:57:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2756e-05 2024-09-18T04:57:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688188+0000", 2024-09-18T04:57:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0290000000000002e-06 2024-09-18T04:57:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688225+0000", 2024-09-18T04:57:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7301000000000001e-05 2024-09-18T04:57:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688324+0000", 2024-09-18T04:57:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9419999999999993e-05 2024-09-18T04:57:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:57:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.700483+0000", 2024-09-18T04:57:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012158771000000001 2024-09-18T04:57:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:57:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723586+0000", 2024-09-18T04:57:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023102952 2024-09-18T04:57:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723634+0000", 2024-09-18T04:57:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7468e-05 2024-09-18T04:57:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:57:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723693+0000", 2024-09-18T04:57:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9539e-05 2024-09-18T04:57:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723735+0000", 2024-09-18T04:57:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1829000000000002e-05 2024-09-18T04:57:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.280 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:120568 3.f 3:f36f9030:::rbd_data.11acae5d5365.0000000000000001:2a [read 407552~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T04:57:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:25.044527+0000", 2024-09-18T04:57:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.1437293349999997, 2024-09-18T04:57:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029750146000000002, 2024-09-18T04:57:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 120568 2024-09-18T04:57:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.281 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044527+0000", 2024-09-18T04:57:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044527+0000", 2024-09-18T04:57:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044527+0000", 2024-09-18T04:57:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:32.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044533+0000", 2024-09-18T04:57:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3029999999999996e-06 2024-09-18T04:57:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:32.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044550+0000", 2024-09-18T04:57:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6702999999999999e-05 2024-09-18T04:57:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044555+0000", 2024-09-18T04:57:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.361e-06 2024-09-18T04:57:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074098+0000", 2024-09-18T04:57:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029543506000000001 2024-09-18T04:57:32.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074193+0000", 2024-09-18T04:57:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4297999999999997e-05 2024-09-18T04:57:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074277+0000", 2024-09-18T04:57:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4389999999999997e-05 2024-09-18T04:57:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:120569 3.f 3:f36f9030:::rbd_data.11acae5d5365.0000000000000001:2a [read 448512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T04:57:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:25.044573+0000", 2024-09-18T04:57:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.1436830819999999, 2024-09-18T04:57:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029916971000000001, 2024-09-18T04:57:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 120569 2024-09-18T04:57:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.292 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044573+0000", 2024-09-18T04:57:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044573+0000", 2024-09-18T04:57:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044573+0000", 2024-09-18T04:57:32.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044578+0000", 2024-09-18T04:57:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4219999999999999e-06 2024-09-18T04:57:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044586+0000", 2024-09-18T04:57:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0870000000000007e-06 2024-09-18T04:57:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044590+0000", 2024-09-18T04:57:32.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.518e-06 2024-09-18T04:57:32.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:32.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074332+0000", 2024-09-18T04:57:32.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029742721 2024-09-18T04:57:32.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074411+0000", 2024-09-18T04:57:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8592999999999998e-05 2024-09-18T04:57:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074490+0000", 2024-09-18T04:57:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9271999999999996e-05 2024-09-18T04:57:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.298 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:120570 3.f 3:f36f9030:::rbd_data.11acae5d5365.0000000000000001:2a [read 464896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T04:57:32.298 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:25.044593+0000", 2024-09-18T04:57:32.298 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.1436634940000001, 2024-09-18T04:57:32.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030073685999999999, 2024-09-18T04:57:32.298 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:32.298 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:32.298 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:32.298 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:32.298 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.298 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 120570 2024-09-18T04:57:32.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044593+0000", 2024-09-18T04:57:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044593+0000", 2024-09-18T04:57:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:32.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044593+0000", 2024-09-18T04:57:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044594+0000", 2024-09-18T04:57:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5850000000000001e-06 2024-09-18T04:57:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044597+0000", 2024-09-18T04:57:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.281e-06 2024-09-18T04:57:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044600+0000", 2024-09-18T04:57:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.475e-06 2024-09-18T04:57:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074519+0000", 2024-09-18T04:57:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.029918962 2024-09-18T04:57:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074594+0000", 2024-09-18T04:57:32.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5174999999999998e-05 2024-09-18T04:57:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074667+0000", 2024-09-18T04:57:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2543e-05 2024-09-18T04:57:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:120571 3.f 3:f36f9030:::rbd_data.11acae5d5365.0000000000000001:2a [read 489472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T04:57:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:25.044602+0000", 2024-09-18T04:57:32.302 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.1436543429999997, 2024-09-18T04:57:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030242781, 2024-09-18T04:57:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 120571 2024-09-18T04:57:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044602+0000", 2024-09-18T04:57:32.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044602+0000", 2024-09-18T04:57:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044602+0000", 2024-09-18T04:57:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:57:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:32.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044603+0000", 2024-09-18T04:57:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.252e-06 2024-09-18T04:57:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044606+0000", 2024-09-18T04:57:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9780000000000001e-06 2024-09-18T04:57:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044608+0000", 2024-09-18T04:57:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4090000000000001e-06 2024-09-18T04:57:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074699+0000", 2024-09-18T04:57:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030090781 2024-09-18T04:57:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074774+0000", 2024-09-18T04:57:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5258000000000003e-05 2024-09-18T04:57:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074845+0000", 2024-09-18T04:57:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0335999999999999e-05 2024-09-18T04:57:32.306 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:120572 3.f 3:f36f9030:::rbd_data.11acae5d5365.0000000000000001:2a [read 505856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-09-18T04:57:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:25.044610+0000", 2024-09-18T04:57:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 7.1436459159999997, 2024-09-18T04:57:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03043088, 2024-09-18T04:57:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:32.307 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 120572 2024-09-18T04:57:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044610+0000", 2024-09-18T04:57:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044610+0000", 2024-09-18T04:57:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044610+0000", 2024-09-18T04:57:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:57:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044611+0000", 2024-09-18T04:57:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2500000000000001e-06 2024-09-18T04:57:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044614+0000", 2024-09-18T04:57:32.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.971e-06 2024-09-18T04:57:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.044617+0000", 2024-09-18T04:57:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4399999999999999e-06 2024-09-18T04:57:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074884+0000", 2024-09-18T04:57:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.030266972 2024-09-18T04:57:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:32.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.074966+0000", 2024-09-18T04:57:32.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2404999999999995e-05 2024-09-18T04:57:32.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:32.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:32.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:32.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:25.075041+0000", 2024-09-18T04:57:32.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5069999999999998e-05 2024-09-18T04:57:32.311 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.311 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:32.311 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.312 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:32.312 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:32.312 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:32.313 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:32.311+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:32.313 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:57:32.414 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:32.412+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:32.421 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:32.421 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:32.421 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:32.421 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:32.433 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:57:32.433 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-18T04:57:32.504 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone41 from 0xeaffdea (overlap 0x47dbac4) to 0x0 2024-09-18T04:57:32.515 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:32.513+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:32.572 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-09-18T04:57:32.575 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:32.575 INFO:teuthology.orchestra.run.smithi002.stdout:5738 write 0x9c1e87a thru 0x9c2bc71 (0xd3f8 bytes) 2024-09-18T04:57:32.603 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:32.603 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:32.603 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:32.603 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:32.603 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:32.612 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:57:32.612 INFO:teuthology.orchestra.run.smithi002.stdout:5739 write 0x95db5cd thru 0x95de843 (0x3277 bytes) 2024-09-18T04:57:32.616 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:32.613+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:32.624 INFO:teuthology.orchestra.run.smithi002.stdout:5741 writesame 0x18b81d thru 0x19b392 data_size 0x3(0xfb76 bytes) 2024-09-18T04:57:32.699 INFO:teuthology.orchestra.run.smithi002.stdout:5742 trunc from 0xeaffdea to 0xd42efc 2024-09-18T04:57:32.716 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:32.714+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:32.759 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9316 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 56.988663422999998, 2024-09-18T04:57:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044027208999999998, 2024-09-18T04:57:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:32.759 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9316 2024-09-18T04:57:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767144+0000", 2024-09-18T04:57:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3070000000000001e-06 2024-09-18T04:57:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767150+0000", 2024-09-18T04:57:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1700000000000002e-06 2024-09-18T04:57:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767163+0000", 2024-09-18T04:57:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2670999999999999e-05 2024-09-18T04:57:32.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810995+0000", 2024-09-18T04:57:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043832310999999999 2024-09-18T04:57:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811087+0000", 2024-09-18T04:57:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1345e-05 2024-09-18T04:57:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:32.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811169+0000", 2024-09-18T04:57:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2848000000000004e-05 2024-09-18T04:57:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9317 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1728512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:57:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 56.988637787999998, 2024-09-18T04:57:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044190170000000001, 2024-09-18T04:57:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:32.764 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9317 2024-09-18T04:57:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:57:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:32.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:57:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767167+0000", 2024-09-18T04:57:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767170+0000", 2024-09-18T04:57:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7259999999999998e-06 2024-09-18T04:57:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767176+0000", 2024-09-18T04:57:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1460000000000001e-06 2024-09-18T04:57:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767182+0000", 2024-09-18T04:57:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1020000000000002e-06 2024-09-18T04:57:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811204+0000", 2024-09-18T04:57:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044022039999999998 2024-09-18T04:57:32.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811286+0000", 2024-09-18T04:57:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1623000000000001e-05 2024-09-18T04:57:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811358+0000", 2024-09-18T04:57:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2125000000000005e-05 2024-09-18T04:57:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9318 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1777664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 56.988619243000002, 2024-09-18T04:57:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044380181999999997, 2024-09-18T04:57:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9318 2024-09-18T04:57:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:32.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:32.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767189+0000", 2024-09-18T04:57:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5909999999999998e-06 2024-09-18T04:57:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767194+0000", 2024-09-18T04:57:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7690000000000002e-06 2024-09-18T04:57:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767199+0000", 2024-09-18T04:57:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189999999999999e-06 2024-09-18T04:57:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811394+0000", 2024-09-18T04:57:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044195350000000001 2024-09-18T04:57:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811483+0000", 2024-09-18T04:57:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8282999999999995e-05 2024-09-18T04:57:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811567+0000", 2024-09-18T04:57:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3855000000000003e-05 2024-09-18T04:57:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9325 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 56.986888704999998, 2024-09-18T04:57:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044063989999999997, 2024-09-18T04:57:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9325 2024-09-18T04:57:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768919+0000", 2024-09-18T04:57:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:57:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768925+0000", 2024-09-18T04:57:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2310000000000001e-06 2024-09-18T04:57:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768929+0000", 2024-09-18T04:57:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4469999999999999e-06 2024-09-18T04:57:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812823+0000", 2024-09-18T04:57:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043893975000000002 2024-09-18T04:57:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812908+0000", 2024-09-18T04:57:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4862000000000002e-05 2024-09-18T04:57:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812981+0000", 2024-09-18T04:57:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2892000000000006e-05 2024-09-18T04:57:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9334 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:57:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 56.984869918999998, 2024-09-18T04:57:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044042853999999999, 2024-09-18T04:57:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9334 2024-09-18T04:57:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:57:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:57:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770935+0000", 2024-09-18T04:57:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770937+0000", 2024-09-18T04:57:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1489999999999999e-06 2024-09-18T04:57:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770940+0000", 2024-09-18T04:57:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9569999999999998e-06 2024-09-18T04:57:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770945+0000", 2024-09-18T04:57:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:57:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814838+0000", 2024-09-18T04:57:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043892635999999999 2024-09-18T04:57:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814920+0000", 2024-09-18T04:57:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2175000000000005e-05 2024-09-18T04:57:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814979+0000", 2024-09-18T04:57:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8372000000000003e-05 2024-09-18T04:57:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9352 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1159168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:57:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 56.982648742000002, 2024-09-18T04:57:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044011269999999998, 2024-09-18T04:57:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9352 2024-09-18T04:57:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:57:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:57:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773156+0000", 2024-09-18T04:57:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773159+0000", 2024-09-18T04:57:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2919999999999998e-06 2024-09-18T04:57:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773165+0000", 2024-09-18T04:57:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0580000000000003e-06 2024-09-18T04:57:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773177+0000", 2024-09-18T04:57:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.204e-05 2024-09-18T04:57:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817082+0000", 2024-09-18T04:57:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043905528999999999 2024-09-18T04:57:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817122+0000", 2024-09-18T04:57:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9258999999999998e-05 2024-09-18T04:57:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817168+0000", 2024-09-18T04:57:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6545000000000002e-05 2024-09-18T04:57:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9353 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1208320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 56.982624444000002, 2024-09-18T04:57:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044105537, 2024-09-18T04:57:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9353 2024-09-18T04:57:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773183+0000", 2024-09-18T04:57:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2929999999999999e-06 2024-09-18T04:57:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773190+0000", 2024-09-18T04:57:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4330000000000001e-06 2024-09-18T04:57:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773194+0000", 2024-09-18T04:57:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.865e-06 2024-09-18T04:57:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817196+0000", 2024-09-18T04:57:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044001708 2024-09-18T04:57:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817236+0000", 2024-09-18T04:57:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0043999999999998e-05 2024-09-18T04:57:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817287+0000", 2024-09-18T04:57:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0605000000000001e-05 2024-09-18T04:57:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9354 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:57:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 56.982606933, 2024-09-18T04:57:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044199796, 2024-09-18T04:57:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9354 2024-09-18T04:57:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:57:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:57:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773198+0000", 2024-09-18T04:57:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773200+0000", 2024-09-18T04:57:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4159999999999998e-06 2024-09-18T04:57:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773206+0000", 2024-09-18T04:57:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.789e-06 2024-09-18T04:57:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773210+0000", 2024-09-18T04:57:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2679999999999996e-06 2024-09-18T04:57:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817314+0000", 2024-09-18T04:57:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044103187000000002 2024-09-18T04:57:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817353+0000", 2024-09-18T04:57:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8973000000000001e-05 2024-09-18T04:57:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817399+0000", 2024-09-18T04:57:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5877999999999997e-05 2024-09-18T04:57:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46591 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2097152~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 43.057750515999999, 2024-09-18T04:57:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043987310000000002, 2024-09-18T04:57:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46591 2024-09-18T04:57:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698057+0000", 2024-09-18T04:57:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.531e-06 2024-09-18T04:57:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698064+0000", 2024-09-18T04:57:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-18T04:57:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698069+0000", 2024-09-18T04:57:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.091e-06 2024-09-18T04:57:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741843+0000", 2024-09-18T04:57:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043773470000000002 2024-09-18T04:57:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741928+0000", 2024-09-18T04:57:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5113999999999999e-05 2024-09-18T04:57:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742042+0000", 2024-09-18T04:57:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114527 2024-09-18T04:57:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46592 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2121728~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:57:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 43.057732088999998, 2024-09-18T04:57:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044077948999999998, 2024-09-18T04:57:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46592 2024-09-18T04:57:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:57:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:57:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698073+0000", 2024-09-18T04:57:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698076+0000", 2024-09-18T04:57:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.481e-06 2024-09-18T04:57:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698082+0000", 2024-09-18T04:57:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8730000000000004e-06 2024-09-18T04:57:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698086+0000", 2024-09-18T04:57:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:57:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742055+0000", 2024-09-18T04:57:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043969170000000002 2024-09-18T04:57:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742093+0000", 2024-09-18T04:57:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7707000000000003e-05 2024-09-18T04:57:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742152+0000", 2024-09-18T04:57:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8323999999999999e-05 2024-09-18T04:57:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46593 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2211840~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 43.057715186000003, 2024-09-18T04:57:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044259917000000003, 2024-09-18T04:57:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46593 2024-09-18T04:57:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698092+0000", 2024-09-18T04:57:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2249999999999999e-06 2024-09-18T04:57:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698098+0000", 2024-09-18T04:57:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:57:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698103+0000", 2024-09-18T04:57:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:57:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742182+0000", 2024-09-18T04:57:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044079508000000003 2024-09-18T04:57:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742257+0000", 2024-09-18T04:57:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4678000000000006e-05 2024-09-18T04:57:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742350+0000", 2024-09-18T04:57:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3531999999999998e-05 2024-09-18T04:57:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46594 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2236416~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 43.057699489000001, 2024-09-18T04:57:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044378984000000003, 2024-09-18T04:57:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46594 2024-09-18T04:57:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698108+0000", 2024-09-18T04:57:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0880000000000002e-06 2024-09-18T04:57:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698114+0000", 2024-09-18T04:57:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6269999999999997e-06 2024-09-18T04:57:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698118+0000", 2024-09-18T04:57:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4050000000000002e-06 2024-09-18T04:57:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742365+0000", 2024-09-18T04:57:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044246799000000003 2024-09-18T04:57:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742407+0000", 2024-09-18T04:57:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2008000000000002e-05 2024-09-18T04:57:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742485+0000", 2024-09-18T04:57:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8405999999999994e-05 2024-09-18T04:57:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46595 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 43.057683416000003, 2024-09-18T04:57:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.04440463, 2024-09-18T04:57:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46595 2024-09-18T04:57:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698124+0000", 2024-09-18T04:57:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1840000000000002e-06 2024-09-18T04:57:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698130+0000", 2024-09-18T04:57:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.947e-06 2024-09-18T04:57:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698136+0000", 2024-09-18T04:57:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6520000000000003e-06 2024-09-18T04:57:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742521+0000", 2024-09-18T04:57:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044385446000000002 2024-09-18T04:57:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742527+0000", 2024-09-18T04:57:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:57:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46596 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 43.057665172, 2024-09-18T04:57:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044418066999999999, 2024-09-18T04:57:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46596 2024-09-18T04:57:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698146+0000", 2024-09-18T04:57:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2779999999999999e-06 2024-09-18T04:57:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698153+0000", 2024-09-18T04:57:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3849999999999999e-06 2024-09-18T04:57:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698164+0000", 2024-09-18T04:57:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1375999999999999e-05 2024-09-18T04:57:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742554+0000", 2024-09-18T04:57:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044389482000000001 2024-09-18T04:57:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742559+0000", 2024-09-18T04:57:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9540000000000003e-06 2024-09-18T04:57:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46597 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:57:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 43.057636807000002, 2024-09-18T04:57:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044416513999999997, 2024-09-18T04:57:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46597 2024-09-18T04:57:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:57:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:57:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698168+0000", 2024-09-18T04:57:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698175+0000", 2024-09-18T04:57:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4520000000000002e-06 2024-09-18T04:57:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698181+0000", 2024-09-18T04:57:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4150000000000004e-06 2024-09-18T04:57:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698186+0000", 2024-09-18T04:57:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7489999999999998e-06 2024-09-18T04:57:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742580+0000", 2024-09-18T04:57:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044394325999999998 2024-09-18T04:57:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742585+0000", 2024-09-18T04:57:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9849999999999997e-06 2024-09-18T04:57:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46598 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:57:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 43.057529330999998, 2024-09-18T04:57:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044335316999999999, 2024-09-18T04:57:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46598 2024-09-18T04:57:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:57:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:57:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698190+0000", 2024-09-18T04:57:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999137 2024-09-18T04:57:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698280+0000", 2024-09-18T04:57:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0409999999999997e-05 2024-09-18T04:57:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698289+0000", 2024-09-18T04:57:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9490000000000002e-06 2024-09-18T04:57:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698296+0000", 2024-09-18T04:57:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2670000000000003e-06 2024-09-18T04:57:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742607+0000", 2024-09-18T04:57:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044311072999999999 2024-09-18T04:57:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742612+0000", 2024-09-18T04:57:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9400000000000001e-06 2024-09-18T04:57:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57986 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3086336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.070683608000003, 2024-09-18T04:57:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044047731999999999, 2024-09-18T04:57:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57986 2024-09-18T04:57:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685124+0000", 2024-09-18T04:57:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2859999999999998e-06 2024-09-18T04:57:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685130+0000", 2024-09-18T04:57:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5400000000000001e-06 2024-09-18T04:57:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685136+0000", 2024-09-18T04:57:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.152e-06 2024-09-18T04:57:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.728976+0000", 2024-09-18T04:57:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043840037999999998 2024-09-18T04:57:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729064+0000", 2024-09-18T04:57:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8709999999999996e-05 2024-09-18T04:57:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729170+0000", 2024-09-18T04:57:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000105386 2024-09-18T04:57:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57987 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3101696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:57:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.070634632999997, 2024-09-18T04:57:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044238123999999997, 2024-09-18T04:57:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57987 2024-09-18T04:57:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:57:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:57:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685170+0000", 2024-09-18T04:57:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685175+0000", 2024-09-18T04:57:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9069999999999998e-06 2024-09-18T04:57:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685187+0000", 2024-09-18T04:57:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.189e-05 2024-09-18T04:57:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685195+0000", 2024-09-18T04:57:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9459999999999998e-06 2024-09-18T04:57:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729212+0000", 2024-09-18T04:57:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044017239 2024-09-18T04:57:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729301+0000", 2024-09-18T04:57:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8887999999999993e-05 2024-09-18T04:57:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729409+0000", 2024-09-18T04:57:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108119 2024-09-18T04:57:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57988 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3117056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:57:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.070592513999998, 2024-09-18T04:57:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044429835000000001, 2024-09-18T04:57:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57988 2024-09-18T04:57:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:57:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:57:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685212+0000", 2024-09-18T04:57:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685216+0000", 2024-09-18T04:57:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7390000000000001e-06 2024-09-18T04:57:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685224+0000", 2024-09-18T04:57:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2260000000000005e-06 2024-09-18T04:57:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685230+0000", 2024-09-18T04:57:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0299999999999999e-06 2024-09-18T04:57:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729454+0000", 2024-09-18T04:57:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044223248999999999 2024-09-18T04:57:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729539+0000", 2024-09-18T04:57:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5029000000000002e-05 2024-09-18T04:57:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729643+0000", 2024-09-18T04:57:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104267 2024-09-18T04:57:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57989 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3132416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:57:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.070569761999998, 2024-09-18T04:57:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044642881000000002, 2024-09-18T04:57:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57989 2024-09-18T04:57:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:57:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:57:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685235+0000", 2024-09-18T04:57:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685238+0000", 2024-09-18T04:57:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.667e-06 2024-09-18T04:57:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685245+0000", 2024-09-18T04:57:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7909999999999999e-06 2024-09-18T04:57:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685250+0000", 2024-09-18T04:57:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3730000000000001e-06 2024-09-18T04:57:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729685+0000", 2024-09-18T04:57:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044434552000000002 2024-09-18T04:57:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729775+0000", 2024-09-18T04:57:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0395000000000004e-05 2024-09-18T04:57:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729879+0000", 2024-09-18T04:57:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010361099999999999 2024-09-18T04:57:32.856 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.856 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:32.856 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.856 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:32.856 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:32.856 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:32.857 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:57:32.858 INFO:teuthology.orchestra.run.smithi002.stdout:5743 write 0xa1c13ba thru 0xa1cb0f5 (0x9d3c bytes) 2024-09-18T04:57:32.858 INFO:teuthology.orchestra.run.smithi002.stdout:5744 clone 43 order 23 su 524288 sc 3 2024-09-18T04:57:32.858 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:32.855+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:32.958 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:32.955+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:33.005 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:33.005 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:33.005 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:33.005 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:33.014 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:57:33.058 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:33.056+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:33.120 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:33.120 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:33.121 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:33.121 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:33.121 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:33.129 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:57:33.170 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:33.168+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:33.259 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:33.257+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:33.278 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14125 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:57:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.148034449999997, 2024-09-18T04:57:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052956470999999998, 2024-09-18T04:57:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14125 2024-09-18T04:57:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:57:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:57:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126849+0000", 2024-09-18T04:57:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126852+0000", 2024-09-18T04:57:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3659999999999999e-06 2024-09-18T04:57:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126858+0000", 2024-09-18T04:57:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4540000000000004e-06 2024-09-18T04:57:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126863+0000", 2024-09-18T04:57:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1780000000000002e-06 2024-09-18T04:57:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179608+0000", 2024-09-18T04:57:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052745159999999999 2024-09-18T04:57:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179703+0000", 2024-09-18T04:57:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4773999999999998e-05 2024-09-18T04:57:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179806+0000", 2024-09-18T04:57:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010296799999999999 2024-09-18T04:57:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14136 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1732608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.145873899000001, 2024-09-18T04:57:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052973412999999997, 2024-09-18T04:57:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14136 2024-09-18T04:57:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129013+0000", 2024-09-18T04:57:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.96e-06 2024-09-18T04:57:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129020+0000", 2024-09-18T04:57:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6809999999999998e-06 2024-09-18T04:57:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129026+0000", 2024-09-18T04:57:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:57:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181873+0000", 2024-09-18T04:57:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052847238999999997 2024-09-18T04:57:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181932+0000", 2024-09-18T04:57:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9339000000000002e-05 2024-09-18T04:57:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181984+0000", 2024-09-18T04:57:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1348999999999999e-05 2024-09-18T04:57:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14137 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1781760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:57:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.145853494000001, 2024-09-18T04:57:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053128914999999999, 2024-09-18T04:57:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14137 2024-09-18T04:57:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:57:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:57:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129030+0000", 2024-09-18T04:57:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129033+0000", 2024-09-18T04:57:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7439999999999999e-06 2024-09-18T04:57:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129039+0000", 2024-09-18T04:57:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.314e-06 2024-09-18T04:57:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129044+0000", 2024-09-18T04:57:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9919999999999998e-06 2024-09-18T04:57:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182015+0000", 2024-09-18T04:57:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052970875000000001 2024-09-18T04:57:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182058+0000", 2024-09-18T04:57:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2997000000000002e-05 2024-09-18T04:57:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182160+0000", 2024-09-18T04:57:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101437 2024-09-18T04:57:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14138 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1830912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:57:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.145835460000001, 2024-09-18T04:57:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053256435999999997, 2024-09-18T04:57:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14138 2024-09-18T04:57:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:57:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:57:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129048+0000", 2024-09-18T04:57:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129056+0000", 2024-09-18T04:57:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.413e-06 2024-09-18T04:57:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129062+0000", 2024-09-18T04:57:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6810000000000001e-06 2024-09-18T04:57:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129068+0000", 2024-09-18T04:57:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0459999999999996e-06 2024-09-18T04:57:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182191+0000", 2024-09-18T04:57:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053123210999999997 2024-09-18T04:57:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182238+0000", 2024-09-18T04:57:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7404999999999998e-05 2024-09-18T04:57:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182305+0000", 2024-09-18T04:57:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7028999999999998e-05 2024-09-18T04:57:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15910 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2236416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.046747928000002, 2024-09-18T04:57:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053241054000000003, 2024-09-18T04:57:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15910 2024-09-18T04:57:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228138+0000", 2024-09-18T04:57:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:57:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228144+0000", 2024-09-18T04:57:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5369999999999997e-06 2024-09-18T04:57:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228149+0000", 2024-09-18T04:57:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1209999999999998e-06 2024-09-18T04:57:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280822+0000", 2024-09-18T04:57:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052673194999999999 2024-09-18T04:57:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280903+0000", 2024-09-18T04:57:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1179000000000003e-05 2024-09-18T04:57:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281377+0000", 2024-09-18T04:57:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047417899999999997 2024-09-18T04:57:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15911 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2285568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.046731196000003, 2024-09-18T04:57:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053415151000000001, 2024-09-18T04:57:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15911 2024-09-18T04:57:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228155+0000", 2024-09-18T04:57:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:57:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228161+0000", 2024-09-18T04:57:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:57:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228166+0000", 2024-09-18T04:57:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2320000000000001e-06 2024-09-18T04:57:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281405+0000", 2024-09-18T04:57:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053239336999999998 2024-09-18T04:57:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281482+0000", 2024-09-18T04:57:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6569999999999994e-05 2024-09-18T04:57:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281568+0000", 2024-09-18T04:57:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6328000000000004e-05 2024-09-18T04:57:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15918 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2662400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.044962214999998, 2024-09-18T04:57:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053091065, 2024-09-18T04:57:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15918 2024-09-18T04:57:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229924+0000", 2024-09-18T04:57:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:57:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229930+0000", 2024-09-18T04:57:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.066e-06 2024-09-18T04:57:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229935+0000", 2024-09-18T04:57:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9119999999999997e-06 2024-09-18T04:57:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282840+0000", 2024-09-18T04:57:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052905119 2024-09-18T04:57:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282922+0000", 2024-09-18T04:57:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2081000000000002e-05 2024-09-18T04:57:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283013+0000", 2024-09-18T04:57:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0828000000000005e-05 2024-09-18T04:57:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15919 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2711552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.044944936999997, 2024-09-18T04:57:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053293021000000003, 2024-09-18T04:57:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15919 2024-09-18T04:57:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229941+0000", 2024-09-18T04:57:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2639999999999998e-06 2024-09-18T04:57:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229947+0000", 2024-09-18T04:57:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9750000000000004e-06 2024-09-18T04:57:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229952+0000", 2024-09-18T04:57:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7929999999999997e-06 2024-09-18T04:57:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283062+0000", 2024-09-18T04:57:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05310952 2024-09-18T04:57:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283162+0000", 2024-09-18T04:57:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100098 2024-09-18T04:57:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283232+0000", 2024-09-18T04:57:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0693999999999998e-05 2024-09-18T04:57:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15920 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2760704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.044928175999999, 2024-09-18T04:57:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053502367000000002, 2024-09-18T04:57:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15920 2024-09-18T04:57:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229958+0000", 2024-09-18T04:57:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1770000000000001e-06 2024-09-18T04:57:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229964+0000", 2024-09-18T04:57:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8470000000000001e-06 2024-09-18T04:57:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229968+0000", 2024-09-18T04:57:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5680000000000001e-06 2024-09-18T04:57:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283269+0000", 2024-09-18T04:57:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053300620999999999 2024-09-18T04:57:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283359+0000", 2024-09-18T04:57:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9971999999999998e-05 2024-09-18T04:57:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283458+0000", 2024-09-18T04:57:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9483999999999998e-05 2024-09-18T04:57:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15921 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2809856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:57:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.044911626000001, 2024-09-18T04:57:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053800977, 2024-09-18T04:57:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15921 2024-09-18T04:57:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:57:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:57:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229972+0000", 2024-09-18T04:57:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229974+0000", 2024-09-18T04:57:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2910000000000002e-06 2024-09-18T04:57:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229982+0000", 2024-09-18T04:57:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7249999999999997e-06 2024-09-18T04:57:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229989+0000", 2024-09-18T04:57:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6429999999999998e-06 2024-09-18T04:57:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283601+0000", 2024-09-18T04:57:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053612211 2024-09-18T04:57:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283681+0000", 2024-09-18T04:57:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0125999999999998e-05 2024-09-18T04:57:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283774+0000", 2024-09-18T04:57:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2362999999999996e-05 2024-09-18T04:57:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15922 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2859008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:57:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.044866687000003, 2024-09-18T04:57:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053882270000000003, 2024-09-18T04:57:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15922 2024-09-18T04:57:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:57:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:57:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230017+0000", 2024-09-18T04:57:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230019+0000", 2024-09-18T04:57:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4210000000000002e-06 2024-09-18T04:57:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230027+0000", 2024-09-18T04:57:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000002e-06 2024-09-18T04:57:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230032+0000", 2024-09-18T04:57:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0089999999999998e-06 2024-09-18T04:57:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283800+0000", 2024-09-18T04:57:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053768418999999998 2024-09-18T04:57:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283849+0000", 2024-09-18T04:57:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9339000000000003e-05 2024-09-18T04:57:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283900+0000", 2024-09-18T04:57:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0421999999999999e-05 2024-09-18T04:57:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15923 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2924544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.044847818000001, 2024-09-18T04:57:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054073016000000002, 2024-09-18T04:57:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15923 2024-09-18T04:57:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230038+0000", 2024-09-18T04:57:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.463e-06 2024-09-18T04:57:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230045+0000", 2024-09-18T04:57:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3570000000000003e-06 2024-09-18T04:57:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230050+0000", 2024-09-18T04:57:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7299999999999996e-06 2024-09-18T04:57:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283928+0000", 2024-09-18T04:57:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053878218999999998 2024-09-18T04:57:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284010+0000", 2024-09-18T04:57:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2577000000000006e-05 2024-09-18T04:57:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284109+0000", 2024-09-18T04:57:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9102000000000001e-05 2024-09-18T04:57:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15924 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.044829348, 2024-09-18T04:57:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054108470999999998, 2024-09-18T04:57:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15924 2024-09-18T04:57:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230061+0000", 2024-09-18T04:57:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8240000000000002e-06 2024-09-18T04:57:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230068+0000", 2024-09-18T04:57:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4320000000000004e-06 2024-09-18T04:57:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230073+0000", 2024-09-18T04:57:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.921e-06 2024-09-18T04:57:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284153+0000", 2024-09-18T04:57:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054080793000000002 2024-09-18T04:57:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284163+0000", 2024-09-18T04:57:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.859e-06 2024-09-18T04:57:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15925 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.044805232999998, 2024-09-18T04:57:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054134632000000002, 2024-09-18T04:57:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15925 2024-09-18T04:57:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230081+0000", 2024-09-18T04:57:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4109999999999999e-06 2024-09-18T04:57:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230087+0000", 2024-09-18T04:57:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.161e-06 2024-09-18T04:57:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230092+0000", 2024-09-18T04:57:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9259999999999999e-06 2024-09-18T04:57:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284204+0000", 2024-09-18T04:57:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054111546000000003 2024-09-18T04:57:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284214+0000", 2024-09-18T04:57:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9639999999999998e-06 2024-09-18T04:57:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15926 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.044787909, 2024-09-18T04:57:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054143193999999999, 2024-09-18T04:57:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15926 2024-09-18T04:57:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230098+0000", 2024-09-18T04:57:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1399999999999998e-06 2024-09-18T04:57:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230104+0000", 2024-09-18T04:57:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-18T04:57:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230109+0000", 2024-09-18T04:57:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9250000000000003e-06 2024-09-18T04:57:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284231+0000", 2024-09-18T04:57:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054122297999999999 2024-09-18T04:57:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284239+0000", 2024-09-18T04:57:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1689999999999992e-06 2024-09-18T04:57:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15927 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.044770862, 2024-09-18T04:57:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054168435000000001, 2024-09-18T04:57:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15927 2024-09-18T04:57:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230116+0000", 2024-09-18T04:57:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5150000000000001e-06 2024-09-18T04:57:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230122+0000", 2024-09-18T04:57:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1190000000000002e-06 2024-09-18T04:57:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230126+0000", 2024-09-18T04:57:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.792e-06 2024-09-18T04:57:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284273+0000", 2024-09-18T04:57:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054146086000000003 2024-09-18T04:57:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284282+0000", 2024-09-18T04:57:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2590000000000006e-06 2024-09-18T04:57:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15928 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:57:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.044753479000001, 2024-09-18T04:57:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054189739000000001, 2024-09-18T04:57:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15928 2024-09-18T04:57:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:57:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:57:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230130+0000", 2024-09-18T04:57:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230133+0000", 2024-09-18T04:57:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.244e-06 2024-09-18T04:57:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230139+0000", 2024-09-18T04:57:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9830000000000001e-06 2024-09-18T04:57:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230143+0000", 2024-09-18T04:57:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7530000000000001e-06 2024-09-18T04:57:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284313+0000", 2024-09-18T04:57:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054169332000000001 2024-09-18T04:57:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284320+0000", 2024-09-18T04:57:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8699999999999992e-06 2024-09-18T04:57:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15929 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.044737038999997, 2024-09-18T04:57:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054218771999999998, 2024-09-18T04:57:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15929 2024-09-18T04:57:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230149+0000", 2024-09-18T04:57:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-18T04:57:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230155+0000", 2024-09-18T04:57:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2469999999999997e-06 2024-09-18T04:57:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230160+0000", 2024-09-18T04:57:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7129999999999996e-06 2024-09-18T04:57:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284355+0000", 2024-09-18T04:57:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054194900999999997 2024-09-18T04:57:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284366+0000", 2024-09-18T04:57:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0993e-05 2024-09-18T04:57:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15930 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.044720038999998, 2024-09-18T04:57:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054244713999999999, 2024-09-18T04:57:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15930 2024-09-18T04:57:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230166+0000", 2024-09-18T04:57:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1009999999999999e-06 2024-09-18T04:57:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230172+0000", 2024-09-18T04:57:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.981e-06 2024-09-18T04:57:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230184+0000", 2024-09-18T04:57:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2449000000000001e-05 2024-09-18T04:57:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284400+0000", 2024-09-18T04:57:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054215452999999997 2024-09-18T04:57:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284409+0000", 2024-09-18T04:57:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0839999999999998e-06 2024-09-18T04:57:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15931 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:57:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.044694249999999, 2024-09-18T04:57:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054261928000000001, 2024-09-18T04:57:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15931 2024-09-18T04:57:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:57:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:57:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230189+0000", 2024-09-18T04:57:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230192+0000", 2024-09-18T04:57:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6599999999999999e-06 2024-09-18T04:57:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230199+0000", 2024-09-18T04:57:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5570000000000001e-06 2024-09-18T04:57:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230204+0000", 2024-09-18T04:57:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2270000000000001e-06 2024-09-18T04:57:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284442+0000", 2024-09-18T04:57:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054238383000000001 2024-09-18T04:57:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284452+0000", 2024-09-18T04:57:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6069999999999997e-06 2024-09-18T04:57:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.377 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:33.379 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:57:33.379 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:33.376+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:33.479 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:33.476+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:33.521 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone42 from 0xa1cb0f6 (overlap 0xd42efc) to 0x3fa7b2 2024-09-18T04:57:33.545 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:33.545 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:33.546 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:33.546 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:33.558 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:57:33.603 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:33.601+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:33.676 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:33.676 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:33.676 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:33.677 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:33.677 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:33.681 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:33.678+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:33.685 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:57:33.697 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-09-18T04:57:33.699 INFO:teuthology.orchestra.run.smithi002.stdout:5746 write 0x7f40450 thru 0x7f4e3c7 (0xdf78 bytes) 2024-09-18T04:57:33.709 INFO:teuthology.orchestra.run.smithi002.stdout:5747 punch from 0x673ddb1 to 0x673e00a, (0x259 bytes) 2024-09-18T04:57:33.709 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:33.709 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:33.711 INFO:teuthology.orchestra.run.smithi002.stdout:5750 trunc from 0xa1cb0f6 to 0xc3f565a 2024-09-18T04:57:33.716 INFO:teuthology.orchestra.run.smithi002.stdout:5752 write 0x710186d thru 0x7110827 (0xefbb bytes) 2024-09-18T04:57:33.723 INFO:teuthology.orchestra.run.smithi002.stdout:5754 punch from 0x1479588 to 0x147e3a2, (0x4e1a bytes) 2024-09-18T04:57:33.727 INFO:teuthology.orchestra.run.smithi002.stdout:5755 trunc from 0xc3f565a to 0x850f57a 2024-09-18T04:57:33.743 INFO:teuthology.orchestra.run.smithi002.stdout:5756 read 0x5eec03d thru 0x5eee65a (0x261e bytes) 2024-09-18T04:57:33.743 INFO:teuthology.orchestra.run.smithi002.stdout:5757 writesame 0x183109a thru 0x1839f59 data_size 0x20(0x8ec0 bytes) 2024-09-18T04:57:33.747 INFO:teuthology.orchestra.run.smithi002.stdout:5758 read 0x6ce9457 thru 0x6ced537 (0x40e1 bytes) 2024-09-18T04:57:33.753 INFO:teuthology.orchestra.run.smithi002.stdout:5760 write 0x85fcee0 thru 0x86084be (0xb5df bytes) 2024-09-18T04:57:33.756 INFO:teuthology.orchestra.run.smithi002.stdout:5761 read 0x76de6a2 thru 0x76e186f (0x31ce bytes) 2024-09-18T04:57:33.756 INFO:teuthology.orchestra.run.smithi002.stdout:5762 read 0x1dcd6ea thru 0x1dda007 (0xc91e bytes) 2024-09-18T04:57:33.769 INFO:teuthology.orchestra.run.smithi002.stdout:5763 writesame 0xedc94b0 thru 0xedd0e4d data_size 0x3(0x799e bytes) 2024-09-18T04:57:33.782 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:33.780+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:33.782 INFO:teuthology.orchestra.run.smithi002.stdout:5764 trunc from 0xedd0e4e to 0x25d7788 2024-09-18T04:57:33.797 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:33.797 INFO:teuthology.orchestra.run.smithi002.stdout:5766 writesame 0x10263c4 thru 0x1032ee5 data_size 0x36(0xcb22 bytes) 2024-09-18T04:57:33.817 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:33.817 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:33.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:105911 3.0 3:041c4dc5:::rbd_data.11ac491a2851.0000000000000006:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e63)", 2024-09-18T04:57:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:10.056416+0000", 2024-09-18T04:57:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.757562945, 2024-09-18T04:57:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.017911119999999999, 2024-09-18T04:57:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 105911 2024-09-18T04:57:33.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.056416+0000", 2024-09-18T04:57:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.056416+0000", 2024-09-18T04:57:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.056414+0000", 2024-09-18T04:57:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999981 2024-09-18T04:57:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.056426+0000", 2024-09-18T04:57:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1584e-05 2024-09-18T04:57:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.056442+0000", 2024-09-18T04:57:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6297e-05 2024-09-18T04:57:33.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.056449+0000", 2024-09-18T04:57:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9070000000000001e-06 2024-09-18T04:57:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.056491+0000", 2024-09-18T04:57:33.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1885000000000003e-05 2024-09-18T04:57:33.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.056546+0000", 2024-09-18T04:57:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4614999999999998e-05 2024-09-18T04:57:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 6", 2024-09-18T04:57:33.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.059571+0000", 2024-09-18T04:57:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0030249389999999999 2024-09-18T04:57:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.074240+0000", 2024-09-18T04:57:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.014668976 2024-09-18T04:57:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:57:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.074250+0000", 2024-09-18T04:57:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0461e-05 2024-09-18T04:57:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:57:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.074285+0000", 2024-09-18T04:57:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4541e-05 2024-09-18T04:57:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:10.074327+0000", 2024-09-18T04:57:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2626000000000002e-05 2024-09-18T04:57:33.825 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:135388 3.4 3:253b93bc:::rbd_data.11ac88109fd7.0000000000000006:2b [read 275456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T04:57:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:27.826419+0000", 2024-09-18T04:57:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9875595260000001, 2024-09-18T04:57:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015753280000000001, 2024-09-18T04:57:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.826 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 135388 2024-09-18T04:57:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826419+0000", 2024-09-18T04:57:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826419+0000", 2024-09-18T04:57:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826419+0000", 2024-09-18T04:57:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826422+0000", 2024-09-18T04:57:33.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.92e-06 2024-09-18T04:57:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826428+0000", 2024-09-18T04:57:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3459999999999999e-06 2024-09-18T04:57:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826433+0000", 2024-09-18T04:57:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4739999999999999e-06 2024-09-18T04:57:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.842020+0000", 2024-09-18T04:57:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015586714999999999 2024-09-18T04:57:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.842111+0000", 2024-09-18T04:57:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1668000000000004e-05 2024-09-18T04:57:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.842173+0000", 2024-09-18T04:57:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1601999999999997e-05 2024-09-18T04:57:33.830 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.831 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.831 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.831 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:135389 3.4 3:253b93bc:::rbd_data.11ac88109fd7.0000000000000006:2b [read 285696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T04:57:33.831 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:27.826437+0000", 2024-09-18T04:57:33.831 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9875417879999997, 2024-09-18T04:57:33.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015891418000000001, 2024-09-18T04:57:33.831 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.831 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.831 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.831 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.832 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.832 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 135389 2024-09-18T04:57:33.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.832 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826437+0000", 2024-09-18T04:57:33.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826437+0000", 2024-09-18T04:57:33.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826437+0000", 2024-09-18T04:57:33.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826439+0000", 2024-09-18T04:57:33.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3870000000000002e-06 2024-09-18T04:57:33.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826446+0000", 2024-09-18T04:57:33.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9789999999999996e-06 2024-09-18T04:57:33.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826452+0000", 2024-09-18T04:57:33.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8730000000000004e-06 2024-09-18T04:57:33.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.842201+0000", 2024-09-18T04:57:33.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015748745000000001 2024-09-18T04:57:33.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.842247+0000", 2024-09-18T04:57:33.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5886000000000001e-05 2024-09-18T04:57:33.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.842329+0000", 2024-09-18T04:57:33.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1965000000000006e-05 2024-09-18T04:57:33.837 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.837 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.837 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.837 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:135390 3.4 3:253b93bc:::rbd_data.11ac88109fd7.0000000000000006:2b [read 295936~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T04:57:33.837 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:27.826457+0000", 2024-09-18T04:57:33.837 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9875223680000005, 2024-09-18T04:57:33.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016021056999999998, 2024-09-18T04:57:33.837 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.837 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.837 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.837 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.838 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.838 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 135390 2024-09-18T04:57:33.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.838 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826457+0000", 2024-09-18T04:57:33.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826457+0000", 2024-09-18T04:57:33.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826456+0000", 2024-09-18T04:57:33.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826459+0000", 2024-09-18T04:57:33.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4899999999999999e-06 2024-09-18T04:57:33.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826465+0000", 2024-09-18T04:57:33.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2700000000000001e-06 2024-09-18T04:57:33.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826470+0000", 2024-09-18T04:57:33.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7729999999999999e-06 2024-09-18T04:57:33.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.842373+0000", 2024-09-18T04:57:33.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015903575999999999 2024-09-18T04:57:33.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.842418+0000", 2024-09-18T04:57:33.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4675999999999997e-05 2024-09-18T04:57:33.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.842478+0000", 2024-09-18T04:57:33.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9654999999999997e-05 2024-09-18T04:57:33.841 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.841 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.841 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.842 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:135391 3.4 3:253b93bc:::rbd_data.11ac88109fd7.0000000000000006:2b [read 306176~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T04:57:33.842 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:27.826487+0000", 2024-09-18T04:57:33.842 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9874924030000001, 2024-09-18T04:57:33.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016136034, 2024-09-18T04:57:33.842 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.842 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.842 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.842 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.842 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.842 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 135391 2024-09-18T04:57:33.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.842 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826487+0000", 2024-09-18T04:57:33.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826487+0000", 2024-09-18T04:57:33.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826486+0000", 2024-09-18T04:57:33.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826489+0000", 2024-09-18T04:57:33.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.32e-06 2024-09-18T04:57:33.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826497+0000", 2024-09-18T04:57:33.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7030000000000006e-06 2024-09-18T04:57:33.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826503+0000", 2024-09-18T04:57:33.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.361e-06 2024-09-18T04:57:33.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.842507+0000", 2024-09-18T04:57:33.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016004421000000001 2024-09-18T04:57:33.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.842567+0000", 2024-09-18T04:57:33.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0189000000000003e-05 2024-09-18T04:57:33.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.842623+0000", 2024-09-18T04:57:33.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5529000000000003e-05 2024-09-18T04:57:33.846 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.846 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.846 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.846 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:135392 3.4 3:253b93bc:::rbd_data.11ac88109fd7.0000000000000006:2b [read 316416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T04:57:33.846 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:27.826511+0000", 2024-09-18T04:57:33.846 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9874680700000003, 2024-09-18T04:57:33.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016338337000000001, 2024-09-18T04:57:33.846 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.846 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.846 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.846 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.847 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.847 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 135392 2024-09-18T04:57:33.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.847 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826511+0000", 2024-09-18T04:57:33.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826511+0000", 2024-09-18T04:57:33.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826510+0000", 2024-09-18T04:57:33.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826513+0000", 2024-09-18T04:57:33.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7549999999999999e-06 2024-09-18T04:57:33.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826521+0000", 2024-09-18T04:57:33.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4669999999999998e-06 2024-09-18T04:57:33.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826526+0000", 2024-09-18T04:57:33.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0780000000000003e-06 2024-09-18T04:57:33.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.842662+0000", 2024-09-18T04:57:33.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016136404 2024-09-18T04:57:33.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.842747+0000", 2024-09-18T04:57:33.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4667999999999996e-05 2024-09-18T04:57:33.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.842849+0000", 2024-09-18T04:57:33.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000102618 2024-09-18T04:57:33.850 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.850 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.850 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.850 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:135393 3.4 3:253b93bc:::rbd_data.11ac88109fd7.0000000000000006:2b [read 326656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T04:57:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:27.826545+0000", 2024-09-18T04:57:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9874337290000001, 2024-09-18T04:57:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016517042999999999, 2024-09-18T04:57:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 135393 2024-09-18T04:57:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826545+0000", 2024-09-18T04:57:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826545+0000", 2024-09-18T04:57:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826545+0000", 2024-09-18T04:57:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826548+0000", 2024-09-18T04:57:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.0510000000000001e-06 2024-09-18T04:57:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826555+0000", 2024-09-18T04:57:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4880000000000001e-06 2024-09-18T04:57:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.826561+0000", 2024-09-18T04:57:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4569999999999996e-06 2024-09-18T04:57:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.842878+0000", 2024-09-18T04:57:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016316924 2024-09-18T04:57:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.842954+0000", 2024-09-18T04:57:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6728000000000001e-05 2024-09-18T04:57:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.843062+0000", 2024-09-18T04:57:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010792 2024-09-18T04:57:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:135902 3.4 3:253b93bc:::rbd_data.11ac88109fd7.0000000000000006:2b [read 288768~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T04:57:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:27.833502+0000", 2024-09-18T04:57:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9804774710000004, 2024-09-18T04:57:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015783231000000002, 2024-09-18T04:57:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 135902 2024-09-18T04:57:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.833502+0000", 2024-09-18T04:57:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.833502+0000", 2024-09-18T04:57:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.833501+0000", 2024-09-18T04:57:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.833504+0000", 2024-09-18T04:57:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.4189999999999998e-06 2024-09-18T04:57:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.833512+0000", 2024-09-18T04:57:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0830000000000004e-06 2024-09-18T04:57:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.833518+0000", 2024-09-18T04:57:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6520000000000003e-06 2024-09-18T04:57:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.849125+0000", 2024-09-18T04:57:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015606484 2024-09-18T04:57:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.849201+0000", 2024-09-18T04:57:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6123000000000003e-05 2024-09-18T04:57:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.849285+0000", 2024-09-18T04:57:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4060000000000005e-05 2024-09-18T04:57:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:135903 3.4 3:253b93bc:::rbd_data.11ac88109fd7.0000000000000006:2b [read 299008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T04:57:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:27.833523+0000", 2024-09-18T04:57:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9804557240000005, 2024-09-18T04:57:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01594595, 2024-09-18T04:57:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 135903 2024-09-18T04:57:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.833523+0000", 2024-09-18T04:57:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.833523+0000", 2024-09-18T04:57:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.833523+0000", 2024-09-18T04:57:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.833525+0000", 2024-09-18T04:57:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2309999999999999e-06 2024-09-18T04:57:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.833531+0000", 2024-09-18T04:57:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3489999999999997e-06 2024-09-18T04:57:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.833536+0000", 2024-09-18T04:57:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7160000000000002e-06 2024-09-18T04:57:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.849312+0000", 2024-09-18T04:57:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015775555 2024-09-18T04:57:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.849383+0000", 2024-09-18T04:57:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1428999999999997e-05 2024-09-18T04:57:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.849469+0000", 2024-09-18T04:57:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6064999999999997e-05 2024-09-18T04:57:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:135904 3.4 3:253b93bc:::rbd_data.11ac88109fd7.0000000000000006:2b [read 309248~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T04:57:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:27.833541+0000", 2024-09-18T04:57:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.980438393, 2024-09-18T04:57:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016144387, 2024-09-18T04:57:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 135904 2024-09-18T04:57:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.833541+0000", 2024-09-18T04:57:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.833541+0000", 2024-09-18T04:57:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.833540+0000", 2024-09-18T04:57:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.833542+0000", 2024-09-18T04:57:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2019999999999998e-06 2024-09-18T04:57:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.833549+0000", 2024-09-18T04:57:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1340000000000001e-06 2024-09-18T04:57:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.833553+0000", 2024-09-18T04:57:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6650000000000002e-06 2024-09-18T04:57:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.849496+0000", 2024-09-18T04:57:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015942378 2024-09-18T04:57:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.849562+0000", 2024-09-18T04:57:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6025000000000006e-05 2024-09-18T04:57:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.849685+0000", 2024-09-18T04:57:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000123311 2024-09-18T04:57:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:135905 3.4 3:253b93bc:::rbd_data.11ac88109fd7.0000000000000006:2b [read 319488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T04:57:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:27.833712+0000", 2024-09-18T04:57:33.869 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9802666069999999, 2024-09-18T04:57:33.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016171031999999998, 2024-09-18T04:57:33.869 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.869 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.869 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.869 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.869 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.869 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 135905 2024-09-18T04:57:33.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.869 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.833712+0000", 2024-09-18T04:57:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.833712+0000", 2024-09-18T04:57:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.833711+0000", 2024-09-18T04:57:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.833717+0000", 2024-09-18T04:57:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4709999999999998e-06 2024-09-18T04:57:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.833744+0000", 2024-09-18T04:57:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7614999999999999e-05 2024-09-18T04:57:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.833752+0000", 2024-09-18T04:57:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5299999999999999e-06 2024-09-18T04:57:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.849711+0000", 2024-09-18T04:57:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015959316000000001 2024-09-18T04:57:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.849790+0000", 2024-09-18T04:57:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8629999999999998e-05 2024-09-18T04:57:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.849883+0000", 2024-09-18T04:57:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3587999999999999e-05 2024-09-18T04:57:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:136154 3.4 3:253b93bc:::rbd_data.11ac88109fd7.0000000000000006:2b [read 238592~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T04:57:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:27.839139+0000", 2024-09-18T04:57:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9748399919999997, 2024-09-18T04:57:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015675283000000002, 2024-09-18T04:57:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 136154 2024-09-18T04:57:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839139+0000", 2024-09-18T04:57:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839139+0000", 2024-09-18T04:57:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839139+0000", 2024-09-18T04:57:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839141+0000", 2024-09-18T04:57:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1959999999999998e-06 2024-09-18T04:57:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839147+0000", 2024-09-18T04:57:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.964e-06 2024-09-18T04:57:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839152+0000", 2024-09-18T04:57:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:57:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.854643+0000", 2024-09-18T04:57:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015491117 2024-09-18T04:57:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.854716+0000", 2024-09-18T04:57:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2889e-05 2024-09-18T04:57:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.854814+0000", 2024-09-18T04:57:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8610999999999995e-05 2024-09-18T04:57:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:136155 3.4 3:253b93bc:::rbd_data.11ac88109fd7.0000000000000006:2b [read 248832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T04:57:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:27.839188+0000", 2024-09-18T04:57:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9747910319999997, 2024-09-18T04:57:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015823067, 2024-09-18T04:57:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 136155 2024-09-18T04:57:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839188+0000", 2024-09-18T04:57:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839188+0000", 2024-09-18T04:57:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839187+0000", 2024-09-18T04:57:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839191+0000", 2024-09-18T04:57:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8419999999999998e-06 2024-09-18T04:57:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839199+0000", 2024-09-18T04:57:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9650000000000008e-06 2024-09-18T04:57:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839205+0000", 2024-09-18T04:57:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6710000000000004e-06 2024-09-18T04:57:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.854843+0000", 2024-09-18T04:57:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015638164 2024-09-18T04:57:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.854923+0000", 2024-09-18T04:57:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0368e-05 2024-09-18T04:57:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.855011+0000", 2024-09-18T04:57:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7787999999999994e-05 2024-09-18T04:57:33.882 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.882 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.882 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.882 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:136156 3.4 3:253b93bc:::rbd_data.11ac88109fd7.0000000000000006:2b [read 259072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T04:57:33.882 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:27.839210+0000", 2024-09-18T04:57:33.882 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9747690809999998, 2024-09-18T04:57:33.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015987322000000002, 2024-09-18T04:57:33.882 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.882 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.882 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.882 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.883 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.883 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 136156 2024-09-18T04:57:33.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.883 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839210+0000", 2024-09-18T04:57:33.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839210+0000", 2024-09-18T04:57:33.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839210+0000", 2024-09-18T04:57:33.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839212+0000", 2024-09-18T04:57:33.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4899999999999999e-06 2024-09-18T04:57:33.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839218+0000", 2024-09-18T04:57:33.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2750000000000001e-06 2024-09-18T04:57:33.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839221+0000", 2024-09-18T04:57:33.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9629999999999998e-06 2024-09-18T04:57:33.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.855048+0000", 2024-09-18T04:57:33.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01582681 2024-09-18T04:57:33.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.855117+0000", 2024-09-18T04:57:33.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8788000000000006e-05 2024-09-18T04:57:33.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.855197+0000", 2024-09-18T04:57:33.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0395000000000005e-05 2024-09-18T04:57:33.886 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.886 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.886 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.886 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:136157 3.4 3:253b93bc:::rbd_data.11ac88109fd7.0000000000000006:2b [read 269312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T04:57:33.887 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:27.839232+0000", 2024-09-18T04:57:33.887 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9747467529999998, 2024-09-18T04:57:33.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016146634999999999, 2024-09-18T04:57:33.887 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.887 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.887 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.887 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.887 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.887 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 136157 2024-09-18T04:57:33.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.887 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839232+0000", 2024-09-18T04:57:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839232+0000", 2024-09-18T04:57:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839232+0000", 2024-09-18T04:57:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839234+0000", 2024-09-18T04:57:33.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4399999999999999e-06 2024-09-18T04:57:33.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839239+0000", 2024-09-18T04:57:33.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3549999999999998e-06 2024-09-18T04:57:33.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839242+0000", 2024-09-18T04:57:33.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.881e-06 2024-09-18T04:57:33.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.855227+0000", 2024-09-18T04:57:33.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.015985737 2024-09-18T04:57:33.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.855299+0000", 2024-09-18T04:57:33.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1699000000000006e-05 2024-09-18T04:57:33.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.855379+0000", 2024-09-18T04:57:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9932000000000007e-05 2024-09-18T04:57:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:136158 3.4 3:253b93bc:::rbd_data.11ac88109fd7.0000000000000006:2b [read 279552~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T04:57:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:27.839244+0000", 2024-09-18T04:57:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.974734915, 2024-09-18T04:57:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016363724999999999, 2024-09-18T04:57:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 136158 2024-09-18T04:57:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839244+0000", 2024-09-18T04:57:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839244+0000", 2024-09-18T04:57:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839244+0000", 2024-09-18T04:57:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839245+0000", 2024-09-18T04:57:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.406e-06 2024-09-18T04:57:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839248+0000", 2024-09-18T04:57:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1489999999999998e-06 2024-09-18T04:57:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839251+0000", 2024-09-18T04:57:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3989999999999998e-06 2024-09-18T04:57:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.855437+0000", 2024-09-18T04:57:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016186645999999999 2024-09-18T04:57:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.855519+0000", 2024-09-18T04:57:33.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1675000000000007e-05 2024-09-18T04:57:33.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.855608+0000", 2024-09-18T04:57:33.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8696999999999995e-05 2024-09-18T04:57:33.895 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.895 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.895 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.895 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:136159 3.4 3:253b93bc:::rbd_data.11ac88109fd7.0000000000000006:2b [read 289792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T04:57:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:27.839483+0000", 2024-09-18T04:57:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9744959629999999, 2024-09-18T04:57:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016326688999999998, 2024-09-18T04:57:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 136159 2024-09-18T04:57:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839483+0000", 2024-09-18T04:57:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839483+0000", 2024-09-18T04:57:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839482+0000", 2024-09-18T04:57:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839487+0000", 2024-09-18T04:57:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9919999999999998e-06 2024-09-18T04:57:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839499+0000", 2024-09-18T04:57:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2211e-05 2024-09-18T04:57:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839506+0000", 2024-09-18T04:57:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8750000000000002e-06 2024-09-18T04:57:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.855638+0000", 2024-09-18T04:57:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016131376999999999 2024-09-18T04:57:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.855711+0000", 2024-09-18T04:57:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3805000000000002e-05 2024-09-18T04:57:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.855810+0000", 2024-09-18T04:57:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8256000000000003e-05 2024-09-18T04:57:33.900 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.900 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.900 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.900 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:136160 3.4 3:253b93bc:::rbd_data.11ac88109fd7.0000000000000006:2b [read 300032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T04:57:33.900 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:27.839523+0000", 2024-09-18T04:57:33.900 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9744564369999997, 2024-09-18T04:57:33.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016482816000000001, 2024-09-18T04:57:33.900 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.900 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.900 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.900 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.901 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.901 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 136160 2024-09-18T04:57:33.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.901 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839523+0000", 2024-09-18T04:57:33.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839523+0000", 2024-09-18T04:57:33.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839522+0000", 2024-09-18T04:57:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839526+0000", 2024-09-18T04:57:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8249999999999998e-06 2024-09-18T04:57:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839537+0000", 2024-09-18T04:57:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1484999999999999e-05 2024-09-18T04:57:33.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839543+0000", 2024-09-18T04:57:33.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9619999999999998e-06 2024-09-18T04:57:33.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.855838+0000", 2024-09-18T04:57:33.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016295113 2024-09-18T04:57:33.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.855915+0000", 2024-09-18T04:57:33.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6703000000000001e-05 2024-09-18T04:57:33.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.856005+0000", 2024-09-18T04:57:33.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0353999999999996e-05 2024-09-18T04:57:33.904 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.904 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.904 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.904 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:136161 3.4 3:253b93bc:::rbd_data.11ac88109fd7.0000000000000006:2b [read 310272~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T04:57:33.904 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:27.839549+0000", 2024-09-18T04:57:33.904 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.974430108, 2024-09-18T04:57:33.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016648196000000001, 2024-09-18T04:57:33.905 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.905 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.905 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.905 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.905 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.905 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 136161 2024-09-18T04:57:33.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.905 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839549+0000", 2024-09-18T04:57:33.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839549+0000", 2024-09-18T04:57:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839548+0000", 2024-09-18T04:57:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839551+0000", 2024-09-18T04:57:33.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.004e-06 2024-09-18T04:57:33.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839558+0000", 2024-09-18T04:57:33.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8410000000000002e-06 2024-09-18T04:57:33.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839563+0000", 2024-09-18T04:57:33.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6890000000000003e-06 2024-09-18T04:57:33.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.856034+0000", 2024-09-18T04:57:33.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016470674000000001 2024-09-18T04:57:33.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.856110+0000", 2024-09-18T04:57:33.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6094999999999996e-05 2024-09-18T04:57:33.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.856197+0000", 2024-09-18T04:57:33.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7377e-05 2024-09-18T04:57:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:136162 3.4 3:253b93bc:::rbd_data.11ac88109fd7.0000000000000006:2b [read 320512~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e80)", 2024-09-18T04:57:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:27.839568+0000", 2024-09-18T04:57:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 5.9744110839999998, 2024-09-18T04:57:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.016843028999999999, 2024-09-18T04:57:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 136162 2024-09-18T04:57:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839568+0000", 2024-09-18T04:57:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839568+0000", 2024-09-18T04:57:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:33.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839568+0000", 2024-09-18T04:57:33.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:33.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:33.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839570+0000", 2024-09-18T04:57:33.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5780000000000001e-06 2024-09-18T04:57:33.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:33.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839581+0000", 2024-09-18T04:57:33.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1362000000000001e-05 2024-09-18T04:57:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.839587+0000", 2024-09-18T04:57:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6149999999999996e-06 2024-09-18T04:57:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.856225+0000", 2024-09-18T04:57:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.01663837 2024-09-18T04:57:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.856304+0000", 2024-09-18T04:57:33.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8078999999999996e-05 2024-09-18T04:57:33.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:33.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:33.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:33.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:27.856411+0000", 2024-09-18T04:57:33.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000107409 2024-09-18T04:57:33.913 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.913 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.913 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.913 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:33.913 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:33.913 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:33.915 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:57:33.917 INFO:teuthology.orchestra.run.smithi002.stdout:5767 read 0x17525ff thru 0x1759572 (0x6f74 bytes) 2024-09-18T04:57:33.917 INFO:teuthology.orchestra.run.smithi002.stdout:5769 write 0x83b1731 thru 0x83bf2b1 (0xdb81 bytes) 2024-09-18T04:57:33.917 INFO:teuthology.orchestra.run.smithi002.stdout:5773 punch from 0x10bac88 to 0x10c9753, (0xeacb bytes) 2024-09-18T04:57:33.917 INFO:teuthology.orchestra.run.smithi002.stdout:5774 punch from 0x38d1441 to 0x38df81b, (0xe3da bytes) 2024-09-18T04:57:33.917 INFO:teuthology.orchestra.run.smithi002.stdout:5775 punch from 0x6c312d to 0x6c8e76, (0x5d49 bytes) 2024-09-18T04:57:33.917 INFO:teuthology.orchestra.run.smithi002.stdout:5776 write 0x1fa1846 thru 0x1fb1648 (0xfe03 bytes) 2024-09-18T04:57:33.917 INFO:teuthology.orchestra.run.smithi002.stdout:5777 punch from 0x2a1dc7e to 0x2a2b433, (0xd7b5 bytes) 2024-09-18T04:57:33.917 INFO:teuthology.orchestra.run.smithi002.stdout:5778 write 0x41a05b5 thru 0x41a78e2 (0x732e bytes) 2024-09-18T04:57:33.917 INFO:teuthology.orchestra.run.smithi002.stdout:5781 read 0x78cd45b thru 0x78d7a3c (0xa5e2 bytes) 2024-09-18T04:57:33.918 INFO:teuthology.orchestra.run.smithi002.stdout:5782 write 0x1346879 thru 0x1352958 (0xc0e0 bytes) 2024-09-18T04:57:33.918 INFO:teuthology.orchestra.run.smithi002.stdout:5783 write 0xdc7bf6c thru 0xdc822d0 (0x6365 bytes) 2024-09-18T04:57:33.918 INFO:teuthology.orchestra.run.smithi002.stdout:5786 trunc from 0xdc822d1 to 0xd108612 2024-09-18T04:57:33.918 INFO:teuthology.orchestra.run.smithi002.stdout:5787 read 0x7926b55 thru 0x7935ea8 (0xf354 bytes) 2024-09-18T04:57:33.918 INFO:teuthology.orchestra.run.smithi002.stdout:5788 read 0x2160c57 thru 0x21692b8 (0x8662 bytes) 2024-09-18T04:57:33.918 INFO:teuthology.orchestra.run.smithi002.stdout:5790 writesame 0x6efa9b0 thru 0x6efaef7 data_size 0xd(0x548 bytes) 2024-09-18T04:57:33.918 INFO:teuthology.orchestra.run.smithi002.stdout:5792 read 0x20fd458 thru 0x20feaba (0x1663 bytes) 2024-09-18T04:57:33.918 INFO:teuthology.orchestra.run.smithi002.stdout:5793 trunc from 0xd108612 to 0x805b1a0 2024-09-18T04:57:33.918 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:33.918 INFO:teuthology.orchestra.run.smithi002.stdout:5795 read 0x16b9b23 thru 0x16bbca3 (0x2181 bytes) 2024-09-18T04:57:33.918 INFO:teuthology.orchestra.run.smithi002.stdout:5796 read 0x7763e82 thru 0x776b70c (0x788b bytes) 2024-09-18T04:57:33.918 INFO:teuthology.orchestra.run.smithi002.stdout:5798 trunc from 0x805b1a0 to 0x203050e 2024-09-18T04:57:33.919 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:33.912+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:33.919 INFO:teuthology.orchestra.run.smithi002.stdout:5801 trunc from 0x203050e to 0x140fd43 2024-09-18T04:57:33.980 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:33.980 INFO:teuthology.orchestra.run.smithi002.stdout:5803 read 0x54e2d6 thru 0x555fb7 (0x7ce2 bytes) 2024-09-18T04:57:33.988 INFO:teuthology.orchestra.run.smithi002.stdout:5804 write 0x36287c7 thru 0x363774a (0xef84 bytes) 2024-09-18T04:57:33.991 INFO:teuthology.orchestra.run.smithi002.stdout:5805 clone 44 order 25 su 1048576 sc 5 2024-09-18T04:57:34.008 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T04:57:34.009 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T04:57:34.009 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-18T04:57:34.009 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T04:57:34.009 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T04:57:34.009 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-09-18T04:57:34.053 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:34.051+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:34.065 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:34.065 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:34.066 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:34.066 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:34.074 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:57:34.110 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:34.111 INFO:teuthology.orchestra.run.smithi099.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-18T04:57:34.111 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:34.117 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:34.115+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:34.184 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:57:34.218 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:34.215+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:34.221 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:34.221 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:34.221 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:34.222 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:34.222 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:34.231 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:57:34.310 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:57:34.310 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:57:34.318 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:34.316+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:34.327 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:57:34.327 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:57:34.343 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:57:34.344 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:57:34.360 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:57:34.360 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:57:34.376 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:57:34.377 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:57:34.381 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:34.382 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:34.391 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:57:34.393 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:57:34.394 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:57:34.410 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:57:34.410 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:57:34.427 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:57:34.428 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:57:34.434 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:34.432+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:34.446 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T04:57:34.501 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:34.502 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:34.511 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:57:34.522 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone43 from 0x363774b (overlap 0x140fd43) to 0x604bfa 2024-09-18T04:57:34.523 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:34.520+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:34.616 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-09-18T04:57:34.621 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:34.619+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:34.647 INFO:teuthology.orchestra.run.smithi002.stdout:5806 writesame 0xbdcfaf0 thru 0xbdd134f data_size 0x3(0x1860 bytes) 2024-09-18T04:57:34.647 INFO:teuthology.orchestra.run.smithi002.stdout:5807 write 0x6d2b0cb thru 0x6d30f40 (0x5e76 bytes) 2024-09-18T04:57:34.660 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:34.660 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:34.663 INFO:teuthology.orchestra.run.smithi002.stdout:5808 write 0xcb551cd thru 0xcb5982c (0x4660 bytes) 2024-09-18T04:57:34.666 INFO:teuthology.orchestra.run.smithi002.stdout:5810 read 0x5a6029a thru 0x5a6eb18 (0xe87f bytes) 2024-09-18T04:57:34.667 INFO:teuthology.orchestra.run.smithi002.stdout:5812 writesame 0x223665c thru 0x2241ea1 data_size 0x67(0xb846 bytes) 2024-09-18T04:57:34.669 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:57:34.677 INFO:teuthology.orchestra.run.smithi002.stdout:5813 write 0xeb8f35f thru 0xeb9dad5 (0xe777 bytes) 2024-09-18T04:57:34.680 INFO:teuthology.orchestra.run.smithi002.stdout:5814 read 0x592ad2a thru 0x59338d6 (0x8bad bytes) 2024-09-18T04:57:34.692 INFO:teuthology.orchestra.run.smithi002.stdout:5816 trunc from 0xeb9dad6 to 0x4d981ac 2024-09-18T04:57:34.721 INFO:teuthology.orchestra.run.smithi002.stdout:5817 write 0x2efa3e2 thru 0x2f081d8 (0xddf7 bytes) 2024-09-18T04:57:34.722 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:34.720+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:34.722 INFO:teuthology.orchestra.run.smithi002.stdout:5818 punch from 0x1b8eb4b to 0x1b9d78f, (0xec44 bytes) 2024-09-18T04:57:34.738 INFO:teuthology.orchestra.run.smithi002.stdout:5819 write 0xc6ed31c thru 0xc6f49f5 (0x76da bytes) 2024-09-18T04:57:34.741 INFO:teuthology.orchestra.run.smithi002.stdout:5821 punch from 0x5ecbc4a to 0x5ed9077, (0xd42d bytes) 2024-09-18T04:57:34.741 INFO:teuthology.orchestra.run.smithi002.stdout:5822 punch from 0xc3be128 to 0xc3c5c79, (0x7b51 bytes) 2024-09-18T04:57:34.741 INFO:teuthology.orchestra.run.smithi002.stdout:5823 write 0x150bcc5 thru 0x1515d25 (0xa061 bytes) 2024-09-18T04:57:34.745 INFO:teuthology.orchestra.run.smithi002.stdout:5826 write 0x84eeab1 thru 0x84f54cc (0x6a1c bytes) 2024-09-18T04:57:34.755 INFO:teuthology.orchestra.run.smithi002.stdout:5827 trunc from 0xc6f49f6 to 0x2ca9765 2024-09-18T04:57:34.777 INFO:teuthology.orchestra.run.smithi002.stdout:5828 trunc from 0x2ca9765 to 0xbeabbda 2024-09-18T04:57:34.781 INFO:teuthology.orchestra.run.smithi002.stdout:5829 write 0x66dc9fe thru 0x66e0cc6 (0x42c9 bytes) 2024-09-18T04:57:34.783 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:34.783 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:34.783 INFO:teuthology.orchestra.run.smithi002.stdout:5830 read 0xb0d3465 thru 0xb0e3319 (0xfeb5 bytes) 2024-09-18T04:57:34.784 INFO:teuthology.orchestra.run.smithi002.stdout:5831 write 0x38e243a thru 0x38e85c1 (0x6188 bytes) 2024-09-18T04:57:34.792 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:57:34.796 INFO:teuthology.orchestra.run.smithi002.stdout:5832 trunc from 0xbeabbda to 0x1fe703c 2024-09-18T04:57:34.803 INFO:teuthology.orchestra.run.smithi002.stdout:5833 read 0x1e46938 thru 0x1e5626b (0xf934 bytes) 2024-09-18T04:57:34.803 INFO:teuthology.orchestra.run.smithi002.stdout:5834 read 0x169c474 thru 0x169f81e (0x33ab bytes) 2024-09-18T04:57:34.804 INFO:teuthology.orchestra.run.smithi002.stdout:5835 read 0x1cae7a7 thru 0x1cbd96c (0xf1c6 bytes) 2024-09-18T04:57:34.804 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:34.804 INFO:teuthology.orchestra.run.smithi002.stdout:5838 read 0x1d8ca25 thru 0x1d993ee (0xc9ca bytes) 2024-09-18T04:57:34.806 INFO:teuthology.orchestra.run.smithi002.stdout:5839 write 0x1011fdc thru 0x1019d7a (0x7d9f bytes) 2024-09-18T04:57:34.806 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:34.811 INFO:teuthology.orchestra.run.smithi002.stdout:5842 write 0x2217569 thru 0x22208b6 (0x934e bytes) 2024-09-18T04:57:34.822 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:34.820+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:34.826 INFO:teuthology.orchestra.run.smithi002.stdout:5843 writesame 0x8a35f84 thru 0x8a41259 data_size 0xb(0xb2d6 bytes) 2024-09-18T04:57:34.828 INFO:teuthology.orchestra.run.smithi002.stdout:5844 punch from 0x69b4b45 to 0x69bc0b0, (0x756b bytes) 2024-09-18T04:57:34.828 INFO:teuthology.orchestra.run.smithi002.stdout:5848 writesame 0x7038ac5 thru 0x7039011 data_size 0x1(0x54d bytes) 2024-09-18T04:57:34.837 INFO:teuthology.orchestra.run.smithi002.stdout:5849 write 0xaa88bf0 thru 0xaa8e044 (0x5455 bytes) 2024-09-18T04:57:34.841 INFO:teuthology.orchestra.run.smithi002.stdout:5851 read 0x49cb760 thru 0x49d5383 (0x9c24 bytes) 2024-09-18T04:57:34.841 INFO:teuthology.orchestra.run.smithi002.stdout:5852 read 0xa462b0d thru 0xa4672fa (0x47ee bytes) 2024-09-18T04:57:34.841 INFO:teuthology.orchestra.run.smithi002.stdout:5853 read 0x852c78e thru 0x8535908 (0x917b bytes) 2024-09-18T04:57:34.851 INFO:teuthology.orchestra.run.smithi002.stdout:5854 write 0xe3fc462 thru 0xe401839 (0x53d8 bytes) 2024-09-18T04:57:34.862 INFO:teuthology.orchestra.run.smithi002.stdout:5855 trunc from 0xe40183a to 0x7b81144 2024-09-18T04:57:34.880 INFO:teuthology.orchestra.run.smithi002.stdout:5856 read 0x18d64ce thru 0x18dc7f7 (0x632a bytes) 2024-09-18T04:57:34.890 INFO:teuthology.orchestra.run.smithi002.stdout:5858 writesame 0x9faafcf thru 0x9fb8db8 data_size 0x41(0xddea bytes) 2024-09-18T04:57:34.891 INFO:teuthology.orchestra.run.smithi002.stdout:5859 read 0x7b390c8 thru 0x7b442e4 (0xb21d bytes) 2024-09-18T04:57:34.892 INFO:teuthology.orchestra.run.smithi002.stdout:5860 writesame 0x545f634 thru 0x54674a0 data_size 0x1(0x7e6d bytes) 2024-09-18T04:57:34.905 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:34.905 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:34.908 INFO:teuthology.orchestra.run.smithi002.stdout:5862 read 0x3035b4e thru 0x3036b6d (0x1020 bytes) 2024-09-18T04:57:34.908 INFO:teuthology.orchestra.run.smithi002.stdout:5863 read 0x6a1cb08 thru 0x6a23668 (0x6b61 bytes) 2024-09-18T04:57:34.909 INFO:teuthology.orchestra.run.smithi002.stdout:5866 write 0x581b0 thru 0x66ae8 (0xe939 bytes) 2024-09-18T04:57:34.915 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:57:34.917 INFO:teuthology.orchestra.run.smithi002.stdout:5869 trunc from 0x9fb8db9 to 0x4ba7173 2024-09-18T04:57:34.923 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:34.921+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:34.927 INFO:teuthology.orchestra.run.smithi002.stdout:5870 punch from 0x1a8a3c5 to 0x1a8fb57, (0x5792 bytes) 2024-09-18T04:57:34.931 INFO:teuthology.orchestra.run.smithi002.stdout:5871 trunc from 0x4ba7173 to 0x90f434d 2024-09-18T04:57:34.938 INFO:teuthology.orchestra.run.smithi002.stdout:5872 writesame 0x13865e6 thru 0x1393203 data_size 0x9(0xcc1e bytes) 2024-09-18T04:57:34.942 INFO:teuthology.orchestra.run.smithi002.stdout:5873 write 0x20744ae thru 0x207e700 (0xa253 bytes) 2024-09-18T04:57:34.946 INFO:teuthology.orchestra.run.smithi002.stdout:5878 trunc from 0x90f434d to 0x6f9939d 2024-09-18T04:57:34.954 INFO:teuthology.orchestra.run.smithi002.stdout:5880 trunc from 0x6f9939d to 0x7ecc81f 2024-09-18T04:57:34.968 INFO:teuthology.orchestra.run.smithi002.stdout:5881 write 0xa8439cf thru 0xa853190 (0xf7c2 bytes) 2024-09-18T04:57:34.972 INFO:teuthology.orchestra.run.smithi002.stdout:5882 punch from 0x97b1b18 to 0x97bfc93, (0xe17b bytes) 2024-09-18T04:57:34.980 INFO:teuthology.orchestra.run.smithi002.stdout:5883 write 0xc9de0e1 thru 0xc9e04cb (0x23eb bytes) 2024-09-18T04:57:34.989 INFO:teuthology.orchestra.run.smithi002.stdout:5884 trunc from 0xc9e04cc to 0x76b2db9 2024-09-18T04:57:35.018 INFO:teuthology.orchestra.run.smithi002.stdout:5886 writesame 0xdf6f94b thru 0xdf6fd4b data_size 0x1(0x401 bytes) 2024-09-18T04:57:35.018 INFO:teuthology.orchestra.run.smithi002.stdout:5888 writesame 0x40c0d42 thru 0x40cdc6b data_size 0x3(0xcf2a bytes) 2024-09-18T04:57:35.022 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:35.022 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:35.024 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:35.022+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:35.027 INFO:teuthology.orchestra.run.smithi002.stdout:5891 write 0xbd1d69a thru 0xbd2aac7 (0xd42e bytes) 2024-09-18T04:57:35.031 INFO:teuthology.orchestra.run.smithi002.stdout:5892 writesame 0x962317b thru 0x9626e9e data_size 0x56(0x3d24 bytes) 2024-09-18T04:57:35.032 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:57:35.033 INFO:teuthology.orchestra.run.smithi002.stdout:5893 read 0x2d68c76 thru 0x2d6a184 (0x150f bytes) 2024-09-18T04:57:35.033 INFO:teuthology.orchestra.run.smithi002.stdout:5894 read 0x1be0347 thru 0x1be1c78 (0x1932 bytes) 2024-09-18T04:57:35.033 INFO:teuthology.orchestra.run.smithi002.stdout:5895 read 0x7f3bf2c thru 0x7f4b0bd (0xf192 bytes) 2024-09-18T04:57:35.033 INFO:teuthology.orchestra.run.smithi002.stdout:5897 read 0xa690e29 thru 0xa6965fc (0x57d4 bytes) 2024-09-18T04:57:35.037 INFO:teuthology.orchestra.run.smithi002.stdout:5899 trunc from 0xdf6fd4c to 0xab38812 2024-09-18T04:57:35.044 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:35.044 INFO:teuthology.orchestra.run.smithi002.stdout:5901 read 0x46d95cd thru 0x46dfb09 (0x653d bytes) 2024-09-18T04:57:35.044 INFO:teuthology.orchestra.run.smithi002.stdout:5902 punch from 0x6453e97 to 0x645e010, (0xa179 bytes) 2024-09-18T04:57:35.044 INFO:teuthology.orchestra.run.smithi002.stdout:5903 writesame 0x753e349 thru 0x754114a data_size 0x27(0x2e02 bytes) 2024-09-18T04:57:35.045 INFO:teuthology.orchestra.run.smithi002.stdout:5904 read 0x9389b53 thru 0x938dc1c (0x40ca bytes) 2024-09-18T04:57:35.046 INFO:teuthology.orchestra.run.smithi002.stdout:5905 trunc from 0xab38812 to 0xb72c1af 2024-09-18T04:57:35.051 INFO:teuthology.orchestra.run.smithi002.stdout:5906 read 0x788cae1 thru 0x7891862 (0x4d82 bytes) 2024-09-18T04:57:35.052 INFO:teuthology.orchestra.run.smithi002.stdout:5907 writesame 0xa6b380c thru 0xa6bf66b data_size 0x10(0xbe60 bytes) 2024-09-18T04:57:35.055 INFO:teuthology.orchestra.run.smithi002.stdout:5909 read 0x36e7a0d thru 0x36e9d4c (0x2340 bytes) 2024-09-18T04:57:35.056 INFO:teuthology.orchestra.run.smithi002.stdout:5910 read 0x7e4b2e3 thru 0x7e4df0d (0x2c2b bytes) 2024-09-18T04:57:35.061 INFO:teuthology.orchestra.run.smithi002.stdout:5911 trunc from 0xb72c1af to 0x703a2ef 2024-09-18T04:57:35.072 INFO:teuthology.orchestra.run.smithi002.stdout:5912 write 0x385f890 thru 0x385f9a1 (0x112 bytes) 2024-09-18T04:57:35.086 INFO:teuthology.orchestra.run.smithi002.stdout:5914 write 0xce5f4e1 thru 0xce61e53 (0x2973 bytes) 2024-09-18T04:57:35.088 INFO:teuthology.orchestra.run.smithi002.stdout:5915 write 0xa2ec820 thru 0xa2ecb26 (0x307 bytes) 2024-09-18T04:57:35.091 INFO:teuthology.orchestra.run.smithi002.stdout:5916 write 0x376315b thru 0x3764c63 (0x1b09 bytes) 2024-09-18T04:57:35.094 INFO:teuthology.orchestra.run.smithi002.stdout:5917 trunc from 0xce61e54 to 0xe54ca82 2024-09-18T04:57:35.099 INFO:teuthology.orchestra.run.smithi002.stdout:5918 clone 45 order 19 su 2048 sc 10 2024-09-18T04:57:35.126 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:35.123+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:35.174 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:35.175 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:35.182 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:57:35.267 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:35.265+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:35.325 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:35.325 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:35.327 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:35.325+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:35.334 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:57:35.428 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:35.426+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:35.449 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-18T04:57:35.481 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:35.481 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:35.490 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:57:35.529 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:35.527+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:35.541 INFO:teuthology.orchestra.run.smithi002.stdout:truncating image image_client.0-clone44 from 0xe54ca82 (overlap 0x1fe703c) to 0xff381e 2024-09-18T04:57:35.630 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:35.628+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:35.644 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:35.644 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:35.654 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:57:35.671 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-09-18T04:57:35.685 INFO:teuthology.orchestra.run.smithi002.stdout:5919 write 0xb1108b8 thru 0xb114fa2 (0x46eb bytes) 2024-09-18T04:57:35.731 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:35.729+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:35.759 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:35.759 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:35.770 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:57:35.788 INFO:teuthology.orchestra.run.smithi002.stdout:5920 write 0xd30ee51 thru 0xd314038 (0x51e8 bytes) 2024-09-18T04:57:35.852 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:35.850+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:35.898 INFO:teuthology.orchestra.run.smithi002.stdout:5921 trunc from 0xe54ca82 to 0x93e2060 2024-09-18T04:57:35.922 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:35.923 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:35.934 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:35.931+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:35.935 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:57:36.035 INFO:teuthology.orchestra.run.smithi002.stdout:5922 trunc from 0x93e2060 to 0xa13883d 2024-09-18T04:57:36.035 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:36.032+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:36.041 INFO:teuthology.orchestra.run.smithi002.stdout:5927 read 0xa269c9 thru 0xa2d5bd (0x6bf5 bytes) 2024-09-18T04:57:36.053 INFO:teuthology.orchestra.run.smithi002.stdout:5928 trunc from 0xa13883d to 0x138f8a0 2024-09-18T04:57:36.127 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:36.127 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:36.136 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:36.134+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:36.140 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:57:36.207 INFO:teuthology.orchestra.run.smithi002.stdout:5929 punch from 0x46835d to 0x46d84e, (0x54f1 bytes) 2024-09-18T04:57:36.224 INFO:teuthology.orchestra.run.smithi002.stdout:5930 write 0xb61e9ea thru 0xb62bdcc (0xd3e3 bytes) 2024-09-18T04:57:36.237 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:36.234+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:36.247 INFO:teuthology.orchestra.run.smithi002.stdout:5931 trunc from 0xb62bdcd to 0xec331d 2024-09-18T04:57:36.274 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:36.274 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:36.286 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:57:36.339 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:36.336+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:36.406 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:36.406 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:36.414 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:57:36.439 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:57:36.440 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:36.438+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:36.533 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:36.534 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:36.541 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:36.538+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:36.543 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:57:36.562 INFO:teuthology.orchestra.run.smithi002.stdout:5933 write 0x24089b thru 0x240ac0 (0x226 bytes) 2024-09-18T04:57:36.593 INFO:teuthology.orchestra.run.smithi002.stdout:5934 trunc from 0xec331d to 0x985b23d 2024-09-18T04:57:36.595 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:57:36.596 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:57:36.600 INFO:teuthology.orchestra.run.smithi002.stdout:5935 trunc from 0x985b23d to 0xb5458aa 2024-09-18T04:57:36.606 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:36.606 INFO:teuthology.orchestra.run.smithi002.stdout:5937 write 0x28b004 thru 0x293210 (0x820d bytes) 2024-09-18T04:57:36.620 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:57:36.621 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:57:36.642 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:36.640+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:36.652 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:57:36.652 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:57:36.681 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:57:36.682 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:57:36.690 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:36.690 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:36.691 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:36.691 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:36.702 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:57:36.704 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:57:36.704 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:57:36.716 INFO:teuthology.orchestra.run.smithi002.stdout:5938 write 0x4c6c024 thru 0x4c73651 (0x762e bytes) 2024-09-18T04:57:36.725 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:57:36.725 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:57:36.738 INFO:teuthology.orchestra.run.smithi002.stdout:5939 trunc from 0xb5458aa to 0x3bbc6cc 2024-09-18T04:57:36.742 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:36.740+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:36.743 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:57:36.744 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:57:36.757 INFO:teuthology.orchestra.run.smithi002.stdout:5940 read 0xf61259 thru 0xf6c133 (0xaedb bytes) 2024-09-18T04:57:36.757 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:36.761 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:57:36.762 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:57:36.768 INFO:teuthology.orchestra.run.smithi002.stdout:5943 trunc from 0x3bbc6cc to 0xde2d0ce 2024-09-18T04:57:36.779 INFO:teuthology.orchestra.run.smithi002.stdout:5944 trunc from 0xde2d0ce to 0xc22082a 2024-09-18T04:57:36.789 INFO:teuthology.orchestra.run.smithi002.stdout:5945 punch from 0x2c90686 to 0x2c964d4, (0x5e4e bytes) 2024-09-18T04:57:36.789 INFO:teuthology.orchestra.run.smithi002.stdout:5947 writesame 0xb674e9d thru 0xb67b9a1 data_size 0x1(0x6b05 bytes) 2024-09-18T04:57:36.802 INFO:teuthology.orchestra.run.smithi002.stdout:5948 read 0x3e391c8 thru 0x3e42027 (0x8e60 bytes) 2024-09-18T04:57:36.803 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:36.803 INFO:teuthology.orchestra.run.smithi002.stdout:5951 read 0x4f447d4 thru 0x4f4eac1 (0xa2ee bytes) 2024-09-18T04:57:36.803 INFO:teuthology.orchestra.run.smithi002.stdout:5952 write 0xa7b2ad3 thru 0xa7ba030 (0x755e bytes) 2024-09-18T04:57:36.816 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:36.816 INFO:teuthology.orchestra.run.smithi002.stdout:5954 read 0x3a125cd thru 0x3a18ac8 (0x64fc bytes) 2024-09-18T04:57:36.816 INFO:teuthology.orchestra.run.smithi002.stdout:5955 read 0x142c890 thru 0x14329fd (0x616e bytes) 2024-09-18T04:57:36.817 INFO:teuthology.orchestra.run.smithi002.stdout:5956 trunc from 0xc22082a to 0xba83a60 2024-09-18T04:57:36.824 INFO:teuthology.orchestra.run.smithi002.stdout:5957 read 0x751e53c thru 0x752211c (0x3be1 bytes) 2024-09-18T04:57:36.824 INFO:teuthology.orchestra.run.smithi002.stdout:5958 punch from 0x93bd57 to 0x9416a1, (0x594a bytes) 2024-09-18T04:57:36.824 INFO:teuthology.orchestra.run.smithi002.stdout:5960 punch from 0x8a2a6f2 to 0x8a36903, (0xc211 bytes) 2024-09-18T04:57:36.824 INFO:teuthology.orchestra.run.smithi002.stdout:5961 read 0x5269d2d thru 0x526d525 (0x37f9 bytes) 2024-09-18T04:57:36.824 INFO:teuthology.orchestra.run.smithi002.stdout:5962 read 0x9e8349d thru 0x9e86efd (0x3a61 bytes) 2024-09-18T04:57:36.824 INFO:teuthology.orchestra.run.smithi002.stdout:5964 read 0x913c4bb thru 0x9142c6e (0x67b4 bytes) 2024-09-18T04:57:36.824 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:36.824 INFO:teuthology.orchestra.run.smithi002.stdout:5966 read 0x9447a48 thru 0x944f764 (0x7d1d bytes) 2024-09-18T04:57:36.825 INFO:teuthology.orchestra.run.smithi002.stdout:5967 read 0xa2c1a0 thru 0xa2f7fa (0x365b bytes) 2024-09-18T04:57:36.825 INFO:teuthology.orchestra.run.smithi002.stdout:5968 read 0x9ed8f5e thru 0x9ee5c89 (0xcd2c bytes) 2024-09-18T04:57:36.825 INFO:teuthology.orchestra.run.smithi002.stdout:5969 read 0x9d874f0 thru 0x9d8f9ab (0x84bc bytes) 2024-09-18T04:57:36.829 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:36.829 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:36.829 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:36.829 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:36.829 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:36.835 INFO:teuthology.orchestra.run.smithi002.stdout:5970 trunc from 0xba83a60 to 0x1eb3b1f 2024-09-18T04:57:36.843 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:57:36.855 INFO:teuthology.orchestra.run.smithi002.stdout:5971 punch from 0x9079e6 to 0x907cd0, (0x2ea bytes) 2024-09-18T04:57:36.855 INFO:teuthology.orchestra.run.smithi002.stdout:5972 read 0x9651d6 thru 0x96dff7 (0x8e22 bytes) 2024-09-18T04:57:36.855 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:36.852+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:36.868 INFO:teuthology.orchestra.run.smithi002.stdout:5973 trunc from 0x1eb3b1f to 0xc0f20fb 2024-09-18T04:57:36.874 INFO:teuthology.orchestra.run.smithi002.stdout:5975 write 0x9970289 thru 0x997495e (0x46d6 bytes) 2024-09-18T04:57:36.887 INFO:teuthology.orchestra.run.smithi002.stdout:5977 read 0x7857891 thru 0x7865af9 (0xe269 bytes) 2024-09-18T04:57:36.887 INFO:teuthology.orchestra.run.smithi002.stdout:5979 read 0xa48f8ee thru 0xa496e5f (0x7572 bytes) 2024-09-18T04:57:36.892 INFO:teuthology.orchestra.run.smithi002.stdout:5980 write 0xc301c6b thru 0xc302c80 (0x1016 bytes) 2024-09-18T04:57:36.897 INFO:teuthology.orchestra.run.smithi002.stdout:5983 read 0x94e88c8 thru 0x94f7f54 (0xf68d bytes) 2024-09-18T04:57:36.898 INFO:teuthology.orchestra.run.smithi002.stdout:5984 read 0x5b4afb9 thru 0x5b4f8c0 (0x4908 bytes) 2024-09-18T04:57:36.898 INFO:teuthology.orchestra.run.smithi002.stdout:5986 write 0xbdc4746 thru 0xbdcb284 (0x6b3f bytes) 2024-09-18T04:57:36.910 INFO:teuthology.orchestra.run.smithi002.stdout:5987 write 0x9f454f7 thru 0x9f48933 (0x343d bytes) 2024-09-18T04:57:36.923 INFO:teuthology.orchestra.run.smithi002.stdout:5989 punch from 0x893fe7f to 0x89476b2, (0x7833 bytes) 2024-09-18T04:57:36.923 INFO:teuthology.orchestra.run.smithi002.stdout:5990 writesame 0xb6290e thru 0xb6554c data_size 0x1(0x2c3f bytes) 2024-09-18T04:57:36.929 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:36.929 INFO:teuthology.orchestra.run.smithi002.stdout:5992 punch from 0xba4297e to 0xba4b160, (0x87e2 bytes) 2024-09-18T04:57:36.929 INFO:teuthology.orchestra.run.smithi002.stdout:5995 read 0xc14ff1d thru 0xc156e8c (0x6f70 bytes) 2024-09-18T04:57:36.929 INFO:teuthology.orchestra.run.smithi002.stdout:5996 punch from 0x5ccda60 to 0x5cdc8f6, (0xee96 bytes) 2024-09-18T04:57:36.930 INFO:teuthology.orchestra.run.smithi002.stdout:compare and write disabled 2024-09-18T04:57:36.930 INFO:teuthology.orchestra.run.smithi002.stdout:5999 punch from 0x6d561d to 0x6d86d6, (0x30b9 bytes) 2024-09-18T04:57:36.945 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:36.943+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:36.974 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:36.974 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:36.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:36.975 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:36.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.975 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14125 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:36.975 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:57:36.975 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.844738923999998, 2024-09-18T04:57:36.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052956470999999998, 2024-09-18T04:57:36.975 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:36.975 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:36.975 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:36.975 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:36.975 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:36.975 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14125 2024-09-18T04:57:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:57:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:57:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126849+0000", 2024-09-18T04:57:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126852+0000", 2024-09-18T04:57:36.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3659999999999999e-06 2024-09-18T04:57:36.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:36.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126858+0000", 2024-09-18T04:57:36.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4540000000000004e-06 2024-09-18T04:57:36.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:36.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126863+0000", 2024-09-18T04:57:36.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1780000000000002e-06 2024-09-18T04:57:36.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:36.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179608+0000", 2024-09-18T04:57:36.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052745159999999999 2024-09-18T04:57:36.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:36.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179703+0000", 2024-09-18T04:57:36.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4773999999999998e-05 2024-09-18T04:57:36.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:36.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179806+0000", 2024-09-18T04:57:36.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010296799999999999 2024-09-18T04:57:36.979 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:36.979 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:36.980 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:36.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.980 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14136 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1732608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:36.980 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:36.980 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.842578373000002, 2024-09-18T04:57:36.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052973412999999997, 2024-09-18T04:57:36.980 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:36.980 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:36.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:36.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:36.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:36.980 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14136 2024-09-18T04:57:36.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.981 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:36.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:36.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:36.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:36.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:36.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:36.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:36.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:36.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:36.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:36.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:36.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129013+0000", 2024-09-18T04:57:36.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.96e-06 2024-09-18T04:57:36.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:36.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129020+0000", 2024-09-18T04:57:36.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6809999999999998e-06 2024-09-18T04:57:36.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:36.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129026+0000", 2024-09-18T04:57:36.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:57:36.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:36.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181873+0000", 2024-09-18T04:57:36.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052847238999999997 2024-09-18T04:57:36.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:36.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181932+0000", 2024-09-18T04:57:36.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9339000000000002e-05 2024-09-18T04:57:36.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181984+0000", 2024-09-18T04:57:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1348999999999999e-05 2024-09-18T04:57:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14137 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1781760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:57:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.842557968000001, 2024-09-18T04:57:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053128914999999999, 2024-09-18T04:57:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14137 2024-09-18T04:57:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:57:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:57:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129030+0000", 2024-09-18T04:57:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129033+0000", 2024-09-18T04:57:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7439999999999999e-06 2024-09-18T04:57:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129039+0000", 2024-09-18T04:57:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.314e-06 2024-09-18T04:57:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129044+0000", 2024-09-18T04:57:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9919999999999998e-06 2024-09-18T04:57:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182015+0000", 2024-09-18T04:57:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052970875000000001 2024-09-18T04:57:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182058+0000", 2024-09-18T04:57:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2997000000000002e-05 2024-09-18T04:57:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182160+0000", 2024-09-18T04:57:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101437 2024-09-18T04:57:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14138 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1830912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:57:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.842539934000001, 2024-09-18T04:57:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053256435999999997, 2024-09-18T04:57:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14138 2024-09-18T04:57:36.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.993 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:36.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:36.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:57:36.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:36.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:36.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:57:36.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:36.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:36.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129048+0000", 2024-09-18T04:57:36.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:36.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:36.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129056+0000", 2024-09-18T04:57:36.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.413e-06 2024-09-18T04:57:36.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:36.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129062+0000", 2024-09-18T04:57:36.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6810000000000001e-06 2024-09-18T04:57:36.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:36.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129068+0000", 2024-09-18T04:57:36.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0459999999999996e-06 2024-09-18T04:57:36.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:36.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182191+0000", 2024-09-18T04:57:36.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053123210999999997 2024-09-18T04:57:36.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:36.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182238+0000", 2024-09-18T04:57:36.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7404999999999998e-05 2024-09-18T04:57:36.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:36.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182305+0000", 2024-09-18T04:57:36.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7028999999999998e-05 2024-09-18T04:57:36.997 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:36.997 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:36.997 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:36.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.997 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15910 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2236416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:36.997 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:36.997 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.743452402000003, 2024-09-18T04:57:36.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053241054000000003, 2024-09-18T04:57:36.997 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:36.997 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:36.998 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:36.998 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:36.998 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:36.998 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15910 2024-09-18T04:57:36.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.998 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:36.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:36.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:36.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:36.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:36.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:36.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:36.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:36.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:36.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:36.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:36.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:36.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:36.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228138+0000", 2024-09-18T04:57:37.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:57:37.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228144+0000", 2024-09-18T04:57:37.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5369999999999997e-06 2024-09-18T04:57:37.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228149+0000", 2024-09-18T04:57:37.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1209999999999998e-06 2024-09-18T04:57:37.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280822+0000", 2024-09-18T04:57:37.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052673194999999999 2024-09-18T04:57:37.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280903+0000", 2024-09-18T04:57:37.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1179000000000003e-05 2024-09-18T04:57:37.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281377+0000", 2024-09-18T04:57:37.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047417899999999997 2024-09-18T04:57:37.002 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.002 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.002 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.002 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15911 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2285568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:37.002 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:37.002 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.743435669999997, 2024-09-18T04:57:37.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053415151000000001, 2024-09-18T04:57:37.002 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.003 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.003 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.003 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.003 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.003 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15911 2024-09-18T04:57:37.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.003 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:37.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:37.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:37.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228155+0000", 2024-09-18T04:57:37.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:57:37.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228161+0000", 2024-09-18T04:57:37.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:57:37.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228166+0000", 2024-09-18T04:57:37.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2320000000000001e-06 2024-09-18T04:57:37.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281405+0000", 2024-09-18T04:57:37.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053239336999999998 2024-09-18T04:57:37.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281482+0000", 2024-09-18T04:57:37.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6569999999999994e-05 2024-09-18T04:57:37.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281568+0000", 2024-09-18T04:57:37.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6328000000000004e-05 2024-09-18T04:57:37.006 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.006 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.006 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.007 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15918 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2662400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:37.007 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:37.007 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.741666688999999, 2024-09-18T04:57:37.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053091065, 2024-09-18T04:57:37.007 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.007 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.007 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.007 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.007 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.007 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15918 2024-09-18T04:57:37.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.007 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:37.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:37.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:37.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229924+0000", 2024-09-18T04:57:37.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:57:37.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229930+0000", 2024-09-18T04:57:37.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.066e-06 2024-09-18T04:57:37.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229935+0000", 2024-09-18T04:57:37.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9119999999999997e-06 2024-09-18T04:57:37.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282840+0000", 2024-09-18T04:57:37.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052905119 2024-09-18T04:57:37.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282922+0000", 2024-09-18T04:57:37.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2081000000000002e-05 2024-09-18T04:57:37.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283013+0000", 2024-09-18T04:57:37.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0828000000000005e-05 2024-09-18T04:57:37.011 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.011 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.011 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.011 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15919 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2711552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:37.011 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:37.011 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.741649410999997, 2024-09-18T04:57:37.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053293021000000003, 2024-09-18T04:57:37.011 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.011 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.012 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.012 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.012 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.012 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15919 2024-09-18T04:57:37.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.012 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:37.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:37.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:37.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229941+0000", 2024-09-18T04:57:37.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2639999999999998e-06 2024-09-18T04:57:37.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229947+0000", 2024-09-18T04:57:37.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9750000000000004e-06 2024-09-18T04:57:37.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229952+0000", 2024-09-18T04:57:37.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7929999999999997e-06 2024-09-18T04:57:37.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283062+0000", 2024-09-18T04:57:37.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05310952 2024-09-18T04:57:37.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283162+0000", 2024-09-18T04:57:37.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100098 2024-09-18T04:57:37.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283232+0000", 2024-09-18T04:57:37.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0693999999999998e-05 2024-09-18T04:57:37.015 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.015 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.015 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.016 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15920 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2760704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:37.016 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:37.016 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.74163265, 2024-09-18T04:57:37.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053502367000000002, 2024-09-18T04:57:37.016 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.016 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.016 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.016 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.016 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.016 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15920 2024-09-18T04:57:37.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.016 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:37.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:37.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:37.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229958+0000", 2024-09-18T04:57:37.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1770000000000001e-06 2024-09-18T04:57:37.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229964+0000", 2024-09-18T04:57:37.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8470000000000001e-06 2024-09-18T04:57:37.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229968+0000", 2024-09-18T04:57:37.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5680000000000001e-06 2024-09-18T04:57:37.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283269+0000", 2024-09-18T04:57:37.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053300620999999999 2024-09-18T04:57:37.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283359+0000", 2024-09-18T04:57:37.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9971999999999998e-05 2024-09-18T04:57:37.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283458+0000", 2024-09-18T04:57:37.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9483999999999998e-05 2024-09-18T04:57:37.020 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.020 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.020 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.020 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15921 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2809856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:37.020 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:57:37.020 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.741616100000002, 2024-09-18T04:57:37.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053800977, 2024-09-18T04:57:37.020 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.020 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.020 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.020 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.021 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.021 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15921 2024-09-18T04:57:37.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.021 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:57:37.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:57:37.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229972+0000", 2024-09-18T04:57:37.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229974+0000", 2024-09-18T04:57:37.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2910000000000002e-06 2024-09-18T04:57:37.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229982+0000", 2024-09-18T04:57:37.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7249999999999997e-06 2024-09-18T04:57:37.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229989+0000", 2024-09-18T04:57:37.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6429999999999998e-06 2024-09-18T04:57:37.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283601+0000", 2024-09-18T04:57:37.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053612211 2024-09-18T04:57:37.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283681+0000", 2024-09-18T04:57:37.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0125999999999998e-05 2024-09-18T04:57:37.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283774+0000", 2024-09-18T04:57:37.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2362999999999996e-05 2024-09-18T04:57:37.024 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.024 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.024 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.025 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15922 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2859008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:37.025 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:57:37.025 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.741571161000003, 2024-09-18T04:57:37.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053882270000000003, 2024-09-18T04:57:37.025 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.025 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.025 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.025 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.025 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.026 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15922 2024-09-18T04:57:37.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.026 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:57:37.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:57:37.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230017+0000", 2024-09-18T04:57:37.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230019+0000", 2024-09-18T04:57:37.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4210000000000002e-06 2024-09-18T04:57:37.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230027+0000", 2024-09-18T04:57:37.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000002e-06 2024-09-18T04:57:37.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230032+0000", 2024-09-18T04:57:37.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0089999999999998e-06 2024-09-18T04:57:37.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283800+0000", 2024-09-18T04:57:37.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053768418999999998 2024-09-18T04:57:37.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283849+0000", 2024-09-18T04:57:37.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9339000000000003e-05 2024-09-18T04:57:37.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283900+0000", 2024-09-18T04:57:37.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0421999999999999e-05 2024-09-18T04:57:37.029 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.029 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.029 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.029 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15923 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2924544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:37.029 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:37.030 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.741552292000002, 2024-09-18T04:57:37.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054073016000000002, 2024-09-18T04:57:37.030 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.030 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.030 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.030 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.030 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.030 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15923 2024-09-18T04:57:37.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.030 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:37.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:37.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:37.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230038+0000", 2024-09-18T04:57:37.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.463e-06 2024-09-18T04:57:37.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230045+0000", 2024-09-18T04:57:37.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3570000000000003e-06 2024-09-18T04:57:37.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230050+0000", 2024-09-18T04:57:37.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7299999999999996e-06 2024-09-18T04:57:37.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283928+0000", 2024-09-18T04:57:37.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053878218999999998 2024-09-18T04:57:37.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284010+0000", 2024-09-18T04:57:37.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2577000000000006e-05 2024-09-18T04:57:37.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284109+0000", 2024-09-18T04:57:37.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9102000000000001e-05 2024-09-18T04:57:37.033 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.033 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.033 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.034 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15924 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:37.034 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:37.034 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.741533822000001, 2024-09-18T04:57:37.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054108470999999998, 2024-09-18T04:57:37.034 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.034 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:37.034 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.034 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.034 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.034 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15924 2024-09-18T04:57:37.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.034 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:37.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:37.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:37.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230061+0000", 2024-09-18T04:57:37.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8240000000000002e-06 2024-09-18T04:57:37.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230068+0000", 2024-09-18T04:57:37.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4320000000000004e-06 2024-09-18T04:57:37.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230073+0000", 2024-09-18T04:57:37.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.921e-06 2024-09-18T04:57:37.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284153+0000", 2024-09-18T04:57:37.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054080793000000002 2024-09-18T04:57:37.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284163+0000", 2024-09-18T04:57:37.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.859e-06 2024-09-18T04:57:37.037 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.037 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.038 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.038 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15925 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:37.038 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:37.038 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.741509706999999, 2024-09-18T04:57:37.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054134632000000002, 2024-09-18T04:57:37.038 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.038 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:37.038 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.038 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.038 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.038 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15925 2024-09-18T04:57:37.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.039 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:37.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:37.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:37.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230081+0000", 2024-09-18T04:57:37.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4109999999999999e-06 2024-09-18T04:57:37.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230087+0000", 2024-09-18T04:57:37.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.161e-06 2024-09-18T04:57:37.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230092+0000", 2024-09-18T04:57:37.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9259999999999999e-06 2024-09-18T04:57:37.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284204+0000", 2024-09-18T04:57:37.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054111546000000003 2024-09-18T04:57:37.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284214+0000", 2024-09-18T04:57:37.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9639999999999998e-06 2024-09-18T04:57:37.042 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.042 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.042 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.042 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15926 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:37.042 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:37.043 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.741492383000001, 2024-09-18T04:57:37.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054143193999999999, 2024-09-18T04:57:37.043 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.043 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:37.043 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.043 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.043 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.043 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15926 2024-09-18T04:57:37.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.043 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:37.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:37.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:37.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230098+0000", 2024-09-18T04:57:37.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1399999999999998e-06 2024-09-18T04:57:37.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230104+0000", 2024-09-18T04:57:37.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-18T04:57:37.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230109+0000", 2024-09-18T04:57:37.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9250000000000003e-06 2024-09-18T04:57:37.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284231+0000", 2024-09-18T04:57:37.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054122297999999999 2024-09-18T04:57:37.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284239+0000", 2024-09-18T04:57:37.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1689999999999992e-06 2024-09-18T04:57:37.046 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.046 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.046 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.046 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15927 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:37.046 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:37.047 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.741475336000001, 2024-09-18T04:57:37.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054168435000000001, 2024-09-18T04:57:37.047 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.047 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:37.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.047 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15927 2024-09-18T04:57:37.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.047 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:37.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:37.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:37.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230116+0000", 2024-09-18T04:57:37.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5150000000000001e-06 2024-09-18T04:57:37.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230122+0000", 2024-09-18T04:57:37.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1190000000000002e-06 2024-09-18T04:57:37.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230126+0000", 2024-09-18T04:57:37.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.792e-06 2024-09-18T04:57:37.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284273+0000", 2024-09-18T04:57:37.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054146086000000003 2024-09-18T04:57:37.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284282+0000", 2024-09-18T04:57:37.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2590000000000006e-06 2024-09-18T04:57:37.050 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.050 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.050 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.050 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15928 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:37.051 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:57:37.051 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.741457953000001, 2024-09-18T04:57:37.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054189739000000001, 2024-09-18T04:57:37.051 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.051 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:37.051 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.051 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.051 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.051 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15928 2024-09-18T04:57:37.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.051 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:57:37.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:57:37.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230130+0000", 2024-09-18T04:57:37.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230133+0000", 2024-09-18T04:57:37.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.244e-06 2024-09-18T04:57:37.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230139+0000", 2024-09-18T04:57:37.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9830000000000001e-06 2024-09-18T04:57:37.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230143+0000", 2024-09-18T04:57:37.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7530000000000001e-06 2024-09-18T04:57:37.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284313+0000", 2024-09-18T04:57:37.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054169332000000001 2024-09-18T04:57:37.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284320+0000", 2024-09-18T04:57:37.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8699999999999992e-06 2024-09-18T04:57:37.055 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.055 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.055 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.055 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15929 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:37.055 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:37.055 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.741441512999998, 2024-09-18T04:57:37.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054218771999999998, 2024-09-18T04:57:37.055 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.055 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:37.055 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.055 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.056 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15929 2024-09-18T04:57:37.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.056 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:37.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:37.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:37.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230149+0000", 2024-09-18T04:57:37.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-18T04:57:37.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230155+0000", 2024-09-18T04:57:37.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2469999999999997e-06 2024-09-18T04:57:37.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230160+0000", 2024-09-18T04:57:37.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7129999999999996e-06 2024-09-18T04:57:37.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284355+0000", 2024-09-18T04:57:37.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054194900999999997 2024-09-18T04:57:37.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284366+0000", 2024-09-18T04:57:37.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0993e-05 2024-09-18T04:57:37.059 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.059 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.059 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.059 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15930 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:37.059 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:37.059 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.741424512999998, 2024-09-18T04:57:37.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054244713999999999, 2024-09-18T04:57:37.059 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.059 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:37.059 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.060 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.060 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.060 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15930 2024-09-18T04:57:37.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.060 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:37.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:37.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:37.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230166+0000", 2024-09-18T04:57:37.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1009999999999999e-06 2024-09-18T04:57:37.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230172+0000", 2024-09-18T04:57:37.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.981e-06 2024-09-18T04:57:37.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230184+0000", 2024-09-18T04:57:37.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2449000000000001e-05 2024-09-18T04:57:37.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284400+0000", 2024-09-18T04:57:37.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054215452999999997 2024-09-18T04:57:37.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284409+0000", 2024-09-18T04:57:37.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0839999999999998e-06 2024-09-18T04:57:37.063 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.063 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.063 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.063 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15931 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:37.063 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:57:37.063 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.741398724, 2024-09-18T04:57:37.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054261928000000001, 2024-09-18T04:57:37.063 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.063 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:37.063 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.063 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.064 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.064 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15931 2024-09-18T04:57:37.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.064 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:57:37.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:57:37.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230189+0000", 2024-09-18T04:57:37.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230192+0000", 2024-09-18T04:57:37.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6599999999999999e-06 2024-09-18T04:57:37.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230199+0000", 2024-09-18T04:57:37.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5570000000000001e-06 2024-09-18T04:57:37.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230204+0000", 2024-09-18T04:57:37.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2270000000000001e-06 2024-09-18T04:57:37.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284442+0000", 2024-09-18T04:57:37.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054238383000000001 2024-09-18T04:57:37.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284452+0000", 2024-09-18T04:57:37.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6069999999999997e-06 2024-09-18T04:57:37.067 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.067 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.067 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.067 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.067 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.067 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:37.068 INFO:teuthology.orchestra.run.smithi002.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-09-18T04:57:37.069 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:57:37.069 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:37.066+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:37.168 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:37.166+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:37.229 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:37.229 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:37.229 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:37.229 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:37.238 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:57:37.270 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:37.267+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:37.343 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:37.344 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:37.344 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:37.344 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:37.344 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:37.353 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:57:37.370 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:37.368+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:37.464 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:37.464 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:37.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:37.464 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:37.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.464 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160713 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 645120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:37.464 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654655+0000", 2024-09-18T04:57:37.464 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.80591972199999995, 2024-09-18T04:57:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039354912999999998, 2024-09-18T04:57:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160713 2024-09-18T04:57:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654655+0000", 2024-09-18T04:57:37.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654655+0000", 2024-09-18T04:57:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654654+0000", 2024-09-18T04:57:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654657+0000", 2024-09-18T04:57:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7049999999999999e-06 2024-09-18T04:57:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654669+0000", 2024-09-18T04:57:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1960000000000001e-05 2024-09-18T04:57:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654674+0000", 2024-09-18T04:57:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8729999999999998e-06 2024-09-18T04:57:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.693848+0000", 2024-09-18T04:57:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039174202999999998 2024-09-18T04:57:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.693926+0000", 2024-09-18T04:57:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8248e-05 2024-09-18T04:57:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694009+0000", 2024-09-18T04:57:37.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3363999999999997e-05 2024-09-18T04:57:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160714 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 665600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654678+0000", 2024-09-18T04:57:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.80589634600000004, 2024-09-18T04:57:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039530155999999997, 2024-09-18T04:57:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.469 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160714 2024-09-18T04:57:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654678+0000", 2024-09-18T04:57:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654678+0000", 2024-09-18T04:57:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654678+0000", 2024-09-18T04:57:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654680+0000", 2024-09-18T04:57:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5229999999999998e-06 2024-09-18T04:57:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654686+0000", 2024-09-18T04:57:37.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9000000000000003e-06 2024-09-18T04:57:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654690+0000", 2024-09-18T04:57:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1060000000000002e-06 2024-09-18T04:57:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694042+0000", 2024-09-18T04:57:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039351490000000003 2024-09-18T04:57:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694142+0000", 2024-09-18T04:57:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100177 2024-09-18T04:57:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694208+0000", 2024-09-18T04:57:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6367999999999999e-05 2024-09-18T04:57:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160715 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:37.473 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:57:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.80588004700000004, 2024-09-18T04:57:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039724306000000001, 2024-09-18T04:57:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160715 2024-09-18T04:57:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:57:37.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:57:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:57:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654696+0000", 2024-09-18T04:57:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1830000000000001e-06 2024-09-18T04:57:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654702+0000", 2024-09-18T04:57:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5110000000000003e-06 2024-09-18T04:57:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654706+0000", 2024-09-18T04:57:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2470000000000002e-06 2024-09-18T04:57:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694245+0000", 2024-09-18T04:57:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039539259 2024-09-18T04:57:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694327+0000", 2024-09-18T04:57:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1817000000000006e-05 2024-09-18T04:57:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694419+0000", 2024-09-18T04:57:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1639999999999997e-05 2024-09-18T04:57:37.477 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160716 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:57:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.80586486199999996, 2024-09-18T04:57:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039842555000000002, 2024-09-18T04:57:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.478 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160716 2024-09-18T04:57:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:57:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:57:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:57:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654711+0000", 2024-09-18T04:57:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.99e-06 2024-09-18T04:57:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654717+0000", 2024-09-18T04:57:37.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6799999999999998e-06 2024-09-18T04:57:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654721+0000", 2024-09-18T04:57:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-18T04:57:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694449+0000", 2024-09-18T04:57:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039727476999999997 2024-09-18T04:57:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694497+0000", 2024-09-18T04:57:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8294000000000002e-05 2024-09-18T04:57:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694552+0000", 2024-09-18T04:57:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5024e-05 2024-09-18T04:57:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160717 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 727040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:57:37.482 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.80582427599999995, 2024-09-18T04:57:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040030957999999998, 2024-09-18T04:57:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160717 2024-09-18T04:57:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:57:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:57:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:57:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654752+0000", 2024-09-18T04:57:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8219999999999998e-06 2024-09-18T04:57:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654769+0000", 2024-09-18T04:57:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6249e-05 2024-09-18T04:57:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654775+0000", 2024-09-18T04:57:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2090000000000002e-06 2024-09-18T04:57:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694589+0000", 2024-09-18T04:57:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039814651999999999 2024-09-18T04:57:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694674+0000", 2024-09-18T04:57:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5032999999999997e-05 2024-09-18T04:57:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694781+0000", 2024-09-18T04:57:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106452 2024-09-18T04:57:37.486 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160718 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 747520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:57:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.805794385, 2024-09-18T04:57:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040152520999999997, 2024-09-18T04:57:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.487 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160718 2024-09-18T04:57:37.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.488 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:57:37.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:57:37.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654779+0000", 2024-09-18T04:57:37.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654782+0000", 2024-09-18T04:57:37.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.633e-06 2024-09-18T04:57:37.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654789+0000", 2024-09-18T04:57:37.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8589999999999999e-06 2024-09-18T04:57:37.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654793+0000", 2024-09-18T04:57:37.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5449999999999997e-06 2024-09-18T04:57:37.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694817+0000", 2024-09-18T04:57:37.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040023940000000001 2024-09-18T04:57:37.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694861+0000", 2024-09-18T04:57:37.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3961999999999997e-05 2024-09-18T04:57:37.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694932+0000", 2024-09-18T04:57:37.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1033999999999998e-05 2024-09-18T04:57:37.492 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.492 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.493 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.493 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160719 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 768000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:37.493 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:57:37.493 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.80577678699999999, 2024-09-18T04:57:37.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040275321000000003, 2024-09-18T04:57:37.493 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.493 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.493 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.493 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.493 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.493 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160719 2024-09-18T04:57:37.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.494 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:57:37.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:57:37.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:57:37.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654799+0000", 2024-09-18T04:57:37.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.204e-06 2024-09-18T04:57:37.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654806+0000", 2024-09-18T04:57:37.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0500000000000003e-06 2024-09-18T04:57:37.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654814+0000", 2024-09-18T04:57:37.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5549999999999997e-06 2024-09-18T04:57:37.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694967+0000", 2024-09-18T04:57:37.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040153393000000002 2024-09-18T04:57:37.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695014+0000", 2024-09-18T04:57:37.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7194999999999998e-05 2024-09-18T04:57:37.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695073+0000", 2024-09-18T04:57:37.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8315e-05 2024-09-18T04:57:37.498 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.498 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.498 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.499 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160720 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 788480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:37.499 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:57:37.499 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.80575449399999999, 2024-09-18T04:57:37.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040546450999999997, 2024-09-18T04:57:37.499 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.499 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.499 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.499 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.499 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.500 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160720 2024-09-18T04:57:37.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.500 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:57:37.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:57:37.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654819+0000", 2024-09-18T04:57:37.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654822+0000", 2024-09-18T04:57:37.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:57:37.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654828+0000", 2024-09-18T04:57:37.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2450000000000003e-06 2024-09-18T04:57:37.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654833+0000", 2024-09-18T04:57:37.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6419999999999998e-06 2024-09-18T04:57:37.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695116+0000", 2024-09-18T04:57:37.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040283126000000002 2024-09-18T04:57:37.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695203+0000", 2024-09-18T04:57:37.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7693000000000002e-05 2024-09-18T04:57:37.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695366+0000", 2024-09-18T04:57:37.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016283700000000001 2024-09-18T04:57:37.507 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.507 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.507 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.508 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160721 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 808960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:37.508 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:57:37.508 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.80573715899999998, 2024-09-18T04:57:37.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040737433000000003, 2024-09-18T04:57:37.508 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.508 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.508 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.508 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.508 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.508 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160721 2024-09-18T04:57:37.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.509 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:57:37.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:57:37.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:57:37.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654839+0000", 2024-09-18T04:57:37.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.43e-06 2024-09-18T04:57:37.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654847+0000", 2024-09-18T04:57:37.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7300000000000005e-06 2024-09-18T04:57:37.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654858+0000", 2024-09-18T04:57:37.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.099e-05 2024-09-18T04:57:37.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695391+0000", 2024-09-18T04:57:37.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040533476999999998 2024-09-18T04:57:37.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695477+0000", 2024-09-18T04:57:37.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5870000000000003e-05 2024-09-18T04:57:37.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695575+0000", 2024-09-18T04:57:37.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7264999999999998e-05 2024-09-18T04:57:37.512 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.512 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.512 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.512 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160722 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 829440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:37.512 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:57:37.512 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.80571162900000004, 2024-09-18T04:57:37.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040875739000000001, 2024-09-18T04:57:37.512 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.513 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.513 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.513 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.513 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.513 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160722 2024-09-18T04:57:37.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.513 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:57:37.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:57:37.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654862+0000", 2024-09-18T04:57:37.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654883+0000", 2024-09-18T04:57:37.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0468000000000001e-05 2024-09-18T04:57:37.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654891+0000", 2024-09-18T04:57:37.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8320000000000006e-06 2024-09-18T04:57:37.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654896+0000", 2024-09-18T04:57:37.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.13e-06 2024-09-18T04:57:37.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695628+0000", 2024-09-18T04:57:37.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040732011999999998 2024-09-18T04:57:37.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695678+0000", 2024-09-18T04:57:37.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0453e-05 2024-09-18T04:57:37.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695738+0000", 2024-09-18T04:57:37.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0164999999999997e-05 2024-09-18T04:57:37.516 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.516 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.516 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.517 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160723 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 849920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:37.517 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:57:37.517 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.80561982300000001, 2024-09-18T04:57:37.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041073576000000001, 2024-09-18T04:57:37.517 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.517 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.517 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.517 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.517 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.517 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160723 2024-09-18T04:57:37.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.517 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:57:37.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:57:37.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:57:37.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654958+0000", 2024-09-18T04:57:37.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7230000000000002e-06 2024-09-18T04:57:37.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654968+0000", 2024-09-18T04:57:37.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0665999999999999e-05 2024-09-18T04:57:37.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654974+0000", 2024-09-18T04:57:37.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9259999999999997e-06 2024-09-18T04:57:37.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695824+0000", 2024-09-18T04:57:37.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040850085000000001 2024-09-18T04:57:37.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695933+0000", 2024-09-18T04:57:37.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108738 2024-09-18T04:57:37.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696028+0000", 2024-09-18T04:57:37.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4988000000000006e-05 2024-09-18T04:57:37.521 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.521 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.521 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.521 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160724 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 870400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:37.521 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:57:37.521 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.80559548000000003, 2024-09-18T04:57:37.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041279300999999997, 2024-09-18T04:57:37.521 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.522 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.522 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.522 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.522 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.522 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160724 2024-09-18T04:57:37.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.522 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:57:37.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:57:37.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654978+0000", 2024-09-18T04:57:37.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654981+0000", 2024-09-18T04:57:37.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4530000000000001e-06 2024-09-18T04:57:37.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654987+0000", 2024-09-18T04:57:37.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6359999999999997e-06 2024-09-18T04:57:37.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654995+0000", 2024-09-18T04:57:37.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9759999999999995e-06 2024-09-18T04:57:37.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696069+0000", 2024-09-18T04:57:37.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041073447999999999 2024-09-18T04:57:37.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696160+0000", 2024-09-18T04:57:37.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0835999999999996e-05 2024-09-18T04:57:37.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696258+0000", 2024-09-18T04:57:37.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8338999999999995e-05 2024-09-18T04:57:37.525 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.525 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.525 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.526 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160725 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 890880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:37.526 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:57:37.526 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.80557441100000005, 2024-09-18T04:57:37.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041552253999999997, 2024-09-18T04:57:37.526 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.526 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.526 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.526 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.526 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.526 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160725 2024-09-18T04:57:37.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.526 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:57:37.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:57:37.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654999+0000", 2024-09-18T04:57:37.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655002+0000", 2024-09-18T04:57:37.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.419e-06 2024-09-18T04:57:37.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655008+0000", 2024-09-18T04:57:37.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2310000000000001e-06 2024-09-18T04:57:37.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655013+0000", 2024-09-18T04:57:37.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8439999999999997e-06 2024-09-18T04:57:37.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696353+0000", 2024-09-18T04:57:37.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041340322999999998 2024-09-18T04:57:37.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696455+0000", 2024-09-18T04:57:37.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000102105 2024-09-18T04:57:37.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696552+0000", 2024-09-18T04:57:37.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6738999999999997e-05 2024-09-18T04:57:37.530 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.530 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.530 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.530 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160726 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:37.530 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:57:37.531 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.80555717900000001, 2024-09-18T04:57:37.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041786440000000001, 2024-09-18T04:57:37.531 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.531 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.531 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.531 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.531 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.531 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160726 2024-09-18T04:57:37.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.532 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:57:37.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:57:37.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:57:37.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655019+0000", 2024-09-18T04:57:37.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1100000000000001e-06 2024-09-18T04:57:37.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655025+0000", 2024-09-18T04:57:37.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2190000000000001e-06 2024-09-18T04:57:37.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655030+0000", 2024-09-18T04:57:37.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5340000000000001e-06 2024-09-18T04:57:37.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696607+0000", 2024-09-18T04:57:37.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041577309 2024-09-18T04:57:37.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696694+0000", 2024-09-18T04:57:37.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7230000000000003e-05 2024-09-18T04:57:37.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696804+0000", 2024-09-18T04:57:37.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0001094 2024-09-18T04:57:37.536 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.536 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.536 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.537 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160727 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 931840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:37.537 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:57:37.537 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.80554044199999997, 2024-09-18T04:57:37.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042000527000000003, 2024-09-18T04:57:37.537 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.537 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.537 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.537 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.537 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.537 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160727 2024-09-18T04:57:37.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.538 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:57:37.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:57:37.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655033+0000", 2024-09-18T04:57:37.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655036+0000", 2024-09-18T04:57:37.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3489999999999999e-06 2024-09-18T04:57:37.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655042+0000", 2024-09-18T04:57:37.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1399999999999997e-06 2024-09-18T04:57:37.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655046+0000", 2024-09-18T04:57:37.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5480000000000003e-06 2024-09-18T04:57:37.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696843+0000", 2024-09-18T04:57:37.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041797103000000002 2024-09-18T04:57:37.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696931+0000", 2024-09-18T04:57:37.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7526000000000002e-05 2024-09-18T04:57:37.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697034+0000", 2024-09-18T04:57:37.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010336999999999999 2024-09-18T04:57:37.541 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.541 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.541 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.542 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160728 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 952320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:37.542 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:57:37.542 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.80552388500000005, 2024-09-18T04:57:37.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042191293999999997, 2024-09-18T04:57:37.542 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.542 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.542 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.542 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.542 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.542 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160728 2024-09-18T04:57:37.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.543 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:57:37.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:57:37.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:57:37.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655052+0000", 2024-09-18T04:57:37.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.232e-06 2024-09-18T04:57:37.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655058+0000", 2024-09-18T04:57:37.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9869999999999996e-06 2024-09-18T04:57:37.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655063+0000", 2024-09-18T04:57:37.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8420000000000004e-06 2024-09-18T04:57:37.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697074+0000", 2024-09-18T04:57:37.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042010446 2024-09-18T04:57:37.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697157+0000", 2024-09-18T04:57:37.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3038e-05 2024-09-18T04:57:37.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697242+0000", 2024-09-18T04:57:37.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5115000000000001e-05 2024-09-18T04:57:37.546 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.546 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.546 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.547 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160729 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 972800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:37.547 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:57:37.547 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.80550712599999996, 2024-09-18T04:57:37.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042367834, 2024-09-18T04:57:37.547 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.547 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.547 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.547 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.547 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.547 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160729 2024-09-18T04:57:37.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.547 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:57:37.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:57:37.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:57:37.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655069+0000", 2024-09-18T04:57:37.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2220000000000001e-06 2024-09-18T04:57:37.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655075+0000", 2024-09-18T04:57:37.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2129999999999996e-06 2024-09-18T04:57:37.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655080+0000", 2024-09-18T04:57:37.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6840000000000004e-06 2024-09-18T04:57:37.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697280+0000", 2024-09-18T04:57:37.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042199819999999999 2024-09-18T04:57:37.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697368+0000", 2024-09-18T04:57:37.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7863999999999998e-05 2024-09-18T04:57:37.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697435+0000", 2024-09-18T04:57:37.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7416000000000006e-05 2024-09-18T04:57:37.551 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.551 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.551 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.551 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160730 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 993280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:37.551 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:57:37.551 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.80549036100000004, 2024-09-18T04:57:37.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042559582999999998, 2024-09-18T04:57:37.551 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.551 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.551 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.551 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.552 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.552 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160730 2024-09-18T04:57:37.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.552 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:57:37.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:57:37.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:57:37.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655086+0000", 2024-09-18T04:57:37.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1500000000000002e-06 2024-09-18T04:57:37.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655092+0000", 2024-09-18T04:57:37.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.896e-06 2024-09-18T04:57:37.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655096+0000", 2024-09-18T04:57:37.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7190000000000001e-06 2024-09-18T04:57:37.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697475+0000", 2024-09-18T04:57:37.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042378644 2024-09-18T04:57:37.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697553+0000", 2024-09-18T04:57:37.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7626999999999994e-05 2024-09-18T04:57:37.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697643+0000", 2024-09-18T04:57:37.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0921000000000006e-05 2024-09-18T04:57:37.557 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.557 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.557 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.557 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160731 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 1013760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:37.557 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:57:37.557 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.80545763100000001, 2024-09-18T04:57:37.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042727764000000001, 2024-09-18T04:57:37.557 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.557 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.558 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.558 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.558 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.558 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160731 2024-09-18T04:57:37.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.558 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:57:37.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:57:37.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655116+0000", 2024-09-18T04:57:37.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655119+0000", 2024-09-18T04:57:37.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1240000000000001e-06 2024-09-18T04:57:37.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655131+0000", 2024-09-18T04:57:37.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2145e-05 2024-09-18T04:57:37.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655138+0000", 2024-09-18T04:57:37.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3369999999999996e-06 2024-09-18T04:57:37.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697688+0000", 2024-09-18T04:57:37.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042550604999999998 2024-09-18T04:57:37.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697781+0000", 2024-09-18T04:57:37.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2867000000000004e-05 2024-09-18T04:57:37.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697844+0000", 2024-09-18T04:57:37.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3236000000000002e-05 2024-09-18T04:57:37.561 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.561 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.561 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.562 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160732 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 1034240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:37.562 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:57:37.562 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.80543178299999996, 2024-09-18T04:57:37.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042857977999999998, 2024-09-18T04:57:37.562 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.562 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.562 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.562 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.562 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.562 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160732 2024-09-18T04:57:37.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.562 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:57:37.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:57:37.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:57:37.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655145+0000", 2024-09-18T04:57:37.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7329999999999999e-06 2024-09-18T04:57:37.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655152+0000", 2024-09-18T04:57:37.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8249999999999999e-06 2024-09-18T04:57:37.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655158+0000", 2024-09-18T04:57:37.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5010000000000001e-06 2024-09-18T04:57:37.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697873+0000", 2024-09-18T04:57:37.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042714457999999997 2024-09-18T04:57:37.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697944+0000", 2024-09-18T04:57:37.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1486e-05 2024-09-18T04:57:37.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.698000+0000", 2024-09-18T04:57:37.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6332000000000002e-05 2024-09-18T04:57:37.566 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.566 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.566 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.566 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.566 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.566 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:37.566 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T04:57:37.568 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:57:37.568 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:37.565+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:37.669 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:37.666+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:37.716 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:37.717 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:37.717 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:37.717 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:37.729 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:57:37.769 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:37.767+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:37.870 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:37.867+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:37.874 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:37.875 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:37.875 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:37.875 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:37.875 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:37.884 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:57:37.971 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:37.969+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:37.992 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:37.992 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:37.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:37.992 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:37.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.992 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11154 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:37.992 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:57:37.993 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 62.099241560000003, 2024-09-18T04:57:37.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040591919999999997, 2024-09-18T04:57:37.993 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.993 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.993 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11154 2024-09-18T04:57:37.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.993 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:57:37.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:57:37.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889400+0000", 2024-09-18T04:57:37.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889403+0000", 2024-09-18T04:57:37.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3999999999999999e-06 2024-09-18T04:57:37.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:37.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889409+0000", 2024-09-18T04:57:37.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3380000000000001e-06 2024-09-18T04:57:37.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:37.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889414+0000", 2024-09-18T04:57:37.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5789999999999997e-06 2024-09-18T04:57:37.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:37.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929811+0000", 2024-09-18T04:57:37.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040397486000000003 2024-09-18T04:57:37.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:37.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929898+0000", 2024-09-18T04:57:37.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6370000000000001e-05 2024-09-18T04:57:37.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:37.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929993+0000", 2024-09-18T04:57:37.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5136000000000005e-05 2024-09-18T04:57:37.997 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.997 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:37.997 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:37.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.997 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11155 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2015232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:37.997 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:57:37.997 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 62.099224382000003, 2024-09-18T04:57:37.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040769156000000001, 2024-09-18T04:57:37.997 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:37.997 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:37.997 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:37.998 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:37.998 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:37.998 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11155 2024-09-18T04:57:37.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.998 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:37.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:37.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:57:37.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:37.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:57:37.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:37.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:37.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:57:37.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:37.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:37.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:37.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889420+0000", 2024-09-18T04:57:37.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.181e-06 2024-09-18T04:57:37.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:37.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:38.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889426+0000", 2024-09-18T04:57:38.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8599999999999998e-06 2024-09-18T04:57:38.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889430+0000", 2024-09-18T04:57:38.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4399999999999998e-06 2024-09-18T04:57:38.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:38.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930020+0000", 2024-09-18T04:57:38.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040589698 2024-09-18T04:57:38.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:38.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930105+0000", 2024-09-18T04:57:38.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4812e-05 2024-09-18T04:57:38.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:38.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930187+0000", 2024-09-18T04:57:38.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2540999999999995e-05 2024-09-18T04:57:38.002 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.002 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:38.002 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.002 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11156 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2064384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:38.002 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:57:38.002 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 62.099184088000001, 2024-09-18T04:57:38.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040934419, 2024-09-18T04:57:38.002 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:38.002 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:38.002 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:38.002 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:38.003 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.003 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11156 2024-09-18T04:57:38.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.003 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:38.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:38.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:57:38.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:38.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:57:38.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:38.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889457+0000", 2024-09-18T04:57:38.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:38.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:38.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889462+0000", 2024-09-18T04:57:38.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8350000000000003e-06 2024-09-18T04:57:38.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:38.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889481+0000", 2024-09-18T04:57:38.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8845999999999999e-05 2024-09-18T04:57:38.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889495+0000", 2024-09-18T04:57:38.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4229e-05 2024-09-18T04:57:38.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:38.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930216+0000", 2024-09-18T04:57:38.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040721081999999999 2024-09-18T04:57:38.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:38.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930299+0000", 2024-09-18T04:57:38.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2995000000000001e-05 2024-09-18T04:57:38.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:38.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930393+0000", 2024-09-18T04:57:38.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3185999999999999e-05 2024-09-18T04:57:38.006 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.007 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:38.007 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.007 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11404 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1609728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:38.007 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:57:38.008 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 62.092379926, 2024-09-18T04:57:38.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040669012999999997, 2024-09-18T04:57:38.008 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:38.008 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:38.008 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:38.008 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:38.008 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.008 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11404 2024-09-18T04:57:38.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.009 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:38.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:38.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:57:38.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:38.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:57:38.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:38.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:57:38.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:57:38.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:38.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896263+0000", 2024-09-18T04:57:38.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2729999999999999e-06 2024-09-18T04:57:38.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:38.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896266+0000", 2024-09-18T04:57:38.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8200000000000001e-06 2024-09-18T04:57:38.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896269+0000", 2024-09-18T04:57:38.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3199999999999998e-06 2024-09-18T04:57:38.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:38.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936538+0000", 2024-09-18T04:57:38.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040269394999999999 2024-09-18T04:57:38.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:38.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936657+0000", 2024-09-18T04:57:38.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119498 2024-09-18T04:57:38.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:38.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936931+0000", 2024-09-18T04:57:38.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027391900000000002 2024-09-18T04:57:38.012 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.012 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:38.012 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.012 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11405 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1658880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:38.012 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:57:38.013 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 62.092371667999998, 2024-09-18T04:57:38.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040891034999999999, 2024-09-18T04:57:38.013 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:38.013 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:38.013 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:38.013 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:38.013 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.013 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11405 2024-09-18T04:57:38.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.013 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:38.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:38.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:57:38.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:38.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:57:38.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:38.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896270+0000", 2024-09-18T04:57:38.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:57:38.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:38.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896272+0000", 2024-09-18T04:57:38.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.124e-06 2024-09-18T04:57:38.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:38.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896274+0000", 2024-09-18T04:57:38.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8660000000000002e-06 2024-09-18T04:57:38.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896285+0000", 2024-09-18T04:57:38.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0101e-05 2024-09-18T04:57:38.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:38.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936986+0000", 2024-09-18T04:57:38.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040701213999999999 2024-09-18T04:57:38.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:38.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937070+0000", 2024-09-18T04:57:38.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4211999999999999e-05 2024-09-18T04:57:38.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:38.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937162+0000", 2024-09-18T04:57:38.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1728e-05 2024-09-18T04:57:38.017 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.017 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:38.017 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.017 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11406 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1708032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:38.017 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:57:38.017 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 62.092355499, 2024-09-18T04:57:38.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041085251000000003, 2024-09-18T04:57:38.017 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:38.017 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:38.018 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:38.018 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:38.018 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.018 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11406 2024-09-18T04:57:38.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.018 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:38.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:38.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:57:38.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:38.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:57:38.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:38.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:57:38.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:57:38.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:38.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896288+0000", 2024-09-18T04:57:38.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2440000000000001e-06 2024-09-18T04:57:38.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:38.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896291+0000", 2024-09-18T04:57:38.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8289999999999999e-06 2024-09-18T04:57:38.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896293+0000", 2024-09-18T04:57:38.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5399999999999998e-06 2024-09-18T04:57:38.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:38.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937201+0000", 2024-09-18T04:57:38.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040907418000000001 2024-09-18T04:57:38.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:38.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937283+0000", 2024-09-18T04:57:38.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2460999999999995e-05 2024-09-18T04:57:38.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:38.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937372+0000", 2024-09-18T04:57:38.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8987999999999996e-05 2024-09-18T04:57:38.021 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.021 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:38.021 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.022 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11407 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:38.022 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:57:38.022 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 62.092345678999997, 2024-09-18T04:57:38.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041438580000000003, 2024-09-18T04:57:38.022 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:38.022 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:38.022 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:38.022 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:38.022 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.022 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11407 2024-09-18T04:57:38.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.023 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:38.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:38.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:57:38.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:38.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:57:38.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:38.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896296+0000", 2024-09-18T04:57:38.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:38.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896298+0000", 2024-09-18T04:57:38.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:57:38.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:38.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896304+0000", 2024-09-18T04:57:38.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6570000000000002e-06 2024-09-18T04:57:38.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896308+0000", 2024-09-18T04:57:38.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0940000000000001e-06 2024-09-18T04:57:38.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:38.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937540+0000", 2024-09-18T04:57:38.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041231469 2024-09-18T04:57:38.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:38.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937625+0000", 2024-09-18T04:57:38.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5767999999999996e-05 2024-09-18T04:57:38.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:38.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937735+0000", 2024-09-18T04:57:38.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010974 2024-09-18T04:57:38.026 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.026 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:38.026 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.026 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11408 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:38.026 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:57:38.026 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 62.09233004, 2024-09-18T04:57:38.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041653055000000001, 2024-09-18T04:57:38.027 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:38.027 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:38.027 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:38.027 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:38.027 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.027 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11408 2024-09-18T04:57:38.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.027 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:38.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:38.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:57:38.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:38.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:57:38.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:38.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:57:38.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.028 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:38.028 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896314+0000", 2024-09-18T04:57:38.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0140000000000001e-06 2024-09-18T04:57:38.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:38.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896320+0000", 2024-09-18T04:57:38.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-18T04:57:38.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896324+0000", 2024-09-18T04:57:38.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4410000000000003e-06 2024-09-18T04:57:38.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:38.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937769+0000", 2024-09-18T04:57:38.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041445243 2024-09-18T04:57:38.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:38.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937861+0000", 2024-09-18T04:57:38.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1471000000000006e-05 2024-09-18T04:57:38.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:38.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937965+0000", 2024-09-18T04:57:38.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104516 2024-09-18T04:57:38.031 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.031 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:38.031 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.031 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11409 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:38.031 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:57:38.031 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 62.092313984, 2024-09-18T04:57:38.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041840467999999999, 2024-09-18T04:57:38.031 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:38.031 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:38.031 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:38.031 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:38.031 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.032 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11409 2024-09-18T04:57:38.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.032 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:38.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:38.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:57:38.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:38.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:57:38.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:38.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:57:38.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:38.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896331+0000", 2024-09-18T04:57:38.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5950000000000001e-06 2024-09-18T04:57:38.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:38.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896336+0000", 2024-09-18T04:57:38.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7259999999999999e-06 2024-09-18T04:57:38.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896341+0000", 2024-09-18T04:57:38.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5619999999999997e-06 2024-09-18T04:57:38.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:38.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937995+0000", 2024-09-18T04:57:38.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041654060999999999 2024-09-18T04:57:38.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:38.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938078+0000", 2024-09-18T04:57:38.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2869999999999998e-05 2024-09-18T04:57:38.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:38.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938169+0000", 2024-09-18T04:57:38.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1040999999999998e-05 2024-09-18T04:57:38.035 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.035 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:38.035 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.035 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11410 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1921024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:38.035 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:57:38.036 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 62.092297500999997, 2024-09-18T04:57:38.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042060342000000001, 2024-09-18T04:57:38.036 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:38.036 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:38.036 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:38.036 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:38.036 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.036 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11410 2024-09-18T04:57:38.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.036 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:38.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:38.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:57:38.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:38.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:57:38.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:38.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896344+0000", 2024-09-18T04:57:38.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:38.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896360+0000", 2024-09-18T04:57:38.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5068e-05 2024-09-18T04:57:38.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:38.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896366+0000", 2024-09-18T04:57:38.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5119999999999997e-06 2024-09-18T04:57:38.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896371+0000", 2024-09-18T04:57:38.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:57:38.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:38.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938198+0000", 2024-09-18T04:57:38.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041826579000000003 2024-09-18T04:57:38.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:38.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938297+0000", 2024-09-18T04:57:38.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9067000000000005e-05 2024-09-18T04:57:38.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:38.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938405+0000", 2024-09-18T04:57:38.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108492 2024-09-18T04:57:38.040 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.040 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:38.040 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.040 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11411 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1970176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:38.040 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:57:38.040 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 62.092266659000003, 2024-09-18T04:57:38.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042259191000000002, 2024-09-18T04:57:38.040 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:38.040 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:38.040 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:38.041 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:38.041 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.041 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11411 2024-09-18T04:57:38.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.041 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:38.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:38.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:57:38.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:38.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:57:38.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:38.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896375+0000", 2024-09-18T04:57:38.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:38.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896377+0000", 2024-09-18T04:57:38.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2699999999999999e-06 2024-09-18T04:57:38.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:38.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896383+0000", 2024-09-18T04:57:38.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6740000000000002e-06 2024-09-18T04:57:38.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896388+0000", 2024-09-18T04:57:38.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:57:38.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:38.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938435+0000", 2024-09-18T04:57:38.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042047169000000002 2024-09-18T04:57:38.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:38.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938534+0000", 2024-09-18T04:57:38.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8939999999999998e-05 2024-09-18T04:57:38.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:38.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938635+0000", 2024-09-18T04:57:38.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101208 2024-09-18T04:57:38.044 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.044 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:38.044 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.045 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11412 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:38.045 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:57:38.045 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 62.092250677000003, 2024-09-18T04:57:38.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042828079999999998, 2024-09-18T04:57:38.045 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:38.045 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:38.045 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:38.045 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:38.045 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.045 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11412 2024-09-18T04:57:38.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.046 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:38.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:38.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:57:38.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:38.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:57:38.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:38.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896391+0000", 2024-09-18T04:57:38.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:38.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896393+0000", 2024-09-18T04:57:38.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0810000000000001e-06 2024-09-18T04:57:38.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:38.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896399+0000", 2024-09-18T04:57:38.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7230000000000001e-06 2024-09-18T04:57:38.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896403+0000", 2024-09-18T04:57:38.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3470000000000001e-06 2024-09-18T04:57:38.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:38.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939043+0000", 2024-09-18T04:57:38.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042639829999999997 2024-09-18T04:57:38.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:38.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939124+0000", 2024-09-18T04:57:38.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0809000000000005e-05 2024-09-18T04:57:38.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:38.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939220+0000", 2024-09-18T04:57:38.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5659e-05 2024-09-18T04:57:38.049 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.049 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:38.049 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.049 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11413 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2068480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:38.049 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:57:38.049 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 62.092234982000001, 2024-09-18T04:57:38.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043042108000000003, 2024-09-18T04:57:38.050 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:38.050 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:38.050 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:38.050 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:38.050 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.050 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11413 2024-09-18T04:57:38.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.050 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:38.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:38.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:57:38.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:38.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:57:38.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:38.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:57:38.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:38.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896409+0000", 2024-09-18T04:57:38.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.165e-06 2024-09-18T04:57:38.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:38.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896415+0000", 2024-09-18T04:57:38.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6149999999999996e-06 2024-09-18T04:57:38.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896419+0000", 2024-09-18T04:57:38.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2649999999999998e-06 2024-09-18T04:57:38.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939262+0000", 2024-09-18T04:57:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042843444000000001 2024-09-18T04:57:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939351+0000", 2024-09-18T04:57:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8944999999999996e-05 2024-09-18T04:57:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939449+0000", 2024-09-18T04:57:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8029000000000006e-05 2024-09-18T04:57:38.053 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.054 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:38.054 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.054 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145409 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:38.054 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:57:38.054 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.4608153340000003, 2024-09-18T04:57:38.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040684845999999997, 2024-09-18T04:57:38.054 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:38.054 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:38.054 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:38.054 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:38.054 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.054 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145409 2024-09-18T04:57:38.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.055 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:38.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:38.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:57:38.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:38.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:57:38.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:38.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:57:38.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:38.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527829+0000", 2024-09-18T04:57:38.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.441e-06 2024-09-18T04:57:38.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:38.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527835+0000", 2024-09-18T04:57:38.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1650000000000003e-06 2024-09-18T04:57:38.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527840+0000", 2024-09-18T04:57:38.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8250000000000004e-06 2024-09-18T04:57:38.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:38.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568404+0000", 2024-09-18T04:57:38.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040563872000000001 2024-09-18T04:57:38.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:38.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568457+0000", 2024-09-18T04:57:38.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2713999999999997e-05 2024-09-18T04:57:38.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:38.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568512+0000", 2024-09-18T04:57:38.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5148000000000001e-05 2024-09-18T04:57:38.058 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.058 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:38.058 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.058 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145411 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:38.059 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:57:38.059 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.4607974730000004, 2024-09-18T04:57:38.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040792108000000001, 2024-09-18T04:57:38.059 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:38.059 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:38.059 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:38.059 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:38.059 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.059 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145411 2024-09-18T04:57:38.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.059 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:38.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:38.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:57:38.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:38.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:57:38.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:38.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527844+0000", 2024-09-18T04:57:38.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:38.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527847+0000", 2024-09-18T04:57:38.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0260000000000002e-06 2024-09-18T04:57:38.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:38.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527852+0000", 2024-09-18T04:57:38.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9889999999999997e-06 2024-09-18T04:57:38.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527857+0000", 2024-09-18T04:57:38.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6410000000000001e-06 2024-09-18T04:57:38.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:38.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568537+0000", 2024-09-18T04:57:38.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040679743999999997 2024-09-18T04:57:38.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:38.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568583+0000", 2024-09-18T04:57:38.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6286999999999999e-05 2024-09-18T04:57:38.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:38.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568637+0000", 2024-09-18T04:57:38.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.376e-05 2024-09-18T04:57:38.063 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.063 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:38.063 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.063 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145413 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1859584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:38.063 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:57:38.063 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.4607810929999996, 2024-09-18T04:57:38.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040919516000000003, 2024-09-18T04:57:38.063 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:38.063 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:38.063 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:38.064 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:38.064 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.064 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145413 2024-09-18T04:57:38.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.064 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:38.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:38.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:57:38.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:38.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:57:38.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:38.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:57:38.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:38.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527863+0000", 2024-09-18T04:57:38.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2630000000000002e-06 2024-09-18T04:57:38.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:38.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527869+0000", 2024-09-18T04:57:38.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9009999999999999e-06 2024-09-18T04:57:38.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527874+0000", 2024-09-18T04:57:38.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0590000000000002e-06 2024-09-18T04:57:38.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:38.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568663+0000", 2024-09-18T04:57:38.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040788805999999997 2024-09-18T04:57:38.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:38.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568716+0000", 2024-09-18T04:57:38.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3501999999999997e-05 2024-09-18T04:57:38.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:38.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568781+0000", 2024-09-18T04:57:38.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4338000000000006e-05 2024-09-18T04:57:38.067 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.067 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:38.067 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.068 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145414 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1908736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:38.068 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:57:38.068 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.4607641349999998, 2024-09-18T04:57:38.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041030234999999998, 2024-09-18T04:57:38.068 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:38.068 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:38.068 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:38.068 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:38.068 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.068 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145414 2024-09-18T04:57:38.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.068 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:38.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:38.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:57:38.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:38.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:57:38.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:38.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:57:38.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:38.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527880+0000", 2024-09-18T04:57:38.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4770000000000002e-06 2024-09-18T04:57:38.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:38.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527886+0000", 2024-09-18T04:57:38.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5759999999999996e-06 2024-09-18T04:57:38.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527890+0000", 2024-09-18T04:57:38.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8180000000000003e-06 2024-09-18T04:57:38.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:38.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568807+0000", 2024-09-18T04:57:38.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040916984000000003 2024-09-18T04:57:38.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:38.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568854+0000", 2024-09-18T04:57:38.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6740999999999999e-05 2024-09-18T04:57:38.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:38.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568908+0000", 2024-09-18T04:57:38.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4228000000000003e-05 2024-09-18T04:57:38.072 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.072 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:38.072 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.073 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145416 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1957888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:38.073 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:57:38.073 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.4607476430000004, 2024-09-18T04:57:38.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041147364999999998, 2024-09-18T04:57:38.073 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:38.073 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:38.073 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:38.073 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:38.073 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.073 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145416 2024-09-18T04:57:38.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.073 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:38.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:38.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:57:38.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:38.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:57:38.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:38.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527894+0000", 2024-09-18T04:57:38.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:38.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527896+0000", 2024-09-18T04:57:38.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1789999999999998e-06 2024-09-18T04:57:38.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:38.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527908+0000", 2024-09-18T04:57:38.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1780000000000001e-05 2024-09-18T04:57:38.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527914+0000", 2024-09-18T04:57:38.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2029999999999997e-06 2024-09-18T04:57:38.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:38.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568935+0000", 2024-09-18T04:57:38.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041020273000000003 2024-09-18T04:57:38.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:38.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568988+0000", 2024-09-18T04:57:38.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2911000000000002e-05 2024-09-18T04:57:38.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:38.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569042+0000", 2024-09-18T04:57:38.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4382000000000002e-05 2024-09-18T04:57:38.077 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.077 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:38.077 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.077 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145418 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 2007040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:38.077 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:57:38.077 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.4607232459999997, 2024-09-18T04:57:38.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041247862000000003, 2024-09-18T04:57:38.078 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:38.078 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:38.078 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:38.078 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:38.078 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.078 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145418 2024-09-18T04:57:38.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.078 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:38.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:38.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:57:38.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:38.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:57:38.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:38.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:57:38.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:38.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527921+0000", 2024-09-18T04:57:38.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.667e-06 2024-09-18T04:57:38.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:38.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527929+0000", 2024-09-18T04:57:38.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.542e-06 2024-09-18T04:57:38.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527934+0000", 2024-09-18T04:57:38.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1440000000000002e-06 2024-09-18T04:57:38.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:38.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569068+0000", 2024-09-18T04:57:38.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041134190000000001 2024-09-18T04:57:38.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:38.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569114+0000", 2024-09-18T04:57:38.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6261999999999999e-05 2024-09-18T04:57:38.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:38.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569167+0000", 2024-09-18T04:57:38.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2506999999999997e-05 2024-09-18T04:57:38.083 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.083 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:38.083 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.083 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145420 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 2056192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:38.083 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:57:38.083 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 4.4607041719999998, 2024-09-18T04:57:38.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041351472, 2024-09-18T04:57:38.083 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:38.083 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:38.083 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:38.083 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:38.083 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.084 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145420 2024-09-18T04:57:38.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.084 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:38.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:38.084 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:57:38.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:38.084 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:57:38.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:38.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:38.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:57:38.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:38.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527940+0000", 2024-09-18T04:57:38.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6570000000000001e-06 2024-09-18T04:57:38.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:38.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527948+0000", 2024-09-18T04:57:38.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8350000000000004e-06 2024-09-18T04:57:38.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527953+0000", 2024-09-18T04:57:38.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0690000000000001e-06 2024-09-18T04:57:38.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:38.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569182+0000", 2024-09-18T04:57:38.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041228734000000003 2024-09-18T04:57:38.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:38.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569234+0000", 2024-09-18T04:57:38.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1663999999999999e-05 2024-09-18T04:57:38.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:38.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:38.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:38.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569290+0000", 2024-09-18T04:57:38.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5937000000000003e-05 2024-09-18T04:57:38.087 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.087 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:38.087 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.087 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:38.087 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:38.087 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:38.089 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:57:38.089 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:38.086+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:38.190 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:38.187+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:38.203 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:38.203 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:38.203 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:57:38.203 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:38.212 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:57:38.291 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:38.289+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:38.354 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:38.355 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:38.355 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:57:38.355 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:57:38.355 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:38.363 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:57:38.406 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:38.404+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:38.465 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-18T04:57:38.468 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:38.468 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:38.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:38.468 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:38.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.469 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72135 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 394240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:38.469 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:57:38.469 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.511282739999999, 2024-09-18T04:57:38.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048559739999999997, 2024-09-18T04:57:38.469 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.469 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.469 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.469 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.469 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.469 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72135 2024-09-18T04:57:38.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.469 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:57:38.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:57:38.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:57:38.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954005+0000", 2024-09-18T04:57:38.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5629999999999999e-06 2024-09-18T04:57:38.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954008+0000", 2024-09-18T04:57:38.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-18T04:57:38.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954012+0000", 2024-09-18T04:57:38.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4790000000000001e-06 2024-09-18T04:57:38.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002418+0000", 2024-09-18T04:57:38.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048406049 2024-09-18T04:57:38.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002460+0000", 2024-09-18T04:57:38.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2188999999999999e-05 2024-09-18T04:57:38.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002564+0000", 2024-09-18T04:57:38.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010377 2024-09-18T04:57:38.473 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.473 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.473 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.473 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72136 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 409600~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:38.473 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:57:38.473 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.511177793000002, 2024-09-18T04:57:38.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048670191000000002, 2024-09-18T04:57:38.473 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.473 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.474 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.474 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.474 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72136 2024-09-18T04:57:38.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.474 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:57:38.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:57:38.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954019+0000", 2024-09-18T04:57:38.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999099 2024-09-18T04:57:38.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954112+0000", 2024-09-18T04:57:38.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3273e-05 2024-09-18T04:57:38.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954121+0000", 2024-09-18T04:57:38.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0289999999999994e-06 2024-09-18T04:57:38.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954128+0000", 2024-09-18T04:57:38.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6189999999999996e-06 2024-09-18T04:57:38.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002604+0000", 2024-09-18T04:57:38.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048476048000000001 2024-09-18T04:57:38.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002682+0000", 2024-09-18T04:57:38.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7955999999999997e-05 2024-09-18T04:57:38.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002779+0000", 2024-09-18T04:57:38.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7372000000000002e-05 2024-09-18T04:57:38.477 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.478 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.478 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.478 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72138 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 440320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:38.478 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:57:38.478 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.510743941000001, 2024-09-18T04:57:38.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048549430999999997, 2024-09-18T04:57:38.478 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.478 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.478 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.478 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.478 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.478 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72138 2024-09-18T04:57:38.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.479 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:57:38.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:57:38.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954542+0000", 2024-09-18T04:57:38.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954545+0000", 2024-09-18T04:57:38.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4360000000000001e-06 2024-09-18T04:57:38.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954551+0000", 2024-09-18T04:57:38.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4949999999999996e-06 2024-09-18T04:57:38.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954557+0000", 2024-09-18T04:57:38.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5099999999999998e-06 2024-09-18T04:57:38.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002993+0000", 2024-09-18T04:57:38.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048436358999999998 2024-09-18T04:57:38.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003039+0000", 2024-09-18T04:57:38.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5884000000000003e-05 2024-09-18T04:57:38.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003092+0000", 2024-09-18T04:57:38.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3208000000000003e-05 2024-09-18T04:57:38.482 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.482 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.482 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.483 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72139 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 455680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:38.483 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:57:38.483 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.510725133000001, 2024-09-18T04:57:38.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048718773, 2024-09-18T04:57:38.483 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.483 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.483 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.483 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.483 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.483 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72139 2024-09-18T04:57:38.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.483 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:57:38.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:57:38.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954561+0000", 2024-09-18T04:57:38.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954564+0000", 2024-09-18T04:57:38.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4099999999999998e-06 2024-09-18T04:57:38.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954570+0000", 2024-09-18T04:57:38.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.178e-06 2024-09-18T04:57:38.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954575+0000", 2024-09-18T04:57:38.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0560000000000004e-06 2024-09-18T04:57:38.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003114+0000", 2024-09-18T04:57:38.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048539212999999998 2024-09-18T04:57:38.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003176+0000", 2024-09-18T04:57:38.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2154000000000002e-05 2024-09-18T04:57:38.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003280+0000", 2024-09-18T04:57:38.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010421600000000001 2024-09-18T04:57:38.488 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.488 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.488 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.488 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72140 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 471040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:38.488 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:57:38.488 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.510469192000002, 2024-09-18T04:57:38.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048670819999999997, 2024-09-18T04:57:38.489 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.489 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.489 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.489 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.489 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.489 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72140 2024-09-18T04:57:38.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.489 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:57:38.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:57:38.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954579+0000", 2024-09-18T04:57:38.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997611 2024-09-18T04:57:38.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954821+0000", 2024-09-18T04:57:38.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024241000000000001 2024-09-18T04:57:38.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954831+0000", 2024-09-18T04:57:38.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8770000000000005e-06 2024-09-18T04:57:38.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954838+0000", 2024-09-18T04:57:38.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0690000000000004e-06 2024-09-18T04:57:38.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003317+0000", 2024-09-18T04:57:38.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048478796999999997 2024-09-18T04:57:38.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003392+0000", 2024-09-18T04:57:38.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4411000000000003e-05 2024-09-18T04:57:38.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003488+0000", 2024-09-18T04:57:38.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6927000000000002e-05 2024-09-18T04:57:38.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.493 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.493 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72141 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 486400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:38.493 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:57:38.493 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.510441708999998, 2024-09-18T04:57:38.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048808114, 2024-09-18T04:57:38.493 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.493 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.493 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.493 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.494 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72141 2024-09-18T04:57:38.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.494 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:57:38.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:57:38.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:57:38.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954847+0000", 2024-09-18T04:57:38.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3800000000000001e-06 2024-09-18T04:57:38.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954854+0000", 2024-09-18T04:57:38.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6080000000000001e-06 2024-09-18T04:57:38.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954859+0000", 2024-09-18T04:57:38.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:57:38.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003526+0000", 2024-09-18T04:57:38.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048666433000000002 2024-09-18T04:57:38.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003599+0000", 2024-09-18T04:57:38.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2854000000000004e-05 2024-09-18T04:57:38.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003653+0000", 2024-09-18T04:57:38.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4511e-05 2024-09-18T04:57:38.498 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.498 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.498 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.498 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72142 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 501760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:38.498 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:57:38.498 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.510402907, 2024-09-18T04:57:38.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048885500999999998, 2024-09-18T04:57:38.498 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.498 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.498 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.498 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.499 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.499 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72142 2024-09-18T04:57:38.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.499 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:57:38.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:57:38.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954864+0000", 2024-09-18T04:57:38.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999804 2024-09-18T04:57:38.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954886+0000", 2024-09-18T04:57:38.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2042999999999999e-05 2024-09-18T04:57:38.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954893+0000", 2024-09-18T04:57:38.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.255e-06 2024-09-18T04:57:38.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954899+0000", 2024-09-18T04:57:38.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8780000000000003e-06 2024-09-18T04:57:38.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003678+0000", 2024-09-18T04:57:38.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048778834 2024-09-18T04:57:38.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003717+0000", 2024-09-18T04:57:38.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8860999999999999e-05 2024-09-18T04:57:38.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003769+0000", 2024-09-18T04:57:38.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2389000000000003e-05 2024-09-18T04:57:38.502 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.502 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.502 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.503 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72143 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 517120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:38.503 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:57:38.503 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.510277281, 2024-09-18T04:57:38.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048951738000000002, 2024-09-18T04:57:38.503 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.503 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.503 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.503 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.503 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.503 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72143 2024-09-18T04:57:38.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.503 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:57:38.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:57:38.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954904+0000", 2024-09-18T04:57:38.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998946 2024-09-18T04:57:38.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955013+0000", 2024-09-18T04:57:38.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108977 2024-09-18T04:57:38.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955021+0000", 2024-09-18T04:57:38.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7570000000000004e-06 2024-09-18T04:57:38.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955025+0000", 2024-09-18T04:57:38.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1219999999999997e-06 2024-09-18T04:57:38.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003792+0000", 2024-09-18T04:57:38.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048767379 2024-09-18T04:57:38.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003860+0000", 2024-09-18T04:57:38.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8033000000000004e-05 2024-09-18T04:57:38.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003961+0000", 2024-09-18T04:57:38.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010103300000000001 2024-09-18T04:57:38.507 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.507 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.507 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.507 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72161 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 13312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:38.507 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:57:38.507 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.510013643999997, 2024-09-18T04:57:38.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048891539999999997, 2024-09-18T04:57:38.507 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.507 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.508 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72161 2024-09-18T04:57:38.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.508 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:57:38.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:57:38.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955028+0000", 2024-09-18T04:57:38.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997549 2024-09-18T04:57:38.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955278+0000", 2024-09-18T04:57:38.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024973299999999998 2024-09-18T04:57:38.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955289+0000", 2024-09-18T04:57:38.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0812000000000001e-05 2024-09-18T04:57:38.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955296+0000", 2024-09-18T04:57:38.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2570000000000001e-06 2024-09-18T04:57:38.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004002+0000", 2024-09-18T04:57:38.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048705904000000001 2024-09-18T04:57:38.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004076+0000", 2024-09-18T04:57:38.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4086999999999997e-05 2024-09-18T04:57:38.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004165+0000", 2024-09-18T04:57:38.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8712e-05 2024-09-18T04:57:38.513 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.513 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.513 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.513 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72163 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 44032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:38.513 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:57:38.513 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.509388205999997, 2024-09-18T04:57:38.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048567011, 2024-09-18T04:57:38.513 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.513 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.513 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.513 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.513 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.514 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72163 2024-09-18T04:57:38.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.514 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:57:38.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:57:38.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955898+0000", 2024-09-18T04:57:38.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955901+0000", 2024-09-18T04:57:38.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.706e-06 2024-09-18T04:57:38.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955907+0000", 2024-09-18T04:57:38.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3559999999999998e-06 2024-09-18T04:57:38.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955912+0000", 2024-09-18T04:57:38.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9640000000000002e-06 2024-09-18T04:57:38.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004377+0000", 2024-09-18T04:57:38.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048465293999999999 2024-09-18T04:57:38.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004418+0000", 2024-09-18T04:57:38.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0163000000000002e-05 2024-09-18T04:57:38.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004466+0000", 2024-09-18T04:57:38.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8004000000000003e-05 2024-09-18T04:57:38.517 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.517 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.517 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.517 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72164 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:38.517 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:57:38.517 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.509369933000002, 2024-09-18T04:57:38.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048707253999999998, 2024-09-18T04:57:38.518 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.518 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.518 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.518 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.518 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.518 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72164 2024-09-18T04:57:38.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.518 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:57:38.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:57:38.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:57:38.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955919+0000", 2024-09-18T04:57:38.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2520000000000002e-06 2024-09-18T04:57:38.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955925+0000", 2024-09-18T04:57:38.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3740000000000003e-06 2024-09-18T04:57:38.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955930+0000", 2024-09-18T04:57:38.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:57:38.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004489+0000", 2024-09-18T04:57:38.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048558581000000003 2024-09-18T04:57:38.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004529+0000", 2024-09-18T04:57:38.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9722999999999999e-05 2024-09-18T04:57:38.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004624+0000", 2024-09-18T04:57:38.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5402999999999994e-05 2024-09-18T04:57:38.521 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.521 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.521 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.522 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72165 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 74752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:38.522 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:57:38.522 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.509095074000001, 2024-09-18T04:57:38.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048633223000000003, 2024-09-18T04:57:38.522 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.522 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.522 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.522 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.522 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.522 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72165 2024-09-18T04:57:38.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.522 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:57:38.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:57:38.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955935+0000", 2024-09-18T04:57:38.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999743 2024-09-18T04:57:38.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956199+0000", 2024-09-18T04:57:38.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000263906 2024-09-18T04:57:38.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956207+0000", 2024-09-18T04:57:38.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2519999999999999e-06 2024-09-18T04:57:38.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956214+0000", 2024-09-18T04:57:38.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7610000000000002e-06 2024-09-18T04:57:38.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004661+0000", 2024-09-18T04:57:38.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048447967000000002 2024-09-18T04:57:38.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004735+0000", 2024-09-18T04:57:38.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3789999999999997e-05 2024-09-18T04:57:38.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004825+0000", 2024-09-18T04:57:38.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9724999999999999e-05 2024-09-18T04:57:38.526 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.526 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.526 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.526 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72166 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 90112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:38.526 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:57:38.526 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.508841539999999, 2024-09-18T04:57:38.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048595737999999999, 2024-09-18T04:57:38.526 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.526 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.526 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.527 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.527 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.527 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72166 2024-09-18T04:57:38.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.527 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:57:38.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:57:38.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956221+0000", 2024-09-18T04:57:38.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997759 2024-09-18T04:57:38.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956448+0000", 2024-09-18T04:57:38.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022685500000000001 2024-09-18T04:57:38.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956456+0000", 2024-09-18T04:57:38.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0099999999999995e-06 2024-09-18T04:57:38.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956462+0000", 2024-09-18T04:57:38.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7200000000000003e-06 2024-09-18T04:57:38.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004862+0000", 2024-09-18T04:57:38.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048399795000000002 2024-09-18T04:57:38.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004941+0000", 2024-09-18T04:57:38.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9414999999999998e-05 2024-09-18T04:57:38.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005041+0000", 2024-09-18T04:57:38.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9942e-05 2024-09-18T04:57:38.530 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.530 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.530 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.530 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72167 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:38.531 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:57:38.531 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.508754502000002, 2024-09-18T04:57:38.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048622894999999999, 2024-09-18T04:57:38.531 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.531 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.531 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.531 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.531 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.531 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72167 2024-09-18T04:57:38.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.531 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:57:38.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:57:38.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956468+0000", 2024-09-18T04:57:38.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999352 2024-09-18T04:57:38.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956535+0000", 2024-09-18T04:57:38.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7440999999999993e-05 2024-09-18T04:57:38.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956544+0000", 2024-09-18T04:57:38.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1039999999999996e-06 2024-09-18T04:57:38.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956550+0000", 2024-09-18T04:57:38.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9850000000000003e-06 2024-09-18T04:57:38.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005066+0000", 2024-09-18T04:57:38.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048515614999999998 2024-09-18T04:57:38.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005107+0000", 2024-09-18T04:57:38.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0843999999999997e-05 2024-09-18T04:57:38.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005155+0000", 2024-09-18T04:57:38.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8686000000000001e-05 2024-09-18T04:57:38.535 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.535 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.535 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.535 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72168 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:38.535 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:57:38.535 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.508593439000002, 2024-09-18T04:57:38.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048585179999999999, 2024-09-18T04:57:38.536 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.536 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.536 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72168 2024-09-18T04:57:38.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.536 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:57:38.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:57:38.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956555+0000", 2024-09-18T04:57:38.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998612 2024-09-18T04:57:38.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956696+0000", 2024-09-18T04:57:38.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000140922 2024-09-18T04:57:38.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956707+0000", 2024-09-18T04:57:38.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1555e-05 2024-09-18T04:57:38.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956714+0000", 2024-09-18T04:57:38.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2709999999999998e-06 2024-09-18T04:57:38.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005177+0000", 2024-09-18T04:57:38.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048463421 2024-09-18T04:57:38.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005214+0000", 2024-09-18T04:57:38.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.676e-05 2024-09-18T04:57:38.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005279+0000", 2024-09-18T04:57:38.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4899000000000003e-05 2024-09-18T04:57:38.539 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.539 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.539 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.540 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72172 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 441344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:38.540 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.957556+0000", 2024-09-18T04:57:38.540 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.507731038999999, 2024-09-18T04:57:38.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048545056000000003, 2024-09-18T04:57:38.540 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.540 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.540 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.540 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.540 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.540 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72172 2024-09-18T04:57:38.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.540 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957556+0000", 2024-09-18T04:57:38.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957556+0000", 2024-09-18T04:57:38.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957555+0000", 2024-09-18T04:57:38.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957558+0000", 2024-09-18T04:57:38.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2900000000000001e-06 2024-09-18T04:57:38.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957564+0000", 2024-09-18T04:57:38.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0700000000000003e-06 2024-09-18T04:57:38.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957569+0000", 2024-09-18T04:57:38.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7870000000000001e-06 2024-09-18T04:57:38.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005922+0000", 2024-09-18T04:57:38.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048353259000000003 2024-09-18T04:57:38.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005999+0000", 2024-09-18T04:57:38.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6889000000000002e-05 2024-09-18T04:57:38.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.006101+0000", 2024-09-18T04:57:38.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102136 2024-09-18T04:57:38.544 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.544 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.544 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.544 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72199 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:38.544 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:57:38.544 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.506061547000002, 2024-09-18T04:57:38.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048626057, 2024-09-18T04:57:38.544 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.544 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.544 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.544 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.545 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.545 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72199 2024-09-18T04:57:38.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.545 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:57:38.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:57:38.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959195+0000", 2024-09-18T04:57:38.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999969 2024-09-18T04:57:38.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959228+0000", 2024-09-18T04:57:38.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3609000000000003e-05 2024-09-18T04:57:38.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959236+0000", 2024-09-18T04:57:38.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.157e-06 2024-09-18T04:57:38.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959243+0000", 2024-09-18T04:57:38.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5910000000000001e-06 2024-09-18T04:57:38.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007684+0000", 2024-09-18T04:57:38.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048440745 2024-09-18T04:57:38.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007762+0000", 2024-09-18T04:57:38.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8360000000000002e-05 2024-09-18T04:57:38.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007851+0000", 2024-09-18T04:57:38.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9388000000000005e-05 2024-09-18T04:57:38.548 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.548 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.548 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.549 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72240 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 428032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:38.549 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:57:38.549 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 41.501968237, 2024-09-18T04:57:38.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048643767999999997, 2024-09-18T04:57:38.549 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.549 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.549 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.549 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.549 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.549 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72240 2024-09-18T04:57:38.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.549 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:57:38.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:57:38.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963240+0000", 2024-09-18T04:57:38.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999218 2024-09-18T04:57:38.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963323+0000", 2024-09-18T04:57:38.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2375999999999999e-05 2024-09-18T04:57:38.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963332+0000", 2024-09-18T04:57:38.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1700000000000003e-06 2024-09-18T04:57:38.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963338+0000", 2024-09-18T04:57:38.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2149999999999998e-06 2024-09-18T04:57:38.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011785+0000", 2024-09-18T04:57:38.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048446493 2024-09-18T04:57:38.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011861+0000", 2024-09-18T04:57:38.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6472999999999998e-05 2024-09-18T04:57:38.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011962+0000", 2024-09-18T04:57:38.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101167 2024-09-18T04:57:38.553 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.553 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.553 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.553 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:117375 3.1 3:80b18d38:::rbd_header.11acae5d5365:head [watch reconnect cookie 140406307615440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T04:57:38.553 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:57:38.553 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.080893700000001, 2024-09-18T04:57:38.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.018908156, 2024-09-18T04:57:38.553 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.553 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.553 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.554 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.554 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.554 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117375 2024-09-18T04:57:38.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.554 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:57:38.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:57:38.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384391+0000", 2024-09-18T04:57:38.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:57:38.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384401+0000", 2024-09-18T04:57:38.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4299999999999995e-06 2024-09-18T04:57:38.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384418+0000", 2024-09-18T04:57:38.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6745e-05 2024-09-18T04:57:38.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384429+0000", 2024-09-18T04:57:38.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1122e-05 2024-09-18T04:57:38.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384496+0000", 2024-09-18T04:57:38.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6841000000000006e-05 2024-09-18T04:57:38.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-18T04:57:38.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384505+0000", 2024-09-18T04:57:38.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4080000000000004e-06 2024-09-18T04:57:38.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.388971+0000", 2024-09-18T04:57:38.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0044660000000000004 2024-09-18T04:57:38.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:57:38.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.388981+0000", 2024-09-18T04:57:38.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0118e-05 2024-09-18T04:57:38.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.399433+0000", 2024-09-18T04:57:38.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.010452055 2024-09-18T04:57:38.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.399594+0000", 2024-09-18T04:57:38.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000160669 2024-09-18T04:57:38.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.403301+0000", 2024-09-18T04:57:38.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037074370000000001 2024-09-18T04:57:38.559 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.559 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.559 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.559 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:117376 3.1 3:80b18d38:::rbd_header.11acae5d5365:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T04:57:38.559 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:57:38.559 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 15.07884205, 2024-09-18T04:57:38.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0196134539999999, 2024-09-18T04:57:38.560 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.560 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:38.560 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.560 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.560 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.560 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117376 2024-09-18T04:57:38.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.560 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:57:38.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:57:38.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386443+0000", 2024-09-18T04:57:38.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:57:38.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386457+0000", 2024-09-18T04:57:38.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4207e-05 2024-09-18T04:57:38.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386478+0000", 2024-09-18T04:57:38.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0349000000000001e-05 2024-09-18T04:57:38.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386503+0000", 2024-09-18T04:57:38.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5582e-05 2024-09-18T04:57:38.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386558+0000", 2024-09-18T04:57:38.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4892999999999998e-05 2024-09-18T04:57:38.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-18T04:57:38.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386562+0000", 2024-09-18T04:57:38.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.313e-06 2024-09-18T04:57:38.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.389010+0000", 2024-09-18T04:57:38.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024480890000000001 2024-09-18T04:57:38.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:57:38.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.389018+0000", 2024-09-18T04:57:38.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8469999999999996e-06 2024-09-18T04:57:38.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400073+0000", 2024-09-18T04:57:38.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.011054286 2024-09-18T04:57:38.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400124+0000", 2024-09-18T04:57:38.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1332999999999998e-05 2024-09-18T04:57:38.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:57:38.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400477+0000", 2024-09-18T04:57:38.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035329300000000002 2024-09-18T04:57:38.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:38.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.401353+0000", 2024-09-18T04:57:38.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00087584200000000005 2024-09-18T04:57:38.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:57:38.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406006+0000", 2024-09-18T04:57:38.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0046534250000000001 2024-09-18T04:57:38.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:57:38.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406037+0000", 2024-09-18T04:57:38.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0559000000000003e-05 2024-09-18T04:57:38.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406058+0000", 2024-09-18T04:57:38.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1260999999999999e-05 2024-09-18T04:57:38.566 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.566 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.566 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.566 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.566 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.567 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:38.569 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:57:38.569 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:38.565+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:38.669 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:38.667+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:38.711 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:57:38.711 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:38.719 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:57:38.770 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:38.767+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:38.777 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:57:38.820 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:38.821 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:38.821 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:57:38.821 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:57:38.821 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:38.829 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:57:38.873 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:38.870+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:38.906 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:57:38.906 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:57:38.922 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:57:38.923 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:57:38.933 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:38.933 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:38.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:38.933 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:38.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.933 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60344 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1456128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:38.933 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:38.933 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.817866328000001, 2024-09-18T04:57:38.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044026324999999998, 2024-09-18T04:57:38.933 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.933 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60344 2024-09-18T04:57:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112224+0000", 2024-09-18T04:57:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1320000000000001e-06 2024-09-18T04:57:38.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112230+0000", 2024-09-18T04:57:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9619999999999998e-06 2024-09-18T04:57:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112234+0000", 2024-09-18T04:57:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7890000000000002e-06 2024-09-18T04:57:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156105+0000", 2024-09-18T04:57:38.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043870531999999997 2024-09-18T04:57:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156170+0000", 2024-09-18T04:57:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5036000000000005e-05 2024-09-18T04:57:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156248+0000", 2024-09-18T04:57:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8295999999999997e-05 2024-09-18T04:57:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.937 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60345 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1471488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:57:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.817849563000003, 2024-09-18T04:57:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044180090999999998, 2024-09-18T04:57:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60345 2024-09-18T04:57:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.938 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:57:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:57:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112238+0000", 2024-09-18T04:57:38.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112240+0000", 2024-09-18T04:57:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9910000000000001e-06 2024-09-18T04:57:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112246+0000", 2024-09-18T04:57:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0859999999999998e-06 2024-09-18T04:57:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112251+0000", 2024-09-18T04:57:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5020000000000003e-06 2024-09-18T04:57:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156274+0000", 2024-09-18T04:57:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044023063000000001 2024-09-18T04:57:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156337+0000", 2024-09-18T04:57:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3077000000000006e-05 2024-09-18T04:57:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156419+0000", 2024-09-18T04:57:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1699000000000005e-05 2024-09-18T04:57:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60346 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1486848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:57:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.817831781999999, 2024-09-18T04:57:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044336331999999999, 2024-09-18T04:57:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.942 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60346 2024-09-18T04:57:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:57:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:57:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112256+0000", 2024-09-18T04:57:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112258+0000", 2024-09-18T04:57:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1720000000000001e-06 2024-09-18T04:57:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112265+0000", 2024-09-18T04:57:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4280000000000001e-06 2024-09-18T04:57:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112269+0000", 2024-09-18T04:57:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.741e-06 2024-09-18T04:57:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156445+0000", 2024-09-18T04:57:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044175341 2024-09-18T04:57:38.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156512+0000", 2024-09-18T04:57:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6723000000000005e-05 2024-09-18T04:57:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156593+0000", 2024-09-18T04:57:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1284000000000003e-05 2024-09-18T04:57:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60347 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1502208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:57:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.817803370999997, 2024-09-18T04:57:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044482560999999997, 2024-09-18T04:57:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60347 2024-09-18T04:57:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.947 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:57:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:57:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112284+0000", 2024-09-18T04:57:38.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112286+0000", 2024-09-18T04:57:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9709999999999998e-06 2024-09-18T04:57:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112290+0000", 2024-09-18T04:57:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9199999999999997e-06 2024-09-18T04:57:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112293+0000", 2024-09-18T04:57:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870000000000001e-06 2024-09-18T04:57:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156617+0000", 2024-09-18T04:57:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044323960000000003 2024-09-18T04:57:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156687+0000", 2024-09-18T04:57:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0141000000000005e-05 2024-09-18T04:57:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156767+0000", 2024-09-18T04:57:38.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0140000000000002e-05 2024-09-18T04:57:38.952 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.952 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.952 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.953 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60348 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1517568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:38.953 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:38.953 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.817792367000003, 2024-09-18T04:57:38.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044645562, 2024-09-18T04:57:38.953 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.953 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.953 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.953 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.953 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.953 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60348 2024-09-18T04:57:38.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.954 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:38.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:38.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:38.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112297+0000", 2024-09-18T04:57:38.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.384e-06 2024-09-18T04:57:38.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112301+0000", 2024-09-18T04:57:38.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0570000000000003e-06 2024-09-18T04:57:38.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112304+0000", 2024-09-18T04:57:38.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6000000000000001e-06 2024-09-18T04:57:38.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156793+0000", 2024-09-18T04:57:38.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044489494999999997 2024-09-18T04:57:38.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156859+0000", 2024-09-18T04:57:38.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5483999999999998e-05 2024-09-18T04:57:38.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156941+0000", 2024-09-18T04:57:38.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2856999999999996e-05 2024-09-18T04:57:38.957 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.957 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.957 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.957 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60349 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1532928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:38.957 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:38.957 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.817782391000001, 2024-09-18T04:57:38.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044832167999999999, 2024-09-18T04:57:38.957 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.958 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.958 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.958 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.958 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.958 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60349 2024-09-18T04:57:38.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.958 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:38.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:38.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:38.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112307+0000", 2024-09-18T04:57:38.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.308e-06 2024-09-18T04:57:38.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112310+0000", 2024-09-18T04:57:38.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-18T04:57:38.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112312+0000", 2024-09-18T04:57:38.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4889999999999998e-06 2024-09-18T04:57:38.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156977+0000", 2024-09-18T04:57:38.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044664598999999999 2024-09-18T04:57:38.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157053+0000", 2024-09-18T04:57:38.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6014999999999996e-05 2024-09-18T04:57:38.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157138+0000", 2024-09-18T04:57:38.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5162999999999998e-05 2024-09-18T04:57:38.961 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.961 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.961 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.962 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60350 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1548288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:38.962 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:38.962 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.817773477000003, 2024-09-18T04:57:38.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045011858000000002, 2024-09-18T04:57:38.962 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.962 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.962 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.962 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.962 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.962 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60350 2024-09-18T04:57:38.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.962 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:38.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:38.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:38.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112316+0000", 2024-09-18T04:57:38.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7579999999999999e-06 2024-09-18T04:57:38.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112321+0000", 2024-09-18T04:57:38.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1649999999999997e-06 2024-09-18T04:57:38.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112325+0000", 2024-09-18T04:57:38.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.726e-06 2024-09-18T04:57:38.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157172+0000", 2024-09-18T04:57:38.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.04484718 2024-09-18T04:57:38.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157242+0000", 2024-09-18T04:57:38.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9675e-05 2024-09-18T04:57:38.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157327+0000", 2024-09-18T04:57:38.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4614e-05 2024-09-18T04:57:38.966 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.966 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.966 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.966 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60351 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1563648~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:38.966 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:38.966 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.817760061000001, 2024-09-18T04:57:38.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045132113000000001, 2024-09-18T04:57:38.966 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.966 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.966 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.967 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.967 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.967 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60351 2024-09-18T04:57:38.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.967 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:38.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:38.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:38.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112330+0000", 2024-09-18T04:57:38.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1660000000000001e-06 2024-09-18T04:57:38.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112336+0000", 2024-09-18T04:57:38.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5940000000000001e-06 2024-09-18T04:57:38.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112340+0000", 2024-09-18T04:57:38.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5589999999999999e-06 2024-09-18T04:57:38.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157365+0000", 2024-09-18T04:57:38.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045024675 2024-09-18T04:57:38.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157409+0000", 2024-09-18T04:57:38.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4356e-05 2024-09-18T04:57:38.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157460+0000", 2024-09-18T04:57:38.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1072000000000002e-05 2024-09-18T04:57:38.970 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.970 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.970 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.970 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60501 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1494016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:38.971 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:38.971 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.811699365000003, 2024-09-18T04:57:38.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044007728000000003, 2024-09-18T04:57:38.971 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.971 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.971 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.971 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.971 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.971 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60501 2024-09-18T04:57:38.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.972 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:38.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:38.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:38.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118391+0000", 2024-09-18T04:57:38.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:57:38.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118396+0000", 2024-09-18T04:57:38.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2680000000000002e-06 2024-09-18T04:57:38.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118400+0000", 2024-09-18T04:57:38.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2080000000000002e-06 2024-09-18T04:57:38.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162317+0000", 2024-09-18T04:57:38.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043916753000000003 2024-09-18T04:57:38.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162351+0000", 2024-09-18T04:57:38.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4372000000000002e-05 2024-09-18T04:57:38.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162397+0000", 2024-09-18T04:57:38.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5207999999999998e-05 2024-09-18T04:57:38.975 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.975 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.975 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.975 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60502 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1509376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:38.975 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:38.975 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.811683887999997, 2024-09-18T04:57:38.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044092438999999997, 2024-09-18T04:57:38.975 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.976 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.976 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.976 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.976 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.976 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60502 2024-09-18T04:57:38.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.976 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:38.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:38.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:38.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118406+0000", 2024-09-18T04:57:38.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:57:38.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118412+0000", 2024-09-18T04:57:38.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.524e-06 2024-09-18T04:57:38.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118416+0000", 2024-09-18T04:57:38.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:57:38.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162418+0000", 2024-09-18T04:57:38.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001508000000002 2024-09-18T04:57:38.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162451+0000", 2024-09-18T04:57:38.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3662999999999999e-05 2024-09-18T04:57:38.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162497+0000", 2024-09-18T04:57:38.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5550000000000003e-05 2024-09-18T04:57:38.979 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.979 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.979 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.980 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60503 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1524736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:38.980 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:57:38.980 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.811667730000003, 2024-09-18T04:57:38.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044178780000000001, 2024-09-18T04:57:38.980 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.980 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.980 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.980 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.980 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.980 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60503 2024-09-18T04:57:38.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.980 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:57:38.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:57:38.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118420+0000", 2024-09-18T04:57:38.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118422+0000", 2024-09-18T04:57:38.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.08e-06 2024-09-18T04:57:38.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118428+0000", 2024-09-18T04:57:38.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.964e-06 2024-09-18T04:57:38.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118433+0000", 2024-09-18T04:57:38.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.425e-06 2024-09-18T04:57:38.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162519+0000", 2024-09-18T04:57:38.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044086082999999998 2024-09-18T04:57:38.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162552+0000", 2024-09-18T04:57:38.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3719e-05 2024-09-18T04:57:38.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162599+0000", 2024-09-18T04:57:38.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6873999999999999e-05 2024-09-18T04:57:38.984 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.984 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.985 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.985 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60504 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1540096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:38.985 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:57:38.985 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.811651625000003, 2024-09-18T04:57:38.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044261921000000003, 2024-09-18T04:57:38.985 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.985 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.985 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.985 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.985 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.985 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60504 2024-09-18T04:57:38.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.986 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:57:38.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:57:38.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118436+0000", 2024-09-18T04:57:38.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118438+0000", 2024-09-18T04:57:38.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0250000000000001e-06 2024-09-18T04:57:38.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118444+0000", 2024-09-18T04:57:38.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4360000000000001e-06 2024-09-18T04:57:38.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118448+0000", 2024-09-18T04:57:38.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.357e-06 2024-09-18T04:57:38.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162620+0000", 2024-09-18T04:57:38.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044171821 2024-09-18T04:57:38.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162653+0000", 2024-09-18T04:57:38.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3479999999999998e-05 2024-09-18T04:57:38.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162699+0000", 2024-09-18T04:57:38.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5113e-05 2024-09-18T04:57:38.991 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.991 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.991 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.991 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60584 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1196032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:38.991 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:57:38.991 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.809858607999999, 2024-09-18T04:57:38.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044048727000000003, 2024-09-18T04:57:38.991 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.992 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.992 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.992 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.992 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.992 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60584 2024-09-18T04:57:38.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.992 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:57:38.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:38.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:57:38.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:38.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120229+0000", 2024-09-18T04:57:38.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:38.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:38.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120231+0000", 2024-09-18T04:57:38.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:57:38.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:38.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120237+0000", 2024-09-18T04:57:38.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8379999999999998e-06 2024-09-18T04:57:38.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:38.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120242+0000", 2024-09-18T04:57:38.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3889999999999998e-06 2024-09-18T04:57:38.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:38.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164187+0000", 2024-09-18T04:57:38.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043945045000000002 2024-09-18T04:57:38.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:38.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164227+0000", 2024-09-18T04:57:38.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0627999999999998e-05 2024-09-18T04:57:38.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:38.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164278+0000", 2024-09-18T04:57:38.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1146000000000001e-05 2024-09-18T04:57:38.997 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.998 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:38.998 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:38.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.998 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60585 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1211392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:38.998 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:57:38.998 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.809842500000002, 2024-09-18T04:57:38.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044389814, 2024-09-18T04:57:38.998 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:38.999 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:38.999 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:38.999 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:38.999 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:38.999 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60585 2024-09-18T04:57:38.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.999 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:38.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:38.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:38.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:57:38.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:38.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:38.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:39.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:57:39.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:39.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:39.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120245+0000", 2024-09-18T04:57:39.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:39.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:39.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120247+0000", 2024-09-18T04:57:39.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0150000000000002e-06 2024-09-18T04:57:39.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:39.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120255+0000", 2024-09-18T04:57:39.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.858e-06 2024-09-18T04:57:39.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:39.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120260+0000", 2024-09-18T04:57:39.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6530000000000002e-06 2024-09-18T04:57:39.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:39.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164489+0000", 2024-09-18T04:57:39.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044229483999999999 2024-09-18T04:57:39.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:39.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164555+0000", 2024-09-18T04:57:39.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.533e-05 2024-09-18T04:57:39.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:39.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164636+0000", 2024-09-18T04:57:39.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0819e-05 2024-09-18T04:57:39.002 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:39.002 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:39.003 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:39.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.003 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60586 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1226752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:39.003 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:39.003 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.809824048999999, 2024-09-18T04:57:39.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044578214999999997, 2024-09-18T04:57:39.003 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:39.003 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:39.003 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:39.003 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:39.003 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:39.003 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60586 2024-09-18T04:57:39.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.004 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:39.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:39.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:39.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:39.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:39.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:39.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:39.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:39.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:39.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:39.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:39.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120266+0000", 2024-09-18T04:57:39.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0660000000000002e-06 2024-09-18T04:57:39.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:39.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120272+0000", 2024-09-18T04:57:39.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-18T04:57:39.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:39.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120276+0000", 2024-09-18T04:57:39.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.476e-06 2024-09-18T04:57:39.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:39.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164673+0000", 2024-09-18T04:57:39.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044396672999999998 2024-09-18T04:57:39.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:39.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164749+0000", 2024-09-18T04:57:39.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5811999999999999e-05 2024-09-18T04:57:39.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:39.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164842+0000", 2024-09-18T04:57:39.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3560000000000005e-05 2024-09-18T04:57:39.007 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:39.007 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:39.007 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:39.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.007 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60587 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1242112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:39.007 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:57:39.007 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.809774949999998, 2024-09-18T04:57:39.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044634794999999998, 2024-09-18T04:57:39.008 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:39.008 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:39.008 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:39.008 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:39.008 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:39.008 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60587 2024-09-18T04:57:39.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.008 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:39.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:39.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:57:39.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:39.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:39.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:57:39.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:39.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:39.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120280+0000", 2024-09-18T04:57:39.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999671 2024-09-18T04:57:39.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:39.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120316+0000", 2024-09-18T04:57:39.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5728000000000001e-05 2024-09-18T04:57:39.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:39.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120329+0000", 2024-09-18T04:57:39.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2914e-05 2024-09-18T04:57:39.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:39.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120335+0000", 2024-09-18T04:57:39.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8289999999999996e-06 2024-09-18T04:57:39.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:39.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164858+0000", 2024-09-18T04:57:39.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044523417000000003 2024-09-18T04:57:39.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:39.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164896+0000", 2024-09-18T04:57:39.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7969999999999997e-05 2024-09-18T04:57:39.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:39.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164948+0000", 2024-09-18T04:57:39.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1996000000000002e-05 2024-09-18T04:57:39.012 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:39.012 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:39.012 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:39.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.012 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60595 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1364992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:39.012 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:57:39.012 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.807931734, 2024-09-18T04:57:39.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001360000000003, 2024-09-18T04:57:39.012 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:39.013 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:39.013 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:39.013 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:39.013 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:39.013 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60595 2024-09-18T04:57:39.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.013 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:39.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:39.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:57:39.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:39.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:39.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:57:39.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:39.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:39.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122156+0000", 2024-09-18T04:57:39.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:39.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:39.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122158+0000", 2024-09-18T04:57:39.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0040000000000002e-06 2024-09-18T04:57:39.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:39.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122164+0000", 2024-09-18T04:57:39.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8719999999999999e-06 2024-09-18T04:57:39.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:39.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122169+0000", 2024-09-18T04:57:39.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5090000000000004e-06 2024-09-18T04:57:39.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:39.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166040+0000", 2024-09-18T04:57:39.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043871226999999999 2024-09-18T04:57:39.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:39.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166096+0000", 2024-09-18T04:57:39.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6495e-05 2024-09-18T04:57:39.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:39.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166158+0000", 2024-09-18T04:57:39.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1626999999999998e-05 2024-09-18T04:57:39.017 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:39.017 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:39.018 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:39.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.018 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60596 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1380352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:39.018 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:39.018 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.807915413000003, 2024-09-18T04:57:39.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044120800000000002, 2024-09-18T04:57:39.018 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:39.018 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:39.018 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:39.018 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:39.019 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:39.019 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60596 2024-09-18T04:57:39.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.019 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:39.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:39.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:39.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:39.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:39.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:39.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:39.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:39.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:39.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:39.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:39.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122175+0000", 2024-09-18T04:57:39.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:57:39.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:39.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122182+0000", 2024-09-18T04:57:39.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3509999999999996e-06 2024-09-18T04:57:39.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:39.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122187+0000", 2024-09-18T04:57:39.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7940000000000002e-06 2024-09-18T04:57:39.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:39.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166184+0000", 2024-09-18T04:57:39.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043997236000000002 2024-09-18T04:57:39.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:39.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166227+0000", 2024-09-18T04:57:39.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2561000000000001e-05 2024-09-18T04:57:39.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:39.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166294+0000", 2024-09-18T04:57:39.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7068000000000002e-05 2024-09-18T04:57:39.023 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:39.023 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:39.023 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:39.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.023 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60597 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1395712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:39.023 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:39.023 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.807897384999997, 2024-09-18T04:57:39.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044242166999999999, 2024-09-18T04:57:39.024 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:39.024 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:39.024 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:39.024 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:39.024 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:39.024 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60597 2024-09-18T04:57:39.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.024 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:39.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:39.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:39.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:39.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:39.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:39.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:39.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:39.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:39.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:39.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:39.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122193+0000", 2024-09-18T04:57:39.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:57:39.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:39.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122199+0000", 2024-09-18T04:57:39.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6449999999999999e-06 2024-09-18T04:57:39.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:39.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122204+0000", 2024-09-18T04:57:39.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4819999999999996e-06 2024-09-18T04:57:39.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:39.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166329+0000", 2024-09-18T04:57:39.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044125059000000001 2024-09-18T04:57:39.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:39.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166370+0000", 2024-09-18T04:57:39.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1012e-05 2024-09-18T04:57:39.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:39.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166433+0000", 2024-09-18T04:57:39.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3178999999999999e-05 2024-09-18T04:57:39.027 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:39.028 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:39.028 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:39.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.028 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60608 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1564672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:39.028 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:57:39.028 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.805899523000001, 2024-09-18T04:57:39.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044082723999999997, 2024-09-18T04:57:39.028 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:39.028 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:39.028 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:39.028 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:39.028 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:39.029 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60608 2024-09-18T04:57:39.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.029 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:39.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:39.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:57:39.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:39.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:39.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:57:39.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:39.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:39.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124188+0000", 2024-09-18T04:57:39.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:39.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:39.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124190+0000", 2024-09-18T04:57:39.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0329999999999998e-06 2024-09-18T04:57:39.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:39.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124196+0000", 2024-09-18T04:57:39.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.575e-06 2024-09-18T04:57:39.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:39.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124201+0000", 2024-09-18T04:57:39.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5000000000000001e-06 2024-09-18T04:57:39.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:39.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168166+0000", 2024-09-18T04:57:39.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043965763999999997 2024-09-18T04:57:39.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:39.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168218+0000", 2024-09-18T04:57:39.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1785e-05 2024-09-18T04:57:39.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:39.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:39.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:39.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168271+0000", 2024-09-18T04:57:39.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3429e-05 2024-09-18T04:57:39.032 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:39.032 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:39.032 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:39.033 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:39.033 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:39.033 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:39.034 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:57:39.034 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:57:39.035 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:57:39.035 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:57:39.036 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:57:39.036 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:57:39.037 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:57:39.037 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:57:39.037 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:57:39.038 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:57:39.038 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:57:39.038 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:57:39.039 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:57:39.039 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:39.032+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:39.119 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T04:57:39.119 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T04:57:39.120 INFO:tasks.thrashosds.thrasher:Removing osd 4, in_osds are: [0, 2, 3, 4, 5, 6, 7] 2024-09-18T04:57:39.120 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 4 2024-09-18T04:57:39.164 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:39.162+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:39.177 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:39.178 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:39.186 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:57:39.235 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:39.233+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:39.287 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:39.287 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:39.296 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:57:39.340 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:39.338+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:39.398 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:39.399 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:39.408 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:57:39.413 INFO:tasks.ceph.mon.a.smithi002.stderr:2024-09-18T04:57:39.411+0000 7f7d3b87f640 -1 mon.a@0(leader).osd e88 definitely_dead 0 2024-09-18T04:57:39.449 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:39.446+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:39.465 INFO:teuthology.orchestra.run.smithi002.stderr:marked out osd.4. 2024-09-18T04:57:39.538 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:39.536+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:39.558 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:39.558 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:39.566 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:57:39.639 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:39.637+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:39.669 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:39.669 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:39.678 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:57:39.741 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:39.739+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:39.820 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:39.821 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:39.829 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:57:39.873 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:39.871+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:39.939 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:39.939 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:39.943 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:39.941+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:39.947 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:57:40.044 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:40.042+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:40.095 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:40.096 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:40.104 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:57:40.145 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:40.143+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:40.246 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:40.243+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:40.250 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:40.251 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:40.260 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:57:40.347 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:40.344+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:40.407 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:40.407 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:40.416 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:57:40.447 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:40.445+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:40.528 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:40.528 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:40.537 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:57:40.548 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:40.545+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:40.569 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T04:57:40.645 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:40.645 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:40.649 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:40.647+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:40.654 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:57:40.751 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:40.748+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:40.802 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:40.802 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:40.810 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:57:40.854 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:40.852+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:40.917 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:40.917 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:40.925 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:57:40.951 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:40.948+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:41.040 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:57:41.073 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:41.073 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:41.073 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:41.070+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:41.083 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:57:41.165 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:57:41.165 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:57:41.166 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:41.163+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:41.182 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:57:41.183 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:57:41.198 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:57:41.199 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:57:41.215 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:57:41.215 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:57:41.232 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:57:41.233 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:57:41.233 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:41.233 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:41.233 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:41.234 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:41.243 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:57:41.249 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:57:41.250 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:57:41.266 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:57:41.267 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:57:41.267 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:41.264+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:41.286 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:57:41.287 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:57:41.355 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:41.352+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:41.356 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:41.357 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:41.366 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:57:41.455 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:41.453+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:41.469 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-18T04:57:41.482 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:41.482 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:41.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:41.482 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:41.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.482 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11154 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:41.482 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:57:41.482 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.588558562000003, 2024-09-18T04:57:41.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040591919999999997, 2024-09-18T04:57:41.482 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:41.483 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:41.483 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:41.483 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:41.483 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:41.483 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11154 2024-09-18T04:57:41.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.483 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:41.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:41.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:57:41.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:41.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:57:41.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:41.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889400+0000", 2024-09-18T04:57:41.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:41.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:41.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889403+0000", 2024-09-18T04:57:41.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3999999999999999e-06 2024-09-18T04:57:41.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889409+0000", 2024-09-18T04:57:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3380000000000001e-06 2024-09-18T04:57:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889414+0000", 2024-09-18T04:57:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5789999999999997e-06 2024-09-18T04:57:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929811+0000", 2024-09-18T04:57:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040397486000000003 2024-09-18T04:57:41.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:41.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929898+0000", 2024-09-18T04:57:41.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6370000000000001e-05 2024-09-18T04:57:41.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929993+0000", 2024-09-18T04:57:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5136000000000005e-05 2024-09-18T04:57:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11155 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2015232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:57:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.588541383999996, 2024-09-18T04:57:41.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040769156000000001, 2024-09-18T04:57:41.488 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:41.488 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:41.488 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:41.488 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:41.488 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:41.488 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11155 2024-09-18T04:57:41.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.488 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:41.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:41.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:57:41.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:41.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:57:41.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:41.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:57:41.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:41.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:41.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889420+0000", 2024-09-18T04:57:41.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.181e-06 2024-09-18T04:57:41.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:41.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889426+0000", 2024-09-18T04:57:41.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8599999999999998e-06 2024-09-18T04:57:41.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:41.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889430+0000", 2024-09-18T04:57:41.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4399999999999998e-06 2024-09-18T04:57:41.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:41.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930020+0000", 2024-09-18T04:57:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040589698 2024-09-18T04:57:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930105+0000", 2024-09-18T04:57:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4812e-05 2024-09-18T04:57:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930187+0000", 2024-09-18T04:57:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2540999999999995e-05 2024-09-18T04:57:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.492 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11156 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2064384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:41.492 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:57:41.492 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.588501089999994, 2024-09-18T04:57:41.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040934419, 2024-09-18T04:57:41.492 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:41.492 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:41.492 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:41.492 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:41.492 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:41.492 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11156 2024-09-18T04:57:41.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.492 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:41.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:57:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:57:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889457+0000", 2024-09-18T04:57:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:41.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889462+0000", 2024-09-18T04:57:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8350000000000003e-06 2024-09-18T04:57:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889481+0000", 2024-09-18T04:57:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8845999999999999e-05 2024-09-18T04:57:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889495+0000", 2024-09-18T04:57:41.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4229e-05 2024-09-18T04:57:41.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:41.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930216+0000", 2024-09-18T04:57:41.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040721081999999999 2024-09-18T04:57:41.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:41.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930299+0000", 2024-09-18T04:57:41.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2995000000000001e-05 2024-09-18T04:57:41.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:41.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930393+0000", 2024-09-18T04:57:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3185999999999999e-05 2024-09-18T04:57:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11404 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1609728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:57:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.581696928, 2024-09-18T04:57:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040669012999999997, 2024-09-18T04:57:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:41.496 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11404 2024-09-18T04:57:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:57:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:57:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:41.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:57:41.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:57:41.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:41.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896263+0000", 2024-09-18T04:57:41.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2729999999999999e-06 2024-09-18T04:57:41.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:41.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896266+0000", 2024-09-18T04:57:41.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8200000000000001e-06 2024-09-18T04:57:41.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:41.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896269+0000", 2024-09-18T04:57:41.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3199999999999998e-06 2024-09-18T04:57:41.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:41.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936538+0000", 2024-09-18T04:57:41.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040269394999999999 2024-09-18T04:57:41.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:41.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936657+0000", 2024-09-18T04:57:41.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119498 2024-09-18T04:57:41.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:41.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936931+0000", 2024-09-18T04:57:41.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027391900000000002 2024-09-18T04:57:41.500 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.500 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:41.500 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.500 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11405 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1658880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:41.500 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:57:41.500 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.581688670000005, 2024-09-18T04:57:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040891034999999999, 2024-09-18T04:57:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11405 2024-09-18T04:57:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:57:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:57:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896270+0000", 2024-09-18T04:57:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:57:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896272+0000", 2024-09-18T04:57:41.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.124e-06 2024-09-18T04:57:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896274+0000", 2024-09-18T04:57:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8660000000000002e-06 2024-09-18T04:57:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896285+0000", 2024-09-18T04:57:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0101e-05 2024-09-18T04:57:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936986+0000", 2024-09-18T04:57:41.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040701213999999999 2024-09-18T04:57:41.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:41.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937070+0000", 2024-09-18T04:57:41.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4211999999999999e-05 2024-09-18T04:57:41.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:41.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937162+0000", 2024-09-18T04:57:41.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1728e-05 2024-09-18T04:57:41.504 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.504 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:41.504 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11406 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1708032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:57:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.581672501, 2024-09-18T04:57:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041085251000000003, 2024-09-18T04:57:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11406 2024-09-18T04:57:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.505 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:57:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:57:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:57:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:57:41.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896288+0000", 2024-09-18T04:57:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2440000000000001e-06 2024-09-18T04:57:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896291+0000", 2024-09-18T04:57:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8289999999999999e-06 2024-09-18T04:57:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896293+0000", 2024-09-18T04:57:41.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5399999999999998e-06 2024-09-18T04:57:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937201+0000", 2024-09-18T04:57:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040907418000000001 2024-09-18T04:57:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937283+0000", 2024-09-18T04:57:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2460999999999995e-05 2024-09-18T04:57:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:41.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937372+0000", 2024-09-18T04:57:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8987999999999996e-05 2024-09-18T04:57:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11407 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:57:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.581662680999997, 2024-09-18T04:57:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041438580000000003, 2024-09-18T04:57:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:41.509 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:41.510 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11407 2024-09-18T04:57:41.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.510 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:41.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:41.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:57:41.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:41.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:57:41.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896296+0000", 2024-09-18T04:57:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896298+0000", 2024-09-18T04:57:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:57:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896304+0000", 2024-09-18T04:57:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6570000000000002e-06 2024-09-18T04:57:41.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:41.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896308+0000", 2024-09-18T04:57:41.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0940000000000001e-06 2024-09-18T04:57:41.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:41.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937540+0000", 2024-09-18T04:57:41.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041231469 2024-09-18T04:57:41.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:41.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937625+0000", 2024-09-18T04:57:41.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5767999999999996e-05 2024-09-18T04:57:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937735+0000", 2024-09-18T04:57:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010974 2024-09-18T04:57:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11408 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:57:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.581647042, 2024-09-18T04:57:41.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041653055000000001, 2024-09-18T04:57:41.514 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:41.514 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:41.514 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:41.514 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:41.514 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:41.514 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11408 2024-09-18T04:57:41.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.514 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:41.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:41.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:57:41.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:57:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:57:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896314+0000", 2024-09-18T04:57:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0140000000000001e-06 2024-09-18T04:57:41.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896320+0000", 2024-09-18T04:57:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-18T04:57:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896324+0000", 2024-09-18T04:57:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4410000000000003e-06 2024-09-18T04:57:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937769+0000", 2024-09-18T04:57:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041445243 2024-09-18T04:57:41.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:41.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937861+0000", 2024-09-18T04:57:41.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1471000000000006e-05 2024-09-18T04:57:41.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:41.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937965+0000", 2024-09-18T04:57:41.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104516 2024-09-18T04:57:41.517 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.517 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:41.518 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.518 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11409 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:41.518 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:57:41.518 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.581630985999993, 2024-09-18T04:57:41.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041840467999999999, 2024-09-18T04:57:41.518 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:41.518 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:41.518 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:41.518 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:41.518 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:41.518 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11409 2024-09-18T04:57:41.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:57:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:57:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:57:41.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:41.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:41.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896331+0000", 2024-09-18T04:57:41.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5950000000000001e-06 2024-09-18T04:57:41.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:41.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896336+0000", 2024-09-18T04:57:41.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7259999999999999e-06 2024-09-18T04:57:41.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:41.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896341+0000", 2024-09-18T04:57:41.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5619999999999997e-06 2024-09-18T04:57:41.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:41.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937995+0000", 2024-09-18T04:57:41.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041654060999999999 2024-09-18T04:57:41.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:41.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938078+0000", 2024-09-18T04:57:41.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2869999999999998e-05 2024-09-18T04:57:41.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:41.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938169+0000", 2024-09-18T04:57:41.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1040999999999998e-05 2024-09-18T04:57:41.522 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.522 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:41.522 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.522 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11410 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1921024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:41.522 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:57:41.522 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.581614502999997, 2024-09-18T04:57:41.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042060342000000001, 2024-09-18T04:57:41.523 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:41.523 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:41.523 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:41.523 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:41.523 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:41.523 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11410 2024-09-18T04:57:41.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.523 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:41.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:41.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:57:41.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:41.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:57:41.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:41.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896344+0000", 2024-09-18T04:57:41.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:41.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:41.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896360+0000", 2024-09-18T04:57:41.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5068e-05 2024-09-18T04:57:41.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:41.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896366+0000", 2024-09-18T04:57:41.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5119999999999997e-06 2024-09-18T04:57:41.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:41.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896371+0000", 2024-09-18T04:57:41.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:57:41.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:41.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938198+0000", 2024-09-18T04:57:41.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041826579000000003 2024-09-18T04:57:41.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:41.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938297+0000", 2024-09-18T04:57:41.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9067000000000005e-05 2024-09-18T04:57:41.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:41.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938405+0000", 2024-09-18T04:57:41.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108492 2024-09-18T04:57:41.526 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.526 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:41.526 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.527 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11411 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1970176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:41.527 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:57:41.527 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.581583660999996, 2024-09-18T04:57:41.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042259191000000002, 2024-09-18T04:57:41.527 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:41.527 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:41.527 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:41.527 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:41.527 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:41.527 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11411 2024-09-18T04:57:41.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.527 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:41.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:41.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:57:41.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:41.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:57:41.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:41.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896375+0000", 2024-09-18T04:57:41.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:41.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:41.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896377+0000", 2024-09-18T04:57:41.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2699999999999999e-06 2024-09-18T04:57:41.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:41.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896383+0000", 2024-09-18T04:57:41.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6740000000000002e-06 2024-09-18T04:57:41.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:41.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896388+0000", 2024-09-18T04:57:41.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:57:41.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:41.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938435+0000", 2024-09-18T04:57:41.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042047169000000002 2024-09-18T04:57:41.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:41.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938534+0000", 2024-09-18T04:57:41.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8939999999999998e-05 2024-09-18T04:57:41.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:41.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938635+0000", 2024-09-18T04:57:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101208 2024-09-18T04:57:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11412 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:57:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.581567679000003, 2024-09-18T04:57:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042828079999999998, 2024-09-18T04:57:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:41.531 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11412 2024-09-18T04:57:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:57:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:57:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896391+0000", 2024-09-18T04:57:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896393+0000", 2024-09-18T04:57:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0810000000000001e-06 2024-09-18T04:57:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896399+0000", 2024-09-18T04:57:41.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7230000000000001e-06 2024-09-18T04:57:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896403+0000", 2024-09-18T04:57:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3470000000000001e-06 2024-09-18T04:57:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939043+0000", 2024-09-18T04:57:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042639829999999997 2024-09-18T04:57:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:41.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939124+0000", 2024-09-18T04:57:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0809000000000005e-05 2024-09-18T04:57:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939220+0000", 2024-09-18T04:57:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5659e-05 2024-09-18T04:57:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11413 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2068480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:57:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 65.581551984000001, 2024-09-18T04:57:41.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043042108000000003, 2024-09-18T04:57:41.536 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:41.536 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:41.536 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:41.536 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:41.536 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:41.536 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11413 2024-09-18T04:57:41.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.536 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:41.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:41.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:57:41.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:41.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:57:41.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:41.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:57:41.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:41.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:41.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896409+0000", 2024-09-18T04:57:41.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.165e-06 2024-09-18T04:57:41.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:41.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896415+0000", 2024-09-18T04:57:41.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6149999999999996e-06 2024-09-18T04:57:41.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:41.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896419+0000", 2024-09-18T04:57:41.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2649999999999998e-06 2024-09-18T04:57:41.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:41.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939262+0000", 2024-09-18T04:57:41.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042843444000000001 2024-09-18T04:57:41.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:41.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939351+0000", 2024-09-18T04:57:41.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8944999999999996e-05 2024-09-18T04:57:41.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:41.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939449+0000", 2024-09-18T04:57:41.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8029000000000006e-05 2024-09-18T04:57:41.540 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.540 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:41.540 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.540 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145409 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:41.540 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:57:41.540 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.9501323360000002, 2024-09-18T04:57:41.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040684845999999997, 2024-09-18T04:57:41.540 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145409 2024-09-18T04:57:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:57:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:57:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:57:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527829+0000", 2024-09-18T04:57:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.441e-06 2024-09-18T04:57:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527835+0000", 2024-09-18T04:57:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1650000000000003e-06 2024-09-18T04:57:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527840+0000", 2024-09-18T04:57:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8250000000000004e-06 2024-09-18T04:57:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568404+0000", 2024-09-18T04:57:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040563872000000001 2024-09-18T04:57:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:41.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568457+0000", 2024-09-18T04:57:41.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2713999999999997e-05 2024-09-18T04:57:41.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:41.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568512+0000", 2024-09-18T04:57:41.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5148000000000001e-05 2024-09-18T04:57:41.544 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.544 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:41.545 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.545 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145411 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:41.545 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:57:41.545 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.9501144750000003, 2024-09-18T04:57:41.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040792108000000001, 2024-09-18T04:57:41.545 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:41.545 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145411 2024-09-18T04:57:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:57:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:57:41.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:41.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527844+0000", 2024-09-18T04:57:41.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:41.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:41.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527847+0000", 2024-09-18T04:57:41.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0260000000000002e-06 2024-09-18T04:57:41.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:41.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527852+0000", 2024-09-18T04:57:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9889999999999997e-06 2024-09-18T04:57:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527857+0000", 2024-09-18T04:57:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6410000000000001e-06 2024-09-18T04:57:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568537+0000", 2024-09-18T04:57:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040679743999999997 2024-09-18T04:57:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568583+0000", 2024-09-18T04:57:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6286999999999999e-05 2024-09-18T04:57:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568637+0000", 2024-09-18T04:57:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.376e-05 2024-09-18T04:57:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145413 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1859584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:41.549 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:57:41.550 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.9500980949999995, 2024-09-18T04:57:41.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040919516000000003, 2024-09-18T04:57:41.550 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:41.550 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:41.550 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:41.550 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:41.550 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:41.550 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145413 2024-09-18T04:57:41.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.550 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:41.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:41.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:57:41.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:41.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:57:41.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:41.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:57:41.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:41.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:41.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527863+0000", 2024-09-18T04:57:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2630000000000002e-06 2024-09-18T04:57:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527869+0000", 2024-09-18T04:57:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9009999999999999e-06 2024-09-18T04:57:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527874+0000", 2024-09-18T04:57:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0590000000000002e-06 2024-09-18T04:57:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568663+0000", 2024-09-18T04:57:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040788805999999997 2024-09-18T04:57:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568716+0000", 2024-09-18T04:57:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3501999999999997e-05 2024-09-18T04:57:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568781+0000", 2024-09-18T04:57:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4338000000000006e-05 2024-09-18T04:57:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.553 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:41.554 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.554 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145414 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1908736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:41.554 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:57:41.554 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.9500811369999997, 2024-09-18T04:57:41.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041030234999999998, 2024-09-18T04:57:41.554 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:41.554 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:41.554 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:41.554 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:41.554 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:41.554 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145414 2024-09-18T04:57:41.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:57:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:57:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:41.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:57:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527880+0000", 2024-09-18T04:57:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4770000000000002e-06 2024-09-18T04:57:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527886+0000", 2024-09-18T04:57:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5759999999999996e-06 2024-09-18T04:57:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:41.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527890+0000", 2024-09-18T04:57:41.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8180000000000003e-06 2024-09-18T04:57:41.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:41.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568807+0000", 2024-09-18T04:57:41.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040916984000000003 2024-09-18T04:57:41.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:41.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568854+0000", 2024-09-18T04:57:41.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6740999999999999e-05 2024-09-18T04:57:41.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:41.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568908+0000", 2024-09-18T04:57:41.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4228000000000003e-05 2024-09-18T04:57:41.558 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.558 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:41.558 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.558 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145416 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1957888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:41.558 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:57:41.558 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.9500646450000003, 2024-09-18T04:57:41.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041147364999999998, 2024-09-18T04:57:41.558 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:41.558 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:41.558 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145416 2024-09-18T04:57:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:57:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:57:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:41.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527894+0000", 2024-09-18T04:57:41.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:41.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:41.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527896+0000", 2024-09-18T04:57:41.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1789999999999998e-06 2024-09-18T04:57:41.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:41.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527908+0000", 2024-09-18T04:57:41.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1780000000000001e-05 2024-09-18T04:57:41.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:41.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527914+0000", 2024-09-18T04:57:41.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2029999999999997e-06 2024-09-18T04:57:41.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:41.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568935+0000", 2024-09-18T04:57:41.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041020273000000003 2024-09-18T04:57:41.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:41.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568988+0000", 2024-09-18T04:57:41.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2911000000000002e-05 2024-09-18T04:57:41.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:41.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569042+0000", 2024-09-18T04:57:41.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4382000000000002e-05 2024-09-18T04:57:41.562 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.562 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:41.562 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.562 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145418 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 2007040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:41.562 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:57:41.562 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.9500402479999996, 2024-09-18T04:57:41.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041247862000000003, 2024-09-18T04:57:41.563 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:41.563 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:41.563 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:41.563 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:41.563 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:41.563 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145418 2024-09-18T04:57:41.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.563 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:41.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:41.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:57:41.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:57:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:57:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527921+0000", 2024-09-18T04:57:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.667e-06 2024-09-18T04:57:41.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527929+0000", 2024-09-18T04:57:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.542e-06 2024-09-18T04:57:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527934+0000", 2024-09-18T04:57:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1440000000000002e-06 2024-09-18T04:57:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569068+0000", 2024-09-18T04:57:41.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041134190000000001 2024-09-18T04:57:41.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:41.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569114+0000", 2024-09-18T04:57:41.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6261999999999999e-05 2024-09-18T04:57:41.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:41.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569167+0000", 2024-09-18T04:57:41.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2506999999999997e-05 2024-09-18T04:57:41.566 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.566 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:41.566 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.567 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145420 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 2056192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:41.567 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:57:41.567 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 7.9500211739999997, 2024-09-18T04:57:41.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041351472, 2024-09-18T04:57:41.567 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:41.567 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:41.567 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:41.567 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:41.567 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:41.567 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145420 2024-09-18T04:57:41.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.567 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:41.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:57:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:57:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:57:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:41.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527940+0000", 2024-09-18T04:57:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6570000000000001e-06 2024-09-18T04:57:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527948+0000", 2024-09-18T04:57:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8350000000000004e-06 2024-09-18T04:57:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527953+0000", 2024-09-18T04:57:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0690000000000001e-06 2024-09-18T04:57:41.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569182+0000", 2024-09-18T04:57:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041228734000000003 2024-09-18T04:57:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569234+0000", 2024-09-18T04:57:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1663999999999999e-05 2024-09-18T04:57:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569290+0000", 2024-09-18T04:57:41.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5937000000000003e-05 2024-09-18T04:57:41.571 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.571 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:41.571 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.571 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:41.571 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:41.571 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:41.573 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:57:41.573 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:41.570+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:41.673 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:41.671+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:41.706 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:41.706 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:41.706 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:57:41.707 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:41.718 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:57:41.803 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:41.800+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:41.875 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:41.873+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:41.888 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:41.888 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:41.888 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:57:41.888 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:57:41.888 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:41.898 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:57:41.982 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:41.980+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:42.072 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:42.072 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:42.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:42.072 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:42.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.072 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72135 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 394240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:42.072 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:57:42.072 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.114831492999997, 2024-09-18T04:57:42.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048559739999999997, 2024-09-18T04:57:42.073 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.073 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.073 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72135 2024-09-18T04:57:42.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.073 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:57:42.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:57:42.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:57:42.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:42.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954005+0000", 2024-09-18T04:57:42.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5629999999999999e-06 2024-09-18T04:57:42.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954008+0000", 2024-09-18T04:57:42.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-18T04:57:42.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954012+0000", 2024-09-18T04:57:42.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4790000000000001e-06 2024-09-18T04:57:42.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002418+0000", 2024-09-18T04:57:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048406049 2024-09-18T04:57:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002460+0000", 2024-09-18T04:57:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2188999999999999e-05 2024-09-18T04:57:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002564+0000", 2024-09-18T04:57:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010377 2024-09-18T04:57:42.076 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.077 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.077 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.077 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72136 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 409600~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:42.077 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:57:42.077 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.114726546, 2024-09-18T04:57:42.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048670191000000002, 2024-09-18T04:57:42.077 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.077 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.078 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.078 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.078 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72136 2024-09-18T04:57:42.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.078 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:57:42.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:57:42.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954019+0000", 2024-09-18T04:57:42.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999099 2024-09-18T04:57:42.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954112+0000", 2024-09-18T04:57:42.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3273e-05 2024-09-18T04:57:42.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954121+0000", 2024-09-18T04:57:42.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0289999999999994e-06 2024-09-18T04:57:42.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954128+0000", 2024-09-18T04:57:42.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6189999999999996e-06 2024-09-18T04:57:42.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002604+0000", 2024-09-18T04:57:42.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048476048000000001 2024-09-18T04:57:42.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002682+0000", 2024-09-18T04:57:42.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7955999999999997e-05 2024-09-18T04:57:42.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002779+0000", 2024-09-18T04:57:42.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7372000000000002e-05 2024-09-18T04:57:42.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.081 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.082 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.082 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72138 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 440320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:42.082 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:57:42.082 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.114292694, 2024-09-18T04:57:42.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048549430999999997, 2024-09-18T04:57:42.082 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.082 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72138 2024-09-18T04:57:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:57:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:57:42.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954542+0000", 2024-09-18T04:57:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954545+0000", 2024-09-18T04:57:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4360000000000001e-06 2024-09-18T04:57:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954551+0000", 2024-09-18T04:57:42.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4949999999999996e-06 2024-09-18T04:57:42.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954557+0000", 2024-09-18T04:57:42.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5099999999999998e-06 2024-09-18T04:57:42.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002993+0000", 2024-09-18T04:57:42.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048436358999999998 2024-09-18T04:57:42.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003039+0000", 2024-09-18T04:57:42.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5884000000000003e-05 2024-09-18T04:57:42.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003092+0000", 2024-09-18T04:57:42.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3208000000000003e-05 2024-09-18T04:57:42.086 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.086 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.086 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.086 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72139 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 455680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:42.086 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:57:42.087 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.114273885999999, 2024-09-18T04:57:42.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048718773, 2024-09-18T04:57:42.087 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.087 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.088 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.089 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72139 2024-09-18T04:57:42.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.089 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:57:42.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:57:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954561+0000", 2024-09-18T04:57:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954564+0000", 2024-09-18T04:57:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4099999999999998e-06 2024-09-18T04:57:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954570+0000", 2024-09-18T04:57:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.178e-06 2024-09-18T04:57:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954575+0000", 2024-09-18T04:57:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0560000000000004e-06 2024-09-18T04:57:42.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003114+0000", 2024-09-18T04:57:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048539212999999998 2024-09-18T04:57:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003176+0000", 2024-09-18T04:57:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2154000000000002e-05 2024-09-18T04:57:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003280+0000", 2024-09-18T04:57:42.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010421600000000001 2024-09-18T04:57:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72140 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 471040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:57:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.114017945000001, 2024-09-18T04:57:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048670819999999997, 2024-09-18T04:57:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.094 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.095 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72140 2024-09-18T04:57:42.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.095 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:57:42.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:57:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954579+0000", 2024-09-18T04:57:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997611 2024-09-18T04:57:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954821+0000", 2024-09-18T04:57:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024241000000000001 2024-09-18T04:57:42.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954831+0000", 2024-09-18T04:57:42.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8770000000000005e-06 2024-09-18T04:57:42.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954838+0000", 2024-09-18T04:57:42.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0690000000000004e-06 2024-09-18T04:57:42.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003317+0000", 2024-09-18T04:57:42.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048478796999999997 2024-09-18T04:57:42.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003392+0000", 2024-09-18T04:57:42.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4411000000000003e-05 2024-09-18T04:57:42.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003488+0000", 2024-09-18T04:57:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6927000000000002e-05 2024-09-18T04:57:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72141 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 486400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:57:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.113990461999997, 2024-09-18T04:57:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048808114, 2024-09-18T04:57:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.099 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.100 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.100 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.100 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.100 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72141 2024-09-18T04:57:42.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:57:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:57:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:57:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954847+0000", 2024-09-18T04:57:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3800000000000001e-06 2024-09-18T04:57:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954854+0000", 2024-09-18T04:57:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6080000000000001e-06 2024-09-18T04:57:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954859+0000", 2024-09-18T04:57:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:57:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003526+0000", 2024-09-18T04:57:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048666433000000002 2024-09-18T04:57:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003599+0000", 2024-09-18T04:57:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2854000000000004e-05 2024-09-18T04:57:42.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003653+0000", 2024-09-18T04:57:42.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4511e-05 2024-09-18T04:57:42.104 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.104 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.104 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.104 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72142 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 501760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:42.104 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:57:42.104 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.113951659999998, 2024-09-18T04:57:42.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048885500999999998, 2024-09-18T04:57:42.105 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.105 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.105 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.105 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.105 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.105 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72142 2024-09-18T04:57:42.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.105 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:57:42.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:57:42.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954864+0000", 2024-09-18T04:57:42.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999804 2024-09-18T04:57:42.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954886+0000", 2024-09-18T04:57:42.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2042999999999999e-05 2024-09-18T04:57:42.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954893+0000", 2024-09-18T04:57:42.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.255e-06 2024-09-18T04:57:42.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954899+0000", 2024-09-18T04:57:42.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8780000000000003e-06 2024-09-18T04:57:42.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003678+0000", 2024-09-18T04:57:42.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048778834 2024-09-18T04:57:42.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003717+0000", 2024-09-18T04:57:42.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8860999999999999e-05 2024-09-18T04:57:42.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003769+0000", 2024-09-18T04:57:42.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2389000000000003e-05 2024-09-18T04:57:42.113 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.113 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.113 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.113 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72143 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 517120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:42.113 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:57:42.113 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.113826033999999, 2024-09-18T04:57:42.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048951738000000002, 2024-09-18T04:57:42.113 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.113 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.113 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.114 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.114 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.114 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72143 2024-09-18T04:57:42.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.114 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:57:42.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:57:42.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954904+0000", 2024-09-18T04:57:42.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998946 2024-09-18T04:57:42.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955013+0000", 2024-09-18T04:57:42.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108977 2024-09-18T04:57:42.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955021+0000", 2024-09-18T04:57:42.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7570000000000004e-06 2024-09-18T04:57:42.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955025+0000", 2024-09-18T04:57:42.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1219999999999997e-06 2024-09-18T04:57:42.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003792+0000", 2024-09-18T04:57:42.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048767379 2024-09-18T04:57:42.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003860+0000", 2024-09-18T04:57:42.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8033000000000004e-05 2024-09-18T04:57:42.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003961+0000", 2024-09-18T04:57:42.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010103300000000001 2024-09-18T04:57:42.117 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.117 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.117 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.117 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72161 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 13312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:42.117 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:57:42.118 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.113562397000003, 2024-09-18T04:57:42.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048891539999999997, 2024-09-18T04:57:42.118 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.118 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.118 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.118 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.118 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.118 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72161 2024-09-18T04:57:42.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.118 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:57:42.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:57:42.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955028+0000", 2024-09-18T04:57:42.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997549 2024-09-18T04:57:42.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955278+0000", 2024-09-18T04:57:42.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024973299999999998 2024-09-18T04:57:42.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955289+0000", 2024-09-18T04:57:42.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0812000000000001e-05 2024-09-18T04:57:42.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955296+0000", 2024-09-18T04:57:42.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2570000000000001e-06 2024-09-18T04:57:42.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004002+0000", 2024-09-18T04:57:42.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048705904000000001 2024-09-18T04:57:42.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004076+0000", 2024-09-18T04:57:42.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4086999999999997e-05 2024-09-18T04:57:42.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004165+0000", 2024-09-18T04:57:42.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8712e-05 2024-09-18T04:57:42.123 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.123 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.123 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.123 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72163 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 44032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:42.123 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:57:42.123 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.112936959000002, 2024-09-18T04:57:42.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048567011, 2024-09-18T04:57:42.124 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.124 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.124 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.125 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.125 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.125 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72163 2024-09-18T04:57:42.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.125 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:57:42.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:57:42.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955898+0000", 2024-09-18T04:57:42.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:42.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955901+0000", 2024-09-18T04:57:42.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.706e-06 2024-09-18T04:57:42.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955907+0000", 2024-09-18T04:57:42.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3559999999999998e-06 2024-09-18T04:57:42.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955912+0000", 2024-09-18T04:57:42.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9640000000000002e-06 2024-09-18T04:57:42.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004377+0000", 2024-09-18T04:57:42.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048465293999999999 2024-09-18T04:57:42.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004418+0000", 2024-09-18T04:57:42.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0163000000000002e-05 2024-09-18T04:57:42.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004466+0000", 2024-09-18T04:57:42.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8004000000000003e-05 2024-09-18T04:57:42.128 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.128 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.128 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.129 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72164 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:42.129 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:57:42.129 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.112918686, 2024-09-18T04:57:42.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048707253999999998, 2024-09-18T04:57:42.129 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.129 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.129 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.129 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.129 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.129 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72164 2024-09-18T04:57:42.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.129 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:57:42.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:57:42.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:57:42.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:42.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955919+0000", 2024-09-18T04:57:42.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2520000000000002e-06 2024-09-18T04:57:42.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955925+0000", 2024-09-18T04:57:42.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3740000000000003e-06 2024-09-18T04:57:42.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955930+0000", 2024-09-18T04:57:42.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:57:42.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004489+0000", 2024-09-18T04:57:42.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048558581000000003 2024-09-18T04:57:42.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004529+0000", 2024-09-18T04:57:42.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9722999999999999e-05 2024-09-18T04:57:42.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004624+0000", 2024-09-18T04:57:42.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5402999999999994e-05 2024-09-18T04:57:42.133 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.133 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.133 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.133 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72165 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 74752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:42.133 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:57:42.133 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.112643826999999, 2024-09-18T04:57:42.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048633223000000003, 2024-09-18T04:57:42.133 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.133 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.133 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.133 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.133 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.134 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72165 2024-09-18T04:57:42.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.134 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:57:42.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:57:42.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955935+0000", 2024-09-18T04:57:42.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999743 2024-09-18T04:57:42.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956199+0000", 2024-09-18T04:57:42.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000263906 2024-09-18T04:57:42.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956207+0000", 2024-09-18T04:57:42.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2519999999999999e-06 2024-09-18T04:57:42.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956214+0000", 2024-09-18T04:57:42.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7610000000000002e-06 2024-09-18T04:57:42.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004661+0000", 2024-09-18T04:57:42.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048447967000000002 2024-09-18T04:57:42.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004735+0000", 2024-09-18T04:57:42.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3789999999999997e-05 2024-09-18T04:57:42.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004825+0000", 2024-09-18T04:57:42.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9724999999999999e-05 2024-09-18T04:57:42.137 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.137 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.137 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.138 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72166 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 90112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:42.138 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:57:42.138 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.112390292999997, 2024-09-18T04:57:42.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048595737999999999, 2024-09-18T04:57:42.138 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.138 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.138 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.138 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.138 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.138 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72166 2024-09-18T04:57:42.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.138 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:57:42.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:57:42.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956221+0000", 2024-09-18T04:57:42.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997759 2024-09-18T04:57:42.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956448+0000", 2024-09-18T04:57:42.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022685500000000001 2024-09-18T04:57:42.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956456+0000", 2024-09-18T04:57:42.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0099999999999995e-06 2024-09-18T04:57:42.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956462+0000", 2024-09-18T04:57:42.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7200000000000003e-06 2024-09-18T04:57:42.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004862+0000", 2024-09-18T04:57:42.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048399795000000002 2024-09-18T04:57:42.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004941+0000", 2024-09-18T04:57:42.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9414999999999998e-05 2024-09-18T04:57:42.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005041+0000", 2024-09-18T04:57:42.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9942e-05 2024-09-18T04:57:42.143 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.143 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.143 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.143 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72167 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:42.143 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:57:42.143 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.112303255, 2024-09-18T04:57:42.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048622894999999999, 2024-09-18T04:57:42.144 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.144 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.144 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.144 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.144 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.144 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72167 2024-09-18T04:57:42.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.144 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:57:42.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:57:42.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956468+0000", 2024-09-18T04:57:42.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999352 2024-09-18T04:57:42.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956535+0000", 2024-09-18T04:57:42.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7440999999999993e-05 2024-09-18T04:57:42.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956544+0000", 2024-09-18T04:57:42.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1039999999999996e-06 2024-09-18T04:57:42.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956550+0000", 2024-09-18T04:57:42.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9850000000000003e-06 2024-09-18T04:57:42.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005066+0000", 2024-09-18T04:57:42.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048515614999999998 2024-09-18T04:57:42.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005107+0000", 2024-09-18T04:57:42.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0843999999999997e-05 2024-09-18T04:57:42.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005155+0000", 2024-09-18T04:57:42.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8686000000000001e-05 2024-09-18T04:57:42.147 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.148 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.148 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.148 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72168 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:42.148 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:57:42.148 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.112142192, 2024-09-18T04:57:42.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048585179999999999, 2024-09-18T04:57:42.148 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.148 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.148 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.148 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.148 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.149 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72168 2024-09-18T04:57:42.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.149 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:57:42.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:57:42.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956555+0000", 2024-09-18T04:57:42.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998612 2024-09-18T04:57:42.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956696+0000", 2024-09-18T04:57:42.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000140922 2024-09-18T04:57:42.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956707+0000", 2024-09-18T04:57:42.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1555e-05 2024-09-18T04:57:42.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956714+0000", 2024-09-18T04:57:42.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2709999999999998e-06 2024-09-18T04:57:42.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005177+0000", 2024-09-18T04:57:42.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048463421 2024-09-18T04:57:42.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005214+0000", 2024-09-18T04:57:42.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.676e-05 2024-09-18T04:57:42.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005279+0000", 2024-09-18T04:57:42.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4899000000000003e-05 2024-09-18T04:57:42.152 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.153 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.153 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.153 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72172 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 441344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:42.153 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.957556+0000", 2024-09-18T04:57:42.153 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.111279791999998, 2024-09-18T04:57:42.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048545056000000003, 2024-09-18T04:57:42.153 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.154 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.154 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.154 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.154 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.154 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72172 2024-09-18T04:57:42.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.154 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957556+0000", 2024-09-18T04:57:42.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957556+0000", 2024-09-18T04:57:42.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957555+0000", 2024-09-18T04:57:42.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:42.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957558+0000", 2024-09-18T04:57:42.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2900000000000001e-06 2024-09-18T04:57:42.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957564+0000", 2024-09-18T04:57:42.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0700000000000003e-06 2024-09-18T04:57:42.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957569+0000", 2024-09-18T04:57:42.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7870000000000001e-06 2024-09-18T04:57:42.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005922+0000", 2024-09-18T04:57:42.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048353259000000003 2024-09-18T04:57:42.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.157 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005999+0000", 2024-09-18T04:57:42.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6889000000000002e-05 2024-09-18T04:57:42.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.006101+0000", 2024-09-18T04:57:42.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102136 2024-09-18T04:57:42.158 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.158 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.158 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.158 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72199 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:42.158 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:57:42.158 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.1096103, 2024-09-18T04:57:42.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048626057, 2024-09-18T04:57:42.158 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.158 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.159 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.159 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.159 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.159 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72199 2024-09-18T04:57:42.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.159 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:57:42.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:57:42.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959195+0000", 2024-09-18T04:57:42.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999969 2024-09-18T04:57:42.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959228+0000", 2024-09-18T04:57:42.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3609000000000003e-05 2024-09-18T04:57:42.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959236+0000", 2024-09-18T04:57:42.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.157e-06 2024-09-18T04:57:42.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959243+0000", 2024-09-18T04:57:42.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5910000000000001e-06 2024-09-18T04:57:42.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007684+0000", 2024-09-18T04:57:42.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048440745 2024-09-18T04:57:42.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007762+0000", 2024-09-18T04:57:42.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8360000000000002e-05 2024-09-18T04:57:42.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007851+0000", 2024-09-18T04:57:42.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9388000000000005e-05 2024-09-18T04:57:42.162 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.162 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.162 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.163 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72240 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 428032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:42.163 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:57:42.163 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 45.105516989999998, 2024-09-18T04:57:42.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048643767999999997, 2024-09-18T04:57:42.163 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.163 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.163 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.163 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.163 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.163 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72240 2024-09-18T04:57:42.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.163 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:57:42.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:57:42.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963240+0000", 2024-09-18T04:57:42.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999218 2024-09-18T04:57:42.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963323+0000", 2024-09-18T04:57:42.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2375999999999999e-05 2024-09-18T04:57:42.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963332+0000", 2024-09-18T04:57:42.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1700000000000003e-06 2024-09-18T04:57:42.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963338+0000", 2024-09-18T04:57:42.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2149999999999998e-06 2024-09-18T04:57:42.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011785+0000", 2024-09-18T04:57:42.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048446493 2024-09-18T04:57:42.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011861+0000", 2024-09-18T04:57:42.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6472999999999998e-05 2024-09-18T04:57:42.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011962+0000", 2024-09-18T04:57:42.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101167 2024-09-18T04:57:42.167 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.167 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.168 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.168 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:117375 3.1 3:80b18d38:::rbd_header.11acae5d5365:head [watch reconnect cookie 140406307615440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T04:57:42.168 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:57:42.168 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.684442452999999, 2024-09-18T04:57:42.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.018908156, 2024-09-18T04:57:42.168 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.168 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.168 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.168 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.168 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.168 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117375 2024-09-18T04:57:42.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.169 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:57:42.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:57:42.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384391+0000", 2024-09-18T04:57:42.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:57:42.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384401+0000", 2024-09-18T04:57:42.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4299999999999995e-06 2024-09-18T04:57:42.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384418+0000", 2024-09-18T04:57:42.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6745e-05 2024-09-18T04:57:42.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384429+0000", 2024-09-18T04:57:42.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1122e-05 2024-09-18T04:57:42.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384496+0000", 2024-09-18T04:57:42.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6841000000000006e-05 2024-09-18T04:57:42.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-18T04:57:42.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384505+0000", 2024-09-18T04:57:42.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4080000000000004e-06 2024-09-18T04:57:42.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.388971+0000", 2024-09-18T04:57:42.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0044660000000000004 2024-09-18T04:57:42.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:57:42.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.388981+0000", 2024-09-18T04:57:42.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0118e-05 2024-09-18T04:57:42.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.399433+0000", 2024-09-18T04:57:42.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.010452055 2024-09-18T04:57:42.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.399594+0000", 2024-09-18T04:57:42.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000160669 2024-09-18T04:57:42.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.403301+0000", 2024-09-18T04:57:42.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037074370000000001 2024-09-18T04:57:42.173 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.173 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.174 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.174 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:117376 3.1 3:80b18d38:::rbd_header.11acae5d5365:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T04:57:42.174 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:57:42.174 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 18.682390803000001, 2024-09-18T04:57:42.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0196134539999999, 2024-09-18T04:57:42.174 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.174 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:42.174 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.174 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.174 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.174 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117376 2024-09-18T04:57:42.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.175 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:57:42.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:57:42.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.175 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.175 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386443+0000", 2024-09-18T04:57:42.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:57:42.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386457+0000", 2024-09-18T04:57:42.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4207e-05 2024-09-18T04:57:42.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386478+0000", 2024-09-18T04:57:42.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0349000000000001e-05 2024-09-18T04:57:42.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386503+0000", 2024-09-18T04:57:42.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5582e-05 2024-09-18T04:57:42.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386558+0000", 2024-09-18T04:57:42.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4892999999999998e-05 2024-09-18T04:57:42.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-18T04:57:42.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386562+0000", 2024-09-18T04:57:42.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.313e-06 2024-09-18T04:57:42.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.389010+0000", 2024-09-18T04:57:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024480890000000001 2024-09-18T04:57:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:57:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.389018+0000", 2024-09-18T04:57:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8469999999999996e-06 2024-09-18T04:57:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400073+0000", 2024-09-18T04:57:42.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.011054286 2024-09-18T04:57:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400124+0000", 2024-09-18T04:57:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1332999999999998e-05 2024-09-18T04:57:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:57:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400477+0000", 2024-09-18T04:57:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035329300000000002 2024-09-18T04:57:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:42.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.401353+0000", 2024-09-18T04:57:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00087584200000000005 2024-09-18T04:57:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:57:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406006+0000", 2024-09-18T04:57:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0046534250000000001 2024-09-18T04:57:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:57:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406037+0000", 2024-09-18T04:57:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0559000000000003e-05 2024-09-18T04:57:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406058+0000", 2024-09-18T04:57:42.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1260999999999999e-05 2024-09-18T04:57:42.181 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.181 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.181 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.181 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.181 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.181 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:42.182 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:57:42.183 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:42.180+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:42.283 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:42.281+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:42.339 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:42.339 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:42.339 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:57:42.339 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:42.349 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:57:42.385 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:42.383+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:42.485 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:42.483+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:42.512 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:42.512 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:57:42.513 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:57:42.513 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:42.522 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:57:42.620 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:42.584+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:42.678 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:42.678 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:42.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:42.678 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:42.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.678 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60344 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1456128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:42.679 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:42.679 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.562926498000003, 2024-09-18T04:57:42.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044026324999999998, 2024-09-18T04:57:42.679 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.679 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.679 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.679 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60344 2024-09-18T04:57:42.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.679 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:42.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:42.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:42.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:42.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112224+0000", 2024-09-18T04:57:42.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1320000000000001e-06 2024-09-18T04:57:42.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112230+0000", 2024-09-18T04:57:42.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9619999999999998e-06 2024-09-18T04:57:42.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112234+0000", 2024-09-18T04:57:42.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7890000000000002e-06 2024-09-18T04:57:42.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156105+0000", 2024-09-18T04:57:42.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043870531999999997 2024-09-18T04:57:42.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156170+0000", 2024-09-18T04:57:42.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5036000000000005e-05 2024-09-18T04:57:42.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156248+0000", 2024-09-18T04:57:42.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8295999999999997e-05 2024-09-18T04:57:42.684 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.684 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.684 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.684 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60345 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1471488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:42.684 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:57:42.684 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.562909732999998, 2024-09-18T04:57:42.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044180090999999998, 2024-09-18T04:57:42.684 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.684 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.685 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.685 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.685 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.685 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60345 2024-09-18T04:57:42.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.685 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:57:42.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:57:42.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112238+0000", 2024-09-18T04:57:42.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:42.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112240+0000", 2024-09-18T04:57:42.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9910000000000001e-06 2024-09-18T04:57:42.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112246+0000", 2024-09-18T04:57:42.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0859999999999998e-06 2024-09-18T04:57:42.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112251+0000", 2024-09-18T04:57:42.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5020000000000003e-06 2024-09-18T04:57:42.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156274+0000", 2024-09-18T04:57:42.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044023063000000001 2024-09-18T04:57:42.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156337+0000", 2024-09-18T04:57:42.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3077000000000006e-05 2024-09-18T04:57:42.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156419+0000", 2024-09-18T04:57:42.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1699000000000005e-05 2024-09-18T04:57:42.689 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.689 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.689 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.689 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60346 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1486848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:42.689 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:57:42.689 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.562891952000001, 2024-09-18T04:57:42.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044336331999999999, 2024-09-18T04:57:42.690 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.690 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.690 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.690 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.690 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.690 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60346 2024-09-18T04:57:42.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.690 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:57:42.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:57:42.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112256+0000", 2024-09-18T04:57:42.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:42.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112258+0000", 2024-09-18T04:57:42.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1720000000000001e-06 2024-09-18T04:57:42.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112265+0000", 2024-09-18T04:57:42.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4280000000000001e-06 2024-09-18T04:57:42.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112269+0000", 2024-09-18T04:57:42.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.741e-06 2024-09-18T04:57:42.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156445+0000", 2024-09-18T04:57:42.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044175341 2024-09-18T04:57:42.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156512+0000", 2024-09-18T04:57:42.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6723000000000005e-05 2024-09-18T04:57:42.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156593+0000", 2024-09-18T04:57:42.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1284000000000003e-05 2024-09-18T04:57:42.694 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.694 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.694 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.694 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60347 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1502208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:42.694 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:57:42.694 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.562863540999999, 2024-09-18T04:57:42.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044482560999999997, 2024-09-18T04:57:42.695 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.695 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.695 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.695 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.695 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.695 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60347 2024-09-18T04:57:42.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.695 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:57:42.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:57:42.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112284+0000", 2024-09-18T04:57:42.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:42.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112286+0000", 2024-09-18T04:57:42.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9709999999999998e-06 2024-09-18T04:57:42.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112290+0000", 2024-09-18T04:57:42.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9199999999999997e-06 2024-09-18T04:57:42.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112293+0000", 2024-09-18T04:57:42.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870000000000001e-06 2024-09-18T04:57:42.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156617+0000", 2024-09-18T04:57:42.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044323960000000003 2024-09-18T04:57:42.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156687+0000", 2024-09-18T04:57:42.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0141000000000005e-05 2024-09-18T04:57:42.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156767+0000", 2024-09-18T04:57:42.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0140000000000002e-05 2024-09-18T04:57:42.700 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.700 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.700 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.700 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60348 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1517568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:42.700 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:42.700 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.562852536999998, 2024-09-18T04:57:42.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044645562, 2024-09-18T04:57:42.700 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.700 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.700 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.701 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.701 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.701 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60348 2024-09-18T04:57:42.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.701 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:42.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:42.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:42.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:42.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112297+0000", 2024-09-18T04:57:42.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.384e-06 2024-09-18T04:57:42.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112301+0000", 2024-09-18T04:57:42.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0570000000000003e-06 2024-09-18T04:57:42.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112304+0000", 2024-09-18T04:57:42.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6000000000000001e-06 2024-09-18T04:57:42.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156793+0000", 2024-09-18T04:57:42.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044489494999999997 2024-09-18T04:57:42.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156859+0000", 2024-09-18T04:57:42.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5483999999999998e-05 2024-09-18T04:57:42.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156941+0000", 2024-09-18T04:57:42.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2856999999999996e-05 2024-09-18T04:57:42.704 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.704 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.704 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.705 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60349 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1532928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:42.705 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:42.705 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.562842560999997, 2024-09-18T04:57:42.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044832167999999999, 2024-09-18T04:57:42.705 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.705 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.705 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.705 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.705 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.705 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60349 2024-09-18T04:57:42.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.706 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:42.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:42.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:42.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:42.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112307+0000", 2024-09-18T04:57:42.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.308e-06 2024-09-18T04:57:42.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112310+0000", 2024-09-18T04:57:42.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-18T04:57:42.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112312+0000", 2024-09-18T04:57:42.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4889999999999998e-06 2024-09-18T04:57:42.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156977+0000", 2024-09-18T04:57:42.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044664598999999999 2024-09-18T04:57:42.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157053+0000", 2024-09-18T04:57:42.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6014999999999996e-05 2024-09-18T04:57:42.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157138+0000", 2024-09-18T04:57:42.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5162999999999998e-05 2024-09-18T04:57:42.709 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.709 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.709 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.709 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60350 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1548288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:42.709 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:42.709 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.562833646999998, 2024-09-18T04:57:42.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045011858000000002, 2024-09-18T04:57:42.710 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.710 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.710 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.710 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.710 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.710 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60350 2024-09-18T04:57:42.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.710 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:42.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:42.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:42.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:42.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112316+0000", 2024-09-18T04:57:42.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7579999999999999e-06 2024-09-18T04:57:42.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112321+0000", 2024-09-18T04:57:42.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1649999999999997e-06 2024-09-18T04:57:42.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112325+0000", 2024-09-18T04:57:42.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.726e-06 2024-09-18T04:57:42.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157172+0000", 2024-09-18T04:57:42.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.04484718 2024-09-18T04:57:42.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157242+0000", 2024-09-18T04:57:42.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9675e-05 2024-09-18T04:57:42.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157327+0000", 2024-09-18T04:57:42.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4614e-05 2024-09-18T04:57:42.714 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.714 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.714 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.714 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60351 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1563648~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:42.714 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:42.714 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.562820231000003, 2024-09-18T04:57:42.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045132113000000001, 2024-09-18T04:57:42.715 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.715 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.715 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.715 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.715 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.715 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60351 2024-09-18T04:57:42.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.715 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:42.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:42.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:42.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:42.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112330+0000", 2024-09-18T04:57:42.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1660000000000001e-06 2024-09-18T04:57:42.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112336+0000", 2024-09-18T04:57:42.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5940000000000001e-06 2024-09-18T04:57:42.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112340+0000", 2024-09-18T04:57:42.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5589999999999999e-06 2024-09-18T04:57:42.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157365+0000", 2024-09-18T04:57:42.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045024675 2024-09-18T04:57:42.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157409+0000", 2024-09-18T04:57:42.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4356e-05 2024-09-18T04:57:42.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157460+0000", 2024-09-18T04:57:42.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1072000000000002e-05 2024-09-18T04:57:42.720 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.721 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.721 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.721 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60501 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1494016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:42.721 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:42.721 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.556759534999998, 2024-09-18T04:57:42.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044007728000000003, 2024-09-18T04:57:42.725 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.725 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.726 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.726 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.726 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.726 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60501 2024-09-18T04:57:42.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.726 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:42.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:42.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:42.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:42.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118391+0000", 2024-09-18T04:57:42.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:57:42.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118396+0000", 2024-09-18T04:57:42.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2680000000000002e-06 2024-09-18T04:57:42.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118400+0000", 2024-09-18T04:57:42.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2080000000000002e-06 2024-09-18T04:57:42.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162317+0000", 2024-09-18T04:57:42.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043916753000000003 2024-09-18T04:57:42.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162351+0000", 2024-09-18T04:57:42.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4372000000000002e-05 2024-09-18T04:57:42.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162397+0000", 2024-09-18T04:57:42.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5207999999999998e-05 2024-09-18T04:57:42.729 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.729 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.729 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.730 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60502 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1509376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:42.730 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:42.730 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.556744058, 2024-09-18T04:57:42.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044092438999999997, 2024-09-18T04:57:42.730 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.730 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.730 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.730 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.730 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.730 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60502 2024-09-18T04:57:42.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.730 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:42.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:42.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:42.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:42.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118406+0000", 2024-09-18T04:57:42.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:57:42.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118412+0000", 2024-09-18T04:57:42.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.524e-06 2024-09-18T04:57:42.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118416+0000", 2024-09-18T04:57:42.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:57:42.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162418+0000", 2024-09-18T04:57:42.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001508000000002 2024-09-18T04:57:42.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162451+0000", 2024-09-18T04:57:42.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3662999999999999e-05 2024-09-18T04:57:42.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162497+0000", 2024-09-18T04:57:42.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5550000000000003e-05 2024-09-18T04:57:42.734 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.734 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.734 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.734 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60503 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1524736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:42.734 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:57:42.734 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.556727899999998, 2024-09-18T04:57:42.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044178780000000001, 2024-09-18T04:57:42.734 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.735 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.735 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.735 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.735 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.735 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60503 2024-09-18T04:57:42.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.735 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:57:42.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:57:42.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118420+0000", 2024-09-18T04:57:42.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:42.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.736 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118422+0000", 2024-09-18T04:57:42.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.08e-06 2024-09-18T04:57:42.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118428+0000", 2024-09-18T04:57:42.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.964e-06 2024-09-18T04:57:42.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118433+0000", 2024-09-18T04:57:42.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.425e-06 2024-09-18T04:57:42.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162519+0000", 2024-09-18T04:57:42.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044086082999999998 2024-09-18T04:57:42.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162552+0000", 2024-09-18T04:57:42.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3719e-05 2024-09-18T04:57:42.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162599+0000", 2024-09-18T04:57:42.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6873999999999999e-05 2024-09-18T04:57:42.738 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.738 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.738 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.739 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60504 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1540096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:42.739 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:57:42.739 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.556711794999998, 2024-09-18T04:57:42.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044261921000000003, 2024-09-18T04:57:42.739 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.739 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.739 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.739 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.739 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.739 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60504 2024-09-18T04:57:42.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.739 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:57:42.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:57:42.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118436+0000", 2024-09-18T04:57:42.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:42.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118438+0000", 2024-09-18T04:57:42.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0250000000000001e-06 2024-09-18T04:57:42.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118444+0000", 2024-09-18T04:57:42.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4360000000000001e-06 2024-09-18T04:57:42.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118448+0000", 2024-09-18T04:57:42.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.357e-06 2024-09-18T04:57:42.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162620+0000", 2024-09-18T04:57:42.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044171821 2024-09-18T04:57:42.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162653+0000", 2024-09-18T04:57:42.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3479999999999998e-05 2024-09-18T04:57:42.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162699+0000", 2024-09-18T04:57:42.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5113e-05 2024-09-18T04:57:42.743 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.743 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.743 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.743 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60584 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1196032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:42.743 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:57:42.743 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.554918778000001, 2024-09-18T04:57:42.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044048727000000003, 2024-09-18T04:57:42.744 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.744 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.744 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.744 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.744 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.744 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60584 2024-09-18T04:57:42.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.744 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:57:42.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:57:42.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120229+0000", 2024-09-18T04:57:42.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:42.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120231+0000", 2024-09-18T04:57:42.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:57:42.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120237+0000", 2024-09-18T04:57:42.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8379999999999998e-06 2024-09-18T04:57:42.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120242+0000", 2024-09-18T04:57:42.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3889999999999998e-06 2024-09-18T04:57:42.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164187+0000", 2024-09-18T04:57:42.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043945045000000002 2024-09-18T04:57:42.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164227+0000", 2024-09-18T04:57:42.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0627999999999998e-05 2024-09-18T04:57:42.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164278+0000", 2024-09-18T04:57:42.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1146000000000001e-05 2024-09-18T04:57:42.748 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.748 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.748 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.748 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60585 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1211392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:42.748 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:57:42.748 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.554902669999997, 2024-09-18T04:57:42.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044389814, 2024-09-18T04:57:42.748 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.749 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.749 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.749 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60585 2024-09-18T04:57:42.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.749 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:57:42.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:57:42.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120245+0000", 2024-09-18T04:57:42.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:42.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120247+0000", 2024-09-18T04:57:42.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0150000000000002e-06 2024-09-18T04:57:42.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120255+0000", 2024-09-18T04:57:42.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.858e-06 2024-09-18T04:57:42.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120260+0000", 2024-09-18T04:57:42.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6530000000000002e-06 2024-09-18T04:57:42.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164489+0000", 2024-09-18T04:57:42.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044229483999999999 2024-09-18T04:57:42.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164555+0000", 2024-09-18T04:57:42.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.533e-05 2024-09-18T04:57:42.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164636+0000", 2024-09-18T04:57:42.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0819e-05 2024-09-18T04:57:42.752 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.752 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.752 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.753 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60586 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1226752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:42.753 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:42.753 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.554884219000002, 2024-09-18T04:57:42.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044578214999999997, 2024-09-18T04:57:42.753 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.753 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.753 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.753 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.753 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.753 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60586 2024-09-18T04:57:42.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.753 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:42.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:42.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:42.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:42.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120266+0000", 2024-09-18T04:57:42.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0660000000000002e-06 2024-09-18T04:57:42.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120272+0000", 2024-09-18T04:57:42.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-18T04:57:42.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120276+0000", 2024-09-18T04:57:42.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.476e-06 2024-09-18T04:57:42.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164673+0000", 2024-09-18T04:57:42.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044396672999999998 2024-09-18T04:57:42.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164749+0000", 2024-09-18T04:57:42.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5811999999999999e-05 2024-09-18T04:57:42.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164842+0000", 2024-09-18T04:57:42.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3560000000000005e-05 2024-09-18T04:57:42.757 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.757 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.757 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.757 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60587 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1242112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:42.757 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:57:42.757 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.55483512, 2024-09-18T04:57:42.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044634794999999998, 2024-09-18T04:57:42.757 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.757 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.757 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.757 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.758 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60587 2024-09-18T04:57:42.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.758 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:57:42.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:57:42.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120280+0000", 2024-09-18T04:57:42.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999671 2024-09-18T04:57:42.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120316+0000", 2024-09-18T04:57:42.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5728000000000001e-05 2024-09-18T04:57:42.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120329+0000", 2024-09-18T04:57:42.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2914e-05 2024-09-18T04:57:42.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120335+0000", 2024-09-18T04:57:42.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8289999999999996e-06 2024-09-18T04:57:42.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164858+0000", 2024-09-18T04:57:42.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044523417000000003 2024-09-18T04:57:42.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164896+0000", 2024-09-18T04:57:42.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7969999999999997e-05 2024-09-18T04:57:42.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164948+0000", 2024-09-18T04:57:42.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1996000000000002e-05 2024-09-18T04:57:42.762 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.762 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.762 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.762 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60595 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1364992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:42.762 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:57:42.763 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.552991904000002, 2024-09-18T04:57:42.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001360000000003, 2024-09-18T04:57:42.763 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.763 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.763 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60595 2024-09-18T04:57:42.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.763 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:57:42.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:57:42.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122156+0000", 2024-09-18T04:57:42.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:42.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122158+0000", 2024-09-18T04:57:42.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0040000000000002e-06 2024-09-18T04:57:42.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122164+0000", 2024-09-18T04:57:42.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8719999999999999e-06 2024-09-18T04:57:42.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122169+0000", 2024-09-18T04:57:42.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5090000000000004e-06 2024-09-18T04:57:42.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166040+0000", 2024-09-18T04:57:42.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043871226999999999 2024-09-18T04:57:42.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166096+0000", 2024-09-18T04:57:42.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6495e-05 2024-09-18T04:57:42.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166158+0000", 2024-09-18T04:57:42.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1626999999999998e-05 2024-09-18T04:57:42.766 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.767 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.767 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.767 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60596 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1380352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:42.767 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:42.767 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.552975582999998, 2024-09-18T04:57:42.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044120800000000002, 2024-09-18T04:57:42.767 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.767 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.767 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.767 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.767 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.767 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60596 2024-09-18T04:57:42.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.768 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:42.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:42.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:42.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:42.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122175+0000", 2024-09-18T04:57:42.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:57:42.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122182+0000", 2024-09-18T04:57:42.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3509999999999996e-06 2024-09-18T04:57:42.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122187+0000", 2024-09-18T04:57:42.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7940000000000002e-06 2024-09-18T04:57:42.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166184+0000", 2024-09-18T04:57:42.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043997236000000002 2024-09-18T04:57:42.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166227+0000", 2024-09-18T04:57:42.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2561000000000001e-05 2024-09-18T04:57:42.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166294+0000", 2024-09-18T04:57:42.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7068000000000002e-05 2024-09-18T04:57:42.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.771 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.771 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60597 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1395712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:42.771 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:42.771 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.552957554999999, 2024-09-18T04:57:42.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044242166999999999, 2024-09-18T04:57:42.771 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.772 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.772 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60597 2024-09-18T04:57:42.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.772 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:42.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:42.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:42.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:42.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122193+0000", 2024-09-18T04:57:42.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:57:42.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122199+0000", 2024-09-18T04:57:42.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6449999999999999e-06 2024-09-18T04:57:42.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122204+0000", 2024-09-18T04:57:42.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4819999999999996e-06 2024-09-18T04:57:42.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166329+0000", 2024-09-18T04:57:42.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044125059000000001 2024-09-18T04:57:42.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166370+0000", 2024-09-18T04:57:42.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1012e-05 2024-09-18T04:57:42.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166433+0000", 2024-09-18T04:57:42.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3178999999999999e-05 2024-09-18T04:57:42.775 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.775 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.775 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.776 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60608 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1564672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:42.776 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:57:42.776 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.550959693000003, 2024-09-18T04:57:42.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044082723999999997, 2024-09-18T04:57:42.776 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:42.776 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:42.776 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:42.776 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:42.776 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:42.776 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60608 2024-09-18T04:57:42.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.776 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:42.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:42.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:57:42.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:42.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:57:42.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:42.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:42.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124188+0000", 2024-09-18T04:57:42.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:42.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:42.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124190+0000", 2024-09-18T04:57:42.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0329999999999998e-06 2024-09-18T04:57:42.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:42.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124196+0000", 2024-09-18T04:57:42.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.575e-06 2024-09-18T04:57:42.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:42.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124201+0000", 2024-09-18T04:57:42.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5000000000000001e-06 2024-09-18T04:57:42.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:42.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168166+0000", 2024-09-18T04:57:42.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043965763999999997 2024-09-18T04:57:42.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:42.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168218+0000", 2024-09-18T04:57:42.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1785e-05 2024-09-18T04:57:42.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:42.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:42.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:42.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168271+0000", 2024-09-18T04:57:42.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3429e-05 2024-09-18T04:57:42.780 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.780 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.780 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.780 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:42.780 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:42.780 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:42.781 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:57:42.782 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:42.779+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:42.882 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:42.879+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:42.895 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:42.895 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:42.895 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:57:42.895 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:42.904 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:57:42.987 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:42.985+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:43.066 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:43.067 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:43.067 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:57:43.067 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:57:43.067 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:43.076 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:57:43.084 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:43.082+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:43.186 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:43.184+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:43.221 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26189 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 237568~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.578056105000002, 2024-09-18T04:57:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022263431, 2024-09-18T04:57:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:43.221 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26189 2024-09-18T04:57:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:43.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639787+0000", 2024-09-18T04:57:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9759999999999998e-06 2024-09-18T04:57:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639792+0000", 2024-09-18T04:57:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5829999999999997e-06 2024-09-18T04:57:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639796+0000", 2024-09-18T04:57:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0690000000000003e-06 2024-09-18T04:57:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.661944+0000", 2024-09-18T04:57:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022147463999999999 2024-09-18T04:57:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.661985+0000", 2024-09-18T04:57:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1360999999999999e-05 2024-09-18T04:57:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662048+0000", 2024-09-18T04:57:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3298000000000003e-05 2024-09-18T04:57:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.225 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26199 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 4096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.578040819000002, 2024-09-18T04:57:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022348776000000001, 2024-09-18T04:57:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26199 2024-09-18T04:57:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:43.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:43.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:43.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:43.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:43.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:43.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:43.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:43.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639806+0000", 2024-09-18T04:57:43.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1539999999999999e-06 2024-09-18T04:57:43.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:43.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639812+0000", 2024-09-18T04:57:43.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5520000000000004e-06 2024-09-18T04:57:43.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:43.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639816+0000", 2024-09-18T04:57:43.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1840000000000001e-06 2024-09-18T04:57:43.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:43.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662063+0000", 2024-09-18T04:57:43.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022247125999999999 2024-09-18T04:57:43.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.229 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:43.229 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662098+0000", 2024-09-18T04:57:43.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4968000000000001e-05 2024-09-18T04:57:43.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:43.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662149+0000", 2024-09-18T04:57:43.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1122000000000003e-05 2024-09-18T04:57:43.230 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.230 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:43.230 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.231 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26200 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 32768~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:43.231 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:43.231 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.578021294000003, 2024-09-18T04:57:43.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022501118000000001, 2024-09-18T04:57:43.231 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:43.231 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:43.231 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:43.231 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:43.231 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:43.231 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26200 2024-09-18T04:57:43.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.232 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:43.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:43.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:43.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:43.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:43.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:43.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639822+0000", 2024-09-18T04:57:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.007e-06 2024-09-18T04:57:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639827+0000", 2024-09-18T04:57:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5709999999999997e-06 2024-09-18T04:57:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:43.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639831+0000", 2024-09-18T04:57:43.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2819999999999998e-06 2024-09-18T04:57:43.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:43.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662181+0000", 2024-09-18T04:57:43.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022349878 2024-09-18T04:57:43.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:43.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662252+0000", 2024-09-18T04:57:43.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0542000000000003e-05 2024-09-18T04:57:43.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:43.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662321+0000", 2024-09-18T04:57:43.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9167999999999999e-05 2024-09-18T04:57:43.235 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.235 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:43.235 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.235 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26201 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 61440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:43.236 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:43.236 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.578006002999999, 2024-09-18T04:57:43.237 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022569783, 2024-09-18T04:57:43.237 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:43.237 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:43.237 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:43.237 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:43.237 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:43.237 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26201 2024-09-18T04:57:43.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.237 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:43.237 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.237 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:43.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:43.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:43.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:43.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:43.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:43.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:43.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.239 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:43.239 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639837+0000", 2024-09-18T04:57:43.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.928e-06 2024-09-18T04:57:43.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:43.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639843+0000", 2024-09-18T04:57:43.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8130000000000001e-06 2024-09-18T04:57:43.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:43.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639847+0000", 2024-09-18T04:57:43.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0999999999999997e-06 2024-09-18T04:57:43.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:43.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662335+0000", 2024-09-18T04:57:43.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022488207999999999 2024-09-18T04:57:43.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:43.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662370+0000", 2024-09-18T04:57:43.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5292e-05 2024-09-18T04:57:43.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:43.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662405+0000", 2024-09-18T04:57:43.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4765999999999999e-05 2024-09-18T04:57:43.241 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.241 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:43.241 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.242 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26202 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 90112~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:43.242 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:57:43.242 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.577990585999999, 2024-09-18T04:57:43.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022655937000000001, 2024-09-18T04:57:43.242 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:43.242 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:43.242 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:43.242 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:43.242 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:43.242 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26202 2024-09-18T04:57:43.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.242 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:43.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:57:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:57:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639850+0000", 2024-09-18T04:57:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:43.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:43.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639852+0000", 2024-09-18T04:57:43.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9700000000000002e-06 2024-09-18T04:57:43.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:43.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639858+0000", 2024-09-18T04:57:43.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.434e-06 2024-09-18T04:57:43.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:43.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639862+0000", 2024-09-18T04:57:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3499999999999999e-06 2024-09-18T04:57:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662429+0000", 2024-09-18T04:57:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022567409 2024-09-18T04:57:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662469+0000", 2024-09-18T04:57:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9801000000000001e-05 2024-09-18T04:57:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:43.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662507+0000", 2024-09-18T04:57:43.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7311000000000002e-05 2024-09-18T04:57:43.246 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.246 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:43.246 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.246 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26203 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 118784~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:43.246 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:57:43.246 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.577975430999999, 2024-09-18T04:57:43.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022730358999999999, 2024-09-18T04:57:43.246 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:43.246 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:43.246 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:43.247 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:43.247 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:43.247 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26203 2024-09-18T04:57:43.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.247 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:43.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:43.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:57:43.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:43.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:57:43.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639865+0000", 2024-09-18T04:57:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639867+0000", 2024-09-18T04:57:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0889999999999998e-06 2024-09-18T04:57:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.248 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:43.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639873+0000", 2024-09-18T04:57:43.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4979999999999997e-06 2024-09-18T04:57:43.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:43.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639877+0000", 2024-09-18T04:57:43.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4510000000000002e-06 2024-09-18T04:57:43.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:43.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662528+0000", 2024-09-18T04:57:43.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022650791999999999 2024-09-18T04:57:43.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662563+0000", 2024-09-18T04:57:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4530000000000003e-05 2024-09-18T04:57:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662596+0000", 2024-09-18T04:57:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3463000000000001e-05 2024-09-18T04:57:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.250 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26204 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 147456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.577960038999997, 2024-09-18T04:57:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022803233999999999, 2024-09-18T04:57:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26204 2024-09-18T04:57:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:43.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639883+0000", 2024-09-18T04:57:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6520000000000002e-06 2024-09-18T04:57:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639888+0000", 2024-09-18T04:57:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2870000000000004e-06 2024-09-18T04:57:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639893+0000", 2024-09-18T04:57:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3839999999999999e-06 2024-09-18T04:57:43.253 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662617+0000", 2024-09-18T04:57:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022723876 2024-09-18T04:57:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662651+0000", 2024-09-18T04:57:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3797000000000002e-05 2024-09-18T04:57:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662684+0000", 2024-09-18T04:57:43.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3828999999999997e-05 2024-09-18T04:57:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26205 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 176128~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.577937077999998, 2024-09-18T04:57:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022919108000000001, 2024-09-18T04:57:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:43.255 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26205 2024-09-18T04:57:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639906+0000", 2024-09-18T04:57:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.531e-06 2024-09-18T04:57:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639913+0000", 2024-09-18T04:57:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.331e-06 2024-09-18T04:57:43.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639920+0000", 2024-09-18T04:57:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5399999999999998e-06 2024-09-18T04:57:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662706+0000", 2024-09-18T04:57:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.02278587 2024-09-18T04:57:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662764+0000", 2024-09-18T04:57:43.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7630000000000002e-05 2024-09-18T04:57:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662823+0000", 2024-09-18T04:57:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9648000000000002e-05 2024-09-18T04:57:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26206 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 204800~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.577916678999998, 2024-09-18T04:57:43.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023047517, 2024-09-18T04:57:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26206 2024-09-18T04:57:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639926+0000", 2024-09-18T04:57:43.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3120000000000001e-06 2024-09-18T04:57:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639932+0000", 2024-09-18T04:57:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4890000000000003e-06 2024-09-18T04:57:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639936+0000", 2024-09-18T04:57:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.07e-06 2024-09-18T04:57:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662850+0000", 2024-09-18T04:57:43.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022913579999999999 2024-09-18T04:57:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662903+0000", 2024-09-18T04:57:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3176e-05 2024-09-18T04:57:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662972+0000", 2024-09-18T04:57:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9270999999999995e-05 2024-09-18T04:57:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26207 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 233472~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:57:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.577901535000002, 2024-09-18T04:57:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023139541999999999, 2024-09-18T04:57:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26207 2024-09-18T04:57:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:43.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:57:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:57:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639939+0000", 2024-09-18T04:57:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:43.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639941+0000", 2024-09-18T04:57:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0080000000000001e-06 2024-09-18T04:57:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639947+0000", 2024-09-18T04:57:43.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.485e-06 2024-09-18T04:57:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639951+0000", 2024-09-18T04:57:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4549999999999997e-06 2024-09-18T04:57:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663005+0000", 2024-09-18T04:57:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023053569999999999 2024-09-18T04:57:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:43.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663042+0000", 2024-09-18T04:57:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7245e-05 2024-09-18T04:57:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663079+0000", 2024-09-18T04:57:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7119000000000002e-05 2024-09-18T04:57:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26208 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 524288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:43.268 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.577884787000002, 2024-09-18T04:57:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023283564999999999, 2024-09-18T04:57:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26208 2024-09-18T04:57:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639958+0000", 2024-09-18T04:57:43.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0899999999999999e-06 2024-09-18T04:57:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639963+0000", 2024-09-18T04:57:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2209999999999996e-06 2024-09-18T04:57:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639967+0000", 2024-09-18T04:57:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1169999999999997e-06 2024-09-18T04:57:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:43.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663114+0000", 2024-09-18T04:57:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023146102000000002 2024-09-18T04:57:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663183+0000", 2024-09-18T04:57:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9686999999999999e-05 2024-09-18T04:57:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663240+0000", 2024-09-18T04:57:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6728000000000003e-05 2024-09-18T04:57:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:43.272 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26209 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 552960~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.577867011000002, 2024-09-18T04:57:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023440966000000001, 2024-09-18T04:57:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26209 2024-09-18T04:57:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.273 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:43.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639976+0000", 2024-09-18T04:57:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4940000000000002e-06 2024-09-18T04:57:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639982+0000", 2024-09-18T04:57:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0560000000000001e-06 2024-09-18T04:57:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639987+0000", 2024-09-18T04:57:43.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-18T04:57:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663278+0000", 2024-09-18T04:57:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023290789999999999 2024-09-18T04:57:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663348+0000", 2024-09-18T04:57:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0837e-05 2024-09-18T04:57:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663415+0000", 2024-09-18T04:57:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6736000000000006e-05 2024-09-18T04:57:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26210 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 581632~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.577850003000002, 2024-09-18T04:57:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023616437000000001, 2024-09-18T04:57:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:43.277 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26210 2024-09-18T04:57:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639993+0000", 2024-09-18T04:57:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.419e-06 2024-09-18T04:57:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640006+0000", 2024-09-18T04:57:43.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2918000000000001e-05 2024-09-18T04:57:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640011+0000", 2024-09-18T04:57:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9320000000000004e-06 2024-09-18T04:57:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663454+0000", 2024-09-18T04:57:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023443035000000001 2024-09-18T04:57:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:43.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663532+0000", 2024-09-18T04:57:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7443999999999999e-05 2024-09-18T04:57:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663608+0000", 2024-09-18T04:57:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6093000000000005e-05 2024-09-18T04:57:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26211 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 610304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:57:43.281 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.577825451000002, 2024-09-18T04:57:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023774337999999999, 2024-09-18T04:57:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26211 2024-09-18T04:57:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:57:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:57:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640015+0000", 2024-09-18T04:57:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640023+0000", 2024-09-18T04:57:43.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3900000000000004e-06 2024-09-18T04:57:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640036+0000", 2024-09-18T04:57:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3091999999999999e-05 2024-09-18T04:57:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640041+0000", 2024-09-18T04:57:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6219999999999997e-06 2024-09-18T04:57:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:43.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663644+0000", 2024-09-18T04:57:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023602643999999999 2024-09-18T04:57:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663718+0000", 2024-09-18T04:57:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4481999999999996e-05 2024-09-18T04:57:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663790+0000", 2024-09-18T04:57:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1589999999999997e-05 2024-09-18T04:57:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:43.285 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26212 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 638976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:57:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.577792410999997, 2024-09-18T04:57:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023924748999999999, 2024-09-18T04:57:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26212 2024-09-18T04:57:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.286 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:57:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:57:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640048+0000", 2024-09-18T04:57:43.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640051+0000", 2024-09-18T04:57:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.095e-06 2024-09-18T04:57:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640060+0000", 2024-09-18T04:57:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8929999999999994e-06 2024-09-18T04:57:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:43.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640065+0000", 2024-09-18T04:57:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0270000000000003e-06 2024-09-18T04:57:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663824+0000", 2024-09-18T04:57:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023758873999999999 2024-09-18T04:57:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663904+0000", 2024-09-18T04:57:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9856000000000003e-05 2024-09-18T04:57:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663974+0000", 2024-09-18T04:57:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9450999999999996e-05 2024-09-18T04:57:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26213 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 667648~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.577771239999997, 2024-09-18T04:57:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024059812999999999, 2024-09-18T04:57:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:43.290 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26213 2024-09-18T04:57:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:43.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640072+0000", 2024-09-18T04:57:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4200000000000001e-06 2024-09-18T04:57:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640078+0000", 2024-09-18T04:57:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0179999999999998e-06 2024-09-18T04:57:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640083+0000", 2024-09-18T04:57:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:57:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664008+0000", 2024-09-18T04:57:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023925287 2024-09-18T04:57:43.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664059+0000", 2024-09-18T04:57:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0730999999999999e-05 2024-09-18T04:57:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664130+0000", 2024-09-18T04:57:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0801000000000002e-05 2024-09-18T04:57:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26214 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 696320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.57775427, 2024-09-18T04:57:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024223738000000002, 2024-09-18T04:57:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26214 2024-09-18T04:57:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:43.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640089+0000", 2024-09-18T04:57:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0849999999999999e-06 2024-09-18T04:57:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640095+0000", 2024-09-18T04:57:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9279999999999998e-06 2024-09-18T04:57:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640099+0000", 2024-09-18T04:57:43.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:57:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664166+0000", 2024-09-18T04:57:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024066513000000001 2024-09-18T04:57:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664239+0000", 2024-09-18T04:57:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3201999999999995e-05 2024-09-18T04:57:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:43.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664311+0000", 2024-09-18T04:57:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1434999999999996e-05 2024-09-18T04:57:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26215 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 724992~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:57:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.577737509000002, 2024-09-18T04:57:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024302284, 2024-09-18T04:57:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:43.299 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26215 2024-09-18T04:57:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:57:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:57:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640103+0000", 2024-09-18T04:57:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640106+0000", 2024-09-18T04:57:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2120000000000002e-06 2024-09-18T04:57:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640111+0000", 2024-09-18T04:57:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7880000000000003e-06 2024-09-18T04:57:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640116+0000", 2024-09-18T04:57:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6099999999999999e-06 2024-09-18T04:57:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664327+0000", 2024-09-18T04:57:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024210941999999999 2024-09-18T04:57:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:43.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664367+0000", 2024-09-18T04:57:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9666000000000003e-05 2024-09-18T04:57:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664406+0000", 2024-09-18T04:57:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9402e-05 2024-09-18T04:57:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26216 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 753664~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:43.303 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.577721015000002, 2024-09-18T04:57:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024452854999999999, 2024-09-18T04:57:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26216 2024-09-18T04:57:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640122+0000", 2024-09-18T04:57:43.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3599999999999999e-06 2024-09-18T04:57:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640128+0000", 2024-09-18T04:57:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6339999999999998e-06 2024-09-18T04:57:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640132+0000", 2024-09-18T04:57:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:57:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:43.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664436+0000", 2024-09-18T04:57:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024303585999999999 2024-09-18T04:57:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664507+0000", 2024-09-18T04:57:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1046999999999999e-05 2024-09-18T04:57:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664573+0000", 2024-09-18T04:57:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6217000000000007e-05 2024-09-18T04:57:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.307 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26217 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 782336~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 62.577704781000001, 2024-09-18T04:57:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024526287000000001, 2024-09-18T04:57:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26217 2024-09-18T04:57:43.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:43.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640140+0000", 2024-09-18T04:57:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2989999999999998e-06 2024-09-18T04:57:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640146+0000", 2024-09-18T04:57:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9569999999999999e-06 2024-09-18T04:57:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:43.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640151+0000", 2024-09-18T04:57:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:57:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664590+0000", 2024-09-18T04:57:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0244389 2024-09-18T04:57:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664628+0000", 2024-09-18T04:57:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7910999999999997e-05 2024-09-18T04:57:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664663+0000", 2024-09-18T04:57:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5024999999999998e-05 2024-09-18T04:57:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.312 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:43.312 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:43.313 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:57:43.314 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:57:43.314 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:43.311+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:43.414 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:43.412+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:43.435 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:57:43.435 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:57:43.452 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:57:43.452 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:57:43.457 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:162105 3.6 3.b74753e6 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e93)", 2024-09-18T04:57:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:43.048686+0000", 2024-09-18T04:57:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.40748731900000001, 2024-09-18T04:57:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.40750789100000001, 2024-09-18T04:57:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "waiting for peered", 2024-09-18T04:57:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 162105 2024-09-18T04:57:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.048686+0000", 2024-09-18T04:57:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.048686+0000", 2024-09-18T04:57:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.048685+0000", 2024-09-18T04:57:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:57:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.048691+0000", 2024-09-18T04:57:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7379999999999999e-06 2024-09-18T04:57:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.048702+0000", 2024-09-18T04:57:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1099e-05 2024-09-18T04:57:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.048710+0000", 2024-09-18T04:57:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1210000000000007e-06 2024-09-18T04:57:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.048747+0000", 2024-09-18T04:57:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7345999999999998e-05 2024-09-18T04:57:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:57:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.048755+0000", 2024-09-18T04:57:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0740000000000004e-06 2024-09-18T04:57:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-18T04:57:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 1 2024-09-18T04:57:43.462 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:43.466 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:57:43.468 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:57:43.468 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:57:43.484 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:57:43.484 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:57:43.501 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:57:43.501 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:57:43.516 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:43.513+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:43.517 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:57:43.517 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:57:43.538 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:57:43.538 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:57:43.551 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:57:43.551 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:57:43.572 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T04:57:43.609 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:43.609 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:43.617 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:57:43.645 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:43.643+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:43.717 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:43.715+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:43.720 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:43.720 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:43.728 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:57:43.819 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:43.816+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:43.835 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:43.836 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:43.848 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:57:43.920 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:43.918+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:43.954 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:43.955 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:43.963 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:57:44.011 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-18T04:57:44.020 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:44.018+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:44.075 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:44.076 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:44.084 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:57:44.128 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:44.126+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:44.191 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:44.192 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:44.200 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:57:44.241 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:44.239+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:44.323 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:44.321+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:44.347 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:44.347 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:44.355 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:57:44.424 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:44.421+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:44.462 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:44.462 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:44.471 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:57:44.485 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 5, 6, 7] out_osds: [1, 4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T04:57:44.485 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T04:57:44.485 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-18T04:57:44.486 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-18T04:57:44.525 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:44.522+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:44.580 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:44.580 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:44.589 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:57:44.635 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:44.633+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:44.654 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-18T04:57:44.699 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:44.699 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:44.708 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:57:44.727 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:44.725+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:44.828 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:44.826+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:44.833 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-18T04:57:44.858 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:44.858 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:44.867 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:57:44.930 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:44.928+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:44.973 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-18T04:57:45.018 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:45.019 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:45.029 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:57:45.030 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:45.027+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:45.124 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-18T04:57:45.145 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:45.145 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:45.145 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:45.143+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:45.155 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:57:45.232 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:45.229+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:45.262 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-18T04:57:45.304 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:45.304 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:45.313 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:57:45.356 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:45.354+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:45.419 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:45.419 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:45.427 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:57:45.433 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:45.431+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:45.534 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:45.532+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:45.562 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:57:45.572 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:45.572 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:45.581 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:57:45.664 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:45.662+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:45.686 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:57:45.686 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:57:45.702 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:57:45.703 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:57:45.718 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:57:45.718 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:57:45.723 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:45.723 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:45.731 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:57:45.735 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:57:45.735 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:57:45.751 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:57:45.751 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:57:45.767 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:57:45.768 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:57:45.773 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:45.770+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:45.784 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:57:45.784 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:57:45.800 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:57:46.267 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:57:46.268 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T04:57:46.269 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:46.269 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:46.269 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:57:46.269 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:46.269 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:57:46.270 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:46.266+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:46.370 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:46.367+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:46.372 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:46.372 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:46.372 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:57:46.372 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:57:46.372 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:46.380 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:57:46.471 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:46.469+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:46.524 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:46.524 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60344 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1456128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.409216241999999, 2024-09-18T04:57:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044026324999999998, 2024-09-18T04:57:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:46.525 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60344 2024-09-18T04:57:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112224+0000", 2024-09-18T04:57:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1320000000000001e-06 2024-09-18T04:57:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112230+0000", 2024-09-18T04:57:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9619999999999998e-06 2024-09-18T04:57:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:46.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112234+0000", 2024-09-18T04:57:46.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7890000000000002e-06 2024-09-18T04:57:46.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:46.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156105+0000", 2024-09-18T04:57:46.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043870531999999997 2024-09-18T04:57:46.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156170+0000", 2024-09-18T04:57:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5036000000000005e-05 2024-09-18T04:57:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156248+0000", 2024-09-18T04:57:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8295999999999997e-05 2024-09-18T04:57:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.529 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60345 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1471488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:57:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.409199477000001, 2024-09-18T04:57:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044180090999999998, 2024-09-18T04:57:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60345 2024-09-18T04:57:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:46.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:57:46.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:46.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:57:46.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:46.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112238+0000", 2024-09-18T04:57:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112240+0000", 2024-09-18T04:57:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9910000000000001e-06 2024-09-18T04:57:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112246+0000", 2024-09-18T04:57:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0859999999999998e-06 2024-09-18T04:57:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:46.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112251+0000", 2024-09-18T04:57:46.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5020000000000003e-06 2024-09-18T04:57:46.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:46.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156274+0000", 2024-09-18T04:57:46.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044023063000000001 2024-09-18T04:57:46.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:46.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156337+0000", 2024-09-18T04:57:46.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3077000000000006e-05 2024-09-18T04:57:46.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:46.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156419+0000", 2024-09-18T04:57:46.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1699000000000005e-05 2024-09-18T04:57:46.534 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.534 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:46.534 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.534 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60346 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1486848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:46.534 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:57:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.409181695999997, 2024-09-18T04:57:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044336331999999999, 2024-09-18T04:57:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60346 2024-09-18T04:57:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:57:46.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:57:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112256+0000", 2024-09-18T04:57:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:46.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112258+0000", 2024-09-18T04:57:46.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1720000000000001e-06 2024-09-18T04:57:46.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:46.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112265+0000", 2024-09-18T04:57:46.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4280000000000001e-06 2024-09-18T04:57:46.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:46.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112269+0000", 2024-09-18T04:57:46.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.741e-06 2024-09-18T04:57:46.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156445+0000", 2024-09-18T04:57:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044175341 2024-09-18T04:57:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156512+0000", 2024-09-18T04:57:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6723000000000005e-05 2024-09-18T04:57:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156593+0000", 2024-09-18T04:57:46.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1284000000000003e-05 2024-09-18T04:57:46.539 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.539 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:46.539 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.539 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60347 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1502208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:46.539 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:57:46.539 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.409153285000002, 2024-09-18T04:57:46.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044482560999999997, 2024-09-18T04:57:46.539 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:46.539 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:46.539 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:46.539 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:46.540 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:46.540 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60347 2024-09-18T04:57:46.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.540 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:46.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:46.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:57:46.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:46.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:57:46.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:46.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112284+0000", 2024-09-18T04:57:46.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:46.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:46.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112286+0000", 2024-09-18T04:57:46.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9709999999999998e-06 2024-09-18T04:57:46.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:46.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112290+0000", 2024-09-18T04:57:46.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9199999999999997e-06 2024-09-18T04:57:46.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:46.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112293+0000", 2024-09-18T04:57:46.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870000000000001e-06 2024-09-18T04:57:46.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:46.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156617+0000", 2024-09-18T04:57:46.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044323960000000003 2024-09-18T04:57:46.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:46.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156687+0000", 2024-09-18T04:57:46.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0141000000000005e-05 2024-09-18T04:57:46.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:46.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156767+0000", 2024-09-18T04:57:46.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0140000000000002e-05 2024-09-18T04:57:46.546 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.547 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:46.547 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.547 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60348 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1517568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:46.547 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:46.547 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.409142281000001, 2024-09-18T04:57:46.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044645562, 2024-09-18T04:57:46.547 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:46.548 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:46.548 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:46.548 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:46.548 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:46.548 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60348 2024-09-18T04:57:46.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.548 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:46.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:46.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:46.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:46.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:46.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:46.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:46.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:46.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:46.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112297+0000", 2024-09-18T04:57:46.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.384e-06 2024-09-18T04:57:46.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:46.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112301+0000", 2024-09-18T04:57:46.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0570000000000003e-06 2024-09-18T04:57:46.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:46.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112304+0000", 2024-09-18T04:57:46.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6000000000000001e-06 2024-09-18T04:57:46.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:46.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156793+0000", 2024-09-18T04:57:46.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044489494999999997 2024-09-18T04:57:46.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:46.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156859+0000", 2024-09-18T04:57:46.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5483999999999998e-05 2024-09-18T04:57:46.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:46.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156941+0000", 2024-09-18T04:57:46.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2856999999999996e-05 2024-09-18T04:57:46.554 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.554 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:46.554 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.554 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60349 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1532928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:46.554 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:46.554 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.409132305, 2024-09-18T04:57:46.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044832167999999999, 2024-09-18T04:57:46.554 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:46.554 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:46.554 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:46.554 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:46.555 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:46.555 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60349 2024-09-18T04:57:46.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.555 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:46.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:46.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:46.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:46.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:46.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:46.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:46.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:46.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:46.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112307+0000", 2024-09-18T04:57:46.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.308e-06 2024-09-18T04:57:46.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:46.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112310+0000", 2024-09-18T04:57:46.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-18T04:57:46.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112312+0000", 2024-09-18T04:57:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4889999999999998e-06 2024-09-18T04:57:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156977+0000", 2024-09-18T04:57:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044664598999999999 2024-09-18T04:57:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157053+0000", 2024-09-18T04:57:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6014999999999996e-05 2024-09-18T04:57:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:46.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157138+0000", 2024-09-18T04:57:46.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5162999999999998e-05 2024-09-18T04:57:46.559 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.559 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:46.559 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.559 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60350 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1548288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:46.560 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:46.560 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.409123391000001, 2024-09-18T04:57:46.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045011858000000002, 2024-09-18T04:57:46.560 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:46.560 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:46.560 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:46.560 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:46.560 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:46.560 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60350 2024-09-18T04:57:46.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.560 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:46.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:46.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:46.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:46.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:46.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:46.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:46.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:46.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:46.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112316+0000", 2024-09-18T04:57:46.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7579999999999999e-06 2024-09-18T04:57:46.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:46.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112321+0000", 2024-09-18T04:57:46.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1649999999999997e-06 2024-09-18T04:57:46.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:46.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112325+0000", 2024-09-18T04:57:46.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.726e-06 2024-09-18T04:57:46.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:46.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157172+0000", 2024-09-18T04:57:46.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.04484718 2024-09-18T04:57:46.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.563 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:46.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157242+0000", 2024-09-18T04:57:46.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9675e-05 2024-09-18T04:57:46.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:46.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157327+0000", 2024-09-18T04:57:46.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4614e-05 2024-09-18T04:57:46.565 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.565 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:46.565 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.566 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60351 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1563648~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:46.566 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:46.567 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.409109975, 2024-09-18T04:57:46.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045132113000000001, 2024-09-18T04:57:46.567 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:46.567 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:46.567 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:46.567 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:46.567 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:46.567 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60351 2024-09-18T04:57:46.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.567 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:46.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:46.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:46.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:46.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:46.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:46.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:46.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:46.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:46.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112330+0000", 2024-09-18T04:57:46.569 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1660000000000001e-06 2024-09-18T04:57:46.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:46.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112336+0000", 2024-09-18T04:57:46.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5940000000000001e-06 2024-09-18T04:57:46.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:46.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112340+0000", 2024-09-18T04:57:46.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5589999999999999e-06 2024-09-18T04:57:46.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:46.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157365+0000", 2024-09-18T04:57:46.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045024675 2024-09-18T04:57:46.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:46.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157409+0000", 2024-09-18T04:57:46.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4356e-05 2024-09-18T04:57:46.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:46.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157460+0000", 2024-09-18T04:57:46.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1072000000000002e-05 2024-09-18T04:57:46.573 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.573 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:46.573 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.574 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60501 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1494016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:46.574 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:46.574 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.403049279000001, 2024-09-18T04:57:46.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044007728000000003, 2024-09-18T04:57:46.574 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:46.574 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:46.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:46.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:46.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:46.574 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60501 2024-09-18T04:57:46.574 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.575 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:46.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:46.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:46.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:46.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:46.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:46.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:46.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:46.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:46.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118391+0000", 2024-09-18T04:57:46.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:57:46.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:46.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118396+0000", 2024-09-18T04:57:46.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2680000000000002e-06 2024-09-18T04:57:46.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:46.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118400+0000", 2024-09-18T04:57:46.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2080000000000002e-06 2024-09-18T04:57:46.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:46.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162317+0000", 2024-09-18T04:57:46.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043916753000000003 2024-09-18T04:57:46.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:46.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162351+0000", 2024-09-18T04:57:46.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4372000000000002e-05 2024-09-18T04:57:46.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:46.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162397+0000", 2024-09-18T04:57:46.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5207999999999998e-05 2024-09-18T04:57:46.578 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.578 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:46.578 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.578 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60502 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1509376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:46.578 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:46.578 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.403033802000003, 2024-09-18T04:57:46.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044092438999999997, 2024-09-18T04:57:46.578 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:46.578 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:46.579 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:46.579 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:46.579 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:46.579 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60502 2024-09-18T04:57:46.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.579 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:46.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:46.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:46.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:46.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:46.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:46.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:46.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:46.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.580 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:46.580 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118406+0000", 2024-09-18T04:57:46.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:57:46.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:46.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118412+0000", 2024-09-18T04:57:46.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.524e-06 2024-09-18T04:57:46.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:46.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118416+0000", 2024-09-18T04:57:46.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:57:46.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:46.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162418+0000", 2024-09-18T04:57:46.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001508000000002 2024-09-18T04:57:46.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:46.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162451+0000", 2024-09-18T04:57:46.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3662999999999999e-05 2024-09-18T04:57:46.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:46.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162497+0000", 2024-09-18T04:57:46.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5550000000000003e-05 2024-09-18T04:57:46.582 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.582 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:46.582 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.583 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60503 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1524736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:46.583 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:57:46.583 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.403017644000002, 2024-09-18T04:57:46.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044178780000000001, 2024-09-18T04:57:46.583 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:46.583 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:46.583 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:46.583 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:46.583 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:46.583 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60503 2024-09-18T04:57:46.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.583 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:46.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:46.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:57:46.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:46.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:57:46.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:46.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118420+0000", 2024-09-18T04:57:46.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:46.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:46.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118422+0000", 2024-09-18T04:57:46.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.08e-06 2024-09-18T04:57:46.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:46.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118428+0000", 2024-09-18T04:57:46.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.964e-06 2024-09-18T04:57:46.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.585 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:46.585 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118433+0000", 2024-09-18T04:57:46.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.425e-06 2024-09-18T04:57:46.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:46.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162519+0000", 2024-09-18T04:57:46.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044086082999999998 2024-09-18T04:57:46.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:46.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162552+0000", 2024-09-18T04:57:46.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3719e-05 2024-09-18T04:57:46.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:46.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162599+0000", 2024-09-18T04:57:46.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6873999999999999e-05 2024-09-18T04:57:46.587 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.587 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:46.587 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.587 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60504 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1540096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:46.587 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:57:46.587 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.403001539000002, 2024-09-18T04:57:46.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044261921000000003, 2024-09-18T04:57:46.587 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:46.587 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:46.587 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:46.587 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:46.587 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:46.588 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60504 2024-09-18T04:57:46.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.588 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:46.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:46.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:57:46.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:46.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:57:46.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:46.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118436+0000", 2024-09-18T04:57:46.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:46.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.590 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:46.590 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118438+0000", 2024-09-18T04:57:46.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0250000000000001e-06 2024-09-18T04:57:46.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:46.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118444+0000", 2024-09-18T04:57:46.591 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4360000000000001e-06 2024-09-18T04:57:46.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:46.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118448+0000", 2024-09-18T04:57:46.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.357e-06 2024-09-18T04:57:46.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:46.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162620+0000", 2024-09-18T04:57:46.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044171821 2024-09-18T04:57:46.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:46.596 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162653+0000", 2024-09-18T04:57:46.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3479999999999998e-05 2024-09-18T04:57:46.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:46.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162699+0000", 2024-09-18T04:57:46.597 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5113e-05 2024-09-18T04:57:46.598 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.598 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:46.598 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.598 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60584 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1196032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:46.599 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:57:46.599 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.401208521999997, 2024-09-18T04:57:46.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044048727000000003, 2024-09-18T04:57:46.599 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:46.599 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:46.600 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:46.600 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:46.601 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:46.601 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60584 2024-09-18T04:57:46.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.602 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:46.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:46.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:57:46.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:46.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:57:46.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:46.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120229+0000", 2024-09-18T04:57:46.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:46.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:46.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120231+0000", 2024-09-18T04:57:46.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:57:46.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:46.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120237+0000", 2024-09-18T04:57:46.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8379999999999998e-06 2024-09-18T04:57:46.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:46.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120242+0000", 2024-09-18T04:57:46.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3889999999999998e-06 2024-09-18T04:57:46.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:46.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164187+0000", 2024-09-18T04:57:46.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043945045000000002 2024-09-18T04:57:46.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:46.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164227+0000", 2024-09-18T04:57:46.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0627999999999998e-05 2024-09-18T04:57:46.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:46.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164278+0000", 2024-09-18T04:57:46.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1146000000000001e-05 2024-09-18T04:57:46.610 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.610 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:46.610 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.610 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60585 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1211392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:46.610 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:57:46.610 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.401192414, 2024-09-18T04:57:46.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044389814, 2024-09-18T04:57:46.611 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:46.611 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:46.611 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:46.611 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:46.611 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:46.611 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60585 2024-09-18T04:57:46.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.611 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:46.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:46.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:57:46.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:46.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:57:46.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:46.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120245+0000", 2024-09-18T04:57:46.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:46.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:46.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120247+0000", 2024-09-18T04:57:46.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0150000000000002e-06 2024-09-18T04:57:46.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:46.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120255+0000", 2024-09-18T04:57:46.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.858e-06 2024-09-18T04:57:46.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:46.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120260+0000", 2024-09-18T04:57:46.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6530000000000002e-06 2024-09-18T04:57:46.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:46.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164489+0000", 2024-09-18T04:57:46.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044229483999999999 2024-09-18T04:57:46.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:46.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164555+0000", 2024-09-18T04:57:46.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.533e-05 2024-09-18T04:57:46.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:46.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164636+0000", 2024-09-18T04:57:46.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0819e-05 2024-09-18T04:57:46.616 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.616 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:46.616 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.617 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60586 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1226752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:46.617 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:46.617 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.401173962999998, 2024-09-18T04:57:46.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044578214999999997, 2024-09-18T04:57:46.617 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:46.617 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:46.617 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:46.617 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:46.617 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:46.617 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60586 2024-09-18T04:57:46.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.618 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:46.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:46.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:46.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:46.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:46.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:46.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:46.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:46.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:46.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120266+0000", 2024-09-18T04:57:46.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0660000000000002e-06 2024-09-18T04:57:46.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:46.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120272+0000", 2024-09-18T04:57:46.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-18T04:57:46.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:46.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120276+0000", 2024-09-18T04:57:46.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.476e-06 2024-09-18T04:57:46.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:46.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164673+0000", 2024-09-18T04:57:46.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044396672999999998 2024-09-18T04:57:46.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:46.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164749+0000", 2024-09-18T04:57:46.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5811999999999999e-05 2024-09-18T04:57:46.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:46.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164842+0000", 2024-09-18T04:57:46.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3560000000000005e-05 2024-09-18T04:57:46.623 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.623 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:46.623 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.625 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60587 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1242112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:46.625 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:57:46.625 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.401124864000003, 2024-09-18T04:57:46.625 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044634794999999998, 2024-09-18T04:57:46.625 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:46.625 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:46.625 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:46.625 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:46.625 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:46.625 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60587 2024-09-18T04:57:46.625 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.626 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:46.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.626 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:46.626 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:57:46.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:46.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:57:46.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:46.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120280+0000", 2024-09-18T04:57:46.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999671 2024-09-18T04:57:46.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:46.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120316+0000", 2024-09-18T04:57:46.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5728000000000001e-05 2024-09-18T04:57:46.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:46.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120329+0000", 2024-09-18T04:57:46.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2914e-05 2024-09-18T04:57:46.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:46.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120335+0000", 2024-09-18T04:57:46.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8289999999999996e-06 2024-09-18T04:57:46.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:46.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164858+0000", 2024-09-18T04:57:46.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044523417000000003 2024-09-18T04:57:46.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:46.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164896+0000", 2024-09-18T04:57:46.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7969999999999997e-05 2024-09-18T04:57:46.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.630 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:46.630 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164948+0000", 2024-09-18T04:57:46.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1996000000000002e-05 2024-09-18T04:57:46.630 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.630 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:46.631 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.631 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.631 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60595 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1364992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:46.631 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:57:46.631 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.399281647999999, 2024-09-18T04:57:46.631 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001360000000003, 2024-09-18T04:57:46.631 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:46.631 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:46.631 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:46.631 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:46.631 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:46.631 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60595 2024-09-18T04:57:46.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.632 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:46.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.632 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:46.632 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:57:46.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:46.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:57:46.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:46.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122156+0000", 2024-09-18T04:57:46.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:46.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.633 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:46.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122158+0000", 2024-09-18T04:57:46.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0040000000000002e-06 2024-09-18T04:57:46.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:46.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122164+0000", 2024-09-18T04:57:46.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8719999999999999e-06 2024-09-18T04:57:46.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:46.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122169+0000", 2024-09-18T04:57:46.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5090000000000004e-06 2024-09-18T04:57:46.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:46.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166040+0000", 2024-09-18T04:57:46.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043871226999999999 2024-09-18T04:57:46.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:46.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166096+0000", 2024-09-18T04:57:46.635 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6495e-05 2024-09-18T04:57:46.635 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.635 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.635 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:46.635 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166158+0000", 2024-09-18T04:57:46.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1626999999999998e-05 2024-09-18T04:57:46.636 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.636 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:46.636 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.636 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60596 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1380352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:46.636 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:46.636 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.399265327000002, 2024-09-18T04:57:46.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044120800000000002, 2024-09-18T04:57:46.637 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:46.637 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:46.637 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:46.637 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:46.637 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:46.637 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60596 2024-09-18T04:57:46.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.637 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:46.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:46.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:46.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:46.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:46.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:46.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:46.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:46.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:46.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122175+0000", 2024-09-18T04:57:46.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:57:46.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:46.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122182+0000", 2024-09-18T04:57:46.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3509999999999996e-06 2024-09-18T04:57:46.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:46.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122187+0000", 2024-09-18T04:57:46.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7940000000000002e-06 2024-09-18T04:57:46.640 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.640 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.640 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:46.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166184+0000", 2024-09-18T04:57:46.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043997236000000002 2024-09-18T04:57:46.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:46.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166227+0000", 2024-09-18T04:57:46.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2561000000000001e-05 2024-09-18T04:57:46.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:46.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166294+0000", 2024-09-18T04:57:46.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7068000000000002e-05 2024-09-18T04:57:46.642 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.642 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:46.642 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.642 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60597 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1395712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:46.642 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:46.642 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.399247299000002, 2024-09-18T04:57:46.643 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044242166999999999, 2024-09-18T04:57:46.643 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:46.643 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:46.643 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:46.643 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:46.643 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:46.643 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60597 2024-09-18T04:57:46.643 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.643 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:46.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:46.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:46.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:46.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:46.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.645 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:46.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:46.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:46.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.646 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:46.646 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122193+0000", 2024-09-18T04:57:46.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:57:46.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:46.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122199+0000", 2024-09-18T04:57:46.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6449999999999999e-06 2024-09-18T04:57:46.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:46.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122204+0000", 2024-09-18T04:57:46.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4819999999999996e-06 2024-09-18T04:57:46.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:46.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166329+0000", 2024-09-18T04:57:46.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044125059000000001 2024-09-18T04:57:46.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:46.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166370+0000", 2024-09-18T04:57:46.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1012e-05 2024-09-18T04:57:46.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:46.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166433+0000", 2024-09-18T04:57:46.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3178999999999999e-05 2024-09-18T04:57:46.649 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.649 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:46.649 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.649 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60608 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1564672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:46.650 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:57:46.650 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.397249436999999, 2024-09-18T04:57:46.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044082723999999997, 2024-09-18T04:57:46.650 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:46.650 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:46.650 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:46.651 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:46.651 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:46.652 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60608 2024-09-18T04:57:46.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.652 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:46.652 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.652 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:46.652 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:57:46.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.652 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:46.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:57:46.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:46.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:46.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124188+0000", 2024-09-18T04:57:46.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:46.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:46.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124190+0000", 2024-09-18T04:57:46.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0329999999999998e-06 2024-09-18T04:57:46.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:46.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124196+0000", 2024-09-18T04:57:46.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.575e-06 2024-09-18T04:57:46.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:46.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124201+0000", 2024-09-18T04:57:46.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5000000000000001e-06 2024-09-18T04:57:46.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:46.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168166+0000", 2024-09-18T04:57:46.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043965763999999997 2024-09-18T04:57:46.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:46.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168218+0000", 2024-09-18T04:57:46.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1785e-05 2024-09-18T04:57:46.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:46.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:46.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:46.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168271+0000", 2024-09-18T04:57:46.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3429e-05 2024-09-18T04:57:46.657 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.657 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:46.658 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.658 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:46.658 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:46.658 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:46.659 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:57:46.660 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:46.657+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:46.760 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:46.758+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:46.763 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:46.764 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:57:46.764 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:46.772 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:57:46.861 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:46.859+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:46.915 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:46.915 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:46.915 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:57:46.915 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:57:46.915 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:46.924 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:57:46.962 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:46.960+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:47.023 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-18T04:57:47.081 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:47.081 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26189 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 237568~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.438420162, 2024-09-18T04:57:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022263431, 2024-09-18T04:57:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.082 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26189 2024-09-18T04:57:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:47.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:57:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639787+0000", 2024-09-18T04:57:47.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9759999999999998e-06 2024-09-18T04:57:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639792+0000", 2024-09-18T04:57:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5829999999999997e-06 2024-09-18T04:57:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639796+0000", 2024-09-18T04:57:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0690000000000003e-06 2024-09-18T04:57:47.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.661944+0000", 2024-09-18T04:57:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022147463999999999 2024-09-18T04:57:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.661985+0000", 2024-09-18T04:57:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1360999999999999e-05 2024-09-18T04:57:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662048+0000", 2024-09-18T04:57:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3298000000000003e-05 2024-09-18T04:57:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26199 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 4096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:47.087 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.438404876000007, 2024-09-18T04:57:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022348776000000001, 2024-09-18T04:57:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26199 2024-09-18T04:57:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:47.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:57:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639806+0000", 2024-09-18T04:57:47.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1539999999999999e-06 2024-09-18T04:57:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639812+0000", 2024-09-18T04:57:47.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5520000000000004e-06 2024-09-18T04:57:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639816+0000", 2024-09-18T04:57:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1840000000000001e-06 2024-09-18T04:57:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662063+0000", 2024-09-18T04:57:47.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022247125999999999 2024-09-18T04:57:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662098+0000", 2024-09-18T04:57:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4968000000000001e-05 2024-09-18T04:57:47.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662149+0000", 2024-09-18T04:57:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1122000000000003e-05 2024-09-18T04:57:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26200 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 32768~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.438385350999994, 2024-09-18T04:57:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022501118000000001, 2024-09-18T04:57:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.095 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.096 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.096 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.096 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26200 2024-09-18T04:57:47.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.096 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:47.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:47.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:57:47.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639822+0000", 2024-09-18T04:57:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.007e-06 2024-09-18T04:57:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639827+0000", 2024-09-18T04:57:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5709999999999997e-06 2024-09-18T04:57:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639831+0000", 2024-09-18T04:57:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2819999999999998e-06 2024-09-18T04:57:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662181+0000", 2024-09-18T04:57:47.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022349878 2024-09-18T04:57:47.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662252+0000", 2024-09-18T04:57:47.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0542000000000003e-05 2024-09-18T04:57:47.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662321+0000", 2024-09-18T04:57:47.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9167999999999999e-05 2024-09-18T04:57:47.100 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.100 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.100 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.100 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26201 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 61440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:47.100 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:47.100 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.438370059999997, 2024-09-18T04:57:47.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022569783, 2024-09-18T04:57:47.100 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26201 2024-09-18T04:57:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:57:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639837+0000", 2024-09-18T04:57:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.928e-06 2024-09-18T04:57:47.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639843+0000", 2024-09-18T04:57:47.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8130000000000001e-06 2024-09-18T04:57:47.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639847+0000", 2024-09-18T04:57:47.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0999999999999997e-06 2024-09-18T04:57:47.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662335+0000", 2024-09-18T04:57:47.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022488207999999999 2024-09-18T04:57:47.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662370+0000", 2024-09-18T04:57:47.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5292e-05 2024-09-18T04:57:47.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662405+0000", 2024-09-18T04:57:47.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4765999999999999e-05 2024-09-18T04:57:47.104 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.105 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.105 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.105 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26202 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 90112~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:47.105 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:57:47.105 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.438354642999997, 2024-09-18T04:57:47.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022655937000000001, 2024-09-18T04:57:47.105 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.105 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.105 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26202 2024-09-18T04:57:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:57:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:57:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639850+0000", 2024-09-18T04:57:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639852+0000", 2024-09-18T04:57:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9700000000000002e-06 2024-09-18T04:57:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639858+0000", 2024-09-18T04:57:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.434e-06 2024-09-18T04:57:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639862+0000", 2024-09-18T04:57:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3499999999999999e-06 2024-09-18T04:57:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662429+0000", 2024-09-18T04:57:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022567409 2024-09-18T04:57:47.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662469+0000", 2024-09-18T04:57:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9801000000000001e-05 2024-09-18T04:57:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662507+0000", 2024-09-18T04:57:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7311000000000002e-05 2024-09-18T04:57:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26203 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 118784~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:57:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.438339487999997, 2024-09-18T04:57:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022730358999999999, 2024-09-18T04:57:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.110 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26203 2024-09-18T04:57:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:57:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:57:47.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639865+0000", 2024-09-18T04:57:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639867+0000", 2024-09-18T04:57:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0889999999999998e-06 2024-09-18T04:57:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639873+0000", 2024-09-18T04:57:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4979999999999997e-06 2024-09-18T04:57:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639877+0000", 2024-09-18T04:57:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4510000000000002e-06 2024-09-18T04:57:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662528+0000", 2024-09-18T04:57:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022650791999999999 2024-09-18T04:57:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662563+0000", 2024-09-18T04:57:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4530000000000003e-05 2024-09-18T04:57:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662596+0000", 2024-09-18T04:57:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3463000000000001e-05 2024-09-18T04:57:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.114 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26204 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 147456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.438324096000002, 2024-09-18T04:57:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022803233999999999, 2024-09-18T04:57:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26204 2024-09-18T04:57:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:57:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639883+0000", 2024-09-18T04:57:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6520000000000002e-06 2024-09-18T04:57:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639888+0000", 2024-09-18T04:57:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2870000000000004e-06 2024-09-18T04:57:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639893+0000", 2024-09-18T04:57:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3839999999999999e-06 2024-09-18T04:57:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662617+0000", 2024-09-18T04:57:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022723876 2024-09-18T04:57:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662651+0000", 2024-09-18T04:57:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3797000000000002e-05 2024-09-18T04:57:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662684+0000", 2024-09-18T04:57:47.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3828999999999997e-05 2024-09-18T04:57:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26205 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 176128~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:47.120 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.438301135000003, 2024-09-18T04:57:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022919108000000001, 2024-09-18T04:57:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26205 2024-09-18T04:57:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:57:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639906+0000", 2024-09-18T04:57:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.531e-06 2024-09-18T04:57:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639913+0000", 2024-09-18T04:57:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.331e-06 2024-09-18T04:57:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639920+0000", 2024-09-18T04:57:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5399999999999998e-06 2024-09-18T04:57:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662706+0000", 2024-09-18T04:57:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.02278587 2024-09-18T04:57:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662764+0000", 2024-09-18T04:57:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7630000000000002e-05 2024-09-18T04:57:47.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662823+0000", 2024-09-18T04:57:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9648000000000002e-05 2024-09-18T04:57:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26206 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 204800~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.438280735999996, 2024-09-18T04:57:47.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023047517, 2024-09-18T04:57:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26206 2024-09-18T04:57:47.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.127 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:47.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:47.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:57:47.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639926+0000", 2024-09-18T04:57:47.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3120000000000001e-06 2024-09-18T04:57:47.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639932+0000", 2024-09-18T04:57:47.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4890000000000003e-06 2024-09-18T04:57:47.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639936+0000", 2024-09-18T04:57:47.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.07e-06 2024-09-18T04:57:47.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662850+0000", 2024-09-18T04:57:47.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022913579999999999 2024-09-18T04:57:47.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662903+0000", 2024-09-18T04:57:47.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3176e-05 2024-09-18T04:57:47.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662972+0000", 2024-09-18T04:57:47.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9270999999999995e-05 2024-09-18T04:57:47.132 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.132 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.132 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.132 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26207 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 233472~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:47.132 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:57:47.132 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.438265591999993, 2024-09-18T04:57:47.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023139541999999999, 2024-09-18T04:57:47.133 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.133 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.133 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.133 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.133 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.133 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26207 2024-09-18T04:57:47.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.133 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:57:47.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:57:47.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639939+0000", 2024-09-18T04:57:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639941+0000", 2024-09-18T04:57:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0080000000000001e-06 2024-09-18T04:57:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639947+0000", 2024-09-18T04:57:47.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.485e-06 2024-09-18T04:57:47.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639951+0000", 2024-09-18T04:57:47.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4549999999999997e-06 2024-09-18T04:57:47.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663005+0000", 2024-09-18T04:57:47.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023053569999999999 2024-09-18T04:57:47.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663042+0000", 2024-09-18T04:57:47.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7245e-05 2024-09-18T04:57:47.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663079+0000", 2024-09-18T04:57:47.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7119000000000002e-05 2024-09-18T04:57:47.138 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.138 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.138 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.138 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26208 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 524288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:47.138 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.438248844, 2024-09-18T04:57:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023283564999999999, 2024-09-18T04:57:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26208 2024-09-18T04:57:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:47.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:47.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:57:47.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639958+0000", 2024-09-18T04:57:47.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0899999999999999e-06 2024-09-18T04:57:47.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639963+0000", 2024-09-18T04:57:47.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2209999999999996e-06 2024-09-18T04:57:47.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639967+0000", 2024-09-18T04:57:47.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1169999999999997e-06 2024-09-18T04:57:47.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663114+0000", 2024-09-18T04:57:47.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023146102000000002 2024-09-18T04:57:47.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663183+0000", 2024-09-18T04:57:47.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9686999999999999e-05 2024-09-18T04:57:47.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663240+0000", 2024-09-18T04:57:47.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6728000000000003e-05 2024-09-18T04:57:47.144 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.144 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.144 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.144 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26209 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 552960~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:47.145 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:47.145 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.438231067999993, 2024-09-18T04:57:47.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023440966000000001, 2024-09-18T04:57:47.145 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.145 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.145 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.145 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.145 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.145 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26209 2024-09-18T04:57:47.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.146 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:47.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:47.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:57:47.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639976+0000", 2024-09-18T04:57:47.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4940000000000002e-06 2024-09-18T04:57:47.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639982+0000", 2024-09-18T04:57:47.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0560000000000001e-06 2024-09-18T04:57:47.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639987+0000", 2024-09-18T04:57:47.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-18T04:57:47.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663278+0000", 2024-09-18T04:57:47.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023290789999999999 2024-09-18T04:57:47.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663348+0000", 2024-09-18T04:57:47.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0837e-05 2024-09-18T04:57:47.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663415+0000", 2024-09-18T04:57:47.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6736000000000006e-05 2024-09-18T04:57:47.150 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.150 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.150 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.150 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26210 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 581632~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.438214060000007, 2024-09-18T04:57:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023616437000000001, 2024-09-18T04:57:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26210 2024-09-18T04:57:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:57:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.153 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.153 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639993+0000", 2024-09-18T04:57:47.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.419e-06 2024-09-18T04:57:47.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640006+0000", 2024-09-18T04:57:47.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2918000000000001e-05 2024-09-18T04:57:47.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640011+0000", 2024-09-18T04:57:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9320000000000004e-06 2024-09-18T04:57:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663454+0000", 2024-09-18T04:57:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023443035000000001 2024-09-18T04:57:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663532+0000", 2024-09-18T04:57:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7443999999999999e-05 2024-09-18T04:57:47.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663608+0000", 2024-09-18T04:57:47.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6093000000000005e-05 2024-09-18T04:57:47.156 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.156 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.156 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.157 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26211 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 610304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:47.157 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:57:47.157 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.438189507999994, 2024-09-18T04:57:47.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023774337999999999, 2024-09-18T04:57:47.157 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.157 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.157 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.157 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26211 2024-09-18T04:57:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:57:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:57:47.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640015+0000", 2024-09-18T04:57:47.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640023+0000", 2024-09-18T04:57:47.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3900000000000004e-06 2024-09-18T04:57:47.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640036+0000", 2024-09-18T04:57:47.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3091999999999999e-05 2024-09-18T04:57:47.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640041+0000", 2024-09-18T04:57:47.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6219999999999997e-06 2024-09-18T04:57:47.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663644+0000", 2024-09-18T04:57:47.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023602643999999999 2024-09-18T04:57:47.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663718+0000", 2024-09-18T04:57:47.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4481999999999996e-05 2024-09-18T04:57:47.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.162 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.162 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663790+0000", 2024-09-18T04:57:47.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1589999999999997e-05 2024-09-18T04:57:47.162 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.162 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.162 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.163 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26212 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 638976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:47.163 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:57:47.163 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.438156468000003, 2024-09-18T04:57:47.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023924748999999999, 2024-09-18T04:57:47.163 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.163 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.163 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.163 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.163 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.164 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26212 2024-09-18T04:57:47.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.164 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:57:47.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:57:47.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640048+0000", 2024-09-18T04:57:47.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640051+0000", 2024-09-18T04:57:47.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.095e-06 2024-09-18T04:57:47.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640060+0000", 2024-09-18T04:57:47.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8929999999999994e-06 2024-09-18T04:57:47.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640065+0000", 2024-09-18T04:57:47.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0270000000000003e-06 2024-09-18T04:57:47.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663824+0000", 2024-09-18T04:57:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023758873999999999 2024-09-18T04:57:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663904+0000", 2024-09-18T04:57:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9856000000000003e-05 2024-09-18T04:57:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663974+0000", 2024-09-18T04:57:47.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9450999999999996e-05 2024-09-18T04:57:47.169 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.169 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.169 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.169 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26213 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 667648~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:47.169 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:47.169 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.438135297000002, 2024-09-18T04:57:47.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024059812999999999, 2024-09-18T04:57:47.170 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.170 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.170 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.170 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.170 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.170 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26213 2024-09-18T04:57:47.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.170 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:47.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:47.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:57:47.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640072+0000", 2024-09-18T04:57:47.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4200000000000001e-06 2024-09-18T04:57:47.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640078+0000", 2024-09-18T04:57:47.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0179999999999998e-06 2024-09-18T04:57:47.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640083+0000", 2024-09-18T04:57:47.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:57:47.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664008+0000", 2024-09-18T04:57:47.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023925287 2024-09-18T04:57:47.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664059+0000", 2024-09-18T04:57:47.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0730999999999999e-05 2024-09-18T04:57:47.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664130+0000", 2024-09-18T04:57:47.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0801000000000002e-05 2024-09-18T04:57:47.175 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.175 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.175 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.175 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26214 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 696320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:47.175 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:47.175 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.438118326999998, 2024-09-18T04:57:47.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024223738000000002, 2024-09-18T04:57:47.175 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.175 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26214 2024-09-18T04:57:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:47.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:57:47.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640089+0000", 2024-09-18T04:57:47.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0849999999999999e-06 2024-09-18T04:57:47.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640095+0000", 2024-09-18T04:57:47.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9279999999999998e-06 2024-09-18T04:57:47.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640099+0000", 2024-09-18T04:57:47.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:57:47.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664166+0000", 2024-09-18T04:57:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024066513000000001 2024-09-18T04:57:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664239+0000", 2024-09-18T04:57:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3201999999999995e-05 2024-09-18T04:57:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664311+0000", 2024-09-18T04:57:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1434999999999996e-05 2024-09-18T04:57:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26215 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 724992~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:57:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.438101566, 2024-09-18T04:57:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024302284, 2024-09-18T04:57:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26215 2024-09-18T04:57:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:57:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:57:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640103+0000", 2024-09-18T04:57:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640106+0000", 2024-09-18T04:57:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2120000000000002e-06 2024-09-18T04:57:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640111+0000", 2024-09-18T04:57:47.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7880000000000003e-06 2024-09-18T04:57:47.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640116+0000", 2024-09-18T04:57:47.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6099999999999999e-06 2024-09-18T04:57:47.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664327+0000", 2024-09-18T04:57:47.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024210941999999999 2024-09-18T04:57:47.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664367+0000", 2024-09-18T04:57:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9666000000000003e-05 2024-09-18T04:57:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664406+0000", 2024-09-18T04:57:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9402e-05 2024-09-18T04:57:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.187 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26216 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 753664~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:47.187 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:47.187 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.438085072000007, 2024-09-18T04:57:47.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024452854999999999, 2024-09-18T04:57:47.187 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.187 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.187 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.187 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.187 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.187 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26216 2024-09-18T04:57:47.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.187 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:47.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:47.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.188 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.188 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:57:47.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640122+0000", 2024-09-18T04:57:47.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3599999999999999e-06 2024-09-18T04:57:47.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640128+0000", 2024-09-18T04:57:47.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6339999999999998e-06 2024-09-18T04:57:47.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640132+0000", 2024-09-18T04:57:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:57:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664436+0000", 2024-09-18T04:57:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024303585999999999 2024-09-18T04:57:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664507+0000", 2024-09-18T04:57:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1046999999999999e-05 2024-09-18T04:57:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664573+0000", 2024-09-18T04:57:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6217000000000007e-05 2024-09-18T04:57:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26217 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 782336~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:57:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.438068838000007, 2024-09-18T04:57:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024526287000000001, 2024-09-18T04:57:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.191 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26217 2024-09-18T04:57:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:57:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640140+0000", 2024-09-18T04:57:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2989999999999998e-06 2024-09-18T04:57:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640146+0000", 2024-09-18T04:57:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9569999999999999e-06 2024-09-18T04:57:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640151+0000", 2024-09-18T04:57:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:57:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664590+0000", 2024-09-18T04:57:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0244389 2024-09-18T04:57:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664628+0000", 2024-09-18T04:57:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7910999999999997e-05 2024-09-18T04:57:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664663+0000", 2024-09-18T04:57:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5024999999999998e-05 2024-09-18T04:57:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.195 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.195 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:47.196 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:47.081+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:47.198 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:57:47.198 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:47.195+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:47.298 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:47.296+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:47.341 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:47.341 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:47.341 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:57:47.342 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:47.349 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:57:47.399 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:47.396+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:47.452 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:47.452 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:47.452 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:57:47.452 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:57:47.452 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:47.460 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:57:47.502 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:47.499+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:47.601 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:47.599+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:47.605 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:47.605 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:47.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:47.605 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:47.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.605 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:57:47.605 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:57:47.605 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.208915257000001, 2024-09-18T04:57:47.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037718511000000003, 2024-09-18T04:57:47.605 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:57:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:57:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-18T04:57:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:57:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:57:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393125+0000", 2024-09-18T04:57:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:57:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393133+0000", 2024-09-18T04:57:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.225e-06 2024-09-18T04:57:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393144+0000", 2024-09-18T04:57:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1279e-05 2024-09-18T04:57:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393153+0000", 2024-09-18T04:57:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4540000000000007e-06 2024-09-18T04:57:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393203+0000", 2024-09-18T04:57:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9963000000000003e-05 2024-09-18T04:57:47.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:57:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393210+0000", 2024-09-18T04:57:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5689999999999999e-06 2024-09-18T04:57:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427399+0000", 2024-09-18T04:57:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034188784999999999 2024-09-18T04:57:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427532+0000", 2024-09-18T04:57:47.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013264700000000001 2024-09-18T04:57:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:57:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427685+0000", 2024-09-18T04:57:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015300599999999999 2024-09-18T04:57:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.428520+0000", 2024-09-18T04:57:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00083502499999999998 2024-09-18T04:57:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:57:47.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430772+0000", 2024-09-18T04:57:47.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022518880000000001 2024-09-18T04:57:47.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:57:47.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430817+0000", 2024-09-18T04:57:47.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5077000000000002e-05 2024-09-18T04:57:47.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430845+0000", 2024-09-18T04:57:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8036999999999999e-05 2024-09-18T04:57:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:62520 3.13 3.84cdb893 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T04:57:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 56.304605373999998, 2024-09-18T04:57:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94452134799999998, 2024-09-18T04:57:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:57:47.612 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 62520 2024-09-18T04:57:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:47.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297441+0000", 2024-09-18T04:57:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9840000000000001e-06 2024-09-18T04:57:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297449+0000", 2024-09-18T04:57:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3999999999999992e-06 2024-09-18T04:57:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297455+0000", 2024-09-18T04:57:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3350000000000003e-06 2024-09-18T04:57:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297552+0000", 2024-09-18T04:57:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6602999999999996e-05 2024-09-18T04:57:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.615 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:57:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297559+0000", 2024-09-18T04:57:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.52e-06 2024-09-18T04:57:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241943+0000", 2024-09-18T04:57:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94438357100000003 2024-09-18T04:57:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241958+0000", 2024-09-18T04:57:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4598e-05 2024-09-18T04:57:47.616 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:113179 3.13 3:cf138017:::rbd_data.11ac5f86a3f9.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T04:57:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:57:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.914814963000001, 2024-09-18T04:57:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036508304999999998, 2024-09-18T04:57:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.617 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 113179 2024-09-18T04:57:47.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.618 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:57:47.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:57:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687226+0000", 2024-09-18T04:57:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:57:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688169+0000", 2024-09-18T04:57:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00094328200000000004 2024-09-18T04:57:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688182+0000", 2024-09-18T04:57:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2756e-05 2024-09-18T04:57:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688188+0000", 2024-09-18T04:57:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0290000000000002e-06 2024-09-18T04:57:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688225+0000", 2024-09-18T04:57:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7301000000000001e-05 2024-09-18T04:57:47.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688324+0000", 2024-09-18T04:57:47.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9419999999999993e-05 2024-09-18T04:57:47.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:57:47.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.700483+0000", 2024-09-18T04:57:47.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012158771000000001 2024-09-18T04:57:47.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:57:47.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723586+0000", 2024-09-18T04:57:47.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023102952 2024-09-18T04:57:47.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:47.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723634+0000", 2024-09-18T04:57:47.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7468e-05 2024-09-18T04:57:47.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:57:47.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723693+0000", 2024-09-18T04:57:47.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9539e-05 2024-09-18T04:57:47.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723735+0000", 2024-09-18T04:57:47.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1829000000000002e-05 2024-09-18T04:57:47.623 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.623 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.623 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.623 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146077 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1400832~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:47.623 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593265+0000", 2024-09-18T04:57:47.623 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.008776585, 2024-09-18T04:57:47.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035380397000000001, 2024-09-18T04:57:47.624 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.624 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.624 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.624 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.624 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.624 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146077 2024-09-18T04:57:47.624 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.624 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.624 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.624 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.624 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593265+0000", 2024-09-18T04:57:47.624 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.626 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.626 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593265+0000", 2024-09-18T04:57:47.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.626 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.626 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.626 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.626 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593265+0000", 2024-09-18T04:57:47.626 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:57:47.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.627 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.627 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.627 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593266+0000", 2024-09-18T04:57:47.627 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1039999999999999e-06 2024-09-18T04:57:47.627 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593269+0000", 2024-09-18T04:57:47.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8760000000000001e-06 2024-09-18T04:57:47.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.628 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.628 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.628 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593271+0000", 2024-09-18T04:57:47.628 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5100000000000001e-06 2024-09-18T04:57:47.628 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628545+0000", 2024-09-18T04:57:47.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035273977999999998 2024-09-18T04:57:47.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628589+0000", 2024-09-18T04:57:47.629 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4088999999999997e-05 2024-09-18T04:57:47.629 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.629 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.629 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.629 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628645+0000", 2024-09-18T04:57:47.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6059e-05 2024-09-18T04:57:47.630 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.630 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.630 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.630 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.630 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.630 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146078 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1449984~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:47.630 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:57:47.630 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.008737119999999, 2024-09-18T04:57:47.630 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035495325000000001, 2024-09-18T04:57:47.630 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.630 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.630 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.630 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.631 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.631 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146078 2024-09-18T04:57:47.631 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.632 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.632 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.632 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:57:47.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.632 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.632 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:57:47.632 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.632 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.632 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.633 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.633 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:57:47.633 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.633 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593306+0000", 2024-09-18T04:57:47.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.686e-06 2024-09-18T04:57:47.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593311+0000", 2024-09-18T04:57:47.634 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8500000000000002e-06 2024-09-18T04:57:47.634 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.634 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.634 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.634 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593314+0000", 2024-09-18T04:57:47.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0290000000000001e-06 2024-09-18T04:57:47.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628674+0000", 2024-09-18T04:57:47.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035360346000000001 2024-09-18T04:57:47.636 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.636 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.636 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.636 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628726+0000", 2024-09-18T04:57:47.636 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2219999999999998e-05 2024-09-18T04:57:47.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.637 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.637 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628800+0000", 2024-09-18T04:57:47.637 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3613000000000001e-05 2024-09-18T04:57:47.637 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.637 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.637 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.637 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.637 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.637 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146079 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1499136~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:47.637 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:57:47.637 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.008725715000001, 2024-09-18T04:57:47.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035604581000000003, 2024-09-18T04:57:47.638 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.638 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.638 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.638 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.638 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.638 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146079 2024-09-18T04:57:47.638 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.638 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.638 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.638 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.638 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:57:47.638 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:57:47.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.639 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:57:47.639 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:57:47.639 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.639 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.639 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.640 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593317+0000", 2024-09-18T04:57:47.640 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1599999999999999e-06 2024-09-18T04:57:47.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.641 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.641 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593320+0000", 2024-09-18T04:57:47.641 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0130000000000002e-06 2024-09-18T04:57:47.641 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.641 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593322+0000", 2024-09-18T04:57:47.642 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6079999999999998e-06 2024-09-18T04:57:47.642 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.642 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.642 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.642 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628825+0000", 2024-09-18T04:57:47.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035502747000000001 2024-09-18T04:57:47.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.644 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628868+0000", 2024-09-18T04:57:47.644 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2719999999999998e-05 2024-09-18T04:57:47.644 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.644 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.644 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.645 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628920+0000", 2024-09-18T04:57:47.645 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2533e-05 2024-09-18T04:57:47.645 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.645 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.645 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.645 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.645 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.645 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146080 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1548288~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:47.645 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:57:47.646 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.008710217000001, 2024-09-18T04:57:47.646 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035755146000000002, 2024-09-18T04:57:47.646 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.646 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.646 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.646 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.646 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.646 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146080 2024-09-18T04:57:47.646 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.646 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:57:47.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.647 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.647 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.647 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:57:47.647 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.647 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:57:47.648 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.648 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.648 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.648 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.648 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593333+0000", 2024-09-18T04:57:47.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.858e-06 2024-09-18T04:57:47.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.649 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593336+0000", 2024-09-18T04:57:47.649 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4910000000000001e-06 2024-09-18T04:57:47.649 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.649 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.649 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593340+0000", 2024-09-18T04:57:47.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2590000000000001e-06 2024-09-18T04:57:47.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628956+0000", 2024-09-18T04:57:47.650 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035616039000000002 2024-09-18T04:57:47.650 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.650 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.650 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.650 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629031+0000", 2024-09-18T04:57:47.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5504000000000001e-05 2024-09-18T04:57:47.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.651 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.651 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629086+0000", 2024-09-18T04:57:47.651 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5429000000000001e-05 2024-09-18T04:57:47.651 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.651 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.651 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.651 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.651 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.652 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146081 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1597440~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:47.652 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:57:47.652 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.008699645, 2024-09-18T04:57:47.652 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035873223000000003, 2024-09-18T04:57:47.652 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.652 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.652 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.652 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.652 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.652 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146081 2024-09-18T04:57:47.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.653 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:57:47.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.653 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.653 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.653 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.653 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:57:47.653 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:57:47.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593343+0000", 2024-09-18T04:57:47.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3319999999999999e-06 2024-09-18T04:57:47.654 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.654 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.654 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593346+0000", 2024-09-18T04:57:47.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9059999999999998e-06 2024-09-18T04:57:47.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593348+0000", 2024-09-18T04:57:47.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4530000000000001e-06 2024-09-18T04:57:47.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629110+0000", 2024-09-18T04:57:47.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035761814000000003 2024-09-18T04:57:47.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629159+0000", 2024-09-18T04:57:47.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8971000000000003e-05 2024-09-18T04:57:47.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629215+0000", 2024-09-18T04:57:47.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6107000000000003e-05 2024-09-18T04:57:47.656 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.656 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.656 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.656 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146082 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1646592~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:47.657 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:57:47.657 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.008683595000001, 2024-09-18T04:57:47.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035987007000000001, 2024-09-18T04:57:47.657 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.657 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.657 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.657 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.657 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.657 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146082 2024-09-18T04:57:47.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.657 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:57:47.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:57:47.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:57:47.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593360+0000", 2024-09-18T04:57:47.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8560000000000001e-06 2024-09-18T04:57:47.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593365+0000", 2024-09-18T04:57:47.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3410000000000002e-06 2024-09-18T04:57:47.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593369+0000", 2024-09-18T04:57:47.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4850000000000003e-06 2024-09-18T04:57:47.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629242+0000", 2024-09-18T04:57:47.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035872464999999999 2024-09-18T04:57:47.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.660 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.660 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629288+0000", 2024-09-18T04:57:47.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6592999999999999e-05 2024-09-18T04:57:47.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629345+0000", 2024-09-18T04:57:47.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6575e-05 2024-09-18T04:57:47.661 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.661 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.661 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.661 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146083 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1695744~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:47.661 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:57:47.661 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.008667748000001, 2024-09-18T04:57:47.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036100587000000003, 2024-09-18T04:57:47.661 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.661 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.662 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.662 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.662 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.662 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146083 2024-09-18T04:57:47.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.662 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:57:47.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:57:47.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593373+0000", 2024-09-18T04:57:47.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593376+0000", 2024-09-18T04:57:47.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.244e-06 2024-09-18T04:57:47.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593379+0000", 2024-09-18T04:57:47.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.619e-06 2024-09-18T04:57:47.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593382+0000", 2024-09-18T04:57:47.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7199999999999998e-06 2024-09-18T04:57:47.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.665 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.665 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629368+0000", 2024-09-18T04:57:47.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035986281000000002 2024-09-18T04:57:47.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629416+0000", 2024-09-18T04:57:47.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8055e-05 2024-09-18T04:57:47.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629474+0000", 2024-09-18T04:57:47.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8176e-05 2024-09-18T04:57:47.667 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.667 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.667 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.667 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146084 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1744896~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:47.667 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:57:47.667 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.008654618, 2024-09-18T04:57:47.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036214349, 2024-09-18T04:57:47.668 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.668 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.668 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.668 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.669 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.669 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146084 2024-09-18T04:57:47.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.669 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:57:47.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:57:47.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:57:47.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593388+0000", 2024-09-18T04:57:47.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.564e-06 2024-09-18T04:57:47.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593392+0000", 2024-09-18T04:57:47.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0579999999999999e-06 2024-09-18T04:57:47.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593395+0000", 2024-09-18T04:57:47.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8890000000000002e-06 2024-09-18T04:57:47.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629500+0000", 2024-09-18T04:57:47.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036105343999999998 2024-09-18T04:57:47.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629547+0000", 2024-09-18T04:57:47.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7036000000000002e-05 2024-09-18T04:57:47.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629601+0000", 2024-09-18T04:57:47.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3866999999999998e-05 2024-09-18T04:57:47.674 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.674 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.674 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.674 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146085 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1794048~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:47.674 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:57:47.674 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.008644231, 2024-09-18T04:57:47.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036331186000000001, 2024-09-18T04:57:47.674 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.674 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.675 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.675 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.675 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.675 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146085 2024-09-18T04:57:47.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.675 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:57:47.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:57:47.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:57:47.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:57:47.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593398+0000", 2024-09-18T04:57:47.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2270000000000001e-06 2024-09-18T04:57:47.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593401+0000", 2024-09-18T04:57:47.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.999e-06 2024-09-18T04:57:47.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593404+0000", 2024-09-18T04:57:47.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.486e-06 2024-09-18T04:57:47.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629624+0000", 2024-09-18T04:57:47.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036219983999999997 2024-09-18T04:57:47.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629672+0000", 2024-09-18T04:57:47.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8368000000000002e-05 2024-09-18T04:57:47.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629729+0000", 2024-09-18T04:57:47.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6359e-05 2024-09-18T04:57:47.682 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.682 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.682 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.682 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146086 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1843200~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:47.682 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:57:47.682 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.008635667, 2024-09-18T04:57:47.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036449098999999999, 2024-09-18T04:57:47.683 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.683 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.683 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.683 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.683 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.683 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146086 2024-09-18T04:57:47.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.683 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:57:47.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:57:47.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:57:47.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:57:47.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593407+0000", 2024-09-18T04:57:47.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1799999999999999e-06 2024-09-18T04:57:47.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593410+0000", 2024-09-18T04:57:47.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.926e-06 2024-09-18T04:57:47.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593413+0000", 2024-09-18T04:57:47.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8250000000000001e-06 2024-09-18T04:57:47.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629755+0000", 2024-09-18T04:57:47.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036342212999999998 2024-09-18T04:57:47.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629803+0000", 2024-09-18T04:57:47.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8619000000000003e-05 2024-09-18T04:57:47.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629855+0000", 2024-09-18T04:57:47.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1536000000000003e-05 2024-09-18T04:57:47.687 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.687 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.687 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.687 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146087 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1892352~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:47.687 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:57:47.687 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.008626742000001, 2024-09-18T04:57:47.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036564608999999998, 2024-09-18T04:57:47.688 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.688 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.688 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.688 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.688 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.688 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146087 2024-09-18T04:57:47.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.688 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:57:47.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:57:47.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:57:47.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:57:47.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593416+0000", 2024-09-18T04:57:47.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.218e-06 2024-09-18T04:57:47.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593419+0000", 2024-09-18T04:57:47.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-18T04:57:47.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593421+0000", 2024-09-18T04:57:47.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4329999999999998e-06 2024-09-18T04:57:47.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629878+0000", 2024-09-18T04:57:47.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036456578000000003 2024-09-18T04:57:47.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629925+0000", 2024-09-18T04:57:47.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7595000000000001e-05 2024-09-18T04:57:47.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629979+0000", 2024-09-18T04:57:47.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4117999999999999e-05 2024-09-18T04:57:47.691 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.692 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.692 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.692 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146088 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1941504~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:47.692 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:57:47.692 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.008604578, 2024-09-18T04:57:47.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036676121999999999, 2024-09-18T04:57:47.692 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.692 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.692 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.692 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.692 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.692 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146088 2024-09-18T04:57:47.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.693 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:57:47.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:57:47.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:57:47.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593438+0000", 2024-09-18T04:57:47.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7570000000000001e-06 2024-09-18T04:57:47.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593442+0000", 2024-09-18T04:57:47.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2880000000000001e-06 2024-09-18T04:57:47.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593444+0000", 2024-09-18T04:57:47.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.593e-06 2024-09-18T04:57:47.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630015+0000", 2024-09-18T04:57:47.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036571125000000003 2024-09-18T04:57:47.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630061+0000", 2024-09-18T04:57:47.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5445000000000003e-05 2024-09-18T04:57:47.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630113+0000", 2024-09-18T04:57:47.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2358000000000003e-05 2024-09-18T04:57:47.696 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.696 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.696 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.697 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146089 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1990656~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:47.697 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:57:47.697 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.008594996999999, 2024-09-18T04:57:47.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036791483999999999, 2024-09-18T04:57:47.697 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.697 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.697 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.697 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.697 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.697 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146089 2024-09-18T04:57:47.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.698 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:57:47.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:57:47.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593446+0000", 2024-09-18T04:57:47.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:57:47.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593448+0000", 2024-09-18T04:57:47.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1939999999999999e-06 2024-09-18T04:57:47.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593451+0000", 2024-09-18T04:57:47.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.766e-06 2024-09-18T04:57:47.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593458+0000", 2024-09-18T04:57:47.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8539999999999999e-06 2024-09-18T04:57:47.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630136+0000", 2024-09-18T04:57:47.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03667757 2024-09-18T04:57:47.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630183+0000", 2024-09-18T04:57:47.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7289000000000001e-05 2024-09-18T04:57:47.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630238+0000", 2024-09-18T04:57:47.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5013000000000003e-05 2024-09-18T04:57:47.702 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.702 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.702 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.703 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146090 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 2039808~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:47.703 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:57:47.703 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.008581117, 2024-09-18T04:57:47.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036904477999999998, 2024-09-18T04:57:47.703 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.703 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.703 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.703 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.703 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.703 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146090 2024-09-18T04:57:47.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.704 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:57:47.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:57:47.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:57:47.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:47.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593461+0000", 2024-09-18T04:57:47.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3680000000000001e-06 2024-09-18T04:57:47.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593465+0000", 2024-09-18T04:57:47.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8120000000000001e-06 2024-09-18T04:57:47.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593468+0000", 2024-09-18T04:57:47.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.582e-06 2024-09-18T04:57:47.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630268+0000", 2024-09-18T04:57:47.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036800449999999998 2024-09-18T04:57:47.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630313+0000", 2024-09-18T04:57:47.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4836000000000002e-05 2024-09-18T04:57:47.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630365+0000", 2024-09-18T04:57:47.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1839999999999998e-05 2024-09-18T04:57:47.709 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.709 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.710 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.710 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146091 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 2088960~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:47.710 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:57:47.710 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 14.008570655, 2024-09-18T04:57:47.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037011534999999998, 2024-09-18T04:57:47.710 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.711 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.711 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.711 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.711 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.711 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146091 2024-09-18T04:57:47.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.711 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:57:47.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:57:47.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:57:47.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:57:47.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593472+0000", 2024-09-18T04:57:47.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1620000000000001e-06 2024-09-18T04:57:47.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593475+0000", 2024-09-18T04:57:47.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0019999999999998e-06 2024-09-18T04:57:47.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593477+0000", 2024-09-18T04:57:47.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-18T04:57:47.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630381+0000", 2024-09-18T04:57:47.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036904014999999998 2024-09-18T04:57:47.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630428+0000", 2024-09-18T04:57:47.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6931000000000002e-05 2024-09-18T04:57:47.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630482+0000", 2024-09-18T04:57:47.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4138999999999998e-05 2024-09-18T04:57:47.717 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.717 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.717 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.717 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:162075 3.13 3:cf3f6048:::rbd_header.11ac6967455f:head [watch reconnect cookie 140406508629968 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T04:57:47.717 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:57:47.717 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.1115978499999999, 2024-09-18T04:57:47.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.53184266599999996, 2024-09-18T04:57:47.717 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.717 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:47.718 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.718 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.718 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.718 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 162075 2024-09-18T04:57:47.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.718 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:57:47.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:57:47.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490442+0000", 2024-09-18T04:57:47.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:57:47.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490453+0000", 2024-09-18T04:57:47.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0613e-05 2024-09-18T04:57:47.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490466+0000", 2024-09-18T04:57:47.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3702e-05 2024-09-18T04:57:47.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490475+0000", 2024-09-18T04:57:47.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2690000000000005e-06 2024-09-18T04:57:47.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490612+0000", 2024-09-18T04:57:47.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013620900000000001 2024-09-18T04:57:47.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-18T04:57:47.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490620+0000", 2024-09-18T04:57:47.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3089999999999996e-06 2024-09-18T04:57:47.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503041+0000", 2024-09-18T04:57:47.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012420512 2024-09-18T04:57:47.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:57:47.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503048+0000", 2024-09-18T04:57:47.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0389999999999999e-06 2024-09-18T04:57:47.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.016122+0000", 2024-09-18T04:57:47.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.51307432900000005 2024-09-18T04:57:47.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.016289+0000", 2024-09-18T04:57:47.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000166875 2024-09-18T04:57:47.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.022286+0000", 2024-09-18T04:57:47.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059976049999999996 2024-09-18T04:57:47.723 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.723 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.723 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.723 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:162076 3.13 3:cf3f6048:::rbd_header.11ac6967455f:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T04:57:47.723 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:57:47.723 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.1114751759999999, 2024-09-18T04:57:47.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.53464117600000005, 2024-09-18T04:57:47.724 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:47.724 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:47.724 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:47.724 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:47.724 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:47.724 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 162076 2024-09-18T04:57:47.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.724 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:47.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:47.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:57:47.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:47.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:57:47.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:47.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:47.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:57:47.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:57:47.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:47.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490571+0000", 2024-09-18T04:57:47.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8259999999999998e-06 2024-09-18T04:57:47.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:47.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490584+0000", 2024-09-18T04:57:47.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2391999999999999e-05 2024-09-18T04:57:47.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:47.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490591+0000", 2024-09-18T04:57:47.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5460000000000003e-06 2024-09-18T04:57:47.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490714+0000", 2024-09-18T04:57:47.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000122543 2024-09-18T04:57:47.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-18T04:57:47.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490731+0000", 2024-09-18T04:57:47.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7405e-05 2024-09-18T04:57:47.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503069+0000", 2024-09-18T04:57:47.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012337836 2024-09-18T04:57:47.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:57:47.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503078+0000", 2024-09-18T04:57:47.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8289999999999997e-06 2024-09-18T04:57:47.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:47.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019018+0000", 2024-09-18T04:57:47.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.51594035900000002 2024-09-18T04:57:47.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:47.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019091+0000", 2024-09-18T04:57:47.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2608999999999995e-05 2024-09-18T04:57:47.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-18T04:57:47.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019421+0000", 2024-09-18T04:57:47.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00033004300000000002 2024-09-18T04:57:47.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:47.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.021644+0000", 2024-09-18T04:57:47.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022229749999999999 2024-09-18T04:57:47.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:57:47.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025113+0000", 2024-09-18T04:57:47.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003469228 2024-09-18T04:57:47.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:57:47.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025167+0000", 2024-09-18T04:57:47.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3569999999999997e-05 2024-09-18T04:57:47.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:47.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:47.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:47.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025208+0000", 2024-09-18T04:57:47.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0821000000000001e-05 2024-09-18T04:57:47.731 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.731 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.731 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.731 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:47.731 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:47.731 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:47.733 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:57:47.734 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:47.730+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:47.834 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:47.832+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:47.845 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:47.845 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:47.845 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:47.846 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:47.855 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:57:47.935 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:47.933+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:47.961 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:47.962 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:47.962 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:47.962 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:47.962 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:47.972 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:57:48.035 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:48.033+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:48.092 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:48.093 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:48.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:48.093 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:48.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.093 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9316 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:48.093 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:48.093 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 72.321714881999995, 2024-09-18T04:57:48.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044027208999999998, 2024-09-18T04:57:48.093 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:48.093 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:48.093 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:48.093 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:48.093 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:48.094 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9316 2024-09-18T04:57:48.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.094 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:48.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:48.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:48.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:48.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:48.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:48.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:48.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:48.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:48.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767144+0000", 2024-09-18T04:57:48.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3070000000000001e-06 2024-09-18T04:57:48.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:48.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767150+0000", 2024-09-18T04:57:48.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1700000000000002e-06 2024-09-18T04:57:48.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:48.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767163+0000", 2024-09-18T04:57:48.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2670999999999999e-05 2024-09-18T04:57:48.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:48.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810995+0000", 2024-09-18T04:57:48.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043832310999999999 2024-09-18T04:57:48.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:48.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811087+0000", 2024-09-18T04:57:48.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1345e-05 2024-09-18T04:57:48.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:48.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811169+0000", 2024-09-18T04:57:48.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2848000000000004e-05 2024-09-18T04:57:48.097 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.097 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:48.097 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.098 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9317 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1728512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:48.098 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:57:48.098 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 72.321689246999995, 2024-09-18T04:57:48.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044190170000000001, 2024-09-18T04:57:48.098 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:48.098 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:48.098 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:48.098 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:48.098 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:48.098 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9317 2024-09-18T04:57:48.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.099 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:48.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:48.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:57:48.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:48.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:57:48.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:48.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767167+0000", 2024-09-18T04:57:48.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:48.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:48.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767170+0000", 2024-09-18T04:57:48.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7259999999999998e-06 2024-09-18T04:57:48.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:48.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767176+0000", 2024-09-18T04:57:48.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1460000000000001e-06 2024-09-18T04:57:48.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:48.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767182+0000", 2024-09-18T04:57:48.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1020000000000002e-06 2024-09-18T04:57:48.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:48.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811204+0000", 2024-09-18T04:57:48.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044022039999999998 2024-09-18T04:57:48.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:48.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811286+0000", 2024-09-18T04:57:48.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1623000000000001e-05 2024-09-18T04:57:48.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.102 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:48.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811358+0000", 2024-09-18T04:57:48.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2125000000000005e-05 2024-09-18T04:57:48.102 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.102 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:48.102 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.103 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9318 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1777664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:48.103 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:48.103 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 72.321670702000006, 2024-09-18T04:57:48.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044380181999999997, 2024-09-18T04:57:48.103 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:48.103 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:48.103 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:48.103 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:48.103 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:48.103 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9318 2024-09-18T04:57:48.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.104 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:48.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:48.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:48.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:48.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:48.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:48.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:48.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:48.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:48.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767189+0000", 2024-09-18T04:57:48.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5909999999999998e-06 2024-09-18T04:57:48.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:48.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767194+0000", 2024-09-18T04:57:48.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7690000000000002e-06 2024-09-18T04:57:48.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:48.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767199+0000", 2024-09-18T04:57:48.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189999999999999e-06 2024-09-18T04:57:48.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:48.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811394+0000", 2024-09-18T04:57:48.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044195350000000001 2024-09-18T04:57:48.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:48.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811483+0000", 2024-09-18T04:57:48.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8282999999999995e-05 2024-09-18T04:57:48.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:48.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811567+0000", 2024-09-18T04:57:48.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3855000000000003e-05 2024-09-18T04:57:48.108 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.108 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:48.108 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.108 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9325 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:48.108 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:48.108 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 72.319940164000002, 2024-09-18T04:57:48.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044063989999999997, 2024-09-18T04:57:48.108 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:48.108 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:48.108 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:48.109 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:48.109 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:48.109 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9325 2024-09-18T04:57:48.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.109 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:48.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:48.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:48.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:48.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:48.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:48.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:48.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:48.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:48.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768919+0000", 2024-09-18T04:57:48.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:57:48.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:48.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768925+0000", 2024-09-18T04:57:48.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2310000000000001e-06 2024-09-18T04:57:48.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:48.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768929+0000", 2024-09-18T04:57:48.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4469999999999999e-06 2024-09-18T04:57:48.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:48.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812823+0000", 2024-09-18T04:57:48.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043893975000000002 2024-09-18T04:57:48.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:48.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812908+0000", 2024-09-18T04:57:48.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4862000000000002e-05 2024-09-18T04:57:48.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:48.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812981+0000", 2024-09-18T04:57:48.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2892000000000006e-05 2024-09-18T04:57:48.113 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.113 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:48.113 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.113 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9334 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:48.115 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:57:48.115 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 72.317921377999994, 2024-09-18T04:57:48.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044042853999999999, 2024-09-18T04:57:48.115 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:48.115 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:48.116 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:48.116 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:48.116 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:48.116 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9334 2024-09-18T04:57:48.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.116 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:48.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:48.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:57:48.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:48.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:57:48.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:48.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770935+0000", 2024-09-18T04:57:48.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:48.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:48.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770937+0000", 2024-09-18T04:57:48.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1489999999999999e-06 2024-09-18T04:57:48.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:48.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770940+0000", 2024-09-18T04:57:48.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9569999999999998e-06 2024-09-18T04:57:48.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:48.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770945+0000", 2024-09-18T04:57:48.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:57:48.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:48.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814838+0000", 2024-09-18T04:57:48.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043892635999999999 2024-09-18T04:57:48.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:48.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814920+0000", 2024-09-18T04:57:48.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2175000000000005e-05 2024-09-18T04:57:48.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:48.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814979+0000", 2024-09-18T04:57:48.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8372000000000003e-05 2024-09-18T04:57:48.120 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.120 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:48.120 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.120 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9352 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1159168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:48.121 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:57:48.121 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 72.315700200999999, 2024-09-18T04:57:48.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044011269999999998, 2024-09-18T04:57:48.121 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:48.121 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:48.121 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:48.121 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:48.121 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:48.121 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9352 2024-09-18T04:57:48.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.121 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:48.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:48.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:57:48.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:48.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:57:48.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:48.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773156+0000", 2024-09-18T04:57:48.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:48.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:48.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773159+0000", 2024-09-18T04:57:48.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2919999999999998e-06 2024-09-18T04:57:48.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:48.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773165+0000", 2024-09-18T04:57:48.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0580000000000003e-06 2024-09-18T04:57:48.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:48.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773177+0000", 2024-09-18T04:57:48.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.204e-05 2024-09-18T04:57:48.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:48.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817082+0000", 2024-09-18T04:57:48.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043905528999999999 2024-09-18T04:57:48.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:48.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817122+0000", 2024-09-18T04:57:48.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9258999999999998e-05 2024-09-18T04:57:48.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:48.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817168+0000", 2024-09-18T04:57:48.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6545000000000002e-05 2024-09-18T04:57:48.125 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.125 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:48.125 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.125 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9353 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1208320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:48.125 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:48.125 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 72.315675902999999, 2024-09-18T04:57:48.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044105537, 2024-09-18T04:57:48.125 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:48.126 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:48.126 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:48.126 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:48.126 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:48.126 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9353 2024-09-18T04:57:48.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.126 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:48.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:48.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:48.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:48.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:48.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:48.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:48.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:48.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:48.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773183+0000", 2024-09-18T04:57:48.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2929999999999999e-06 2024-09-18T04:57:48.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:48.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773190+0000", 2024-09-18T04:57:48.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4330000000000001e-06 2024-09-18T04:57:48.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:48.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773194+0000", 2024-09-18T04:57:48.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.865e-06 2024-09-18T04:57:48.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:48.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817196+0000", 2024-09-18T04:57:48.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044001708 2024-09-18T04:57:48.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:48.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817236+0000", 2024-09-18T04:57:48.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0043999999999998e-05 2024-09-18T04:57:48.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:48.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817287+0000", 2024-09-18T04:57:48.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0605000000000001e-05 2024-09-18T04:57:48.130 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.130 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:48.131 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.131 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9354 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:48.131 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:57:48.131 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 72.315658392000003, 2024-09-18T04:57:48.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044199796, 2024-09-18T04:57:48.131 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:48.131 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:48.131 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:48.131 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:48.131 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:48.131 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9354 2024-09-18T04:57:48.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.132 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:48.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:48.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:57:48.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:48.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:57:48.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:48.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773198+0000", 2024-09-18T04:57:48.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:48.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:48.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773200+0000", 2024-09-18T04:57:48.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4159999999999998e-06 2024-09-18T04:57:48.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:48.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773206+0000", 2024-09-18T04:57:48.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.789e-06 2024-09-18T04:57:48.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:48.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773210+0000", 2024-09-18T04:57:48.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2679999999999996e-06 2024-09-18T04:57:48.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:48.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817314+0000", 2024-09-18T04:57:48.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044103187000000002 2024-09-18T04:57:48.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:48.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817353+0000", 2024-09-18T04:57:48.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8973000000000001e-05 2024-09-18T04:57:48.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:48.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817399+0000", 2024-09-18T04:57:48.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5877999999999997e-05 2024-09-18T04:57:48.135 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.135 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:48.135 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.135 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46591 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2097152~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:48.135 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:48.136 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.390801975000002, 2024-09-18T04:57:48.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043987310000000002, 2024-09-18T04:57:48.136 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:48.136 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:48.136 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:48.136 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:48.136 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:48.136 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46591 2024-09-18T04:57:48.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.136 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:48.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:48.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:48.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:48.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:48.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:48.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:48.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:48.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:48.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698057+0000", 2024-09-18T04:57:48.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.531e-06 2024-09-18T04:57:48.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:48.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698064+0000", 2024-09-18T04:57:48.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-18T04:57:48.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:48.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698069+0000", 2024-09-18T04:57:48.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.091e-06 2024-09-18T04:57:48.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:48.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741843+0000", 2024-09-18T04:57:48.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043773470000000002 2024-09-18T04:57:48.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:48.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741928+0000", 2024-09-18T04:57:48.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5113999999999999e-05 2024-09-18T04:57:48.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:48.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742042+0000", 2024-09-18T04:57:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114527 2024-09-18T04:57:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46592 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2121728~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:57:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.390783548000002, 2024-09-18T04:57:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044077948999999998, 2024-09-18T04:57:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:48.143 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:48.144 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:48.144 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46592 2024-09-18T04:57:48.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.144 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:48.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:48.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:57:48.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:48.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:57:48.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698073+0000", 2024-09-18T04:57:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698076+0000", 2024-09-18T04:57:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.481e-06 2024-09-18T04:57:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698082+0000", 2024-09-18T04:57:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8730000000000004e-06 2024-09-18T04:57:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698086+0000", 2024-09-18T04:57:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:57:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742055+0000", 2024-09-18T04:57:48.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043969170000000002 2024-09-18T04:57:48.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742093+0000", 2024-09-18T04:57:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7707000000000003e-05 2024-09-18T04:57:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742152+0000", 2024-09-18T04:57:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8323999999999999e-05 2024-09-18T04:57:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46593 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2211840~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.390766644999999, 2024-09-18T04:57:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044259917000000003, 2024-09-18T04:57:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46593 2024-09-18T04:57:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.149 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:48.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:48.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:48.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:48.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:48.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:48.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:48.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:48.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:48.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698092+0000", 2024-09-18T04:57:48.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2249999999999999e-06 2024-09-18T04:57:48.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:48.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698098+0000", 2024-09-18T04:57:48.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:57:48.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:48.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698103+0000", 2024-09-18T04:57:48.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:57:48.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742182+0000", 2024-09-18T04:57:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044079508000000003 2024-09-18T04:57:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742257+0000", 2024-09-18T04:57:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4678000000000006e-05 2024-09-18T04:57:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:48.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742350+0000", 2024-09-18T04:57:48.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3531999999999998e-05 2024-09-18T04:57:48.155 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.155 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:48.155 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.155 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46594 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2236416~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:48.155 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:48.155 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.390750947999997, 2024-09-18T04:57:48.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044378984000000003, 2024-09-18T04:57:48.156 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:48.156 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:48.156 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:48.156 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:48.156 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:48.156 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46594 2024-09-18T04:57:48.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.156 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:48.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:48.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:48.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:48.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698108+0000", 2024-09-18T04:57:48.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0880000000000002e-06 2024-09-18T04:57:48.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:48.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698114+0000", 2024-09-18T04:57:48.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6269999999999997e-06 2024-09-18T04:57:48.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:48.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698118+0000", 2024-09-18T04:57:48.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4050000000000002e-06 2024-09-18T04:57:48.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:48.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742365+0000", 2024-09-18T04:57:48.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044246799000000003 2024-09-18T04:57:48.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:48.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742407+0000", 2024-09-18T04:57:48.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2008000000000002e-05 2024-09-18T04:57:48.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:48.160 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742485+0000", 2024-09-18T04:57:48.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8405999999999994e-05 2024-09-18T04:57:48.160 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.160 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:48.160 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.161 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46595 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:48.161 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:48.161 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.390734875, 2024-09-18T04:57:48.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.04440463, 2024-09-18T04:57:48.161 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:48.161 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:48.161 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:48.161 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:48.161 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:48.162 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46595 2024-09-18T04:57:48.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.162 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:48.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:48.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:48.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:48.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:48.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698124+0000", 2024-09-18T04:57:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1840000000000002e-06 2024-09-18T04:57:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:48.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698130+0000", 2024-09-18T04:57:48.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.947e-06 2024-09-18T04:57:48.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:48.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698136+0000", 2024-09-18T04:57:48.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6520000000000003e-06 2024-09-18T04:57:48.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742521+0000", 2024-09-18T04:57:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044385446000000002 2024-09-18T04:57:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742527+0000", 2024-09-18T04:57:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:57:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46596 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.390716630999997, 2024-09-18T04:57:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044418066999999999, 2024-09-18T04:57:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46596 2024-09-18T04:57:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.166 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:48.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:48.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:48.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:48.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:48.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:48.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:48.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:48.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:48.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698146+0000", 2024-09-18T04:57:48.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2779999999999999e-06 2024-09-18T04:57:48.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:48.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698153+0000", 2024-09-18T04:57:48.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3849999999999999e-06 2024-09-18T04:57:48.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:48.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698164+0000", 2024-09-18T04:57:48.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1375999999999999e-05 2024-09-18T04:57:48.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:48.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742554+0000", 2024-09-18T04:57:48.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044389482000000001 2024-09-18T04:57:48.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742559+0000", 2024-09-18T04:57:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9540000000000003e-06 2024-09-18T04:57:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46597 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:57:48.170 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.390688265999998, 2024-09-18T04:57:48.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044416513999999997, 2024-09-18T04:57:48.171 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:48.171 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:48.171 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:48.171 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:48.171 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:48.171 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46597 2024-09-18T04:57:48.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.171 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:48.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:48.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:57:48.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:48.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:57:48.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:48.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698168+0000", 2024-09-18T04:57:48.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:48.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:48.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698175+0000", 2024-09-18T04:57:48.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4520000000000002e-06 2024-09-18T04:57:48.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:48.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698181+0000", 2024-09-18T04:57:48.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4150000000000004e-06 2024-09-18T04:57:48.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:48.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698186+0000", 2024-09-18T04:57:48.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7489999999999998e-06 2024-09-18T04:57:48.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:48.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742580+0000", 2024-09-18T04:57:48.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044394325999999998 2024-09-18T04:57:48.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:48.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742585+0000", 2024-09-18T04:57:48.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9849999999999997e-06 2024-09-18T04:57:48.175 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.175 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:48.175 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.175 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46598 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:48.175 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:57:48.175 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 58.390580790000001, 2024-09-18T04:57:48.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044335316999999999, 2024-09-18T04:57:48.175 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:48.176 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:48.176 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:48.176 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:48.176 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:48.176 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46598 2024-09-18T04:57:48.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.176 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:48.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:48.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:57:48.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:48.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:57:48.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:48.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698190+0000", 2024-09-18T04:57:48.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999137 2024-09-18T04:57:48.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:48.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698280+0000", 2024-09-18T04:57:48.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0409999999999997e-05 2024-09-18T04:57:48.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:48.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698289+0000", 2024-09-18T04:57:48.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9490000000000002e-06 2024-09-18T04:57:48.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:48.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698296+0000", 2024-09-18T04:57:48.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2670000000000003e-06 2024-09-18T04:57:48.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:48.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742607+0000", 2024-09-18T04:57:48.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044311072999999999 2024-09-18T04:57:48.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:48.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742612+0000", 2024-09-18T04:57:48.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9400000000000001e-06 2024-09-18T04:57:48.180 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.180 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:48.180 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.180 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57986 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3086336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:48.180 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:48.180 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.403735067, 2024-09-18T04:57:48.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044047731999999999, 2024-09-18T04:57:48.183 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:48.183 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:48.183 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:48.183 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:48.183 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:48.183 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57986 2024-09-18T04:57:48.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.183 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:48.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:48.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:48.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:48.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:48.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:48.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:48.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:48.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:48.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685124+0000", 2024-09-18T04:57:48.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2859999999999998e-06 2024-09-18T04:57:48.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:48.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685130+0000", 2024-09-18T04:57:48.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5400000000000001e-06 2024-09-18T04:57:48.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:48.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685136+0000", 2024-09-18T04:57:48.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.152e-06 2024-09-18T04:57:48.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:48.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.728976+0000", 2024-09-18T04:57:48.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043840037999999998 2024-09-18T04:57:48.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:48.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729064+0000", 2024-09-18T04:57:48.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8709999999999996e-05 2024-09-18T04:57:48.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:48.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729170+0000", 2024-09-18T04:57:48.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000105386 2024-09-18T04:57:48.187 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.187 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:48.187 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.187 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57987 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3101696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:48.187 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:57:48.187 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.403686092000001, 2024-09-18T04:57:48.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044238123999999997, 2024-09-18T04:57:48.187 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:48.187 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:48.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:48.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:48.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:48.188 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57987 2024-09-18T04:57:48.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.188 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:48.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:48.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:57:48.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:48.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:57:48.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:48.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685170+0000", 2024-09-18T04:57:48.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:48.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:48.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685175+0000", 2024-09-18T04:57:48.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9069999999999998e-06 2024-09-18T04:57:48.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:48.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685187+0000", 2024-09-18T04:57:48.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.189e-05 2024-09-18T04:57:48.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:48.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685195+0000", 2024-09-18T04:57:48.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9459999999999998e-06 2024-09-18T04:57:48.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:48.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729212+0000", 2024-09-18T04:57:48.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044017239 2024-09-18T04:57:48.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:48.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729301+0000", 2024-09-18T04:57:48.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8887999999999993e-05 2024-09-18T04:57:48.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:48.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729409+0000", 2024-09-18T04:57:48.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108119 2024-09-18T04:57:48.191 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.191 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:48.191 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.191 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57988 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3117056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:48.192 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:57:48.192 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.403643973000001, 2024-09-18T04:57:48.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044429835000000001, 2024-09-18T04:57:48.192 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:48.192 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:48.192 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:48.192 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:48.192 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:48.192 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57988 2024-09-18T04:57:48.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.192 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:48.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:48.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:57:48.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:48.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:57:48.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:48.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685212+0000", 2024-09-18T04:57:48.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:48.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:48.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685216+0000", 2024-09-18T04:57:48.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7390000000000001e-06 2024-09-18T04:57:48.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:48.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685224+0000", 2024-09-18T04:57:48.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2260000000000005e-06 2024-09-18T04:57:48.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:48.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685230+0000", 2024-09-18T04:57:48.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0299999999999999e-06 2024-09-18T04:57:48.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:48.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729454+0000", 2024-09-18T04:57:48.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044223248999999999 2024-09-18T04:57:48.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:48.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729539+0000", 2024-09-18T04:57:48.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5029000000000002e-05 2024-09-18T04:57:48.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:48.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729643+0000", 2024-09-18T04:57:48.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104267 2024-09-18T04:57:48.195 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.195 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:48.196 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.196 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57989 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3132416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:48.196 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:57:48.196 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.403621221000002, 2024-09-18T04:57:48.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044642881000000002, 2024-09-18T04:57:48.196 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:48.196 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:48.196 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:48.196 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:48.196 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:48.196 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57989 2024-09-18T04:57:48.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.197 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:48.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:48.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:57:48.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:48.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:57:48.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:48.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:48.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685235+0000", 2024-09-18T04:57:48.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:48.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:48.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685238+0000", 2024-09-18T04:57:48.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.667e-06 2024-09-18T04:57:48.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:48.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685245+0000", 2024-09-18T04:57:48.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7909999999999999e-06 2024-09-18T04:57:48.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:48.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685250+0000", 2024-09-18T04:57:48.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3730000000000001e-06 2024-09-18T04:57:48.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:48.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729685+0000", 2024-09-18T04:57:48.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044434552000000002 2024-09-18T04:57:48.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:48.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729775+0000", 2024-09-18T04:57:48.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0395000000000004e-05 2024-09-18T04:57:48.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:48.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:48.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:48.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729879+0000", 2024-09-18T04:57:48.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010361099999999999 2024-09-18T04:57:48.200 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.200 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:48.200 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.200 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:48.200 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:48.200 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:48.202 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:57:48.202 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:48.199+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:48.268 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:57:48.352 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:48.352 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:48.352 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:48.352 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:48.361 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:57:48.382 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:48.327+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:48.404 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:48.400+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:48.414 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:57:48.414 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:57:48.431 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:57:48.431 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:57:48.447 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:57:48.447 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:57:48.463 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:57:48.464 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:57:48.480 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:57:48.480 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:57:48.490 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:48.491 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:48.496 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:57:48.496 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:57:48.500 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:57:48.513 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:57:48.514 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:57:48.515 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:48.511+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:48.529 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:57:48.530 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:57:48.604 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:48.602+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:48.621 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:48.621 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:48.630 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:57:48.705 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:48.703+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:48.736 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:48.736 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:48.745 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:57:48.806 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:48.804+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:48.892 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:48.892 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:48.901 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:57:48.945 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:48.943+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:49.008 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:49.008 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:49.009 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:49.006+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:49.016 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:57:49.044 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T04:57:49.109 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:49.106+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:49.185 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:49.185 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:49.195 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:57:49.210 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:49.207+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:49.310 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:49.308+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:49.342 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:49.342 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:49.351 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:57:49.411 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:49.409+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:49.497 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:49.497 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:49.506 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:57:49.511 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:49.509+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:49.610 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:49.610 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:49.612 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:49.610+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:49.618 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:57:49.713 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:49.710+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:49.720 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:49.720 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:49.728 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:57:49.814 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:49.811+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:49.831 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:49.831 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:49.839 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:57:49.915 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:49.913+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:49.982 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:49.983 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:49.991 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:57:50.035 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-18T04:57:50.036 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:50.034+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:50.102 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:50.103 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:50.112 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:57:50.117 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:50.115+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:50.218 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:50.216+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:50.254 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:50.254 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:50.263 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:57:50.320 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:50.316+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:50.399 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 5, 6, 7] out_osds: [1, 4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T04:57:50.400 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T04:57:50.400 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-18T04:57:50.401 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T04:57:50.407 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:50.407 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:50.416 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:57:50.446 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:50.442+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:50.518 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:50.518 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:50.521 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:50.518+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:50.526 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:57:50.541 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:57:50.622 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:50.620+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:50.628 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:50.629 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:50.637 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:57:50.666 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:57:50.666 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:57:50.682 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:57:50.682 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:57:50.692 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:57:50.692 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:50.692 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 100, 2024-09-18T04:57:50.692 INFO:teuthology.orchestra.run.smithi002.stdout: "fsid": "c0761d0c-69d5-4b64-b8b9-c7d098b2d06d", 2024-09-18T04:57:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: "created": "2024-09-18T04:55:58.216538+0000", 2024-09-18T04:57:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: "modified": "2024-09-18T04:57:50.028535+0000", 2024-09-18T04:57:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: "last_up_change": "2024-09-18T04:56:08.369924+0000", 2024-09-18T04:57:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: "last_in_change": "2024-09-18T04:57:39.460089+0000", 2024-09-18T04:57:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T04:57:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_num": 5804032, 2024-09-18T04:57:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_set": [ 2024-09-18T04:57:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: "nodeep-scrub", 2024-09-18T04:57:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: "pglog_hardlimit", 2024-09-18T04:57:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: "purged_snapdirs", 2024-09-18T04:57:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_deletes", 2024-09-18T04:57:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: "sortbitwise" 2024-09-18T04:57:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:57:50.693 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_version": 4, 2024-09-18T04:57:50.694 INFO:teuthology.orchestra.run.smithi002.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T04:57:50.694 INFO:teuthology.orchestra.run.smithi002.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T04:57:50.694 INFO:teuthology.orchestra.run.smithi002.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T04:57:50.694 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_snapshot": "", 2024-09-18T04:57:50.694 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_max": 3, 2024-09-18T04:57:50.694 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd": 8, 2024-09-18T04:57:50.694 INFO:teuthology.orchestra.run.smithi002.stdout: "require_min_compat_client": "luminous", 2024-09-18T04:57:50.694 INFO:teuthology.orchestra.run.smithi002.stdout: "min_compat_client": "luminous", 2024-09-18T04:57:50.694 INFO:teuthology.orchestra.run.smithi002.stdout: "require_osd_release": "reef", 2024-09-18T04:57:50.694 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_crimson": false, 2024-09-18T04:57:50.694 INFO:teuthology.orchestra.run.smithi002.stdout: "pools": [ 2024-09-18T04:57:50.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.694 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 1, 2024-09-18T04:57:50.695 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "rbd", 2024-09-18T04:57:50.695 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-18T04:56:08.983344+0000", 2024-09-18T04:57:50.695 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-18T04:57:50.695 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T04:57:50.695 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-18T04:57:50.695 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-18T04:57:50.695 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-18T04:57:50.695 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-18T04:57:50.695 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-18T04:57:50.695 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-18T04:57:50.695 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T04:57:50.695 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T04:57:50.695 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-18T04:57:50.696 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-18T04:57:50.696 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 8, 2024-09-18T04:57:50.696 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 8, 2024-09-18T04:57:50.696 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 8, 2024-09-18T04:57:50.696 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 8, 2024-09-18T04:57:50.696 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 8, 2024-09-18T04:57:50.696 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-18T04:57:50.696 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-18T04:57:50.696 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-18T04:57:50.696 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-18T04:57:50.696 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-18T04:57:50.696 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-18T04:57:50.697 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-18T04:57:50.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.697 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "17", 2024-09-18T04:57:50.697 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-18T04:57:50.697 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T04:57:50.697 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T04:57:50.697 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-18T04:57:50.697 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-18T04:57:50.697 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 3, 2024-09-18T04:57:50.697 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 17, 2024-09-18T04:57:50.697 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-18T04:57:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-18T04:57:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-18T04:57:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-18T04:57:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-18T04:57:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-18T04:57:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-18T04:57:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-18T04:57:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-18T04:57:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-18T04:57:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-18T04:57:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T04:57:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T04:57:50.698 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T04:57:50.699 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-18T04:57:50.699 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-18T04:57:50.699 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-18T04:57:50.699 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-18T04:57:50.699 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-18T04:57:50.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.699 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-18T04:57:50.699 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-18T04:57:50.699 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-18T04:57:50.699 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-18T04:57:50.699 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-18T04:57:50.699 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T04:57:50.699 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-18T04:57:50.700 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-18T04:57:50.700 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-18T04:57:50.700 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-18T04:57:50.700 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-18T04:57:50.700 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-18T04:57:50.700 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-18T04:57:50.700 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-18T04:57:50.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.700 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-18T04:57:50.700 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 3.4900000095367432, 2024-09-18T04:57:50.700 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 3.4900000095367432, 2024-09-18T04:57:50.701 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.86000001430511475, 2024-09-18T04:57:50.701 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 3, 2024-09-18T04:57:50.701 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 3, 2024-09-18T04:57:50.701 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 0.875, 2024-09-18T04:57:50.701 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-18T04:57:50.701 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-18T04:57:50.701 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.701 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 2, 2024-09-18T04:57:50.702 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": ".mgr", 2024-09-18T04:57:50.702 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-18T04:56:10.313988+0000", 2024-09-18T04:57:50.702 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-18T04:57:50.702 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-18T04:57:50.702 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-18T04:57:50.702 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-18T04:57:50.702 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-18T04:57:50.702 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-18T04:57:50.702 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-18T04:57:50.702 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-18T04:57:50.702 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T04:57:50.703 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T04:57:50.703 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-18T04:57:50.703 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-18T04:57:50.703 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 1, 2024-09-18T04:57:50.703 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 1, 2024-09-18T04:57:50.703 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 1, 2024-09-18T04:57:50.703 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 1, 2024-09-18T04:57:50.703 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 1, 2024-09-18T04:57:50.703 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-18T04:57:50.703 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-18T04:57:50.703 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-18T04:57:50.704 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-18T04:57:50.704 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-18T04:57:50.704 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-18T04:57:50.704 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-18T04:57:50.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.704 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "17", 2024-09-18T04:57:50.704 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-18T04:57:50.704 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T04:57:50.704 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T04:57:50.704 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-18T04:57:50.704 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-18T04:57:50.704 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-18T04:57:50.705 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-18T04:57:50.705 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-18T04:57:50.705 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-18T04:57:50.705 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-18T04:57:50.705 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-18T04:57:50.705 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-18T04:57:50.705 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-18T04:57:50.705 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-18T04:57:50.705 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-18T04:57:50.705 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-18T04:57:50.705 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-18T04:57:50.706 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-18T04:57:50.706 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T04:57:50.706 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T04:57:50.706 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T04:57:50.706 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-18T04:57:50.706 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-18T04:57:50.706 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-18T04:57:50.706 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-18T04:57:50.706 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-18T04:57:50.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.706 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-18T04:57:50.706 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-18T04:57:50.707 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-18T04:57:50.707 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-18T04:57:50.707 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-18T04:57:50.707 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T04:57:50.707 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-18T04:57:50.707 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-18T04:57:50.707 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-18T04:57:50.707 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-18T04:57:50.707 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-18T04:57:50.707 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-18T04:57:50.707 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_max": 32, 2024-09-18T04:57:50.707 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 1 2024-09-18T04:57:50.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.708 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-18T04:57:50.708 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": {} 2024-09-18T04:57:50.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.708 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-18T04:57:50.708 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 6.9000000953674316, 2024-09-18T04:57:50.708 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 6.9000000953674316, 2024-09-18T04:57:50.708 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.28999999165534973, 2024-09-18T04:57:50.708 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 2, 2024-09-18T04:57:50.708 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 2, 2024-09-18T04:57:50.708 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 0.875, 2024-09-18T04:57:50.708 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-18T04:57:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-18T04:57:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-18T04:57:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "pool_client.0", 2024-09-18T04:57:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-18T04:56:19.562579+0000", 2024-09-18T04:57:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-18T04:57:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T04:57:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-18T04:57:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-18T04:57:50.709 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-18T04:57:50.710 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-18T04:57:50.710 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-18T04:57:50.710 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-18T04:57:50.710 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T04:57:50.710 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T04:57:50.710 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-18T04:57:50.710 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-18T04:57:50.712 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 32, 2024-09-18T04:57:50.712 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 32, 2024-09-18T04:57:50.712 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 32, 2024-09-18T04:57:50.712 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 32, 2024-09-18T04:57:50.712 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 32, 2024-09-18T04:57:50.712 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-18T04:57:50.712 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-18T04:57:50.713 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-18T04:57:50.713 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-18T04:57:50.713 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-18T04:57:50.713 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-18T04:57:50.713 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-18T04:57:50.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.713 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "100", 2024-09-18T04:57:50.713 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-18T04:57:50.713 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T04:57:50.713 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T04:57:50.714 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-18T04:57:50.714 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-18T04:57:50.714 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 57, 2024-09-18T04:57:50.714 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 100, 2024-09-18T04:57:50.714 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-18T04:57:50.714 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-18T04:57:50.714 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-18T04:57:50.715 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-18T04:57:50.715 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-18T04:57:50.715 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-18T04:57:50.715 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-18T04:57:50.715 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-18T04:57:50.715 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-18T04:57:50.715 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-18T04:57:50.715 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-18T04:57:50.715 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T04:57:50.715 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T04:57:50.715 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T04:57:50.715 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-18T04:57:50.716 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-18T04:57:50.716 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-18T04:57:50.716 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-18T04:57:50.716 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-18T04:57:50.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.716 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-18T04:57:50.716 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-18T04:57:50.716 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-18T04:57:50.716 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-18T04:57:50.716 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-18T04:57:50.716 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T04:57:50.716 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-18T04:57:50.716 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-18T04:57:50.717 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-18T04:57:50.717 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-18T04:57:50.717 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-18T04:57:50.717 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-18T04:57:50.717 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-18T04:57:50.717 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-18T04:57:50.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.717 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-18T04:57:50.717 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 2.190000057220459, 2024-09-18T04:57:50.717 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 2.190000057220459, 2024-09-18T04:57:50.717 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.86000001430511475, 2024-09-18T04:57:50.717 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 1.8799999952316284, 2024-09-18T04:57:50.717 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 1.8799999952316284, 2024-09-18T04:57:50.717 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 0.875, 2024-09-18T04:57:50.718 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-18T04:57:50.718 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-09-18T04:57:50.718 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.718 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.718 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:57:50.718 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-18T04:57:50.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.718 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-18T04:57:50.718 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "c43f8024-3a3e-424e-aa54-398310ceb70d", 2024-09-18T04:57:50.718 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:57:50.718 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:57:50.718 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:57:50.718 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 0, 2024-09-18T04:57:50.719 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:57:50.719 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:57:50.719 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:57:50.719 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 93, 2024-09-18T04:57:50.719 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:57:50.719 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:57:50.719 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:57:50.719 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.719 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.719 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6824", 2024-09-18T04:57:50.719 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:57:50.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.720 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.720 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6825", 2024-09-18T04:57:50.720 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:57:50.720 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.720 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.720 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:57:50.720 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.720 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.720 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6826", 2024-09-18T04:57:50.720 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:57:50.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.721 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.721 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6827", 2024-09-18T04:57:50.721 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:57:50.721 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.721 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.721 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:57:50.721 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.721 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.721 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6830", 2024-09-18T04:57:50.721 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:57:50.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.722 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.722 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6831", 2024-09-18T04:57:50.722 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:57:50.722 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.722 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.722 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:57:50.722 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.722 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.722 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6828", 2024-09-18T04:57:50.722 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:57:50.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.723 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.723 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6829", 2024-09-18T04:57:50.723 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:57:50.723 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.723 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.723 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6825/4106704527", 2024-09-18T04:57:50.723 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6827/4106704527", 2024-09-18T04:57:50.723 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6831/4106704527", 2024-09-18T04:57:50.723 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6829/4106704527", 2024-09-18T04:57:50.723 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:57:50.724 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:57:50.724 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:57:50.724 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.724 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-18T04:57:50.724 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "c664b970-145c-4b6f-8852-2f6dcf1fa30d", 2024-09-18T04:57:50.724 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:57:50.724 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 0, 2024-09-18T04:57:50.724 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 0, 2024-09-18T04:57:50.724 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:57:50.724 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:57:50.724 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:57:50.725 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:57:50.725 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 19, 2024-09-18T04:57:50.725 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:57:50.725 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:57:50.725 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:57:50.725 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.725 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.725 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6808", 2024-09-18T04:57:50.725 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:57:50.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.725 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.726 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6809", 2024-09-18T04:57:50.726 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:57:50.726 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.726 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.726 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:57:50.726 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.726 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.726 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6810", 2024-09-18T04:57:50.726 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:57:50.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.727 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.727 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6811", 2024-09-18T04:57:50.727 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:57:50.727 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.727 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.727 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:57:50.727 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.728 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.728 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6814", 2024-09-18T04:57:50.728 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:57:50.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.728 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.728 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6815", 2024-09-18T04:57:50.728 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:57:50.728 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.728 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.729 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:57:50.729 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.729 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.729 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6812", 2024-09-18T04:57:50.729 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:57:50.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.729 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.730 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6813", 2024-09-18T04:57:50.730 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:57:50.730 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.730 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.730 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6809/1781959259", 2024-09-18T04:57:50.730 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6811/1781959259", 2024-09-18T04:57:50.730 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6815/1781959259", 2024-09-18T04:57:50.730 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6813/1781959259", 2024-09-18T04:57:50.730 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:57:50.730 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:57:50.731 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:57:50.731 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.731 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-18T04:57:50.731 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "0609ea8c-d975-490e-8844-04a3e4512efd", 2024-09-18T04:57:50.731 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:57:50.731 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:57:50.731 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:57:50.731 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:57:50.731 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:57:50.732 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:57:50.732 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:57:50.732 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 92, 2024-09-18T04:57:50.732 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:57:50.732 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:57:50.732 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:57:50.732 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.732 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.732 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6800", 2024-09-18T04:57:50.732 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:57:50.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.733 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.733 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6801", 2024-09-18T04:57:50.733 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:57:50.733 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.733 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.733 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:57:50.733 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.734 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.734 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6802", 2024-09-18T04:57:50.734 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:57:50.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.734 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.734 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6803", 2024-09-18T04:57:50.734 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:57:50.734 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.734 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.735 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:57:50.735 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.735 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.735 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6806", 2024-09-18T04:57:50.735 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:57:50.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.735 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.736 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6807", 2024-09-18T04:57:50.736 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:57:50.736 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.736 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.736 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:57:50.736 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.737 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.737 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6804", 2024-09-18T04:57:50.737 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:57:50.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.737 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.737 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6805", 2024-09-18T04:57:50.737 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:57:50.737 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.737 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.738 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6801/1153570802", 2024-09-18T04:57:50.738 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6803/1153570802", 2024-09-18T04:57:50.738 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6807/1153570802", 2024-09-18T04:57:50.738 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6805/1153570802", 2024-09-18T04:57:50.738 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:57:50.739 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:57:50.739 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:57:50.739 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.739 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-18T04:57:50.739 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "c6bd5ac8-ab89-4fa2-b843-3b166d1b238b", 2024-09-18T04:57:50.739 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:57:50.739 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:57:50.739 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:57:50.739 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:57:50.739 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:57:50.740 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:57:50.740 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:57:50.740 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 92, 2024-09-18T04:57:50.740 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:57:50.740 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:57:50.740 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:57:50.740 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.740 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.741 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6816", 2024-09-18T04:57:50.741 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:57:50.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.741 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.741 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6817", 2024-09-18T04:57:50.741 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:57:50.741 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.741 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.741 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:57:50.741 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.742 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.742 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6818", 2024-09-18T04:57:50.742 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:57:50.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.742 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.742 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6819", 2024-09-18T04:57:50.742 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:57:50.743 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.743 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.743 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:57:50.743 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.743 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.743 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6822", 2024-09-18T04:57:50.743 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:57:50.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.743 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.744 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6823", 2024-09-18T04:57:50.744 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:57:50.744 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.744 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.744 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:57:50.744 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.744 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.744 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6820", 2024-09-18T04:57:50.745 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:57:50.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.745 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.745 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6821", 2024-09-18T04:57:50.745 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:57:50.745 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.745 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.745 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6817/2008321375", 2024-09-18T04:57:50.745 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6819/2008321375", 2024-09-18T04:57:50.746 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6823/2008321375", 2024-09-18T04:57:50.746 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6821/2008321375", 2024-09-18T04:57:50.746 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:57:50.746 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:57:50.746 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:57:50.746 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.746 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-18T04:57:50.747 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "89d6a371-4f4e-465e-a023-d5083cce6e6c", 2024-09-18T04:57:50.747 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:57:50.747 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 0, 2024-09-18T04:57:50.747 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 0, 2024-09-18T04:57:50.747 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:57:50.747 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:57:50.747 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:57:50.747 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:57:50.747 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 77, 2024-09-18T04:57:50.748 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:57:50.748 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:57:50.748 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:57:50.748 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.748 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.748 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6824", 2024-09-18T04:57:50.748 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:57:50.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.748 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.749 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6825", 2024-09-18T04:57:50.749 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:57:50.749 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.749 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.749 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:57:50.749 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.749 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.749 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6826", 2024-09-18T04:57:50.749 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:57:50.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.750 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.750 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6827", 2024-09-18T04:57:50.750 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:57:50.750 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.750 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.750 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:57:50.750 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.750 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.750 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6830", 2024-09-18T04:57:50.750 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:57:50.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.751 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.751 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6831", 2024-09-18T04:57:50.751 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:57:50.751 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.751 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.751 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:57:50.751 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.751 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.751 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6828", 2024-09-18T04:57:50.751 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:57:50.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.752 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.752 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6829", 2024-09-18T04:57:50.752 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:57:50.752 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.752 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.752 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6825/2939896598", 2024-09-18T04:57:50.752 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6827/2939896598", 2024-09-18T04:57:50.752 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6831/2939896598", 2024-09-18T04:57:50.752 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6829/2939896598", 2024-09-18T04:57:50.752 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:57:50.752 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:57:50.753 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:57:50.753 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.753 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-18T04:57:50.753 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "da99686b-d979-4753-b4aa-a3b26e9a0497", 2024-09-18T04:57:50.753 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:57:50.753 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:57:50.753 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:57:50.753 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:57:50.753 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:57:50.753 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:57:50.753 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:57:50.754 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 92, 2024-09-18T04:57:50.754 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:57:50.754 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:57:50.754 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:57:50.754 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.754 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.754 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6816", 2024-09-18T04:57:50.754 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:57:50.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.754 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.754 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6817", 2024-09-18T04:57:50.755 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:57:50.755 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.755 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.755 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:57:50.755 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.755 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.755 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6818", 2024-09-18T04:57:50.755 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:57:50.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.755 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.755 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6819", 2024-09-18T04:57:50.756 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:57:50.756 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.756 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.756 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:57:50.756 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.756 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.756 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6822", 2024-09-18T04:57:50.756 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:57:50.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.757 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.757 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6823", 2024-09-18T04:57:50.757 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:57:50.757 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.757 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.757 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:57:50.757 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.757 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.757 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6820", 2024-09-18T04:57:50.758 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:57:50.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.758 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.758 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6821", 2024-09-18T04:57:50.758 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:57:50.758 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.758 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.758 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6817/3440879618", 2024-09-18T04:57:50.758 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6819/3440879618", 2024-09-18T04:57:50.758 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6823/3440879618", 2024-09-18T04:57:50.758 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6821/3440879618", 2024-09-18T04:57:50.759 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:57:50.759 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:57:50.759 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:57:50.759 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.759 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-18T04:57:50.759 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "ceec6aa6-603d-4706-bfea-3c07dbdaed3a", 2024-09-18T04:57:50.759 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:57:50.759 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:57:50.759 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:57:50.759 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:57:50.759 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:57:50.760 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:57:50.760 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:57:50.760 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 94, 2024-09-18T04:57:50.760 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:57:50.760 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:57:50.760 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:57:50.760 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.760 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.760 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6800", 2024-09-18T04:57:50.760 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:57:50.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.761 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.761 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6801", 2024-09-18T04:57:50.761 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:57:50.761 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.761 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.761 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:57:50.761 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.761 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.761 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6802", 2024-09-18T04:57:50.761 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:57:50.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.762 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.762 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6803", 2024-09-18T04:57:50.762 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:57:50.762 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.762 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.762 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:57:50.762 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.763 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.763 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6806", 2024-09-18T04:57:50.763 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:57:50.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.763 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.763 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6807", 2024-09-18T04:57:50.763 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:57:50.763 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.763 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.764 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:57:50.764 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.764 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.764 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6804", 2024-09-18T04:57:50.764 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:57:50.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.764 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.765 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6805", 2024-09-18T04:57:50.765 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:57:50.765 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.765 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.765 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6801/2493830504", 2024-09-18T04:57:50.765 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6803/2493830504", 2024-09-18T04:57:50.765 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6807/2493830504", 2024-09-18T04:57:50.765 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6805/2493830504", 2024-09-18T04:57:50.765 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:57:50.765 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:57:50.766 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:57:50.766 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.766 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-18T04:57:50.766 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "6779dfad-f60b-41f4-b666-a99db6dc1814", 2024-09-18T04:57:50.766 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:57:50.766 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:57:50.766 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:57:50.766 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:57:50.766 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:57:50.767 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:57:50.767 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:57:50.767 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 93, 2024-09-18T04:57:50.767 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:57:50.767 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:57:50.767 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:57:50.767 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.767 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.767 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6808", 2024-09-18T04:57:50.767 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:57:50.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.768 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.768 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6809", 2024-09-18T04:57:50.768 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:57:50.768 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.768 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.768 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:57:50.768 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.769 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.769 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6810", 2024-09-18T04:57:50.769 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:57:50.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.769 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.769 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6811", 2024-09-18T04:57:50.769 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:57:50.769 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.770 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.770 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:57:50.770 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.770 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.770 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6814", 2024-09-18T04:57:50.770 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:57:50.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.771 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.771 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6815", 2024-09-18T04:57:50.771 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:57:50.771 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.771 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.771 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:57:50.771 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:57:50.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.772 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:57:50.772 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6812", 2024-09-18T04:57:50.772 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:57:50.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.772 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:57:50.772 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6813", 2024-09-18T04:57:50.773 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:57:50.773 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.773 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.773 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6809/91038858", 2024-09-18T04:57:50.773 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6811/91038858", 2024-09-18T04:57:50.773 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6815/91038858", 2024-09-18T04:57:50.773 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6813/91038858", 2024-09-18T04:57:50.773 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:57:50.773 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:57:50.773 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:57:50.773 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.773 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.774 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:57:50.774 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_xinfo": [ 2024-09-18T04:57:50.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.774 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-18T04:57:50.774 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:57:50.774 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:57:50.774 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:57:50.774 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:57:50.774 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:57:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.468213+0000", 2024-09-18T04:57:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:57:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-18T04:57:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:57:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:57:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:57:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:57:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 65536, 2024-09-18T04:57:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.538610+0000", 2024-09-18T04:57:50.775 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:57:50.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.776 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-18T04:57:50.776 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:57:50.776 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:57:50.776 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:57:50.776 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:57:50.776 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:57:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.500881+0000", 2024-09-18T04:57:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:57:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-18T04:57:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:57:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:57:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:57:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:57:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:57:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.474179+0000", 2024-09-18T04:57:50.777 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:57:50.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.778 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-18T04:57:50.778 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:57:50.778 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:57:50.778 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:57:50.778 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:57:50.778 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 65536, 2024-09-18T04:57:50.778 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.483737+0000", 2024-09-18T04:57:50.778 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:57:50.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.779 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-18T04:57:50.779 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:57:50.779 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:57:50.779 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:57:50.779 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:57:50.779 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:57:50.779 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.460693+0000", 2024-09-18T04:57:50.779 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:57:50.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.780 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-18T04:57:50.780 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:57:50.780 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:57:50.780 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:57:50.780 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:57:50.780 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:57:50.780 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.475970+0000", 2024-09-18T04:57:50.781 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:57:50.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.781 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-18T04:57:50.781 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:57:50.781 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:57:50.781 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:57:50.781 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:57:50.781 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:57:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.519287+0000", 2024-09-18T04:57:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:57:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:57:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap": [ 2024-09-18T04:57:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "1.0", 2024-09-18T04:57:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-18T04:57:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: 6, 2024-09-18T04:57:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: 4 2024-09-18T04:57:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "1.4", 2024-09-18T04:57:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-18T04:57:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: 6, 2024-09-18T04:57:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: 7 2024-09-18T04:57:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:57:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_items": [], 2024-09-18T04:57:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_primaries": [], 2024-09-18T04:57:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_temp": [], 2024-09-18T04:57:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_temp": [], 2024-09-18T04:57:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: "blocklist": {}, 2024-09-18T04:57:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: "range_blocklist": {}, 2024-09-18T04:57:50.783 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profiles": { 2024-09-18T04:57:50.784 INFO:teuthology.orchestra.run.smithi002.stdout: "default": { 2024-09-18T04:57:50.784 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-failure-domain": "osd", 2024-09-18T04:57:50.784 INFO:teuthology.orchestra.run.smithi002.stdout: "k": "2", 2024-09-18T04:57:50.784 INFO:teuthology.orchestra.run.smithi002.stdout: "m": "1", 2024-09-18T04:57:50.784 INFO:teuthology.orchestra.run.smithi002.stdout: "plugin": "jerasure", 2024-09-18T04:57:50.784 INFO:teuthology.orchestra.run.smithi002.stdout: "technique": "reed_sol_van" 2024-09-18T04:57:50.784 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:50.784 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps_queue": [ 2024-09-18T04:57:50.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.784 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-18T04:57:50.784 INFO:teuthology.orchestra.run.smithi002.stdout: "snaps": [ 2024-09-18T04:57:50.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.785 INFO:teuthology.orchestra.run.smithi002.stdout: "begin": 40, 2024-09-18T04:57:50.785 INFO:teuthology.orchestra.run.smithi002.stdout: "length": 7 2024-09-18T04:57:50.785 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.785 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.785 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.785 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:57:50.785 INFO:teuthology.orchestra.run.smithi002.stdout: "new_removed_snaps": [ 2024-09-18T04:57:50.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.785 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-18T04:57:50.785 INFO:teuthology.orchestra.run.smithi002.stdout: "snaps": [ 2024-09-18T04:57:50.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:50.785 INFO:teuthology.orchestra.run.smithi002.stdout: "begin": 40, 2024-09-18T04:57:50.785 INFO:teuthology.orchestra.run.smithi002.stdout: "length": 1 2024-09-18T04:57:50.786 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.786 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:50.786 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.786 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:57:50.786 INFO:teuthology.orchestra.run.smithi002.stdout: "new_purged_snaps": [], 2024-09-18T04:57:50.786 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_node_flags": {}, 2024-09-18T04:57:50.786 INFO:teuthology.orchestra.run.smithi002.stdout: "device_class_flags": {}, 2024-09-18T04:57:50.786 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode": { 2024-09-18T04:57:50.786 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_enabled": false, 2024-09-18T04:57:50.786 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_bucket_count": 0, 2024-09-18T04:57:50.786 INFO:teuthology.orchestra.run.smithi002.stdout: "degraded_stretch_mode": 0, 2024-09-18T04:57:50.786 INFO:teuthology.orchestra.run.smithi002.stdout: "recovering_stretch_mode": 0, 2024-09-18T04:57:50.786 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_bucket": 0 2024-09-18T04:57:50.787 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:50.787 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:50.788 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:50.788 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:50.788 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:57:50.788 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:50.788 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:57:50.788 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:57:50.790 INFO:tasks.thrashosds.thrasher:j is {'epoch': 100, 'fsid': 'c0761d0c-69d5-4b64-b8b9-c7d098b2d06d', 'created': '2024-09-18T04:55:58.216538+0000', 'modified': '2024-09-18T04:57:50.028535+0000', 'last_up_change': '2024-09-18T04:56:08.369924+0000', 'last_in_change': '2024-09-18T04:57:39.460089+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-18T04:56:08.983344+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 3.490000009536743, 'score_stable': 3.490000009536743, 'optimal_score': 0.8600000143051147, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-18T04:56:10.313988+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T04:56:19.562579+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 57, 'snap_epoch': 100, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2.190000057220459, 'score_stable': 2.190000057220459, 'optimal_score': 0.8600000143051147, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'c43f8024-3a3e-424e-aa54-398310ceb70d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 93, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6824', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6825', 'nonce': 4106704527}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6826', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6827', 'nonce': 4106704527}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6830', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6831', 'nonce': 4106704527}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6828', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6829', 'nonce': 4106704527}]}, 'public_addr': '172.21.15.2:6825/4106704527', 'cluster_addr': '172.21.15.2:6827/4106704527', 'heartbeat_back_addr': '172.21.15.2:6831/4106704527', 'heartbeat_front_addr': '172.21.15.2:6829/4106704527', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c664b970-145c-4b6f-8852-2f6dcf1fa30d', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 19, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6808', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6809', 'nonce': 1781959259}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6810', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6811', 'nonce': 1781959259}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6814', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6815', 'nonce': 1781959259}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6812', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6813', 'nonce': 1781959259}]}, 'public_addr': '172.21.15.2:6809/1781959259', 'cluster_addr': '172.21.15.2:6811/1781959259', 'heartbeat_back_addr': '172.21.15.2:6815/1781959259', 'heartbeat_front_addr': '172.21.15.2:6813/1781959259', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0609ea8c-d975-490e-8844-04a3e4512efd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 92, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6800', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6801', 'nonce': 1153570802}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6802', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6803', 'nonce': 1153570802}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6806', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6807', 'nonce': 1153570802}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6804', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6805', 'nonce': 1153570802}]}, 'public_addr': '172.21.15.2:6801/1153570802', 'cluster_addr': '172.21.15.2:6803/1153570802', 'heartbeat_back_addr': '172.21.15.2:6807/1153570802', 'heartbeat_front_addr': '172.21.15.2:6805/1153570802', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'c6bd5ac8-ab89-4fa2-b843-3b166d1b238b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 92, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6816', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6817', 'nonce': 2008321375}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6818', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6819', 'nonce': 2008321375}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6822', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6823', 'nonce': 2008321375}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6820', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6821', 'nonce': 2008321375}]}, 'public_addr': '172.21.15.2:6817/2008321375', 'cluster_addr': '172.21.15.2:6819/2008321375', 'heartbeat_back_addr': '172.21.15.2:6823/2008321375', 'heartbeat_front_addr': '172.21.15.2:6821/2008321375', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '89d6a371-4f4e-465e-a023-d5083cce6e6c', 'up': 1, 'in': 0, 'weight': 0, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 77, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6824', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6825', 'nonce': 2939896598}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6826', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6827', 'nonce': 2939896598}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6830', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6831', 'nonce': 2939896598}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6828', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6829', 'nonce': 2939896598}]}, 'public_addr': '172.21.15.99:6825/2939896598', 'cluster_addr': '172.21.15.99:6827/2939896598', 'heartbeat_back_addr': '172.21.15.99:6831/2939896598', 'heartbeat_front_addr': '172.21.15.99:6829/2939896598', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'da99686b-d979-4753-b4aa-a3b26e9a0497', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 92, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6816', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6817', 'nonce': 3440879618}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6818', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6819', 'nonce': 3440879618}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6822', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6823', 'nonce': 3440879618}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6820', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6821', 'nonce': 3440879618}]}, 'public_addr': '172.21.15.99:6817/3440879618', 'cluster_addr': '172.21.15.99:6819/3440879618', 'heartbeat_back_addr': '172.21.15.99:6823/3440879618', 'heartbeat_front_addr': '172.21.15.99:6821/3440879618', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ceec6aa6-603d-4706-bfea-3c07dbdaed3a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 94, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6800', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6801', 'nonce': 2493830504}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6802', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6803', 'nonce': 2493830504}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6806', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6807', 'nonce': 2493830504}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6804', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6805', 'nonce': 2493830504}]}, 'public_addr': '172.21.15.99:6801/2493830504', 'cluster_addr': '172.21.15.99:6803/2493830504', 'heartbeat_back_addr': '172.21.15.99:6807/2493830504', 'heartbeat_front_addr': '172.21.15.99:6805/2493830504', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '6779dfad-f60b-41f4-b666-a99db6dc1814', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 93, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6808', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6809', 'nonce': 91038858}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6810', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6811', 'nonce': 91038858}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6814', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6815', 'nonce': 91038858}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6812', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6813', 'nonce': 91038858}]}, 'public_addr': '172.21.15.99:6809/91038858', 'cluster_addr': '172.21.15.99:6811/91038858', 'heartbeat_back_addr': '172.21.15.99:6815/91038858', 'heartbeat_front_addr': '172.21.15.99:6813/91038858', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.468213+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.538610+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.500881+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.474179+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 65536, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.483737+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.460693+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.475970+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.519287+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '1.0', 'osds': [6, 4]}, {'pgid': '1.4', 'osds': [6, 7]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 40, 'length': 7}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 40, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T04:57:50.790 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 1.0 2024-09-18T04:57:50.790 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 1.0 2024-09-18T04:57:50.790 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:57:50.791 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:57:50.791 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:57:50.791 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:57:50.792 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:57:50.792 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:57:50.792 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:57:50.792 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:57:50.793 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:57:50.793 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:57:50.794 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:50.785+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:50.795 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:57:50.889 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:50.885+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:50.896 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:57:50.896 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:57:50.896 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:50.905 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:57:50.990 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:50.988+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:51.009 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:51.009 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:51.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:51.010 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:51.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.010 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:57:51.010 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:57:51.010 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 99.612823352000007, 2024-09-18T04:57:51.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037718511000000003, 2024-09-18T04:57:51.010 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:51.010 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:51.010 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:51.010 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:57:51.011 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:57:51.011 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-18T04:57:51.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.011 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:51.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:51.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:57:51.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:51.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:57:51.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:51.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393125+0000", 2024-09-18T04:57:51.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:57:51.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:51.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393133+0000", 2024-09-18T04:57:51.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.225e-06 2024-09-18T04:57:51.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:51.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393144+0000", 2024-09-18T04:57:51.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1279e-05 2024-09-18T04:57:51.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393153+0000", 2024-09-18T04:57:51.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4540000000000007e-06 2024-09-18T04:57:51.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:51.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393203+0000", 2024-09-18T04:57:51.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9963000000000003e-05 2024-09-18T04:57:51.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:57:51.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393210+0000", 2024-09-18T04:57:51.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5689999999999999e-06 2024-09-18T04:57:51.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:51.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427399+0000", 2024-09-18T04:57:51.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034188784999999999 2024-09-18T04:57:51.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:51.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427532+0000", 2024-09-18T04:57:51.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013264700000000001 2024-09-18T04:57:51.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:57:51.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427685+0000", 2024-09-18T04:57:51.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015300599999999999 2024-09-18T04:57:51.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:51.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.428520+0000", 2024-09-18T04:57:51.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00083502499999999998 2024-09-18T04:57:51.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:57:51.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430772+0000", 2024-09-18T04:57:51.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022518880000000001 2024-09-18T04:57:51.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:57:51.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430817+0000", 2024-09-18T04:57:51.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5077000000000002e-05 2024-09-18T04:57:51.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:51.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430845+0000", 2024-09-18T04:57:51.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8036999999999999e-05 2024-09-18T04:57:51.017 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.018 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:51.018 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.018 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:62520 3.13 3.84cdb893 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T04:57:51.018 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:51.018 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 59.708513469000003, 2024-09-18T04:57:51.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94452134799999998, 2024-09-18T04:57:51.018 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:51.018 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:57:51.018 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:51.019 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:51.019 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.019 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 62520 2024-09-18T04:57:51.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.019 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:51.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:51.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:51.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:51.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:51.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:51.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:57:51.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:51.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297441+0000", 2024-09-18T04:57:51.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9840000000000001e-06 2024-09-18T04:57:51.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:51.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297449+0000", 2024-09-18T04:57:51.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3999999999999992e-06 2024-09-18T04:57:51.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297455+0000", 2024-09-18T04:57:51.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3350000000000003e-06 2024-09-18T04:57:51.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:51.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297552+0000", 2024-09-18T04:57:51.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6602999999999996e-05 2024-09-18T04:57:51.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:57:51.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297559+0000", 2024-09-18T04:57:51.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.52e-06 2024-09-18T04:57:51.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:51.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241943+0000", 2024-09-18T04:57:51.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94438357100000003 2024-09-18T04:57:51.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:51.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241958+0000", 2024-09-18T04:57:51.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4598e-05 2024-09-18T04:57:51.027 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.027 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:51.027 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.028 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:113179 3.13 3:cf138017:::rbd_data.11ac5f86a3f9.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T04:57:51.028 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:57:51.028 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.318723058000003, 2024-09-18T04:57:51.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036508304999999998, 2024-09-18T04:57:51.028 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:51.028 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:51.028 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:51.028 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:51.028 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.029 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 113179 2024-09-18T04:57:51.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.029 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:51.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:51.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:57:51.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:51.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:57:51.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:51.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687226+0000", 2024-09-18T04:57:51.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:57:51.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:51.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688169+0000", 2024-09-18T04:57:51.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00094328200000000004 2024-09-18T04:57:51.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:51.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688182+0000", 2024-09-18T04:57:51.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2756e-05 2024-09-18T04:57:51.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688188+0000", 2024-09-18T04:57:51.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0290000000000002e-06 2024-09-18T04:57:51.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:51.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688225+0000", 2024-09-18T04:57:51.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7301000000000001e-05 2024-09-18T04:57:51.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:51.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688324+0000", 2024-09-18T04:57:51.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9419999999999993e-05 2024-09-18T04:57:51.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:57:51.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.700483+0000", 2024-09-18T04:57:51.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012158771000000001 2024-09-18T04:57:51.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:57:51.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723586+0000", 2024-09-18T04:57:51.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023102952 2024-09-18T04:57:51.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:51.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723634+0000", 2024-09-18T04:57:51.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7468e-05 2024-09-18T04:57:51.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:57:51.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723693+0000", 2024-09-18T04:57:51.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9539e-05 2024-09-18T04:57:51.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:51.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723735+0000", 2024-09-18T04:57:51.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1829000000000002e-05 2024-09-18T04:57:51.034 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.034 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:51.035 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.035 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146077 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1400832~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:51.035 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593265+0000", 2024-09-18T04:57:51.035 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.412684680000002, 2024-09-18T04:57:51.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035380397000000001, 2024-09-18T04:57:51.035 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:51.035 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:51.035 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:51.035 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:51.035 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.035 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146077 2024-09-18T04:57:51.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.036 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:51.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:51.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593265+0000", 2024-09-18T04:57:51.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:51.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593265+0000", 2024-09-18T04:57:51.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:51.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593265+0000", 2024-09-18T04:57:51.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:57:51.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:51.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593266+0000", 2024-09-18T04:57:51.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1039999999999999e-06 2024-09-18T04:57:51.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:51.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593269+0000", 2024-09-18T04:57:51.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8760000000000001e-06 2024-09-18T04:57:51.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593271+0000", 2024-09-18T04:57:51.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5100000000000001e-06 2024-09-18T04:57:51.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:51.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628545+0000", 2024-09-18T04:57:51.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035273977999999998 2024-09-18T04:57:51.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:51.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628589+0000", 2024-09-18T04:57:51.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4088999999999997e-05 2024-09-18T04:57:51.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:51.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628645+0000", 2024-09-18T04:57:51.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6059e-05 2024-09-18T04:57:51.039 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.039 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:51.039 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.039 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146078 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1449984~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:51.039 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:57:51.039 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.412645215000001, 2024-09-18T04:57:51.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035495325000000001, 2024-09-18T04:57:51.039 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:51.039 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:51.040 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:51.040 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:51.040 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.040 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146078 2024-09-18T04:57:51.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.040 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:51.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:51.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:57:51.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:51.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:57:51.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:51.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:57:51.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:51.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593306+0000", 2024-09-18T04:57:51.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.686e-06 2024-09-18T04:57:51.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:51.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593311+0000", 2024-09-18T04:57:51.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8500000000000002e-06 2024-09-18T04:57:51.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593314+0000", 2024-09-18T04:57:51.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0290000000000001e-06 2024-09-18T04:57:51.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:51.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628674+0000", 2024-09-18T04:57:51.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035360346000000001 2024-09-18T04:57:51.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:51.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628726+0000", 2024-09-18T04:57:51.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2219999999999998e-05 2024-09-18T04:57:51.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:51.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628800+0000", 2024-09-18T04:57:51.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3613000000000001e-05 2024-09-18T04:57:51.043 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.043 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:51.043 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.043 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146079 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1499136~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:51.044 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:57:51.044 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.412633809999999, 2024-09-18T04:57:51.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035604581000000003, 2024-09-18T04:57:51.044 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:51.044 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:51.044 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:51.044 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:51.044 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.044 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146079 2024-09-18T04:57:51.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.044 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:51.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:51.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:57:51.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:51.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:57:51.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:51.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:57:51.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:57:51.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:51.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593317+0000", 2024-09-18T04:57:51.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1599999999999999e-06 2024-09-18T04:57:51.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:51.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593320+0000", 2024-09-18T04:57:51.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0130000000000002e-06 2024-09-18T04:57:51.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593322+0000", 2024-09-18T04:57:51.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6079999999999998e-06 2024-09-18T04:57:51.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:51.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628825+0000", 2024-09-18T04:57:51.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035502747000000001 2024-09-18T04:57:51.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:51.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628868+0000", 2024-09-18T04:57:51.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2719999999999998e-05 2024-09-18T04:57:51.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:51.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628920+0000", 2024-09-18T04:57:51.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2533e-05 2024-09-18T04:57:51.048 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.048 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:51.048 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.048 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146080 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1548288~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:51.048 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:57:51.048 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.412618311999999, 2024-09-18T04:57:51.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035755146000000002, 2024-09-18T04:57:51.049 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:51.049 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:51.049 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:51.049 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:51.049 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.049 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146080 2024-09-18T04:57:51.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.049 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:51.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:51.049 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:57:51.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:51.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:57:51.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:51.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:57:51.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:51.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593333+0000", 2024-09-18T04:57:51.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.858e-06 2024-09-18T04:57:51.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:51.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593336+0000", 2024-09-18T04:57:51.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4910000000000001e-06 2024-09-18T04:57:51.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593340+0000", 2024-09-18T04:57:51.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2590000000000001e-06 2024-09-18T04:57:51.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:51.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628956+0000", 2024-09-18T04:57:51.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035616039000000002 2024-09-18T04:57:51.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:51.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629031+0000", 2024-09-18T04:57:51.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5504000000000001e-05 2024-09-18T04:57:51.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:51.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629086+0000", 2024-09-18T04:57:51.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5429000000000001e-05 2024-09-18T04:57:51.052 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.052 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:51.052 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.053 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146081 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1597440~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:51.053 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:57:51.053 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.412607739999999, 2024-09-18T04:57:51.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035873223000000003, 2024-09-18T04:57:51.053 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:51.053 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:51.053 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:51.053 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:51.053 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.053 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146081 2024-09-18T04:57:51.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.053 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:51.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:51.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:57:51.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:51.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:57:51.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:51.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:57:51.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:51.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593343+0000", 2024-09-18T04:57:51.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3319999999999999e-06 2024-09-18T04:57:51.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:51.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593346+0000", 2024-09-18T04:57:51.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9059999999999998e-06 2024-09-18T04:57:51.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593348+0000", 2024-09-18T04:57:51.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4530000000000001e-06 2024-09-18T04:57:51.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:51.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629110+0000", 2024-09-18T04:57:51.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035761814000000003 2024-09-18T04:57:51.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:51.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629159+0000", 2024-09-18T04:57:51.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8971000000000003e-05 2024-09-18T04:57:51.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:51.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629215+0000", 2024-09-18T04:57:51.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6107000000000003e-05 2024-09-18T04:57:51.057 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.057 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:51.057 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.057 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146082 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1646592~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:51.057 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:57:51.057 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.412591689999999, 2024-09-18T04:57:51.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035987007000000001, 2024-09-18T04:57:51.058 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:51.058 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:51.058 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:51.058 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:51.058 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.058 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146082 2024-09-18T04:57:51.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.058 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:51.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:51.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:57:51.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:51.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:57:51.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:51.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:57:51.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:51.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593360+0000", 2024-09-18T04:57:51.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8560000000000001e-06 2024-09-18T04:57:51.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:51.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593365+0000", 2024-09-18T04:57:51.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3410000000000002e-06 2024-09-18T04:57:51.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593369+0000", 2024-09-18T04:57:51.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4850000000000003e-06 2024-09-18T04:57:51.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:51.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629242+0000", 2024-09-18T04:57:51.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035872464999999999 2024-09-18T04:57:51.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:51.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629288+0000", 2024-09-18T04:57:51.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6592999999999999e-05 2024-09-18T04:57:51.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:51.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629345+0000", 2024-09-18T04:57:51.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6575e-05 2024-09-18T04:57:51.063 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.063 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:51.063 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.063 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146083 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1695744~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:51.063 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:57:51.063 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.412575842999999, 2024-09-18T04:57:51.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036100587000000003, 2024-09-18T04:57:51.063 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:51.063 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:51.064 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:51.064 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:51.064 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.064 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146083 2024-09-18T04:57:51.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.064 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:51.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:51.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:57:51.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:51.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:57:51.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:51.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593373+0000", 2024-09-18T04:57:51.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:51.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593376+0000", 2024-09-18T04:57:51.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.244e-06 2024-09-18T04:57:51.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:51.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593379+0000", 2024-09-18T04:57:51.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.619e-06 2024-09-18T04:57:51.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593382+0000", 2024-09-18T04:57:51.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7199999999999998e-06 2024-09-18T04:57:51.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:51.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629368+0000", 2024-09-18T04:57:51.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035986281000000002 2024-09-18T04:57:51.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:51.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629416+0000", 2024-09-18T04:57:51.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8055e-05 2024-09-18T04:57:51.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:51.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629474+0000", 2024-09-18T04:57:51.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8176e-05 2024-09-18T04:57:51.067 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.067 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:51.068 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.068 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146084 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1744896~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:51.068 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:57:51.068 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.412562713, 2024-09-18T04:57:51.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036214349, 2024-09-18T04:57:51.068 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:51.068 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:51.068 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:51.068 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:51.068 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.068 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146084 2024-09-18T04:57:51.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.069 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:51.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:51.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:57:51.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:51.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:57:51.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:51.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:57:51.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:51.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593388+0000", 2024-09-18T04:57:51.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.564e-06 2024-09-18T04:57:51.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:51.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593392+0000", 2024-09-18T04:57:51.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0579999999999999e-06 2024-09-18T04:57:51.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593395+0000", 2024-09-18T04:57:51.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8890000000000002e-06 2024-09-18T04:57:51.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:51.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629500+0000", 2024-09-18T04:57:51.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036105343999999998 2024-09-18T04:57:51.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:51.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629547+0000", 2024-09-18T04:57:51.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7036000000000002e-05 2024-09-18T04:57:51.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:51.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629601+0000", 2024-09-18T04:57:51.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3866999999999998e-05 2024-09-18T04:57:51.072 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.072 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:51.072 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.072 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146085 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1794048~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:51.072 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:57:51.072 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.412552326, 2024-09-18T04:57:51.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036331186000000001, 2024-09-18T04:57:51.072 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:51.072 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:51.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:51.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:51.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.073 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146085 2024-09-18T04:57:51.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.073 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:51.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:51.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:57:51.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:51.073 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:57:51.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:51.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:57:51.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:57:51.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:51.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593398+0000", 2024-09-18T04:57:51.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2270000000000001e-06 2024-09-18T04:57:51.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:51.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593401+0000", 2024-09-18T04:57:51.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.999e-06 2024-09-18T04:57:51.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593404+0000", 2024-09-18T04:57:51.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.486e-06 2024-09-18T04:57:51.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:51.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629624+0000", 2024-09-18T04:57:51.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036219983999999997 2024-09-18T04:57:51.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:51.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629672+0000", 2024-09-18T04:57:51.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8368000000000002e-05 2024-09-18T04:57:51.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:51.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629729+0000", 2024-09-18T04:57:51.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6359e-05 2024-09-18T04:57:51.076 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.076 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:51.076 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.077 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146086 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1843200~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:51.077 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:57:51.077 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.412543761999999, 2024-09-18T04:57:51.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036449098999999999, 2024-09-18T04:57:51.077 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:51.077 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:51.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:51.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:51.077 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.077 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146086 2024-09-18T04:57:51.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.077 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:51.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:51.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:57:51.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:51.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:57:51.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:51.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:57:51.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:57:51.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:51.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593407+0000", 2024-09-18T04:57:51.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1799999999999999e-06 2024-09-18T04:57:51.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:51.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593410+0000", 2024-09-18T04:57:51.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.926e-06 2024-09-18T04:57:51.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593413+0000", 2024-09-18T04:57:51.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8250000000000001e-06 2024-09-18T04:57:51.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:51.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629755+0000", 2024-09-18T04:57:51.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036342212999999998 2024-09-18T04:57:51.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:51.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629803+0000", 2024-09-18T04:57:51.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8619000000000003e-05 2024-09-18T04:57:51.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:51.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629855+0000", 2024-09-18T04:57:51.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1536000000000003e-05 2024-09-18T04:57:51.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.081 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:51.081 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.081 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146087 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1892352~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:51.081 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:57:51.081 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.412534836999999, 2024-09-18T04:57:51.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036564608999999998, 2024-09-18T04:57:51.081 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:51.081 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:51.081 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:51.081 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:51.081 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.082 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146087 2024-09-18T04:57:51.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.082 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:51.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:51.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:57:51.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.082 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:51.082 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:57:51.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:51.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:57:51.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:57:51.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:51.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593416+0000", 2024-09-18T04:57:51.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.218e-06 2024-09-18T04:57:51.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:51.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593419+0000", 2024-09-18T04:57:51.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-18T04:57:51.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593421+0000", 2024-09-18T04:57:51.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4329999999999998e-06 2024-09-18T04:57:51.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:51.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629878+0000", 2024-09-18T04:57:51.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036456578000000003 2024-09-18T04:57:51.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:51.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629925+0000", 2024-09-18T04:57:51.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7595000000000001e-05 2024-09-18T04:57:51.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:51.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629979+0000", 2024-09-18T04:57:51.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4117999999999999e-05 2024-09-18T04:57:51.085 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.085 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:51.085 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.085 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146088 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1941504~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:51.085 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:57:51.085 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.412512672999998, 2024-09-18T04:57:51.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036676121999999999, 2024-09-18T04:57:51.086 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:51.086 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:51.086 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:51.086 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:51.086 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.086 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146088 2024-09-18T04:57:51.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.086 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:51.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:51.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:57:51.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:51.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:57:51.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:51.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:57:51.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:51.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593438+0000", 2024-09-18T04:57:51.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7570000000000001e-06 2024-09-18T04:57:51.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:51.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593442+0000", 2024-09-18T04:57:51.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2880000000000001e-06 2024-09-18T04:57:51.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593444+0000", 2024-09-18T04:57:51.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.593e-06 2024-09-18T04:57:51.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:51.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630015+0000", 2024-09-18T04:57:51.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036571125000000003 2024-09-18T04:57:51.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:51.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630061+0000", 2024-09-18T04:57:51.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5445000000000003e-05 2024-09-18T04:57:51.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:51.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630113+0000", 2024-09-18T04:57:51.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2358000000000003e-05 2024-09-18T04:57:51.089 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.089 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:51.089 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.090 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146089 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1990656~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:51.090 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:57:51.090 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.412503092000001, 2024-09-18T04:57:51.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036791483999999999, 2024-09-18T04:57:51.090 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:51.090 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:51.090 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:51.090 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:51.090 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.090 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146089 2024-09-18T04:57:51.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.091 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:51.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:51.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:57:51.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:51.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:57:51.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.091 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:51.091 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593446+0000", 2024-09-18T04:57:51.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:57:51.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:51.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593448+0000", 2024-09-18T04:57:51.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1939999999999999e-06 2024-09-18T04:57:51.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:51.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593451+0000", 2024-09-18T04:57:51.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.766e-06 2024-09-18T04:57:51.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593458+0000", 2024-09-18T04:57:51.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8539999999999999e-06 2024-09-18T04:57:51.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:51.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630136+0000", 2024-09-18T04:57:51.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03667757 2024-09-18T04:57:51.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:51.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630183+0000", 2024-09-18T04:57:51.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7289000000000001e-05 2024-09-18T04:57:51.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:51.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630238+0000", 2024-09-18T04:57:51.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5013000000000003e-05 2024-09-18T04:57:51.094 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.094 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:51.094 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.094 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146090 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 2039808~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:51.094 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:57:51.094 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.412489212000001, 2024-09-18T04:57:51.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036904477999999998, 2024-09-18T04:57:51.095 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:51.095 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:51.095 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:51.095 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:51.095 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.095 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146090 2024-09-18T04:57:51.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.095 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:51.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:51.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:57:51.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:51.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:57:51.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:51.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:57:51.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:51.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593461+0000", 2024-09-18T04:57:51.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3680000000000001e-06 2024-09-18T04:57:51.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:51.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593465+0000", 2024-09-18T04:57:51.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8120000000000001e-06 2024-09-18T04:57:51.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593468+0000", 2024-09-18T04:57:51.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.582e-06 2024-09-18T04:57:51.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:51.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630268+0000", 2024-09-18T04:57:51.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036800449999999998 2024-09-18T04:57:51.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:51.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630313+0000", 2024-09-18T04:57:51.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4836000000000002e-05 2024-09-18T04:57:51.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:51.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630365+0000", 2024-09-18T04:57:51.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1839999999999998e-05 2024-09-18T04:57:51.098 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.098 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:51.098 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.099 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146091 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 2088960~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:51.099 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:57:51.099 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 17.412478749999998, 2024-09-18T04:57:51.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037011534999999998, 2024-09-18T04:57:51.099 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:51.099 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:51.099 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:51.099 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:51.099 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.099 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146091 2024-09-18T04:57:51.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.100 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:51.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:51.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:57:51.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:51.100 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:57:51.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.100 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:51.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:57:51.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:57:51.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:51.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593472+0000", 2024-09-18T04:57:51.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1620000000000001e-06 2024-09-18T04:57:51.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:51.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593475+0000", 2024-09-18T04:57:51.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0019999999999998e-06 2024-09-18T04:57:51.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593477+0000", 2024-09-18T04:57:51.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-18T04:57:51.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:51.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630381+0000", 2024-09-18T04:57:51.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036904014999999998 2024-09-18T04:57:51.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:51.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630428+0000", 2024-09-18T04:57:51.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6931000000000002e-05 2024-09-18T04:57:51.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:51.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630482+0000", 2024-09-18T04:57:51.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4138999999999998e-05 2024-09-18T04:57:51.103 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.103 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:51.103 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.103 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:162075 3.13 3:cf3f6048:::rbd_header.11ac6967455f:head [watch reconnect cookie 140406508629968 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T04:57:51.103 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:57:51.103 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.5155059449999992, 2024-09-18T04:57:51.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.53184266599999996, 2024-09-18T04:57:51.103 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:51.104 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:51.104 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:51.104 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:51.104 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.104 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 162075 2024-09-18T04:57:51.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.104 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:51.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:51.104 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:57:51.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.104 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:51.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:57:51.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:51.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490442+0000", 2024-09-18T04:57:51.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:57:51.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:51.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490453+0000", 2024-09-18T04:57:51.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0613e-05 2024-09-18T04:57:51.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:51.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490466+0000", 2024-09-18T04:57:51.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3702e-05 2024-09-18T04:57:51.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490475+0000", 2024-09-18T04:57:51.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2690000000000005e-06 2024-09-18T04:57:51.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:51.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490612+0000", 2024-09-18T04:57:51.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013620900000000001 2024-09-18T04:57:51.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-18T04:57:51.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490620+0000", 2024-09-18T04:57:51.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3089999999999996e-06 2024-09-18T04:57:51.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:51.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503041+0000", 2024-09-18T04:57:51.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012420512 2024-09-18T04:57:51.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:57:51.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503048+0000", 2024-09-18T04:57:51.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0389999999999999e-06 2024-09-18T04:57:51.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:51.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.016122+0000", 2024-09-18T04:57:51.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.51307432900000005 2024-09-18T04:57:51.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:51.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.016289+0000", 2024-09-18T04:57:51.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000166875 2024-09-18T04:57:51.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:51.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.022286+0000", 2024-09-18T04:57:51.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059976049999999996 2024-09-18T04:57:51.109 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.109 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:51.109 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.109 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:162076 3.13 3:cf3f6048:::rbd_header.11ac6967455f:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T04:57:51.109 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:57:51.109 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.5153832709999993, 2024-09-18T04:57:51.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.53464117600000005, 2024-09-18T04:57:51.109 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:51.109 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:51.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:51.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:51.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.110 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 162076 2024-09-18T04:57:51.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.110 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:51.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:51.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:57:51.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:51.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:57:51.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:51.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:51.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:57:51.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:57:51.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:51.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490571+0000", 2024-09-18T04:57:51.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8259999999999998e-06 2024-09-18T04:57:51.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:51.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490584+0000", 2024-09-18T04:57:51.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2391999999999999e-05 2024-09-18T04:57:51.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490591+0000", 2024-09-18T04:57:51.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5460000000000003e-06 2024-09-18T04:57:51.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:51.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490714+0000", 2024-09-18T04:57:51.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000122543 2024-09-18T04:57:51.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-18T04:57:51.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490731+0000", 2024-09-18T04:57:51.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7405e-05 2024-09-18T04:57:51.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:51.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503069+0000", 2024-09-18T04:57:51.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012337836 2024-09-18T04:57:51.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:57:51.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503078+0000", 2024-09-18T04:57:51.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8289999999999997e-06 2024-09-18T04:57:51.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:51.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019018+0000", 2024-09-18T04:57:51.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.51594035900000002 2024-09-18T04:57:51.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:51.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019091+0000", 2024-09-18T04:57:51.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2608999999999995e-05 2024-09-18T04:57:51.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-18T04:57:51.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019421+0000", 2024-09-18T04:57:51.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00033004300000000002 2024-09-18T04:57:51.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:51.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.021644+0000", 2024-09-18T04:57:51.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022229749999999999 2024-09-18T04:57:51.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:57:51.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025113+0000", 2024-09-18T04:57:51.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003469228 2024-09-18T04:57:51.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:57:51.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025167+0000", 2024-09-18T04:57:51.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3569999999999997e-05 2024-09-18T04:57:51.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:51.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:51.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:51.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025208+0000", 2024-09-18T04:57:51.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0821000000000001e-05 2024-09-18T04:57:51.116 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.116 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:51.116 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.116 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:51.116 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:51.116 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:51.118 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:51.115+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:51.118 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:57:51.227 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:51.223+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:51.230 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:51.231 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:51.231 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:51.231 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:51.239 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:57:51.328 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:51.321+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:51.363 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:51.363 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:51.363 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:51.363 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:51.364 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:51.372 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:57:51.432 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:51.429+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:51.530 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:51.527+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:51.549 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:51.550 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:51.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:51.550 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:51.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.550 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9316 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:51.551 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:51.551 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.778984944000001, 2024-09-18T04:57:51.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044027208999999998, 2024-09-18T04:57:51.551 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:51.551 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:51.551 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:51.551 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:51.551 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.551 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9316 2024-09-18T04:57:51.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.551 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:51.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:51.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:51.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:51.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:51.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:51.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:57:51.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:51.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767144+0000", 2024-09-18T04:57:51.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3070000000000001e-06 2024-09-18T04:57:51.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:51.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767150+0000", 2024-09-18T04:57:51.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1700000000000002e-06 2024-09-18T04:57:51.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767163+0000", 2024-09-18T04:57:51.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2670999999999999e-05 2024-09-18T04:57:51.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:51.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810995+0000", 2024-09-18T04:57:51.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043832310999999999 2024-09-18T04:57:51.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:51.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811087+0000", 2024-09-18T04:57:51.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1345e-05 2024-09-18T04:57:51.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:51.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811169+0000", 2024-09-18T04:57:51.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2848000000000004e-05 2024-09-18T04:57:51.555 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.555 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:51.555 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.555 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9317 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1728512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:51.555 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:57:51.555 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.778959309000001, 2024-09-18T04:57:51.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044190170000000001, 2024-09-18T04:57:51.555 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:51.555 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:51.555 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:51.556 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:51.556 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.556 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9317 2024-09-18T04:57:51.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.556 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:51.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:51.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:57:51.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:51.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:57:51.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:51.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767167+0000", 2024-09-18T04:57:51.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:51.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767170+0000", 2024-09-18T04:57:51.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7259999999999998e-06 2024-09-18T04:57:51.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:51.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767176+0000", 2024-09-18T04:57:51.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1460000000000001e-06 2024-09-18T04:57:51.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767182+0000", 2024-09-18T04:57:51.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1020000000000002e-06 2024-09-18T04:57:51.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:51.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811204+0000", 2024-09-18T04:57:51.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044022039999999998 2024-09-18T04:57:51.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:51.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811286+0000", 2024-09-18T04:57:51.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1623000000000001e-05 2024-09-18T04:57:51.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:51.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811358+0000", 2024-09-18T04:57:51.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2125000000000005e-05 2024-09-18T04:57:51.562 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.562 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:51.562 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.562 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9318 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1777664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:51.562 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:51.562 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.778940763999998, 2024-09-18T04:57:51.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044380181999999997, 2024-09-18T04:57:51.562 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:51.563 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:51.563 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:51.563 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:51.563 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.563 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9318 2024-09-18T04:57:51.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.563 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:51.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:51.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:51.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:51.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:51.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:51.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:57:51.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:51.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767189+0000", 2024-09-18T04:57:51.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5909999999999998e-06 2024-09-18T04:57:51.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:51.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767194+0000", 2024-09-18T04:57:51.566 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7690000000000002e-06 2024-09-18T04:57:51.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767199+0000", 2024-09-18T04:57:51.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189999999999999e-06 2024-09-18T04:57:51.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:51.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811394+0000", 2024-09-18T04:57:51.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044195350000000001 2024-09-18T04:57:51.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:51.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811483+0000", 2024-09-18T04:57:51.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8282999999999995e-05 2024-09-18T04:57:51.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:51.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811567+0000", 2024-09-18T04:57:51.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3855000000000003e-05 2024-09-18T04:57:51.569 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.569 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:51.569 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.569 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9325 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:51.569 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:51.569 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.777210225999994, 2024-09-18T04:57:51.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044063989999999997, 2024-09-18T04:57:51.570 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:51.570 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:51.570 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:51.570 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:51.570 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.570 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9325 2024-09-18T04:57:51.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.570 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:51.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:51.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:51.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:51.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:51.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:51.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:57:51.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:51.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768919+0000", 2024-09-18T04:57:51.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:57:51.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:51.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768925+0000", 2024-09-18T04:57:51.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2310000000000001e-06 2024-09-18T04:57:51.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768929+0000", 2024-09-18T04:57:51.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4469999999999999e-06 2024-09-18T04:57:51.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:51.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812823+0000", 2024-09-18T04:57:51.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043893975000000002 2024-09-18T04:57:51.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:51.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812908+0000", 2024-09-18T04:57:51.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4862000000000002e-05 2024-09-18T04:57:51.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.574 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:51.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812981+0000", 2024-09-18T04:57:51.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2892000000000006e-05 2024-09-18T04:57:51.574 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.574 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:51.575 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.576 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9334 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:51.576 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:57:51.576 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.77519144, 2024-09-18T04:57:51.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044042853999999999, 2024-09-18T04:57:51.577 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:51.577 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:51.577 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:51.577 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:51.577 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.577 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9334 2024-09-18T04:57:51.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.578 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:51.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:51.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:57:51.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:51.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:57:51.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:51.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770935+0000", 2024-09-18T04:57:51.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:51.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770937+0000", 2024-09-18T04:57:51.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1489999999999999e-06 2024-09-18T04:57:51.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.580 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:51.580 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770940+0000", 2024-09-18T04:57:51.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9569999999999998e-06 2024-09-18T04:57:51.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770945+0000", 2024-09-18T04:57:51.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:57:51.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:51.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814838+0000", 2024-09-18T04:57:51.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043892635999999999 2024-09-18T04:57:51.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:51.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814920+0000", 2024-09-18T04:57:51.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2175000000000005e-05 2024-09-18T04:57:51.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:51.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814979+0000", 2024-09-18T04:57:51.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8372000000000003e-05 2024-09-18T04:57:51.583 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.584 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:51.584 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.584 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9352 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1159168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:51.585 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:57:51.585 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.772970263000005, 2024-09-18T04:57:51.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044011269999999998, 2024-09-18T04:57:51.585 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:51.585 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:51.586 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:51.586 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:51.586 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.587 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9352 2024-09-18T04:57:51.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.587 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:51.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:51.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:57:51.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:51.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:57:51.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:51.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773156+0000", 2024-09-18T04:57:51.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:51.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773159+0000", 2024-09-18T04:57:51.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2919999999999998e-06 2024-09-18T04:57:51.590 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:51.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773165+0000", 2024-09-18T04:57:51.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0580000000000003e-06 2024-09-18T04:57:51.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.591 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.591 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773177+0000", 2024-09-18T04:57:51.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.204e-05 2024-09-18T04:57:51.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:51.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817082+0000", 2024-09-18T04:57:51.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043905528999999999 2024-09-18T04:57:51.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:51.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817122+0000", 2024-09-18T04:57:51.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9258999999999998e-05 2024-09-18T04:57:51.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:51.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817168+0000", 2024-09-18T04:57:51.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6545000000000002e-05 2024-09-18T04:57:51.594 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.594 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:51.594 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.595 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9353 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1208320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:51.595 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:51.595 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.772945965000005, 2024-09-18T04:57:51.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044105537, 2024-09-18T04:57:51.596 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:51.596 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:51.596 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:51.596 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:51.596 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.596 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9353 2024-09-18T04:57:51.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.596 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:51.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:51.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:51.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:51.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:51.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:51.601 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:57:51.601 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.601 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.601 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.601 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:51.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773183+0000", 2024-09-18T04:57:51.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2929999999999999e-06 2024-09-18T04:57:51.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.602 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.602 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:51.602 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773190+0000", 2024-09-18T04:57:51.602 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4330000000000001e-06 2024-09-18T04:57:51.602 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.603 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.603 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773194+0000", 2024-09-18T04:57:51.603 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.865e-06 2024-09-18T04:57:51.603 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.603 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:51.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817196+0000", 2024-09-18T04:57:51.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044001708 2024-09-18T04:57:51.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.604 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:51.604 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817236+0000", 2024-09-18T04:57:51.604 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0043999999999998e-05 2024-09-18T04:57:51.604 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.604 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.605 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:51.605 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817287+0000", 2024-09-18T04:57:51.605 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0605000000000001e-05 2024-09-18T04:57:51.605 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.605 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:51.605 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.605 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.605 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.605 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9354 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:51.605 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:57:51.606 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.772928453999995, 2024-09-18T04:57:51.606 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044199796, 2024-09-18T04:57:51.606 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:51.606 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:51.606 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:51.606 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:51.606 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.607 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9354 2024-09-18T04:57:51.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.607 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:51.607 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.607 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:51.607 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:57:51.607 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.607 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:51.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:57:51.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.608 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.608 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.608 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:51.608 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773198+0000", 2024-09-18T04:57:51.608 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:51.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773200+0000", 2024-09-18T04:57:51.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4159999999999998e-06 2024-09-18T04:57:51.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.609 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.609 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:51.609 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773206+0000", 2024-09-18T04:57:51.609 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.789e-06 2024-09-18T04:57:51.609 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773210+0000", 2024-09-18T04:57:51.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2679999999999996e-06 2024-09-18T04:57:51.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:51.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817314+0000", 2024-09-18T04:57:51.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044103187000000002 2024-09-18T04:57:51.610 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.610 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.610 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:51.610 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817353+0000", 2024-09-18T04:57:51.610 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8973000000000001e-05 2024-09-18T04:57:51.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.611 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:51.611 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817399+0000", 2024-09-18T04:57:51.611 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5877999999999997e-05 2024-09-18T04:57:51.611 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.611 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:51.611 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.611 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.611 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.611 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46591 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2097152~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:51.612 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:51.612 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.848072037000001, 2024-09-18T04:57:51.612 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043987310000000002, 2024-09-18T04:57:51.612 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:51.612 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:51.612 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:51.612 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:51.612 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.612 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46591 2024-09-18T04:57:51.612 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.612 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:51.612 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.612 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:51.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:51.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:51.613 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:51.613 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.613 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.613 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.613 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:51.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:57:51.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.614 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:51.614 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698057+0000", 2024-09-18T04:57:51.614 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.531e-06 2024-09-18T04:57:51.614 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.614 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:51.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698064+0000", 2024-09-18T04:57:51.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-18T04:57:51.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.615 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.615 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698069+0000", 2024-09-18T04:57:51.615 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.091e-06 2024-09-18T04:57:51.615 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.615 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:51.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741843+0000", 2024-09-18T04:57:51.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043773470000000002 2024-09-18T04:57:51.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.616 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:51.616 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741928+0000", 2024-09-18T04:57:51.616 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5113999999999999e-05 2024-09-18T04:57:51.616 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.616 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.617 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:51.617 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742042+0000", 2024-09-18T04:57:51.617 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114527 2024-09-18T04:57:51.617 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.617 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:51.617 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.617 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.617 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.617 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46592 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2121728~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:51.617 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:57:51.618 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.848053610000001, 2024-09-18T04:57:51.618 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044077948999999998, 2024-09-18T04:57:51.618 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:51.618 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:51.618 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:51.618 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:51.618 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.618 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46592 2024-09-18T04:57:51.618 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.618 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:51.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:51.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:57:51.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.619 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:51.619 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:57:51.619 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.619 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.619 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:51.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698073+0000", 2024-09-18T04:57:51.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.620 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:51.620 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698076+0000", 2024-09-18T04:57:51.620 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.481e-06 2024-09-18T04:57:51.620 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.620 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:51.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698082+0000", 2024-09-18T04:57:51.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8730000000000004e-06 2024-09-18T04:57:51.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698086+0000", 2024-09-18T04:57:51.621 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:57:51.621 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.621 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.621 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:51.621 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742055+0000", 2024-09-18T04:57:51.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043969170000000002 2024-09-18T04:57:51.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:51.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742093+0000", 2024-09-18T04:57:51.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7707000000000003e-05 2024-09-18T04:57:51.622 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.622 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.622 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:51.622 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742152+0000", 2024-09-18T04:57:51.622 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8323999999999999e-05 2024-09-18T04:57:51.622 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.623 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:51.623 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.623 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.623 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.623 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46593 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2211840~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:51.623 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:51.623 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.848036706999999, 2024-09-18T04:57:51.623 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044259917000000003, 2024-09-18T04:57:51.623 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:51.623 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:51.623 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:51.623 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:51.623 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.623 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46593 2024-09-18T04:57:51.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.624 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:51.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:51.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:51.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.624 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.624 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.624 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:51.624 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:51.624 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:51.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:57:51.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:51.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698092+0000", 2024-09-18T04:57:51.625 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2249999999999999e-06 2024-09-18T04:57:51.625 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.625 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.625 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:51.625 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698098+0000", 2024-09-18T04:57:51.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:57:51.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698103+0000", 2024-09-18T04:57:51.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:57:51.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.626 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:51.626 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742182+0000", 2024-09-18T04:57:51.626 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044079508000000003 2024-09-18T04:57:51.626 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.626 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:51.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742257+0000", 2024-09-18T04:57:51.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4678000000000006e-05 2024-09-18T04:57:51.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.627 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:51.627 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742350+0000", 2024-09-18T04:57:51.627 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3531999999999998e-05 2024-09-18T04:57:51.627 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.627 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:51.627 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.627 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.627 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.628 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46594 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2236416~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:51.628 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:51.628 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.848021009999997, 2024-09-18T04:57:51.628 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044378984000000003, 2024-09-18T04:57:51.628 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:51.628 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:51.628 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:51.628 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:51.628 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.628 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46594 2024-09-18T04:57:51.628 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.628 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:51.628 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:51.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:51.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:51.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:51.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.629 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.629 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:51.629 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:57:51.629 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.629 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:51.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698108+0000", 2024-09-18T04:57:51.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0880000000000002e-06 2024-09-18T04:57:51.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:51.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698114+0000", 2024-09-18T04:57:51.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6269999999999997e-06 2024-09-18T04:57:51.630 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.630 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.630 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698118+0000", 2024-09-18T04:57:51.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4050000000000002e-06 2024-09-18T04:57:51.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:51.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742365+0000", 2024-09-18T04:57:51.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044246799000000003 2024-09-18T04:57:51.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:51.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742407+0000", 2024-09-18T04:57:51.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2008000000000002e-05 2024-09-18T04:57:51.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:51.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742485+0000", 2024-09-18T04:57:51.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8405999999999994e-05 2024-09-18T04:57:51.632 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.632 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:51.632 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.632 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46595 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:51.632 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:51.632 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.848004936999999, 2024-09-18T04:57:51.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.04440463, 2024-09-18T04:57:51.632 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:51.632 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:51.633 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:51.633 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:51.633 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.633 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46595 2024-09-18T04:57:51.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.633 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:51.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:51.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:51.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:51.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:51.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:51.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:57:51.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:51.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698124+0000", 2024-09-18T04:57:51.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1840000000000002e-06 2024-09-18T04:57:51.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:51.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698130+0000", 2024-09-18T04:57:51.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.947e-06 2024-09-18T04:57:51.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.635 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698136+0000", 2024-09-18T04:57:51.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6520000000000003e-06 2024-09-18T04:57:51.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:51.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742521+0000", 2024-09-18T04:57:51.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044385446000000002 2024-09-18T04:57:51.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:51.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742527+0000", 2024-09-18T04:57:51.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:57:51.636 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.637 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:51.637 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.637 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46596 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:51.637 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:51.637 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.847986693000003, 2024-09-18T04:57:51.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044418066999999999, 2024-09-18T04:57:51.637 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:51.637 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:51.637 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:51.637 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:51.638 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.638 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46596 2024-09-18T04:57:51.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.638 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:51.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:51.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:51.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:51.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:51.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:51.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:57:51.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:51.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698146+0000", 2024-09-18T04:57:51.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2779999999999999e-06 2024-09-18T04:57:51.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:51.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698153+0000", 2024-09-18T04:57:51.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3849999999999999e-06 2024-09-18T04:57:51.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698164+0000", 2024-09-18T04:57:51.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1375999999999999e-05 2024-09-18T04:57:51.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:51.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742554+0000", 2024-09-18T04:57:51.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044389482000000001 2024-09-18T04:57:51.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:51.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742559+0000", 2024-09-18T04:57:51.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9540000000000003e-06 2024-09-18T04:57:51.641 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.641 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:51.641 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.642 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46597 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:51.642 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:57:51.642 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.847958327999997, 2024-09-18T04:57:51.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044416513999999997, 2024-09-18T04:57:51.642 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:51.642 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:51.642 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:51.642 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:51.642 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.643 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46597 2024-09-18T04:57:51.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.643 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:51.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:51.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:57:51.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:51.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:57:51.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:51.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698168+0000", 2024-09-18T04:57:51.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.644 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.644 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:51.644 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698175+0000", 2024-09-18T04:57:51.644 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4520000000000002e-06 2024-09-18T04:57:51.644 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:51.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698181+0000", 2024-09-18T04:57:51.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4150000000000004e-06 2024-09-18T04:57:51.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698186+0000", 2024-09-18T04:57:51.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7489999999999998e-06 2024-09-18T04:57:51.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:51.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742580+0000", 2024-09-18T04:57:51.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044394325999999998 2024-09-18T04:57:51.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.646 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:51.646 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742585+0000", 2024-09-18T04:57:51.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9849999999999997e-06 2024-09-18T04:57:51.646 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.646 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:51.646 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.647 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46598 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:57:51.647 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:57:51.647 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.847850852000001, 2024-09-18T04:57:51.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044335316999999999, 2024-09-18T04:57:51.647 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:51.647 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:51.647 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:51.647 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:51.647 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.647 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46598 2024-09-18T04:57:51.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.648 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:51.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:51.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:57:51.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:51.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:57:51.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:51.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698190+0000", 2024-09-18T04:57:51.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999137 2024-09-18T04:57:51.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:51.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698280+0000", 2024-09-18T04:57:51.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0409999999999997e-05 2024-09-18T04:57:51.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:51.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698289+0000", 2024-09-18T04:57:51.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9490000000000002e-06 2024-09-18T04:57:51.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698296+0000", 2024-09-18T04:57:51.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2670000000000003e-06 2024-09-18T04:57:51.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:51.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742607+0000", 2024-09-18T04:57:51.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044311072999999999 2024-09-18T04:57:51.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:51.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742612+0000", 2024-09-18T04:57:51.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9400000000000001e-06 2024-09-18T04:57:51.651 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.651 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:51.651 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.651 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57986 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3086336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:51.651 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:51.652 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.861005128999999, 2024-09-18T04:57:51.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044047731999999999, 2024-09-18T04:57:51.652 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:51.652 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:51.652 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:51.652 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:51.652 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.652 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57986 2024-09-18T04:57:51.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.652 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:51.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:51.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:51.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:51.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:51.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:51.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:57:51.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:51.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685124+0000", 2024-09-18T04:57:51.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2859999999999998e-06 2024-09-18T04:57:51.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:51.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685130+0000", 2024-09-18T04:57:51.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5400000000000001e-06 2024-09-18T04:57:51.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685136+0000", 2024-09-18T04:57:51.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.152e-06 2024-09-18T04:57:51.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:51.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.728976+0000", 2024-09-18T04:57:51.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043840037999999998 2024-09-18T04:57:51.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:51.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729064+0000", 2024-09-18T04:57:51.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8709999999999996e-05 2024-09-18T04:57:51.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:51.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729170+0000", 2024-09-18T04:57:51.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000105386 2024-09-18T04:57:51.656 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.656 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:51.656 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.656 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57987 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3101696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:51.656 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:57:51.656 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.860956154, 2024-09-18T04:57:51.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044238123999999997, 2024-09-18T04:57:51.657 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:51.657 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:51.657 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:51.657 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:51.657 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.657 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57987 2024-09-18T04:57:51.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.657 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:51.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:51.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:57:51.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:51.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:57:51.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:51.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685170+0000", 2024-09-18T04:57:51.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:51.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:51.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685175+0000", 2024-09-18T04:57:51.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9069999999999998e-06 2024-09-18T04:57:51.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:51.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685187+0000", 2024-09-18T04:57:51.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.189e-05 2024-09-18T04:57:51.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685195+0000", 2024-09-18T04:57:51.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9459999999999998e-06 2024-09-18T04:57:51.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:51.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729212+0000", 2024-09-18T04:57:51.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044017239 2024-09-18T04:57:51.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:51.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729301+0000", 2024-09-18T04:57:51.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8887999999999993e-05 2024-09-18T04:57:51.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:51.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729409+0000", 2024-09-18T04:57:51.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108119 2024-09-18T04:57:51.661 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.661 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:51.661 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.661 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57988 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3117056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:51.661 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:57:51.661 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.860914035, 2024-09-18T04:57:51.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044429835000000001, 2024-09-18T04:57:51.661 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:51.662 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:51.662 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:51.662 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:51.662 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.662 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57988 2024-09-18T04:57:51.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.662 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:51.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:51.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:57:51.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:51.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:57:51.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:51.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685212+0000", 2024-09-18T04:57:51.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:51.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685216+0000", 2024-09-18T04:57:51.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7390000000000001e-06 2024-09-18T04:57:51.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:51.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685224+0000", 2024-09-18T04:57:51.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2260000000000005e-06 2024-09-18T04:57:51.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685230+0000", 2024-09-18T04:57:51.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0299999999999999e-06 2024-09-18T04:57:51.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:51.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729454+0000", 2024-09-18T04:57:51.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044223248999999999 2024-09-18T04:57:51.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:51.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729539+0000", 2024-09-18T04:57:51.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5029000000000002e-05 2024-09-18T04:57:51.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:51.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729643+0000", 2024-09-18T04:57:51.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104267 2024-09-18T04:57:51.666 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.666 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:51.667 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.667 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57989 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3132416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:51.667 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:57:51.667 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 60.860891283000001, 2024-09-18T04:57:51.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044642881000000002, 2024-09-18T04:57:51.667 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:51.667 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:51.667 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:51.667 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:51.668 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:51.668 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57989 2024-09-18T04:57:51.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.668 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:51.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:51.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:57:51.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:51.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:57:51.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:51.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:51.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685235+0000", 2024-09-18T04:57:51.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:51.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:51.669 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685238+0000", 2024-09-18T04:57:51.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.667e-06 2024-09-18T04:57:51.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:51.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685245+0000", 2024-09-18T04:57:51.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7909999999999999e-06 2024-09-18T04:57:51.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:51.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685250+0000", 2024-09-18T04:57:51.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3730000000000001e-06 2024-09-18T04:57:51.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:51.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729685+0000", 2024-09-18T04:57:51.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044434552000000002 2024-09-18T04:57:51.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:51.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729775+0000", 2024-09-18T04:57:51.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0395000000000004e-05 2024-09-18T04:57:51.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:51.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:51.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:51.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729879+0000", 2024-09-18T04:57:51.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010361099999999999 2024-09-18T04:57:51.671 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.671 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:51.672 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.672 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:51.672 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:51.672 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:51.672 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:57:51.682 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:51.679+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:51.775 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:51.771+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:51.795 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:51.796 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:51.796 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:51.796 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:51.804 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:57:51.904 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:51.898+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:51.953 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:51.953 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:51.953 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:51.953 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:51.953 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:51.962 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:57:52.006 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:52.003+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:52.048 INFO:teuthology.orchestra.run.smithi002.stderr:clear 1.0 pg_upmap mapping 2024-09-18T04:57:52.056 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T04:57:52.107 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:52.103+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:52.125 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:52.126 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:52.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:52.126 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:52.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.126 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14125 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:52.126 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:57:52.126 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.994983628, 2024-09-18T04:57:52.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052956470999999998, 2024-09-18T04:57:52.126 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.126 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.127 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.127 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.127 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.127 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14125 2024-09-18T04:57:52.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.127 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:57:52.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:57:52.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126849+0000", 2024-09-18T04:57:52.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126852+0000", 2024-09-18T04:57:52.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3659999999999999e-06 2024-09-18T04:57:52.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126858+0000", 2024-09-18T04:57:52.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4540000000000004e-06 2024-09-18T04:57:52.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126863+0000", 2024-09-18T04:57:52.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1780000000000002e-06 2024-09-18T04:57:52.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179608+0000", 2024-09-18T04:57:52.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052745159999999999 2024-09-18T04:57:52.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179703+0000", 2024-09-18T04:57:52.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4773999999999998e-05 2024-09-18T04:57:52.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179806+0000", 2024-09-18T04:57:52.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010296799999999999 2024-09-18T04:57:52.131 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.131 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.131 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.132 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14136 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1732608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:52.132 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:52.132 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.992823076999997, 2024-09-18T04:57:52.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052973412999999997, 2024-09-18T04:57:52.132 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.132 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.132 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.132 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.132 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.132 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14136 2024-09-18T04:57:52.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.133 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:52.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:52.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:52.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129013+0000", 2024-09-18T04:57:52.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.96e-06 2024-09-18T04:57:52.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129020+0000", 2024-09-18T04:57:52.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6809999999999998e-06 2024-09-18T04:57:52.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129026+0000", 2024-09-18T04:57:52.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:57:52.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181873+0000", 2024-09-18T04:57:52.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052847238999999997 2024-09-18T04:57:52.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181932+0000", 2024-09-18T04:57:52.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9339000000000002e-05 2024-09-18T04:57:52.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.138 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.138 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181984+0000", 2024-09-18T04:57:52.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1348999999999999e-05 2024-09-18T04:57:52.138 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.138 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.138 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.138 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14137 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1781760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:52.139 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:57:52.139 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.992802671999996, 2024-09-18T04:57:52.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053128914999999999, 2024-09-18T04:57:52.139 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.139 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.139 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.139 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.139 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.139 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14137 2024-09-18T04:57:52.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.139 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:57:52.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:57:52.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129030+0000", 2024-09-18T04:57:52.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129033+0000", 2024-09-18T04:57:52.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7439999999999999e-06 2024-09-18T04:57:52.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129039+0000", 2024-09-18T04:57:52.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.314e-06 2024-09-18T04:57:52.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129044+0000", 2024-09-18T04:57:52.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9919999999999998e-06 2024-09-18T04:57:52.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182015+0000", 2024-09-18T04:57:52.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052970875000000001 2024-09-18T04:57:52.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182058+0000", 2024-09-18T04:57:52.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2997000000000002e-05 2024-09-18T04:57:52.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.143 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182160+0000", 2024-09-18T04:57:52.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101437 2024-09-18T04:57:52.144 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.144 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.144 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.144 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14138 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1830912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:52.144 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:57:52.145 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.992784638000003, 2024-09-18T04:57:52.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053256435999999997, 2024-09-18T04:57:52.145 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.145 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.145 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.145 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.145 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.145 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14138 2024-09-18T04:57:52.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.145 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:57:52.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:57:52.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129048+0000", 2024-09-18T04:57:52.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129056+0000", 2024-09-18T04:57:52.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.413e-06 2024-09-18T04:57:52.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129062+0000", 2024-09-18T04:57:52.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6810000000000001e-06 2024-09-18T04:57:52.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129068+0000", 2024-09-18T04:57:52.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0459999999999996e-06 2024-09-18T04:57:52.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182191+0000", 2024-09-18T04:57:52.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053123210999999997 2024-09-18T04:57:52.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182238+0000", 2024-09-18T04:57:52.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7404999999999998e-05 2024-09-18T04:57:52.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182305+0000", 2024-09-18T04:57:52.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7028999999999998e-05 2024-09-18T04:57:52.149 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.149 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.149 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.150 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15910 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2236416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:52.150 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:52.150 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.893697106000005, 2024-09-18T04:57:52.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053241054000000003, 2024-09-18T04:57:52.150 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.152 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.152 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.152 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.152 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.152 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15910 2024-09-18T04:57:52.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.153 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:52.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:52.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:52.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228138+0000", 2024-09-18T04:57:52.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:57:52.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228144+0000", 2024-09-18T04:57:52.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5369999999999997e-06 2024-09-18T04:57:52.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228149+0000", 2024-09-18T04:57:52.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1209999999999998e-06 2024-09-18T04:57:52.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280822+0000", 2024-09-18T04:57:52.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052673194999999999 2024-09-18T04:57:52.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280903+0000", 2024-09-18T04:57:52.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1179000000000003e-05 2024-09-18T04:57:52.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281377+0000", 2024-09-18T04:57:52.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047417899999999997 2024-09-18T04:57:52.158 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.158 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.158 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.159 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15911 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2285568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:52.159 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:52.159 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.893680373999999, 2024-09-18T04:57:52.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053415151000000001, 2024-09-18T04:57:52.159 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.159 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.159 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.160 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.160 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.160 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15911 2024-09-18T04:57:52.160 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.160 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.160 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.161 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:52.161 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:52.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.162 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:52.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.162 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228155+0000", 2024-09-18T04:57:52.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:57:52.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228161+0000", 2024-09-18T04:57:52.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:57:52.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228166+0000", 2024-09-18T04:57:52.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2320000000000001e-06 2024-09-18T04:57:52.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281405+0000", 2024-09-18T04:57:52.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053239336999999998 2024-09-18T04:57:52.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281482+0000", 2024-09-18T04:57:52.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6569999999999994e-05 2024-09-18T04:57:52.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281568+0000", 2024-09-18T04:57:52.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6328000000000004e-05 2024-09-18T04:57:52.165 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.165 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.165 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.165 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15918 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2662400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:52.165 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:52.165 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.891911393000001, 2024-09-18T04:57:52.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053091065, 2024-09-18T04:57:52.166 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.166 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.166 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.166 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.166 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.166 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15918 2024-09-18T04:57:52.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.166 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:52.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:52.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:52.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229924+0000", 2024-09-18T04:57:52.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:57:52.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229930+0000", 2024-09-18T04:57:52.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.066e-06 2024-09-18T04:57:52.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229935+0000", 2024-09-18T04:57:52.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9119999999999997e-06 2024-09-18T04:57:52.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282840+0000", 2024-09-18T04:57:52.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052905119 2024-09-18T04:57:52.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282922+0000", 2024-09-18T04:57:52.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2081000000000002e-05 2024-09-18T04:57:52.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283013+0000", 2024-09-18T04:57:52.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0828000000000005e-05 2024-09-18T04:57:52.170 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.170 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.170 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.171 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15919 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2711552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:52.171 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:52.171 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.891894114999999, 2024-09-18T04:57:52.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053293021000000003, 2024-09-18T04:57:52.171 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.171 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.171 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.171 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.171 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.171 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15919 2024-09-18T04:57:52.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.171 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:52.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:52.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:52.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229941+0000", 2024-09-18T04:57:52.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2639999999999998e-06 2024-09-18T04:57:52.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229947+0000", 2024-09-18T04:57:52.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9750000000000004e-06 2024-09-18T04:57:52.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229952+0000", 2024-09-18T04:57:52.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7929999999999997e-06 2024-09-18T04:57:52.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283062+0000", 2024-09-18T04:57:52.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05310952 2024-09-18T04:57:52.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283162+0000", 2024-09-18T04:57:52.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100098 2024-09-18T04:57:52.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283232+0000", 2024-09-18T04:57:52.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0693999999999998e-05 2024-09-18T04:57:52.175 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.176 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.176 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.176 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15920 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2760704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:52.176 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:52.176 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.891877354000002, 2024-09-18T04:57:52.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053502367000000002, 2024-09-18T04:57:52.176 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.176 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.176 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.176 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.176 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.177 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15920 2024-09-18T04:57:52.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.177 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:52.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:52.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:52.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229958+0000", 2024-09-18T04:57:52.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1770000000000001e-06 2024-09-18T04:57:52.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229964+0000", 2024-09-18T04:57:52.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8470000000000001e-06 2024-09-18T04:57:52.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229968+0000", 2024-09-18T04:57:52.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5680000000000001e-06 2024-09-18T04:57:52.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283269+0000", 2024-09-18T04:57:52.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053300620999999999 2024-09-18T04:57:52.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283359+0000", 2024-09-18T04:57:52.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9971999999999998e-05 2024-09-18T04:57:52.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283458+0000", 2024-09-18T04:57:52.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9483999999999998e-05 2024-09-18T04:57:52.181 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.181 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.181 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.181 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15921 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2809856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:52.181 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:57:52.181 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.891860804000004, 2024-09-18T04:57:52.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053800977, 2024-09-18T04:57:52.182 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.182 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.182 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.182 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.182 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.182 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15921 2024-09-18T04:57:52.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.182 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:57:52.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:57:52.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229972+0000", 2024-09-18T04:57:52.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229974+0000", 2024-09-18T04:57:52.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2910000000000002e-06 2024-09-18T04:57:52.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229982+0000", 2024-09-18T04:57:52.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7249999999999997e-06 2024-09-18T04:57:52.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229989+0000", 2024-09-18T04:57:52.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6429999999999998e-06 2024-09-18T04:57:52.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283601+0000", 2024-09-18T04:57:52.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053612211 2024-09-18T04:57:52.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283681+0000", 2024-09-18T04:57:52.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0125999999999998e-05 2024-09-18T04:57:52.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283774+0000", 2024-09-18T04:57:52.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2362999999999996e-05 2024-09-18T04:57:52.185 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.186 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.186 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.186 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15922 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2859008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:52.186 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:57:52.186 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.891815864999998, 2024-09-18T04:57:52.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053882270000000003, 2024-09-18T04:57:52.186 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.186 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.186 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.188 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.189 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.189 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15922 2024-09-18T04:57:52.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.189 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:57:52.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:57:52.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230017+0000", 2024-09-18T04:57:52.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230019+0000", 2024-09-18T04:57:52.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4210000000000002e-06 2024-09-18T04:57:52.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.191 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230027+0000", 2024-09-18T04:57:52.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000002e-06 2024-09-18T04:57:52.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.191 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230032+0000", 2024-09-18T04:57:52.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0089999999999998e-06 2024-09-18T04:57:52.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283800+0000", 2024-09-18T04:57:52.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053768418999999998 2024-09-18T04:57:52.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283849+0000", 2024-09-18T04:57:52.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9339000000000003e-05 2024-09-18T04:57:52.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283900+0000", 2024-09-18T04:57:52.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0421999999999999e-05 2024-09-18T04:57:52.193 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.194 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.194 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.194 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15923 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2924544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:52.194 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:52.194 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.891796995999997, 2024-09-18T04:57:52.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054073016000000002, 2024-09-18T04:57:52.194 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.194 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.194 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.194 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.194 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.194 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15923 2024-09-18T04:57:52.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.195 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:52.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:52.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:52.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230038+0000", 2024-09-18T04:57:52.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.463e-06 2024-09-18T04:57:52.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230045+0000", 2024-09-18T04:57:52.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3570000000000003e-06 2024-09-18T04:57:52.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230050+0000", 2024-09-18T04:57:52.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7299999999999996e-06 2024-09-18T04:57:52.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283928+0000", 2024-09-18T04:57:52.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053878218999999998 2024-09-18T04:57:52.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284010+0000", 2024-09-18T04:57:52.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2577000000000006e-05 2024-09-18T04:57:52.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284109+0000", 2024-09-18T04:57:52.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9102000000000001e-05 2024-09-18T04:57:52.199 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.199 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.199 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.200 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15924 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:52.200 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:52.200 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.891778525999996, 2024-09-18T04:57:52.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054108470999999998, 2024-09-18T04:57:52.200 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.200 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:52.200 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.200 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.200 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.200 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15924 2024-09-18T04:57:52.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.200 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:52.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:52.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230061+0000", 2024-09-18T04:57:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8240000000000002e-06 2024-09-18T04:57:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230068+0000", 2024-09-18T04:57:52.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4320000000000004e-06 2024-09-18T04:57:52.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230073+0000", 2024-09-18T04:57:52.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.921e-06 2024-09-18T04:57:52.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284153+0000", 2024-09-18T04:57:52.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054080793000000002 2024-09-18T04:57:52.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284163+0000", 2024-09-18T04:57:52.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.859e-06 2024-09-18T04:57:52.205 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.205 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.205 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.205 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15925 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:52.205 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:52.205 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.891754410999994, 2024-09-18T04:57:52.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054134632000000002, 2024-09-18T04:57:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15925 2024-09-18T04:57:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:52.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230081+0000", 2024-09-18T04:57:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4109999999999999e-06 2024-09-18T04:57:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230087+0000", 2024-09-18T04:57:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.161e-06 2024-09-18T04:57:52.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230092+0000", 2024-09-18T04:57:52.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9259999999999999e-06 2024-09-18T04:57:52.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284204+0000", 2024-09-18T04:57:52.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054111546000000003 2024-09-18T04:57:52.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284214+0000", 2024-09-18T04:57:52.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9639999999999998e-06 2024-09-18T04:57:52.210 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.210 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.210 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15926 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.891737086999996, 2024-09-18T04:57:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054143193999999999, 2024-09-18T04:57:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15926 2024-09-18T04:57:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:52.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:52.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:52.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230098+0000", 2024-09-18T04:57:52.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1399999999999998e-06 2024-09-18T04:57:52.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230104+0000", 2024-09-18T04:57:52.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-18T04:57:52.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230109+0000", 2024-09-18T04:57:52.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9250000000000003e-06 2024-09-18T04:57:52.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284231+0000", 2024-09-18T04:57:52.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054122297999999999 2024-09-18T04:57:52.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284239+0000", 2024-09-18T04:57:52.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1689999999999992e-06 2024-09-18T04:57:52.214 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.215 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.215 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.215 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15927 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:52.215 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:52.215 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.891720039999996, 2024-09-18T04:57:52.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054168435000000001, 2024-09-18T04:57:52.215 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.215 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:52.215 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.215 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.215 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.215 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15927 2024-09-18T04:57:52.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.216 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:52.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:52.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:52.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230116+0000", 2024-09-18T04:57:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5150000000000001e-06 2024-09-18T04:57:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230122+0000", 2024-09-18T04:57:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1190000000000002e-06 2024-09-18T04:57:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230126+0000", 2024-09-18T04:57:52.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.792e-06 2024-09-18T04:57:52.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284273+0000", 2024-09-18T04:57:52.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054146086000000003 2024-09-18T04:57:52.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284282+0000", 2024-09-18T04:57:52.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2590000000000006e-06 2024-09-18T04:57:52.220 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.220 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.220 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.220 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15928 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:52.220 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:57:52.220 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.891702656999996, 2024-09-18T04:57:52.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054189739000000001, 2024-09-18T04:57:52.220 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.221 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:52.221 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.221 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15928 2024-09-18T04:57:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:57:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:57:52.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230130+0000", 2024-09-18T04:57:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230133+0000", 2024-09-18T04:57:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.244e-06 2024-09-18T04:57:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230139+0000", 2024-09-18T04:57:52.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9830000000000001e-06 2024-09-18T04:57:52.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230143+0000", 2024-09-18T04:57:52.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7530000000000001e-06 2024-09-18T04:57:52.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284313+0000", 2024-09-18T04:57:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054169332000000001 2024-09-18T04:57:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284320+0000", 2024-09-18T04:57:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8699999999999992e-06 2024-09-18T04:57:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15929 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:52.225 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.891686217, 2024-09-18T04:57:52.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054218771999999998, 2024-09-18T04:57:52.226 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.226 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:52.226 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.226 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.226 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.226 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15929 2024-09-18T04:57:52.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.226 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:52.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:52.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:52.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230149+0000", 2024-09-18T04:57:52.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-18T04:57:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230155+0000", 2024-09-18T04:57:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2469999999999997e-06 2024-09-18T04:57:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230160+0000", 2024-09-18T04:57:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7129999999999996e-06 2024-09-18T04:57:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284355+0000", 2024-09-18T04:57:52.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054194900999999997 2024-09-18T04:57:52.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284366+0000", 2024-09-18T04:57:52.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0993e-05 2024-09-18T04:57:52.231 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.231 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.231 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.231 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15930 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:52.231 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:52.231 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.891669217, 2024-09-18T04:57:52.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054244713999999999, 2024-09-18T04:57:52.231 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.231 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:52.232 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.232 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.232 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.232 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15930 2024-09-18T04:57:52.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.232 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230166+0000", 2024-09-18T04:57:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1009999999999999e-06 2024-09-18T04:57:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230172+0000", 2024-09-18T04:57:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.981e-06 2024-09-18T04:57:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230184+0000", 2024-09-18T04:57:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2449000000000001e-05 2024-09-18T04:57:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284400+0000", 2024-09-18T04:57:52.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054215452999999997 2024-09-18T04:57:52.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284409+0000", 2024-09-18T04:57:52.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0839999999999998e-06 2024-09-18T04:57:52.235 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.235 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.235 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.236 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15931 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:52.236 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:57:52.236 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.891643427999995, 2024-09-18T04:57:52.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054261928000000001, 2024-09-18T04:57:52.236 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.236 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:52.236 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.236 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.236 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.237 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15931 2024-09-18T04:57:52.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.237 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:57:52.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:57:52.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230189+0000", 2024-09-18T04:57:52.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230192+0000", 2024-09-18T04:57:52.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6599999999999999e-06 2024-09-18T04:57:52.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230199+0000", 2024-09-18T04:57:52.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5570000000000001e-06 2024-09-18T04:57:52.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230204+0000", 2024-09-18T04:57:52.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2270000000000001e-06 2024-09-18T04:57:52.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284442+0000", 2024-09-18T04:57:52.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054238383000000001 2024-09-18T04:57:52.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284452+0000", 2024-09-18T04:57:52.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6069999999999997e-06 2024-09-18T04:57:52.241 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.241 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.241 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.241 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.241 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.241 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:52.243 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:57:52.243 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:52.240+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:52.343 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:52.340+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:52.392 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:52.392 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:52.392 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:52.392 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:52.401 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:57:52.444 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:52.442+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:52.545 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:52.543+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:52.550 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:52.550 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:52.550 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:52.550 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:52.550 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:52.561 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:57:52.647 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:52.645+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:52.671 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:52.671 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:52.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:52.671 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:52.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160713 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 645120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654655+0000", 2024-09-18T04:57:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.013391203000001, 2024-09-18T04:57:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039354912999999998, 2024-09-18T04:57:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160713 2024-09-18T04:57:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654655+0000", 2024-09-18T04:57:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654655+0000", 2024-09-18T04:57:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654654+0000", 2024-09-18T04:57:52.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654657+0000", 2024-09-18T04:57:52.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7049999999999999e-06 2024-09-18T04:57:52.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654669+0000", 2024-09-18T04:57:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1960000000000001e-05 2024-09-18T04:57:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654674+0000", 2024-09-18T04:57:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8729999999999998e-06 2024-09-18T04:57:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.693848+0000", 2024-09-18T04:57:52.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039174202999999998 2024-09-18T04:57:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.693926+0000", 2024-09-18T04:57:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8248e-05 2024-09-18T04:57:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694009+0000", 2024-09-18T04:57:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3363999999999997e-05 2024-09-18T04:57:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.676 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.677 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160714 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 665600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:52.677 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654678+0000", 2024-09-18T04:57:52.677 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.013367827, 2024-09-18T04:57:52.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039530155999999997, 2024-09-18T04:57:52.677 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.677 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.677 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.677 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160714 2024-09-18T04:57:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654678+0000", 2024-09-18T04:57:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654678+0000", 2024-09-18T04:57:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654678+0000", 2024-09-18T04:57:52.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654680+0000", 2024-09-18T04:57:52.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5229999999999998e-06 2024-09-18T04:57:52.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654686+0000", 2024-09-18T04:57:52.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9000000000000003e-06 2024-09-18T04:57:52.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654690+0000", 2024-09-18T04:57:52.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1060000000000002e-06 2024-09-18T04:57:52.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694042+0000", 2024-09-18T04:57:52.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039351490000000003 2024-09-18T04:57:52.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694142+0000", 2024-09-18T04:57:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100177 2024-09-18T04:57:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694208+0000", 2024-09-18T04:57:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6367999999999999e-05 2024-09-18T04:57:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160715 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:57:52.683 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.013351528000001, 2024-09-18T04:57:52.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039724306000000001, 2024-09-18T04:57:52.683 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.683 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.683 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.683 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.683 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.683 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160715 2024-09-18T04:57:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:57:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:57:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:57:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654696+0000", 2024-09-18T04:57:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1830000000000001e-06 2024-09-18T04:57:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654702+0000", 2024-09-18T04:57:52.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5110000000000003e-06 2024-09-18T04:57:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654706+0000", 2024-09-18T04:57:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2470000000000002e-06 2024-09-18T04:57:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694245+0000", 2024-09-18T04:57:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039539259 2024-09-18T04:57:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694327+0000", 2024-09-18T04:57:52.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1817000000000006e-05 2024-09-18T04:57:52.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694419+0000", 2024-09-18T04:57:52.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1639999999999997e-05 2024-09-18T04:57:52.687 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.687 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.687 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.688 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160716 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:52.688 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:57:52.688 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.013336342999999, 2024-09-18T04:57:52.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039842555000000002, 2024-09-18T04:57:52.690 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.690 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.690 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.691 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.691 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.691 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160716 2024-09-18T04:57:52.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.692 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:57:52.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:57:52.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:57:52.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654711+0000", 2024-09-18T04:57:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.99e-06 2024-09-18T04:57:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654717+0000", 2024-09-18T04:57:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6799999999999998e-06 2024-09-18T04:57:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654721+0000", 2024-09-18T04:57:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-18T04:57:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694449+0000", 2024-09-18T04:57:52.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039727476999999997 2024-09-18T04:57:52.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694497+0000", 2024-09-18T04:57:52.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8294000000000002e-05 2024-09-18T04:57:52.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694552+0000", 2024-09-18T04:57:52.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5024e-05 2024-09-18T04:57:52.698 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.699 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.699 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.699 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160717 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 727040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:52.699 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:57:52.699 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.013295757000002, 2024-09-18T04:57:52.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040030957999999998, 2024-09-18T04:57:52.699 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.699 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.699 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.699 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.700 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160717 2024-09-18T04:57:52.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.700 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:57:52.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:57:52.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:57:52.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654752+0000", 2024-09-18T04:57:52.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8219999999999998e-06 2024-09-18T04:57:52.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654769+0000", 2024-09-18T04:57:52.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6249e-05 2024-09-18T04:57:52.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654775+0000", 2024-09-18T04:57:52.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2090000000000002e-06 2024-09-18T04:57:52.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694589+0000", 2024-09-18T04:57:52.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039814651999999999 2024-09-18T04:57:52.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694674+0000", 2024-09-18T04:57:52.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5032999999999997e-05 2024-09-18T04:57:52.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694781+0000", 2024-09-18T04:57:52.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106452 2024-09-18T04:57:52.704 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.704 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.704 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.704 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160718 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 747520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:52.704 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:57:52.704 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.013265866000001, 2024-09-18T04:57:52.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040152520999999997, 2024-09-18T04:57:52.704 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.704 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.705 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.705 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.705 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.705 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160718 2024-09-18T04:57:52.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.705 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:57:52.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:57:52.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654779+0000", 2024-09-18T04:57:52.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654782+0000", 2024-09-18T04:57:52.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.633e-06 2024-09-18T04:57:52.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654789+0000", 2024-09-18T04:57:52.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8589999999999999e-06 2024-09-18T04:57:52.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654793+0000", 2024-09-18T04:57:52.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5449999999999997e-06 2024-09-18T04:57:52.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694817+0000", 2024-09-18T04:57:52.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040023940000000001 2024-09-18T04:57:52.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694861+0000", 2024-09-18T04:57:52.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3961999999999997e-05 2024-09-18T04:57:52.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694932+0000", 2024-09-18T04:57:52.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1033999999999998e-05 2024-09-18T04:57:52.708 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.708 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.708 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.709 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160719 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 768000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:52.709 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:57:52.709 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.013248268000002, 2024-09-18T04:57:52.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040275321000000003, 2024-09-18T04:57:52.709 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.709 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.709 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.709 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.709 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.709 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160719 2024-09-18T04:57:52.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.709 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:57:52.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:57:52.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:57:52.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654799+0000", 2024-09-18T04:57:52.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.204e-06 2024-09-18T04:57:52.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654806+0000", 2024-09-18T04:57:52.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0500000000000003e-06 2024-09-18T04:57:52.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654814+0000", 2024-09-18T04:57:52.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5549999999999997e-06 2024-09-18T04:57:52.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694967+0000", 2024-09-18T04:57:52.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040153393000000002 2024-09-18T04:57:52.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695014+0000", 2024-09-18T04:57:52.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7194999999999998e-05 2024-09-18T04:57:52.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695073+0000", 2024-09-18T04:57:52.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8315e-05 2024-09-18T04:57:52.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.713 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.713 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160720 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 788480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:52.713 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:57:52.713 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.013225975000001, 2024-09-18T04:57:52.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040546450999999997, 2024-09-18T04:57:52.713 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.713 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.713 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.714 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.714 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.714 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160720 2024-09-18T04:57:52.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.714 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:57:52.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:57:52.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654819+0000", 2024-09-18T04:57:52.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654822+0000", 2024-09-18T04:57:52.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:57:52.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654828+0000", 2024-09-18T04:57:52.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2450000000000003e-06 2024-09-18T04:57:52.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654833+0000", 2024-09-18T04:57:52.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6419999999999998e-06 2024-09-18T04:57:52.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695116+0000", 2024-09-18T04:57:52.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040283126000000002 2024-09-18T04:57:52.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695203+0000", 2024-09-18T04:57:52.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7693000000000002e-05 2024-09-18T04:57:52.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695366+0000", 2024-09-18T04:57:52.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016283700000000001 2024-09-18T04:57:52.717 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.717 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.717 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.718 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160721 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 808960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:52.718 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:57:52.718 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.013208639999998, 2024-09-18T04:57:52.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040737433000000003, 2024-09-18T04:57:52.718 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.718 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.718 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.718 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.718 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.718 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160721 2024-09-18T04:57:52.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.718 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:57:52.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:57:52.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:57:52.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654839+0000", 2024-09-18T04:57:52.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.43e-06 2024-09-18T04:57:52.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654847+0000", 2024-09-18T04:57:52.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7300000000000005e-06 2024-09-18T04:57:52.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654858+0000", 2024-09-18T04:57:52.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.099e-05 2024-09-18T04:57:52.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695391+0000", 2024-09-18T04:57:52.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040533476999999998 2024-09-18T04:57:52.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695477+0000", 2024-09-18T04:57:52.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5870000000000003e-05 2024-09-18T04:57:52.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695575+0000", 2024-09-18T04:57:52.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7264999999999998e-05 2024-09-18T04:57:52.722 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.722 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.722 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.722 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160722 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 829440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:52.722 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:57:52.722 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.01318311, 2024-09-18T04:57:52.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040875739000000001, 2024-09-18T04:57:52.722 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.722 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.723 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.723 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.723 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.723 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160722 2024-09-18T04:57:52.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.723 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:57:52.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:57:52.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654862+0000", 2024-09-18T04:57:52.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654883+0000", 2024-09-18T04:57:52.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0468000000000001e-05 2024-09-18T04:57:52.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654891+0000", 2024-09-18T04:57:52.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8320000000000006e-06 2024-09-18T04:57:52.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654896+0000", 2024-09-18T04:57:52.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.13e-06 2024-09-18T04:57:52.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695628+0000", 2024-09-18T04:57:52.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040732011999999998 2024-09-18T04:57:52.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695678+0000", 2024-09-18T04:57:52.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0453e-05 2024-09-18T04:57:52.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.727 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.727 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695738+0000", 2024-09-18T04:57:52.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0164999999999997e-05 2024-09-18T04:57:52.727 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.727 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.727 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.727 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160723 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 849920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:52.727 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:57:52.728 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.013091304, 2024-09-18T04:57:52.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041073576000000001, 2024-09-18T04:57:52.728 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.728 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.728 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.728 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.728 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.728 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160723 2024-09-18T04:57:52.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.728 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:57:52.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:57:52.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:57:52.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654958+0000", 2024-09-18T04:57:52.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7230000000000002e-06 2024-09-18T04:57:52.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654968+0000", 2024-09-18T04:57:52.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0665999999999999e-05 2024-09-18T04:57:52.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654974+0000", 2024-09-18T04:57:52.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9259999999999997e-06 2024-09-18T04:57:52.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695824+0000", 2024-09-18T04:57:52.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040850085000000001 2024-09-18T04:57:52.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695933+0000", 2024-09-18T04:57:52.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108738 2024-09-18T04:57:52.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696028+0000", 2024-09-18T04:57:52.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4988000000000006e-05 2024-09-18T04:57:52.732 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.732 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.732 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.732 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160724 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 870400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:52.732 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:57:52.732 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.013066961, 2024-09-18T04:57:52.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041279300999999997, 2024-09-18T04:57:52.732 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.732 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.732 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.732 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.733 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.733 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160724 2024-09-18T04:57:52.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.733 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:57:52.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:57:52.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654978+0000", 2024-09-18T04:57:52.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654981+0000", 2024-09-18T04:57:52.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4530000000000001e-06 2024-09-18T04:57:52.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654987+0000", 2024-09-18T04:57:52.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6359999999999997e-06 2024-09-18T04:57:52.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654995+0000", 2024-09-18T04:57:52.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9759999999999995e-06 2024-09-18T04:57:52.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.736 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696069+0000", 2024-09-18T04:57:52.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041073447999999999 2024-09-18T04:57:52.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696160+0000", 2024-09-18T04:57:52.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0835999999999996e-05 2024-09-18T04:57:52.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696258+0000", 2024-09-18T04:57:52.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8338999999999995e-05 2024-09-18T04:57:52.738 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.738 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.738 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.738 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160725 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 890880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:52.738 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:57:52.738 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.013045892000001, 2024-09-18T04:57:52.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041552253999999997, 2024-09-18T04:57:52.738 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.739 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.739 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.739 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.739 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.739 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160725 2024-09-18T04:57:52.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.740 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:57:52.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:57:52.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654999+0000", 2024-09-18T04:57:52.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.741 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.741 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655002+0000", 2024-09-18T04:57:52.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.419e-06 2024-09-18T04:57:52.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655008+0000", 2024-09-18T04:57:52.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2310000000000001e-06 2024-09-18T04:57:52.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655013+0000", 2024-09-18T04:57:52.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8439999999999997e-06 2024-09-18T04:57:52.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696353+0000", 2024-09-18T04:57:52.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041340322999999998 2024-09-18T04:57:52.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696455+0000", 2024-09-18T04:57:52.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000102105 2024-09-18T04:57:52.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696552+0000", 2024-09-18T04:57:52.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6738999999999997e-05 2024-09-18T04:57:52.744 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.744 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.744 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.744 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160726 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:52.744 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:57:52.744 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.01302866, 2024-09-18T04:57:52.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041786440000000001, 2024-09-18T04:57:52.745 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.745 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.745 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.745 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.745 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.745 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160726 2024-09-18T04:57:52.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.745 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:57:52.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.746 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:57:52.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:57:52.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655019+0000", 2024-09-18T04:57:52.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1100000000000001e-06 2024-09-18T04:57:52.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655025+0000", 2024-09-18T04:57:52.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2190000000000001e-06 2024-09-18T04:57:52.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655030+0000", 2024-09-18T04:57:52.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5340000000000001e-06 2024-09-18T04:57:52.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696607+0000", 2024-09-18T04:57:52.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041577309 2024-09-18T04:57:52.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696694+0000", 2024-09-18T04:57:52.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7230000000000003e-05 2024-09-18T04:57:52.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696804+0000", 2024-09-18T04:57:52.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0001094 2024-09-18T04:57:52.750 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.750 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.750 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.751 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160727 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 931840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:52.751 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:57:52.751 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.013011923000001, 2024-09-18T04:57:52.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042000527000000003, 2024-09-18T04:57:52.751 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.751 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.751 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.751 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.751 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.752 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160727 2024-09-18T04:57:52.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.752 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:57:52.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:57:52.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655033+0000", 2024-09-18T04:57:52.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655036+0000", 2024-09-18T04:57:52.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3489999999999999e-06 2024-09-18T04:57:52.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655042+0000", 2024-09-18T04:57:52.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1399999999999997e-06 2024-09-18T04:57:52.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655046+0000", 2024-09-18T04:57:52.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5480000000000003e-06 2024-09-18T04:57:52.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696843+0000", 2024-09-18T04:57:52.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041797103000000002 2024-09-18T04:57:52.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696931+0000", 2024-09-18T04:57:52.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7526000000000002e-05 2024-09-18T04:57:52.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697034+0000", 2024-09-18T04:57:52.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010336999999999999 2024-09-18T04:57:52.757 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.757 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.757 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.758 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160728 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 952320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:52.758 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:57:52.758 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.012995365999998, 2024-09-18T04:57:52.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042191293999999997, 2024-09-18T04:57:52.758 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.758 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.758 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.758 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.758 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.758 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160728 2024-09-18T04:57:52.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.758 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:57:52.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:57:52.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:57:52.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655052+0000", 2024-09-18T04:57:52.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.232e-06 2024-09-18T04:57:52.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655058+0000", 2024-09-18T04:57:52.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9869999999999996e-06 2024-09-18T04:57:52.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655063+0000", 2024-09-18T04:57:52.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8420000000000004e-06 2024-09-18T04:57:52.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697074+0000", 2024-09-18T04:57:52.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042010446 2024-09-18T04:57:52.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697157+0000", 2024-09-18T04:57:52.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3038e-05 2024-09-18T04:57:52.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697242+0000", 2024-09-18T04:57:52.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5115000000000001e-05 2024-09-18T04:57:52.762 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.763 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.763 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.763 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160729 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 972800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:52.763 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:57:52.763 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.012978607000001, 2024-09-18T04:57:52.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042367834, 2024-09-18T04:57:52.763 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.763 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.763 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.763 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.763 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.764 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160729 2024-09-18T04:57:52.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.764 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:57:52.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:57:52.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:57:52.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655069+0000", 2024-09-18T04:57:52.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2220000000000001e-06 2024-09-18T04:57:52.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655075+0000", 2024-09-18T04:57:52.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2129999999999996e-06 2024-09-18T04:57:52.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655080+0000", 2024-09-18T04:57:52.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6840000000000004e-06 2024-09-18T04:57:52.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697280+0000", 2024-09-18T04:57:52.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042199819999999999 2024-09-18T04:57:52.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697368+0000", 2024-09-18T04:57:52.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7863999999999998e-05 2024-09-18T04:57:52.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697435+0000", 2024-09-18T04:57:52.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7416000000000006e-05 2024-09-18T04:57:52.767 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.767 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.767 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.768 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160730 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 993280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:52.768 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:57:52.768 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.012961841999999, 2024-09-18T04:57:52.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042559582999999998, 2024-09-18T04:57:52.768 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.768 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.768 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.768 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.768 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.769 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160730 2024-09-18T04:57:52.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.769 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:57:52.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:57:52.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:57:52.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655086+0000", 2024-09-18T04:57:52.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1500000000000002e-06 2024-09-18T04:57:52.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655092+0000", 2024-09-18T04:57:52.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.896e-06 2024-09-18T04:57:52.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655096+0000", 2024-09-18T04:57:52.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7190000000000001e-06 2024-09-18T04:57:52.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697475+0000", 2024-09-18T04:57:52.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042378644 2024-09-18T04:57:52.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697553+0000", 2024-09-18T04:57:52.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7626999999999994e-05 2024-09-18T04:57:52.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697643+0000", 2024-09-18T04:57:52.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0921000000000006e-05 2024-09-18T04:57:52.773 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.773 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.773 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.773 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160731 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 1013760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:52.773 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:57:52.774 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.012929111999998, 2024-09-18T04:57:52.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042727764000000001, 2024-09-18T04:57:52.774 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.774 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.774 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.774 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.774 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.774 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160731 2024-09-18T04:57:52.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.774 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:57:52.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:57:52.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655116+0000", 2024-09-18T04:57:52.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655119+0000", 2024-09-18T04:57:52.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1240000000000001e-06 2024-09-18T04:57:52.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655131+0000", 2024-09-18T04:57:52.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2145e-05 2024-09-18T04:57:52.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655138+0000", 2024-09-18T04:57:52.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3369999999999996e-06 2024-09-18T04:57:52.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697688+0000", 2024-09-18T04:57:52.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042550604999999998 2024-09-18T04:57:52.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697781+0000", 2024-09-18T04:57:52.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2867000000000004e-05 2024-09-18T04:57:52.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697844+0000", 2024-09-18T04:57:52.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3236000000000002e-05 2024-09-18T04:57:52.779 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.779 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.779 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.779 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160732 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 1034240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:52.779 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:57:52.779 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 16.012903263999998, 2024-09-18T04:57:52.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042857977999999998, 2024-09-18T04:57:52.779 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:52.779 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:52.780 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:52.780 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:52.780 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:52.780 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160732 2024-09-18T04:57:52.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.780 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:52.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:52.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:57:52.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:52.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:57:52.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:52.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:52.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:57:52.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:52.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:52.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655145+0000", 2024-09-18T04:57:52.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7329999999999999e-06 2024-09-18T04:57:52.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:52.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655152+0000", 2024-09-18T04:57:52.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8249999999999999e-06 2024-09-18T04:57:52.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:52.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655158+0000", 2024-09-18T04:57:52.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5010000000000001e-06 2024-09-18T04:57:52.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:52.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697873+0000", 2024-09-18T04:57:52.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042714457999999997 2024-09-18T04:57:52.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:52.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697944+0000", 2024-09-18T04:57:52.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1486e-05 2024-09-18T04:57:52.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:52.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:52.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:52.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.698000+0000", 2024-09-18T04:57:52.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6332000000000002e-05 2024-09-18T04:57:52.785 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.785 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.785 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.785 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:52.785 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:52.785 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:52.786 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:57:52.787 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:52.784+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:52.796 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:57:52.888 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:52.886+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:52.939 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:52.939 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:52.940 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:52.940 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:52.949 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:57:52.955 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:57:52.957 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:57:52.989 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:52.987+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:53.011 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-18T04:57:53.065 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:53.065 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:53.065 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:53.065 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:53.066 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:53.076 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:57:53.091 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:53.088+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:53.175 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:57:53.175 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:57:53.189 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:57:53.190 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:57:53.192 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:53.188+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:53.205 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:57:53.206 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:57:53.222 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:57:53.222 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:57:53.226 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:53.226 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:53.237 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:57:53.240 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:57:53.240 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:57:53.256 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:57:53.257 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:57:53.274 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:57:53.274 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:57:53.327 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:53.324+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:53.341 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:53.341 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:53.350 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:57:53.394 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:53.392+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:53.452 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:53.453 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:53.462 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:57:53.495 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:53.491+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:53.595 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:53.592+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:53.606 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:53.606 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:53.615 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:57:53.697 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:53.694+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:53.717 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:53.717 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:53.725 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:57:53.809 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:53.807+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:53.867 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:53.867 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:53.876 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:57:53.917 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:53.914+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:54.000 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:53.997+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:54.017 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:54.017 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:54.026 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:57:54.101 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:54.099+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:54.139 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:54.140 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:54.150 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:57:54.234 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:54.232+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:54.293 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:54.294 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:54.303 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:57:54.304 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:54.301+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:54.404 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:54.402+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:54.445 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:54.446 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:54.454 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:57:54.505 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:54.503+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:54.556 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:54.556 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:54.564 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:57:54.608 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:54.604+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:54.707 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:54.704+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:54.709 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:54.709 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:54.718 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:57:54.808 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:54.805+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:54.861 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:54.861 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:54.870 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:57:54.909 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:54.907+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:54.991 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:54.991 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:55.000 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:57:55.027 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T04:57:55.044 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:55.041+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:55.111 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:55.108+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:55.118 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:55.118 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:55.128 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:57:55.213 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:55.210+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:55.281 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:55.282 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:55.286 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:57:55.291 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:57:55.332 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:55.329+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:55.411 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:57:55.412 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:57:55.413 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:55.409+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:55.429 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:57:55.429 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:57:55.459 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:55.459 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:55.459 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:55.459 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:55.462 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:57:55.462 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:57:55.471 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:57:55.482 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:57:55.482 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:57:55.498 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:57:55.499 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:57:55.514 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:55.512+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:55.515 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:57:55.515 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:57:55.531 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:57:55.532 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:57:55.548 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:57:55.548 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:57:55.590 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:55.590 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:55.590 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:55.590 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:55.590 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:55.599 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:57:55.615 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:55.613+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:55.716 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:55.714+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:55.748 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:55.748 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:55.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:55.748 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:55.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.748 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14125 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:55.748 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:57:55.749 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 79.618002304000001, 2024-09-18T04:57:55.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052956470999999998, 2024-09-18T04:57:55.749 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:55.749 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:55.749 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:55.749 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:55.749 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:55.749 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14125 2024-09-18T04:57:55.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.749 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:55.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:55.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:57:55.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.750 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:55.750 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:57:55.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:55.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126849+0000", 2024-09-18T04:57:55.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:55.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:55.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126852+0000", 2024-09-18T04:57:55.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3659999999999999e-06 2024-09-18T04:57:55.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:55.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126858+0000", 2024-09-18T04:57:55.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4540000000000004e-06 2024-09-18T04:57:55.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:55.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126863+0000", 2024-09-18T04:57:55.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1780000000000002e-06 2024-09-18T04:57:55.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:55.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179608+0000", 2024-09-18T04:57:55.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052745159999999999 2024-09-18T04:57:55.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:55.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179703+0000", 2024-09-18T04:57:55.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4773999999999998e-05 2024-09-18T04:57:55.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:55.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179806+0000", 2024-09-18T04:57:55.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010296799999999999 2024-09-18T04:57:55.754 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.754 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:55.754 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.754 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14136 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1732608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:55.754 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:55.754 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 79.615841752999998, 2024-09-18T04:57:55.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052973412999999997, 2024-09-18T04:57:55.754 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:55.754 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:55.755 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:55.755 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:55.755 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:55.755 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14136 2024-09-18T04:57:55.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.755 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:55.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:55.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:55.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:55.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:55.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.757 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:55.757 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:57:55.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:55.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:55.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129013+0000", 2024-09-18T04:57:55.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.96e-06 2024-09-18T04:57:55.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:55.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129020+0000", 2024-09-18T04:57:55.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6809999999999998e-06 2024-09-18T04:57:55.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:55.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129026+0000", 2024-09-18T04:57:55.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:57:55.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:55.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181873+0000", 2024-09-18T04:57:55.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052847238999999997 2024-09-18T04:57:55.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:55.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181932+0000", 2024-09-18T04:57:55.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9339000000000002e-05 2024-09-18T04:57:55.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:55.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181984+0000", 2024-09-18T04:57:55.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1348999999999999e-05 2024-09-18T04:57:55.760 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.760 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:55.760 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.761 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14137 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1781760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:55.761 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:57:55.761 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 79.615821347999997, 2024-09-18T04:57:55.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053128914999999999, 2024-09-18T04:57:55.761 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:55.761 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:55.761 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:55.762 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:55.762 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:55.762 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14137 2024-09-18T04:57:55.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.762 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:55.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:55.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:57:55.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:55.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:57:55.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:55.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129030+0000", 2024-09-18T04:57:55.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:55.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:55.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129033+0000", 2024-09-18T04:57:55.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7439999999999999e-06 2024-09-18T04:57:55.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:55.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129039+0000", 2024-09-18T04:57:55.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.314e-06 2024-09-18T04:57:55.764 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.764 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:55.764 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129044+0000", 2024-09-18T04:57:55.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9919999999999998e-06 2024-09-18T04:57:55.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:55.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182015+0000", 2024-09-18T04:57:55.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052970875000000001 2024-09-18T04:57:55.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:55.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182058+0000", 2024-09-18T04:57:55.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2997000000000002e-05 2024-09-18T04:57:55.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:55.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182160+0000", 2024-09-18T04:57:55.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101437 2024-09-18T04:57:55.766 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.766 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:55.766 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.767 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14138 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1830912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:55.767 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:57:55.767 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 79.615803314000004, 2024-09-18T04:57:55.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053256435999999997, 2024-09-18T04:57:55.767 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:55.767 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:55.767 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:55.767 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:55.767 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:55.768 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14138 2024-09-18T04:57:55.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.768 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:55.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:55.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:57:55.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:55.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:57:55.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.769 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:55.769 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129048+0000", 2024-09-18T04:57:55.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:55.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:55.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129056+0000", 2024-09-18T04:57:55.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.413e-06 2024-09-18T04:57:55.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:55.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129062+0000", 2024-09-18T04:57:55.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6810000000000001e-06 2024-09-18T04:57:55.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:55.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129068+0000", 2024-09-18T04:57:55.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0459999999999996e-06 2024-09-18T04:57:55.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:55.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182191+0000", 2024-09-18T04:57:55.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053123210999999997 2024-09-18T04:57:55.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:55.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182238+0000", 2024-09-18T04:57:55.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7404999999999998e-05 2024-09-18T04:57:55.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:55.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182305+0000", 2024-09-18T04:57:55.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7028999999999998e-05 2024-09-18T04:57:55.772 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.772 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:55.772 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.773 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15910 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2236416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:55.773 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:55.773 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 79.516715782000006, 2024-09-18T04:57:55.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053241054000000003, 2024-09-18T04:57:55.773 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:55.773 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:55.773 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:55.773 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:55.773 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:55.773 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15910 2024-09-18T04:57:55.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.773 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:55.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:55.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:55.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:55.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:55.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:55.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:57:55.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:55.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:55.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228138+0000", 2024-09-18T04:57:55.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:57:55.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:55.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228144+0000", 2024-09-18T04:57:55.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5369999999999997e-06 2024-09-18T04:57:55.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:55.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228149+0000", 2024-09-18T04:57:55.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1209999999999998e-06 2024-09-18T04:57:55.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:55.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280822+0000", 2024-09-18T04:57:55.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052673194999999999 2024-09-18T04:57:55.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:55.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280903+0000", 2024-09-18T04:57:55.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1179000000000003e-05 2024-09-18T04:57:55.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:55.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281377+0000", 2024-09-18T04:57:55.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047417899999999997 2024-09-18T04:57:55.777 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.777 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:55.777 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.777 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15911 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2285568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:55.777 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:55.777 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 79.51669905, 2024-09-18T04:57:55.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053415151000000001, 2024-09-18T04:57:55.778 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:55.778 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:55.778 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:55.778 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:55.778 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:55.778 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15911 2024-09-18T04:57:55.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.778 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:55.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.778 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:55.778 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:55.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:55.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:55.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:55.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:57:55.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:55.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:55.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228155+0000", 2024-09-18T04:57:55.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:57:55.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:55.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228161+0000", 2024-09-18T04:57:55.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:57:55.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:55.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228166+0000", 2024-09-18T04:57:55.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2320000000000001e-06 2024-09-18T04:57:55.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:55.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281405+0000", 2024-09-18T04:57:55.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053239336999999998 2024-09-18T04:57:55.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:55.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281482+0000", 2024-09-18T04:57:55.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6569999999999994e-05 2024-09-18T04:57:55.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:55.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281568+0000", 2024-09-18T04:57:55.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6328000000000004e-05 2024-09-18T04:57:55.782 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.782 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:55.782 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.782 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15918 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2662400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:55.782 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:55.782 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 79.514930069000002, 2024-09-18T04:57:55.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053091065, 2024-09-18T04:57:55.782 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:55.782 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:55.782 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:55.782 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:55.783 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:55.783 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15918 2024-09-18T04:57:55.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.783 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:55.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:55.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:55.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:55.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:55.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:55.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:57:55.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:55.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:55.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229924+0000", 2024-09-18T04:57:55.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:57:55.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:55.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229930+0000", 2024-09-18T04:57:55.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.066e-06 2024-09-18T04:57:55.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:55.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229935+0000", 2024-09-18T04:57:55.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9119999999999997e-06 2024-09-18T04:57:55.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:55.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282840+0000", 2024-09-18T04:57:55.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052905119 2024-09-18T04:57:55.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:55.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282922+0000", 2024-09-18T04:57:55.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2081000000000002e-05 2024-09-18T04:57:55.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:55.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283013+0000", 2024-09-18T04:57:55.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0828000000000005e-05 2024-09-18T04:57:55.786 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.786 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:55.786 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.787 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15919 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2711552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:55.787 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:55.787 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 79.514912791, 2024-09-18T04:57:55.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053293021000000003, 2024-09-18T04:57:55.787 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:55.787 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:55.787 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:55.787 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:55.787 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:55.787 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15919 2024-09-18T04:57:55.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.787 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:55.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:55.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:55.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:55.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:55.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:55.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:57:55.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:55.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:55.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229941+0000", 2024-09-18T04:57:55.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2639999999999998e-06 2024-09-18T04:57:55.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:55.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229947+0000", 2024-09-18T04:57:55.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9750000000000004e-06 2024-09-18T04:57:55.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:55.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229952+0000", 2024-09-18T04:57:55.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7929999999999997e-06 2024-09-18T04:57:55.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:55.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283062+0000", 2024-09-18T04:57:55.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05310952 2024-09-18T04:57:55.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:55.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283162+0000", 2024-09-18T04:57:55.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100098 2024-09-18T04:57:55.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:55.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283232+0000", 2024-09-18T04:57:55.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0693999999999998e-05 2024-09-18T04:57:55.791 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.791 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:55.791 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.791 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15920 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2760704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:55.792 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:55.792 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 79.514896030000003, 2024-09-18T04:57:55.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053502367000000002, 2024-09-18T04:57:55.792 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:55.792 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:55.792 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:55.792 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:55.792 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:55.792 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15920 2024-09-18T04:57:55.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.793 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:55.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:55.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:55.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:55.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:55.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:55.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:57:55.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:55.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:55.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229958+0000", 2024-09-18T04:57:55.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1770000000000001e-06 2024-09-18T04:57:55.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:55.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229964+0000", 2024-09-18T04:57:55.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8470000000000001e-06 2024-09-18T04:57:55.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:55.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229968+0000", 2024-09-18T04:57:55.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5680000000000001e-06 2024-09-18T04:57:55.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:55.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283269+0000", 2024-09-18T04:57:55.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053300620999999999 2024-09-18T04:57:55.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:55.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283359+0000", 2024-09-18T04:57:55.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9971999999999998e-05 2024-09-18T04:57:55.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:55.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283458+0000", 2024-09-18T04:57:55.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9483999999999998e-05 2024-09-18T04:57:55.796 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.796 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:55.796 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.796 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15921 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2809856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:55.796 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:57:55.796 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 79.514879480000005, 2024-09-18T04:57:55.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053800977, 2024-09-18T04:57:55.797 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:55.797 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:55.797 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:55.797 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:55.797 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:55.797 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15921 2024-09-18T04:57:55.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.797 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:55.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:55.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:57:55.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:55.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:57:55.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:55.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229972+0000", 2024-09-18T04:57:55.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:55.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:55.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229974+0000", 2024-09-18T04:57:55.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2910000000000002e-06 2024-09-18T04:57:55.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:55.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229982+0000", 2024-09-18T04:57:55.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7249999999999997e-06 2024-09-18T04:57:55.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:55.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229989+0000", 2024-09-18T04:57:55.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6429999999999998e-06 2024-09-18T04:57:55.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:55.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283601+0000", 2024-09-18T04:57:55.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053612211 2024-09-18T04:57:55.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:55.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283681+0000", 2024-09-18T04:57:55.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0125999999999998e-05 2024-09-18T04:57:55.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:55.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283774+0000", 2024-09-18T04:57:55.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2362999999999996e-05 2024-09-18T04:57:55.801 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.801 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:55.801 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.802 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15922 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2859008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:55.802 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:57:55.802 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 79.514834540999999, 2024-09-18T04:57:55.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053882270000000003, 2024-09-18T04:57:55.802 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:55.802 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:55.802 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:55.802 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:55.802 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:55.803 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15922 2024-09-18T04:57:55.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.803 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:55.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:55.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:57:55.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:55.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:57:55.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:55.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230017+0000", 2024-09-18T04:57:55.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:55.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:55.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230019+0000", 2024-09-18T04:57:55.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4210000000000002e-06 2024-09-18T04:57:55.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:55.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230027+0000", 2024-09-18T04:57:55.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000002e-06 2024-09-18T04:57:55.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:55.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230032+0000", 2024-09-18T04:57:55.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0089999999999998e-06 2024-09-18T04:57:55.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:55.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283800+0000", 2024-09-18T04:57:55.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053768418999999998 2024-09-18T04:57:55.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:55.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283849+0000", 2024-09-18T04:57:55.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9339000000000003e-05 2024-09-18T04:57:55.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:55.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283900+0000", 2024-09-18T04:57:55.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0421999999999999e-05 2024-09-18T04:57:55.807 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.808 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:55.808 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.808 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15923 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2924544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:55.808 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:55.808 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 79.514815671999997, 2024-09-18T04:57:55.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054073016000000002, 2024-09-18T04:57:55.809 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:55.809 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:55.809 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:55.809 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:55.809 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:55.809 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15923 2024-09-18T04:57:55.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.809 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:55.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:55.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:55.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:55.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:55.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:55.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:57:55.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:55.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:55.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230038+0000", 2024-09-18T04:57:55.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.463e-06 2024-09-18T04:57:55.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:55.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230045+0000", 2024-09-18T04:57:55.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3570000000000003e-06 2024-09-18T04:57:55.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:55.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230050+0000", 2024-09-18T04:57:55.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7299999999999996e-06 2024-09-18T04:57:55.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:55.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283928+0000", 2024-09-18T04:57:55.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053878218999999998 2024-09-18T04:57:55.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:55.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284010+0000", 2024-09-18T04:57:55.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2577000000000006e-05 2024-09-18T04:57:55.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:55.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284109+0000", 2024-09-18T04:57:55.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9102000000000001e-05 2024-09-18T04:57:55.813 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.813 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:55.813 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.813 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15924 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:55.813 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:55.813 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 79.514797201999997, 2024-09-18T04:57:55.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054108470999999998, 2024-09-18T04:57:55.813 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:55.813 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:55.813 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:55.813 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:55.814 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:55.814 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15924 2024-09-18T04:57:55.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.814 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:55.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:55.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:55.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:55.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:55.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:55.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:57:55.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:55.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:55.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230061+0000", 2024-09-18T04:57:55.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8240000000000002e-06 2024-09-18T04:57:55.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:55.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230068+0000", 2024-09-18T04:57:55.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4320000000000004e-06 2024-09-18T04:57:55.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:55.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230073+0000", 2024-09-18T04:57:55.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.921e-06 2024-09-18T04:57:55.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:55.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284153+0000", 2024-09-18T04:57:55.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054080793000000002 2024-09-18T04:57:55.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:55.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284163+0000", 2024-09-18T04:57:55.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.859e-06 2024-09-18T04:57:55.817 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.817 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:55.817 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.817 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15925 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:55.817 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:55.817 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 79.514773086999995, 2024-09-18T04:57:55.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054134632000000002, 2024-09-18T04:57:55.817 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:55.817 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:55.818 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:55.818 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:55.818 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:55.818 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15925 2024-09-18T04:57:55.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.818 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:55.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:55.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:55.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:55.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:55.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:55.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:57:55.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:55.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:55.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230081+0000", 2024-09-18T04:57:55.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4109999999999999e-06 2024-09-18T04:57:55.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:55.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230087+0000", 2024-09-18T04:57:55.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.161e-06 2024-09-18T04:57:55.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:55.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230092+0000", 2024-09-18T04:57:55.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9259999999999999e-06 2024-09-18T04:57:55.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:55.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284204+0000", 2024-09-18T04:57:55.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054111546000000003 2024-09-18T04:57:55.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:55.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284214+0000", 2024-09-18T04:57:55.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9639999999999998e-06 2024-09-18T04:57:55.821 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.821 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:55.821 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.821 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15926 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:55.821 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:55.821 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 79.514755762999997, 2024-09-18T04:57:55.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054143193999999999, 2024-09-18T04:57:55.822 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:55.822 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:55.822 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:55.822 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:55.822 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:55.822 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15926 2024-09-18T04:57:55.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.822 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:55.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:55.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:55.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:55.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:55.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:55.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:57:55.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:55.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:55.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230098+0000", 2024-09-18T04:57:55.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1399999999999998e-06 2024-09-18T04:57:55.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:55.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230104+0000", 2024-09-18T04:57:55.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-18T04:57:55.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:55.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230109+0000", 2024-09-18T04:57:55.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9250000000000003e-06 2024-09-18T04:57:55.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:55.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284231+0000", 2024-09-18T04:57:55.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054122297999999999 2024-09-18T04:57:55.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:55.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284239+0000", 2024-09-18T04:57:55.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1689999999999992e-06 2024-09-18T04:57:55.825 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.825 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:55.826 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.826 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15927 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:55.826 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:55.826 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 79.514738715999997, 2024-09-18T04:57:55.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054168435000000001, 2024-09-18T04:57:55.826 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:55.826 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:55.826 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:55.826 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:55.826 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:55.826 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15927 2024-09-18T04:57:55.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.827 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:55.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:55.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:55.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:55.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:55.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:55.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:57:55.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:55.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:55.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230116+0000", 2024-09-18T04:57:55.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5150000000000001e-06 2024-09-18T04:57:55.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:55.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230122+0000", 2024-09-18T04:57:55.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1190000000000002e-06 2024-09-18T04:57:55.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:55.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230126+0000", 2024-09-18T04:57:55.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.792e-06 2024-09-18T04:57:55.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:55.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284273+0000", 2024-09-18T04:57:55.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054146086000000003 2024-09-18T04:57:55.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:55.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284282+0000", 2024-09-18T04:57:55.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2590000000000006e-06 2024-09-18T04:57:55.830 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.830 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:55.830 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.830 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15928 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:55.831 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:57:55.831 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 79.514721332999997, 2024-09-18T04:57:55.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054189739000000001, 2024-09-18T04:57:55.831 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:55.831 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:55.831 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:55.831 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:55.831 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:55.831 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15928 2024-09-18T04:57:55.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.831 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:55.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:55.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:57:55.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:55.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:57:55.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:55.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230130+0000", 2024-09-18T04:57:55.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:55.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:55.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230133+0000", 2024-09-18T04:57:55.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.244e-06 2024-09-18T04:57:55.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:55.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230139+0000", 2024-09-18T04:57:55.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9830000000000001e-06 2024-09-18T04:57:55.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:55.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230143+0000", 2024-09-18T04:57:55.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7530000000000001e-06 2024-09-18T04:57:55.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:55.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284313+0000", 2024-09-18T04:57:55.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054169332000000001 2024-09-18T04:57:55.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:55.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284320+0000", 2024-09-18T04:57:55.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8699999999999992e-06 2024-09-18T04:57:55.835 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.835 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:55.835 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.836 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15929 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:55.836 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:55.836 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 79.514704893000001, 2024-09-18T04:57:55.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054218771999999998, 2024-09-18T04:57:55.836 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:55.836 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:55.836 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:55.837 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:55.837 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:55.837 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15929 2024-09-18T04:57:55.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.837 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:55.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:55.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:55.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:55.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:55.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:55.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:57:55.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:55.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:55.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230149+0000", 2024-09-18T04:57:55.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-18T04:57:55.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:55.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230155+0000", 2024-09-18T04:57:55.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2469999999999997e-06 2024-09-18T04:57:55.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:55.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230160+0000", 2024-09-18T04:57:55.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7129999999999996e-06 2024-09-18T04:57:55.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:55.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284355+0000", 2024-09-18T04:57:55.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054194900999999997 2024-09-18T04:57:55.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:55.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284366+0000", 2024-09-18T04:57:55.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0993e-05 2024-09-18T04:57:55.842 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.842 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:55.842 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.843 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15930 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:55.843 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:55.843 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 79.514687893000001, 2024-09-18T04:57:55.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054244713999999999, 2024-09-18T04:57:55.843 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:55.843 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:55.843 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:55.843 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:55.843 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:55.843 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15930 2024-09-18T04:57:55.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.843 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:55.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:55.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:55.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:55.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:55.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:55.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:57:55.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:55.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:55.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230166+0000", 2024-09-18T04:57:55.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1009999999999999e-06 2024-09-18T04:57:55.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:55.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230172+0000", 2024-09-18T04:57:55.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.981e-06 2024-09-18T04:57:55.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:55.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230184+0000", 2024-09-18T04:57:55.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2449000000000001e-05 2024-09-18T04:57:55.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:55.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284400+0000", 2024-09-18T04:57:55.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054215452999999997 2024-09-18T04:57:55.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:55.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284409+0000", 2024-09-18T04:57:55.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0839999999999998e-06 2024-09-18T04:57:55.847 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.847 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:55.849 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.849 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15931 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:55.849 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:57:55.849 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 79.514662103999996, 2024-09-18T04:57:55.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054261928000000001, 2024-09-18T04:57:55.849 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:55.849 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:57:55.849 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:55.850 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:55.850 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:55.850 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15931 2024-09-18T04:57:55.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.850 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:55.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:55.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:57:55.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:55.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:57:55.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:55.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:55.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230189+0000", 2024-09-18T04:57:55.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:55.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:55.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230192+0000", 2024-09-18T04:57:55.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6599999999999999e-06 2024-09-18T04:57:55.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:55.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230199+0000", 2024-09-18T04:57:55.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5570000000000001e-06 2024-09-18T04:57:55.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:55.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230204+0000", 2024-09-18T04:57:55.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2270000000000001e-06 2024-09-18T04:57:55.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:55.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284442+0000", 2024-09-18T04:57:55.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054238383000000001 2024-09-18T04:57:55.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:55.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:55.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:55.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284452+0000", 2024-09-18T04:57:55.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6069999999999997e-06 2024-09-18T04:57:55.853 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.853 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:55.853 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.853 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:55.853 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:55.853 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:55.854 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:57:55.855 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:55.852+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:55.956 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:55.953+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:55.961 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:55.961 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:55.961 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:55.961 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:55.970 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:57:56.056 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:56.054+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:56.067 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-18T04:57:56.084 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:56.084 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:56.084 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:56.084 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:56.084 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:56.095 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:57:56.180 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:56.177+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:56.248 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:56.248 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:56.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:56.248 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:56.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.249 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160713 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 645120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:56.249 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654655+0000", 2024-09-18T04:57:56.249 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.590416786999999, 2024-09-18T04:57:56.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039354912999999998, 2024-09-18T04:57:56.249 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.249 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.249 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.249 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.249 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.249 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160713 2024-09-18T04:57:56.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.249 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654655+0000", 2024-09-18T04:57:56.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654655+0000", 2024-09-18T04:57:56.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654654+0000", 2024-09-18T04:57:56.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654657+0000", 2024-09-18T04:57:56.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7049999999999999e-06 2024-09-18T04:57:56.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654669+0000", 2024-09-18T04:57:56.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1960000000000001e-05 2024-09-18T04:57:56.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654674+0000", 2024-09-18T04:57:56.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8729999999999998e-06 2024-09-18T04:57:56.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.693848+0000", 2024-09-18T04:57:56.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039174202999999998 2024-09-18T04:57:56.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.693926+0000", 2024-09-18T04:57:56.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8248e-05 2024-09-18T04:57:56.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694009+0000", 2024-09-18T04:57:56.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3363999999999997e-05 2024-09-18T04:57:56.253 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.253 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.253 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.253 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160714 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 665600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:56.253 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654678+0000", 2024-09-18T04:57:56.253 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.590393411000001, 2024-09-18T04:57:56.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039530155999999997, 2024-09-18T04:57:56.253 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.254 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.254 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.254 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.254 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.254 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160714 2024-09-18T04:57:56.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.254 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654678+0000", 2024-09-18T04:57:56.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654678+0000", 2024-09-18T04:57:56.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654678+0000", 2024-09-18T04:57:56.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654680+0000", 2024-09-18T04:57:56.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5229999999999998e-06 2024-09-18T04:57:56.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654686+0000", 2024-09-18T04:57:56.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9000000000000003e-06 2024-09-18T04:57:56.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654690+0000", 2024-09-18T04:57:56.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1060000000000002e-06 2024-09-18T04:57:56.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694042+0000", 2024-09-18T04:57:56.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039351490000000003 2024-09-18T04:57:56.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694142+0000", 2024-09-18T04:57:56.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100177 2024-09-18T04:57:56.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694208+0000", 2024-09-18T04:57:56.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6367999999999999e-05 2024-09-18T04:57:56.258 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.258 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.258 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.258 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160715 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:56.258 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:57:56.258 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.590377111999999, 2024-09-18T04:57:56.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039724306000000001, 2024-09-18T04:57:56.258 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.258 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.258 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.258 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.259 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.259 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160715 2024-09-18T04:57:56.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.259 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:57:56.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:57:56.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:57:56.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654696+0000", 2024-09-18T04:57:56.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1830000000000001e-06 2024-09-18T04:57:56.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654702+0000", 2024-09-18T04:57:56.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5110000000000003e-06 2024-09-18T04:57:56.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654706+0000", 2024-09-18T04:57:56.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2470000000000002e-06 2024-09-18T04:57:56.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694245+0000", 2024-09-18T04:57:56.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039539259 2024-09-18T04:57:56.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694327+0000", 2024-09-18T04:57:56.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1817000000000006e-05 2024-09-18T04:57:56.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694419+0000", 2024-09-18T04:57:56.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1639999999999997e-05 2024-09-18T04:57:56.262 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.262 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.262 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.263 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160716 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:56.263 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:57:56.263 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.590361927, 2024-09-18T04:57:56.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039842555000000002, 2024-09-18T04:57:56.263 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.263 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.263 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.263 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.263 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.263 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160716 2024-09-18T04:57:56.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.263 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:57:56.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:57:56.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:57:56.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654711+0000", 2024-09-18T04:57:56.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.99e-06 2024-09-18T04:57:56.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654717+0000", 2024-09-18T04:57:56.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6799999999999998e-06 2024-09-18T04:57:56.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654721+0000", 2024-09-18T04:57:56.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-18T04:57:56.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694449+0000", 2024-09-18T04:57:56.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039727476999999997 2024-09-18T04:57:56.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694497+0000", 2024-09-18T04:57:56.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8294000000000002e-05 2024-09-18T04:57:56.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694552+0000", 2024-09-18T04:57:56.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5024e-05 2024-09-18T04:57:56.267 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.267 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.267 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.267 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160717 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 727040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:56.267 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:57:56.268 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.590321340999999, 2024-09-18T04:57:56.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040030957999999998, 2024-09-18T04:57:56.268 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.268 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.268 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.268 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.268 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.268 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160717 2024-09-18T04:57:56.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.268 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:57:56.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:57:56.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:57:56.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654752+0000", 2024-09-18T04:57:56.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8219999999999998e-06 2024-09-18T04:57:56.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654769+0000", 2024-09-18T04:57:56.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6249e-05 2024-09-18T04:57:56.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654775+0000", 2024-09-18T04:57:56.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2090000000000002e-06 2024-09-18T04:57:56.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694589+0000", 2024-09-18T04:57:56.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039814651999999999 2024-09-18T04:57:56.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694674+0000", 2024-09-18T04:57:56.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5032999999999997e-05 2024-09-18T04:57:56.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694781+0000", 2024-09-18T04:57:56.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106452 2024-09-18T04:57:56.272 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.272 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.272 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.272 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160718 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 747520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:56.272 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:57:56.272 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.590291449999999, 2024-09-18T04:57:56.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040152520999999997, 2024-09-18T04:57:56.272 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.272 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.273 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.273 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.273 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.273 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160718 2024-09-18T04:57:56.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.273 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:57:56.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:57:56.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654779+0000", 2024-09-18T04:57:56.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654782+0000", 2024-09-18T04:57:56.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.633e-06 2024-09-18T04:57:56.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654789+0000", 2024-09-18T04:57:56.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8589999999999999e-06 2024-09-18T04:57:56.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654793+0000", 2024-09-18T04:57:56.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5449999999999997e-06 2024-09-18T04:57:56.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694817+0000", 2024-09-18T04:57:56.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040023940000000001 2024-09-18T04:57:56.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694861+0000", 2024-09-18T04:57:56.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3961999999999997e-05 2024-09-18T04:57:56.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694932+0000", 2024-09-18T04:57:56.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1033999999999998e-05 2024-09-18T04:57:56.276 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.276 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.276 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.276 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160719 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 768000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:56.276 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:57:56.277 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.590273851999999, 2024-09-18T04:57:56.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040275321000000003, 2024-09-18T04:57:56.277 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.277 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.277 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.277 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.277 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.277 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160719 2024-09-18T04:57:56.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.277 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:57:56.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:57:56.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:57:56.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654799+0000", 2024-09-18T04:57:56.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.204e-06 2024-09-18T04:57:56.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654806+0000", 2024-09-18T04:57:56.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0500000000000003e-06 2024-09-18T04:57:56.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654814+0000", 2024-09-18T04:57:56.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5549999999999997e-06 2024-09-18T04:57:56.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694967+0000", 2024-09-18T04:57:56.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040153393000000002 2024-09-18T04:57:56.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695014+0000", 2024-09-18T04:57:56.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7194999999999998e-05 2024-09-18T04:57:56.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695073+0000", 2024-09-18T04:57:56.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8315e-05 2024-09-18T04:57:56.280 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.281 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.281 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.281 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160720 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 788480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:56.281 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:57:56.281 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.590251558999999, 2024-09-18T04:57:56.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040546450999999997, 2024-09-18T04:57:56.281 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.281 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.281 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.281 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.281 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.282 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160720 2024-09-18T04:57:56.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.282 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:57:56.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:57:56.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654819+0000", 2024-09-18T04:57:56.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654822+0000", 2024-09-18T04:57:56.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:57:56.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654828+0000", 2024-09-18T04:57:56.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2450000000000003e-06 2024-09-18T04:57:56.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654833+0000", 2024-09-18T04:57:56.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6419999999999998e-06 2024-09-18T04:57:56.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695116+0000", 2024-09-18T04:57:56.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040283126000000002 2024-09-18T04:57:56.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695203+0000", 2024-09-18T04:57:56.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7693000000000002e-05 2024-09-18T04:57:56.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695366+0000", 2024-09-18T04:57:56.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016283700000000001 2024-09-18T04:57:56.285 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.285 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.285 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.286 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160721 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 808960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:56.286 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:57:56.286 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.590234224, 2024-09-18T04:57:56.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040737433000000003, 2024-09-18T04:57:56.286 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.286 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.286 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.286 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.286 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.286 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160721 2024-09-18T04:57:56.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.286 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:57:56.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:57:56.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:57:56.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654839+0000", 2024-09-18T04:57:56.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.43e-06 2024-09-18T04:57:56.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654847+0000", 2024-09-18T04:57:56.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7300000000000005e-06 2024-09-18T04:57:56.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654858+0000", 2024-09-18T04:57:56.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.099e-05 2024-09-18T04:57:56.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695391+0000", 2024-09-18T04:57:56.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040533476999999998 2024-09-18T04:57:56.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695477+0000", 2024-09-18T04:57:56.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5870000000000003e-05 2024-09-18T04:57:56.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695575+0000", 2024-09-18T04:57:56.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7264999999999998e-05 2024-09-18T04:57:56.290 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.290 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.290 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.290 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160722 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 829440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:56.290 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:57:56.290 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.590208694000001, 2024-09-18T04:57:56.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040875739000000001, 2024-09-18T04:57:56.290 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.290 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.290 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.290 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.290 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.291 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160722 2024-09-18T04:57:56.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.291 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:57:56.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:57:56.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654862+0000", 2024-09-18T04:57:56.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654883+0000", 2024-09-18T04:57:56.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0468000000000001e-05 2024-09-18T04:57:56.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654891+0000", 2024-09-18T04:57:56.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8320000000000006e-06 2024-09-18T04:57:56.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654896+0000", 2024-09-18T04:57:56.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.13e-06 2024-09-18T04:57:56.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695628+0000", 2024-09-18T04:57:56.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040732011999999998 2024-09-18T04:57:56.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695678+0000", 2024-09-18T04:57:56.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0453e-05 2024-09-18T04:57:56.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695738+0000", 2024-09-18T04:57:56.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0164999999999997e-05 2024-09-18T04:57:56.294 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.294 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.294 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.294 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160723 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 849920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:56.295 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:57:56.295 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.590116888000001, 2024-09-18T04:57:56.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041073576000000001, 2024-09-18T04:57:56.295 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.295 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.295 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.295 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.295 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.295 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160723 2024-09-18T04:57:56.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.296 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:57:56.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:57:56.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:57:56.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654958+0000", 2024-09-18T04:57:56.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7230000000000002e-06 2024-09-18T04:57:56.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654968+0000", 2024-09-18T04:57:56.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0665999999999999e-05 2024-09-18T04:57:56.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654974+0000", 2024-09-18T04:57:56.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9259999999999997e-06 2024-09-18T04:57:56.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695824+0000", 2024-09-18T04:57:56.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040850085000000001 2024-09-18T04:57:56.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695933+0000", 2024-09-18T04:57:56.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108738 2024-09-18T04:57:56.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696028+0000", 2024-09-18T04:57:56.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4988000000000006e-05 2024-09-18T04:57:56.299 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.299 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.299 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.299 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160724 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 870400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:56.299 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:57:56.299 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.590092545000001, 2024-09-18T04:57:56.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041279300999999997, 2024-09-18T04:57:56.299 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.299 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.300 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.300 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.300 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.300 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160724 2024-09-18T04:57:56.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.300 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:57:56.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:57:56.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654978+0000", 2024-09-18T04:57:56.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654981+0000", 2024-09-18T04:57:56.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4530000000000001e-06 2024-09-18T04:57:56.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654987+0000", 2024-09-18T04:57:56.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6359999999999997e-06 2024-09-18T04:57:56.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654995+0000", 2024-09-18T04:57:56.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9759999999999995e-06 2024-09-18T04:57:56.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696069+0000", 2024-09-18T04:57:56.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041073447999999999 2024-09-18T04:57:56.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696160+0000", 2024-09-18T04:57:56.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0835999999999996e-05 2024-09-18T04:57:56.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696258+0000", 2024-09-18T04:57:56.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8338999999999995e-05 2024-09-18T04:57:56.303 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.303 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.303 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.303 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160725 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 890880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:56.303 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:57:56.304 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.590071475999999, 2024-09-18T04:57:56.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041552253999999997, 2024-09-18T04:57:56.304 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.304 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.304 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.304 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.304 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.304 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160725 2024-09-18T04:57:56.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.304 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:57:56.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:57:56.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654999+0000", 2024-09-18T04:57:56.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655002+0000", 2024-09-18T04:57:56.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.419e-06 2024-09-18T04:57:56.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655008+0000", 2024-09-18T04:57:56.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2310000000000001e-06 2024-09-18T04:57:56.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655013+0000", 2024-09-18T04:57:56.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8439999999999997e-06 2024-09-18T04:57:56.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696353+0000", 2024-09-18T04:57:56.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041340322999999998 2024-09-18T04:57:56.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696455+0000", 2024-09-18T04:57:56.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000102105 2024-09-18T04:57:56.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696552+0000", 2024-09-18T04:57:56.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6738999999999997e-05 2024-09-18T04:57:56.308 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.308 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.308 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.308 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160726 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:56.308 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:57:56.308 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.590054244000001, 2024-09-18T04:57:56.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041786440000000001, 2024-09-18T04:57:56.309 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.309 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.309 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.309 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.309 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.309 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160726 2024-09-18T04:57:56.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.309 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:57:56.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:57:56.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:57:56.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655019+0000", 2024-09-18T04:57:56.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1100000000000001e-06 2024-09-18T04:57:56.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655025+0000", 2024-09-18T04:57:56.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2190000000000001e-06 2024-09-18T04:57:56.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655030+0000", 2024-09-18T04:57:56.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5340000000000001e-06 2024-09-18T04:57:56.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696607+0000", 2024-09-18T04:57:56.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041577309 2024-09-18T04:57:56.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696694+0000", 2024-09-18T04:57:56.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7230000000000003e-05 2024-09-18T04:57:56.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696804+0000", 2024-09-18T04:57:56.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0001094 2024-09-18T04:57:56.313 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.313 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.313 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.313 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160727 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 931840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:56.313 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:57:56.313 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.590037507000002, 2024-09-18T04:57:56.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042000527000000003, 2024-09-18T04:57:56.313 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.313 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.313 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.313 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.314 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.314 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160727 2024-09-18T04:57:56.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.314 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:57:56.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:57:56.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655033+0000", 2024-09-18T04:57:56.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655036+0000", 2024-09-18T04:57:56.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3489999999999999e-06 2024-09-18T04:57:56.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655042+0000", 2024-09-18T04:57:56.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1399999999999997e-06 2024-09-18T04:57:56.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655046+0000", 2024-09-18T04:57:56.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5480000000000003e-06 2024-09-18T04:57:56.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696843+0000", 2024-09-18T04:57:56.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041797103000000002 2024-09-18T04:57:56.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696931+0000", 2024-09-18T04:57:56.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7526000000000002e-05 2024-09-18T04:57:56.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697034+0000", 2024-09-18T04:57:56.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010336999999999999 2024-09-18T04:57:56.317 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.317 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.317 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.317 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160728 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 952320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:56.318 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:57:56.318 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.59002095, 2024-09-18T04:57:56.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042191293999999997, 2024-09-18T04:57:56.318 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.318 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.318 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.318 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.318 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.318 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160728 2024-09-18T04:57:56.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.318 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:57:56.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:57:56.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:57:56.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655052+0000", 2024-09-18T04:57:56.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.232e-06 2024-09-18T04:57:56.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655058+0000", 2024-09-18T04:57:56.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9869999999999996e-06 2024-09-18T04:57:56.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655063+0000", 2024-09-18T04:57:56.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8420000000000004e-06 2024-09-18T04:57:56.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697074+0000", 2024-09-18T04:57:56.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042010446 2024-09-18T04:57:56.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697157+0000", 2024-09-18T04:57:56.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3038e-05 2024-09-18T04:57:56.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697242+0000", 2024-09-18T04:57:56.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5115000000000001e-05 2024-09-18T04:57:56.322 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.322 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.322 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.323 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160729 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 972800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:56.323 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:57:56.323 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.590004190999998, 2024-09-18T04:57:56.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042367834, 2024-09-18T04:57:56.323 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.323 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.323 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.323 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.323 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.323 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160729 2024-09-18T04:57:56.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.323 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:57:56.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:57:56.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:57:56.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655069+0000", 2024-09-18T04:57:56.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2220000000000001e-06 2024-09-18T04:57:56.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655075+0000", 2024-09-18T04:57:56.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2129999999999996e-06 2024-09-18T04:57:56.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655080+0000", 2024-09-18T04:57:56.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6840000000000004e-06 2024-09-18T04:57:56.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697280+0000", 2024-09-18T04:57:56.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042199819999999999 2024-09-18T04:57:56.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697368+0000", 2024-09-18T04:57:56.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7863999999999998e-05 2024-09-18T04:57:56.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697435+0000", 2024-09-18T04:57:56.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7416000000000006e-05 2024-09-18T04:57:56.327 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.327 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.327 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.328 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160730 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 993280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:56.328 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:57:56.328 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.589987426, 2024-09-18T04:57:56.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042559582999999998, 2024-09-18T04:57:56.328 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.328 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.328 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.328 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.328 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.328 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160730 2024-09-18T04:57:56.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.328 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:57:56.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:57:56.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:57:56.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655086+0000", 2024-09-18T04:57:56.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1500000000000002e-06 2024-09-18T04:57:56.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655092+0000", 2024-09-18T04:57:56.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.896e-06 2024-09-18T04:57:56.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655096+0000", 2024-09-18T04:57:56.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7190000000000001e-06 2024-09-18T04:57:56.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697475+0000", 2024-09-18T04:57:56.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042378644 2024-09-18T04:57:56.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697553+0000", 2024-09-18T04:57:56.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7626999999999994e-05 2024-09-18T04:57:56.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697643+0000", 2024-09-18T04:57:56.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0921000000000006e-05 2024-09-18T04:57:56.332 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.332 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.332 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.332 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160731 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 1013760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:56.332 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:57:56.333 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.589954696, 2024-09-18T04:57:56.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042727764000000001, 2024-09-18T04:57:56.333 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.333 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.333 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.333 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.333 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.333 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160731 2024-09-18T04:57:56.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.333 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:57:56.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:57:56.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655116+0000", 2024-09-18T04:57:56.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655119+0000", 2024-09-18T04:57:56.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1240000000000001e-06 2024-09-18T04:57:56.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655131+0000", 2024-09-18T04:57:56.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2145e-05 2024-09-18T04:57:56.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655138+0000", 2024-09-18T04:57:56.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3369999999999996e-06 2024-09-18T04:57:56.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697688+0000", 2024-09-18T04:57:56.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042550604999999998 2024-09-18T04:57:56.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697781+0000", 2024-09-18T04:57:56.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2867000000000004e-05 2024-09-18T04:57:56.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697844+0000", 2024-09-18T04:57:56.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3236000000000002e-05 2024-09-18T04:57:56.337 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.337 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.337 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.337 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160732 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 1034240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:57:56.337 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:57:56.337 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 19.589928848, 2024-09-18T04:57:56.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042857977999999998, 2024-09-18T04:57:56.337 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.337 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.337 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.338 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.338 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.338 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160732 2024-09-18T04:57:56.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.338 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:57:56.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:57:56.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:57:56.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655145+0000", 2024-09-18T04:57:56.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7329999999999999e-06 2024-09-18T04:57:56.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655152+0000", 2024-09-18T04:57:56.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8249999999999999e-06 2024-09-18T04:57:56.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655158+0000", 2024-09-18T04:57:56.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5010000000000001e-06 2024-09-18T04:57:56.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697873+0000", 2024-09-18T04:57:56.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042714457999999997 2024-09-18T04:57:56.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697944+0000", 2024-09-18T04:57:56.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1486e-05 2024-09-18T04:57:56.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.698000+0000", 2024-09-18T04:57:56.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6332000000000002e-05 2024-09-18T04:57:56.341 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.341 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.341 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.341 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.341 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.341 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:56.343 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:57:56.343 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:56.340+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:56.444 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:56.442+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:56.491 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:56.491 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:56.491 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:57:56.491 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:56.500 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:57:56.545 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:56.542+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:56.606 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:56.607 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:57:56.607 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:57:56.607 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:57:56.607 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:56.615 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:57:56.646 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:56.644+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:56.746 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:56.743+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:56.763 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:57:56.763 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:57:56.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:57:56.764 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:57:56.764 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.764 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11154 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:56.764 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:57:56.764 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.871220817999998, 2024-09-18T04:57:56.764 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040591919999999997, 2024-09-18T04:57:56.764 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.764 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.764 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.764 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.764 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.764 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11154 2024-09-18T04:57:56.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.765 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:57:56.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.765 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:57:56.765 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.765 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.765 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.765 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889400+0000", 2024-09-18T04:57:56.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.766 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889403+0000", 2024-09-18T04:57:56.766 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3999999999999999e-06 2024-09-18T04:57:56.766 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.766 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.766 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889409+0000", 2024-09-18T04:57:56.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3380000000000001e-06 2024-09-18T04:57:56.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889414+0000", 2024-09-18T04:57:56.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5789999999999997e-06 2024-09-18T04:57:56.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.767 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.767 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929811+0000", 2024-09-18T04:57:56.767 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040397486000000003 2024-09-18T04:57:56.767 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.767 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929898+0000", 2024-09-18T04:57:56.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6370000000000001e-05 2024-09-18T04:57:56.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.768 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.768 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929993+0000", 2024-09-18T04:57:56.768 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5136000000000005e-05 2024-09-18T04:57:56.768 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.768 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.768 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.768 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.768 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.768 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11155 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2015232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:56.769 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:57:56.769 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.871203640000004, 2024-09-18T04:57:56.769 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040769156000000001, 2024-09-18T04:57:56.769 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.769 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.769 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.769 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.769 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.769 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11155 2024-09-18T04:57:56.769 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.769 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.769 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:57:56.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:57:56.770 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.770 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.770 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.770 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.770 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:57:56.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889420+0000", 2024-09-18T04:57:56.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.181e-06 2024-09-18T04:57:56.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.771 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889426+0000", 2024-09-18T04:57:56.771 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8599999999999998e-06 2024-09-18T04:57:56.771 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.771 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.771 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889430+0000", 2024-09-18T04:57:56.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4399999999999998e-06 2024-09-18T04:57:56.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930020+0000", 2024-09-18T04:57:56.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040589698 2024-09-18T04:57:56.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.772 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.772 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930105+0000", 2024-09-18T04:57:56.772 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4812e-05 2024-09-18T04:57:56.772 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.772 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.773 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.773 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930187+0000", 2024-09-18T04:57:56.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2540999999999995e-05 2024-09-18T04:57:56.773 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.773 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.773 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.773 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.773 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11156 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2064384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:56.773 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:57:56.773 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.871163346000003, 2024-09-18T04:57:56.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040934419, 2024-09-18T04:57:56.774 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.774 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.774 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.774 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.774 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.774 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11156 2024-09-18T04:57:56.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.774 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:57:56.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:57:56.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889457+0000", 2024-09-18T04:57:56.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:57:56.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889462+0000", 2024-09-18T04:57:56.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8350000000000003e-06 2024-09-18T04:57:56.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889481+0000", 2024-09-18T04:57:56.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8845999999999999e-05 2024-09-18T04:57:56.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889495+0000", 2024-09-18T04:57:56.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4229e-05 2024-09-18T04:57:56.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930216+0000", 2024-09-18T04:57:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040721081999999999 2024-09-18T04:57:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930299+0000", 2024-09-18T04:57:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2995000000000001e-05 2024-09-18T04:57:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930393+0000", 2024-09-18T04:57:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3185999999999999e-05 2024-09-18T04:57:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.777 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11404 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1609728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:57:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.864359183999994, 2024-09-18T04:57:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040669012999999997, 2024-09-18T04:57:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11404 2024-09-18T04:57:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.778 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:57:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:57:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:57:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:57:56.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896263+0000", 2024-09-18T04:57:56.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2729999999999999e-06 2024-09-18T04:57:56.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896266+0000", 2024-09-18T04:57:56.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8200000000000001e-06 2024-09-18T04:57:56.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896269+0000", 2024-09-18T04:57:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3199999999999998e-06 2024-09-18T04:57:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936538+0000", 2024-09-18T04:57:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040269394999999999 2024-09-18T04:57:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936657+0000", 2024-09-18T04:57:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119498 2024-09-18T04:57:56.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936931+0000", 2024-09-18T04:57:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027391900000000002 2024-09-18T04:57:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11405 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1658880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:57:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.864350926, 2024-09-18T04:57:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040891034999999999, 2024-09-18T04:57:56.782 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11405 2024-09-18T04:57:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:57:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:57:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896270+0000", 2024-09-18T04:57:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:57:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896272+0000", 2024-09-18T04:57:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.124e-06 2024-09-18T04:57:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896274+0000", 2024-09-18T04:57:56.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8660000000000002e-06 2024-09-18T04:57:56.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896285+0000", 2024-09-18T04:57:56.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0101e-05 2024-09-18T04:57:56.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936986+0000", 2024-09-18T04:57:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040701213999999999 2024-09-18T04:57:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937070+0000", 2024-09-18T04:57:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4211999999999999e-05 2024-09-18T04:57:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937162+0000", 2024-09-18T04:57:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1728e-05 2024-09-18T04:57:56.786 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11406 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1708032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:57:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.864334756999995, 2024-09-18T04:57:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041085251000000003, 2024-09-18T04:57:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11406 2024-09-18T04:57:56.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:57:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:57:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:57:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:57:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896288+0000", 2024-09-18T04:57:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2440000000000001e-06 2024-09-18T04:57:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896291+0000", 2024-09-18T04:57:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8289999999999999e-06 2024-09-18T04:57:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896293+0000", 2024-09-18T04:57:56.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5399999999999998e-06 2024-09-18T04:57:56.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937201+0000", 2024-09-18T04:57:56.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040907418000000001 2024-09-18T04:57:56.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937283+0000", 2024-09-18T04:57:56.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2460999999999995e-05 2024-09-18T04:57:56.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937372+0000", 2024-09-18T04:57:56.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8987999999999996e-05 2024-09-18T04:57:56.791 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.791 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.791 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.791 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11407 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:56.791 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:57:56.791 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.864324937000006, 2024-09-18T04:57:56.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041438580000000003, 2024-09-18T04:57:56.792 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.792 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.792 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.792 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.792 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.792 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11407 2024-09-18T04:57:56.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.792 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:57:56.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:57:56.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896296+0000", 2024-09-18T04:57:56.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896298+0000", 2024-09-18T04:57:56.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:57:56.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896304+0000", 2024-09-18T04:57:56.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6570000000000002e-06 2024-09-18T04:57:56.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896308+0000", 2024-09-18T04:57:56.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0940000000000001e-06 2024-09-18T04:57:56.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937540+0000", 2024-09-18T04:57:56.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041231469 2024-09-18T04:57:56.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937625+0000", 2024-09-18T04:57:56.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5767999999999996e-05 2024-09-18T04:57:56.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937735+0000", 2024-09-18T04:57:56.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010974 2024-09-18T04:57:56.795 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.795 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.795 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.796 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11408 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:56.796 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:57:56.796 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.864309297999995, 2024-09-18T04:57:56.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041653055000000001, 2024-09-18T04:57:56.796 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.796 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.796 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.796 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.797 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.797 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11408 2024-09-18T04:57:56.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.797 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:57:56.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:57:56.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:57:56.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896314+0000", 2024-09-18T04:57:56.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0140000000000001e-06 2024-09-18T04:57:56.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896320+0000", 2024-09-18T04:57:56.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-18T04:57:56.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896324+0000", 2024-09-18T04:57:56.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4410000000000003e-06 2024-09-18T04:57:56.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937769+0000", 2024-09-18T04:57:56.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041445243 2024-09-18T04:57:56.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937861+0000", 2024-09-18T04:57:56.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1471000000000006e-05 2024-09-18T04:57:56.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937965+0000", 2024-09-18T04:57:56.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104516 2024-09-18T04:57:56.801 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.801 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.801 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.802 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11409 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:56.802 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:57:56.802 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.864293242000002, 2024-09-18T04:57:56.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041840467999999999, 2024-09-18T04:57:56.802 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.802 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.802 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.802 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.802 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.802 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11409 2024-09-18T04:57:56.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.802 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:57:56.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:57:56.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:57:56.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896331+0000", 2024-09-18T04:57:56.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5950000000000001e-06 2024-09-18T04:57:56.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896336+0000", 2024-09-18T04:57:56.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7259999999999999e-06 2024-09-18T04:57:56.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896341+0000", 2024-09-18T04:57:56.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5619999999999997e-06 2024-09-18T04:57:56.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937995+0000", 2024-09-18T04:57:56.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041654060999999999 2024-09-18T04:57:56.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938078+0000", 2024-09-18T04:57:56.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2869999999999998e-05 2024-09-18T04:57:56.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938169+0000", 2024-09-18T04:57:56.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1040999999999998e-05 2024-09-18T04:57:56.806 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.806 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.807 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.807 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11410 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1921024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:56.807 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:57:56.807 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.864276759000006, 2024-09-18T04:57:56.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042060342000000001, 2024-09-18T04:57:56.807 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.807 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.807 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.807 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.807 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.807 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11410 2024-09-18T04:57:56.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.808 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:57:56.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:57:56.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896344+0000", 2024-09-18T04:57:56.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896360+0000", 2024-09-18T04:57:56.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5068e-05 2024-09-18T04:57:56.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896366+0000", 2024-09-18T04:57:56.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5119999999999997e-06 2024-09-18T04:57:56.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896371+0000", 2024-09-18T04:57:56.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:57:56.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938198+0000", 2024-09-18T04:57:56.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041826579000000003 2024-09-18T04:57:56.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938297+0000", 2024-09-18T04:57:56.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9067000000000005e-05 2024-09-18T04:57:56.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938405+0000", 2024-09-18T04:57:56.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108492 2024-09-18T04:57:56.812 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.812 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.812 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.813 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11411 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1970176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:56.813 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:57:56.813 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.864245917000005, 2024-09-18T04:57:56.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042259191000000002, 2024-09-18T04:57:56.813 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.813 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.813 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.813 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.813 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.813 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11411 2024-09-18T04:57:56.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.813 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:57:56.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:57:56.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896375+0000", 2024-09-18T04:57:56.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896377+0000", 2024-09-18T04:57:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2699999999999999e-06 2024-09-18T04:57:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896383+0000", 2024-09-18T04:57:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6740000000000002e-06 2024-09-18T04:57:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896388+0000", 2024-09-18T04:57:56.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:57:56.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938435+0000", 2024-09-18T04:57:56.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042047169000000002 2024-09-18T04:57:56.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938534+0000", 2024-09-18T04:57:56.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8939999999999998e-05 2024-09-18T04:57:56.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938635+0000", 2024-09-18T04:57:56.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101208 2024-09-18T04:57:56.817 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.817 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.817 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.817 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11412 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:56.817 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:57:56.817 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.864229934999997, 2024-09-18T04:57:56.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042828079999999998, 2024-09-18T04:57:56.817 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.817 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.817 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11412 2024-09-18T04:57:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:57:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:57:56.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896391+0000", 2024-09-18T04:57:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896393+0000", 2024-09-18T04:57:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0810000000000001e-06 2024-09-18T04:57:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896399+0000", 2024-09-18T04:57:56.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7230000000000001e-06 2024-09-18T04:57:56.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896403+0000", 2024-09-18T04:57:56.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3470000000000001e-06 2024-09-18T04:57:56.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939043+0000", 2024-09-18T04:57:56.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042639829999999997 2024-09-18T04:57:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939124+0000", 2024-09-18T04:57:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0809000000000005e-05 2024-09-18T04:57:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939220+0000", 2024-09-18T04:57:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5659e-05 2024-09-18T04:57:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11413 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2068480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:57:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:57:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.864214239999995, 2024-09-18T04:57:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043042108000000003, 2024-09-18T04:57:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11413 2024-09-18T04:57:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:57:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:57:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:57:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896409+0000", 2024-09-18T04:57:56.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.165e-06 2024-09-18T04:57:56.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896415+0000", 2024-09-18T04:57:56.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6149999999999996e-06 2024-09-18T04:57:56.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896419+0000", 2024-09-18T04:57:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2649999999999998e-06 2024-09-18T04:57:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939262+0000", 2024-09-18T04:57:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042843444000000001 2024-09-18T04:57:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939351+0000", 2024-09-18T04:57:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8944999999999996e-05 2024-09-18T04:57:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939449+0000", 2024-09-18T04:57:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8029000000000006e-05 2024-09-18T04:57:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145409 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:57:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.232794592000001, 2024-09-18T04:57:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040684845999999997, 2024-09-18T04:57:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.826 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.827 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.827 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.827 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145409 2024-09-18T04:57:56.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.827 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:57:56.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:57:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:57:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527829+0000", 2024-09-18T04:57:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.441e-06 2024-09-18T04:57:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527835+0000", 2024-09-18T04:57:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1650000000000003e-06 2024-09-18T04:57:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527840+0000", 2024-09-18T04:57:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8250000000000004e-06 2024-09-18T04:57:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568404+0000", 2024-09-18T04:57:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040563872000000001 2024-09-18T04:57:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568457+0000", 2024-09-18T04:57:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2713999999999997e-05 2024-09-18T04:57:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568512+0000", 2024-09-18T04:57:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5148000000000001e-05 2024-09-18T04:57:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.830 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145411 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:57:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.232776731000001, 2024-09-18T04:57:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040792108000000001, 2024-09-18T04:57:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145411 2024-09-18T04:57:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:57:56.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:57:56.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527844+0000", 2024-09-18T04:57:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527847+0000", 2024-09-18T04:57:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0260000000000002e-06 2024-09-18T04:57:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527852+0000", 2024-09-18T04:57:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9889999999999997e-06 2024-09-18T04:57:56.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527857+0000", 2024-09-18T04:57:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6410000000000001e-06 2024-09-18T04:57:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568537+0000", 2024-09-18T04:57:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040679743999999997 2024-09-18T04:57:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568583+0000", 2024-09-18T04:57:56.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6286999999999999e-05 2024-09-18T04:57:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568637+0000", 2024-09-18T04:57:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.376e-05 2024-09-18T04:57:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145413 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1859584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:57:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.232760351, 2024-09-18T04:57:56.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040919516000000003, 2024-09-18T04:57:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145413 2024-09-18T04:57:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:57:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:57:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:57:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527863+0000", 2024-09-18T04:57:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2630000000000002e-06 2024-09-18T04:57:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527869+0000", 2024-09-18T04:57:56.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9009999999999999e-06 2024-09-18T04:57:56.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527874+0000", 2024-09-18T04:57:56.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0590000000000002e-06 2024-09-18T04:57:56.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568663+0000", 2024-09-18T04:57:56.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040788805999999997 2024-09-18T04:57:56.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568716+0000", 2024-09-18T04:57:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3501999999999997e-05 2024-09-18T04:57:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568781+0000", 2024-09-18T04:57:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4338000000000006e-05 2024-09-18T04:57:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145414 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1908736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:57:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.232743393, 2024-09-18T04:57:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041030234999999998, 2024-09-18T04:57:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145414 2024-09-18T04:57:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:57:56.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:57:56.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:57:56.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527880+0000", 2024-09-18T04:57:56.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4770000000000002e-06 2024-09-18T04:57:56.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527886+0000", 2024-09-18T04:57:56.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5759999999999996e-06 2024-09-18T04:57:56.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527890+0000", 2024-09-18T04:57:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8180000000000003e-06 2024-09-18T04:57:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568807+0000", 2024-09-18T04:57:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040916984000000003 2024-09-18T04:57:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568854+0000", 2024-09-18T04:57:56.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6740999999999999e-05 2024-09-18T04:57:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568908+0000", 2024-09-18T04:57:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4228000000000003e-05 2024-09-18T04:57:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145416 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1957888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:57:56.844 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.232726900999999, 2024-09-18T04:57:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041147364999999998, 2024-09-18T04:57:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145416 2024-09-18T04:57:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:57:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:57:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527894+0000", 2024-09-18T04:57:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527896+0000", 2024-09-18T04:57:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1789999999999998e-06 2024-09-18T04:57:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527908+0000", 2024-09-18T04:57:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1780000000000001e-05 2024-09-18T04:57:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527914+0000", 2024-09-18T04:57:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2029999999999997e-06 2024-09-18T04:57:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568935+0000", 2024-09-18T04:57:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041020273000000003 2024-09-18T04:57:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568988+0000", 2024-09-18T04:57:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2911000000000002e-05 2024-09-18T04:57:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569042+0000", 2024-09-18T04:57:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4382000000000002e-05 2024-09-18T04:57:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.848 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145418 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 2007040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:57:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.232702503999999, 2024-09-18T04:57:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041247862000000003, 2024-09-18T04:57:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.849 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145418 2024-09-18T04:57:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:57:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:57:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:57:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527921+0000", 2024-09-18T04:57:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.667e-06 2024-09-18T04:57:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527929+0000", 2024-09-18T04:57:56.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.542e-06 2024-09-18T04:57:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527934+0000", 2024-09-18T04:57:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1440000000000002e-06 2024-09-18T04:57:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569068+0000", 2024-09-18T04:57:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041134190000000001 2024-09-18T04:57:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569114+0000", 2024-09-18T04:57:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6261999999999999e-05 2024-09-18T04:57:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569167+0000", 2024-09-18T04:57:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2506999999999997e-05 2024-09-18T04:57:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145420 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 2056192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:57:56.853 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:57:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 23.232683430000002, 2024-09-18T04:57:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041351472, 2024-09-18T04:57:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:57:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:57:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:57:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:57:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145420 2024-09-18T04:57:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:57:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:57:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:57:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:57:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:57:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:57:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:57:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:57:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:57:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527940+0000", 2024-09-18T04:57:56.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6570000000000001e-06 2024-09-18T04:57:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:57:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527948+0000", 2024-09-18T04:57:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8350000000000004e-06 2024-09-18T04:57:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:57:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527953+0000", 2024-09-18T04:57:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0690000000000001e-06 2024-09-18T04:57:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:57:56.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569182+0000", 2024-09-18T04:57:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041228734000000003 2024-09-18T04:57:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:57:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569234+0000", 2024-09-18T04:57:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1663999999999999e-05 2024-09-18T04:57:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:57:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:57:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:57:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569290+0000", 2024-09-18T04:57:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5937000000000003e-05 2024-09-18T04:57:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.857 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.858 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:57:56.858 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:57:56.858 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:57:56.859 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:57:56.860 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:56.856+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:56.960 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:56.958+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:56.972 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:56.972 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:56.972 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:57:56.973 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:56.982 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:57:57.061 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:57.059+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:57.064 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 5, 6, 7] out_osds: [1, 4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T04:57:57.064 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T04:57:57.064 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-18T04:57:57.064 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-09-18T04:57:57.137 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:57.137 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:57.137 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:57:57.138 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:57:57.138 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:57.147 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:57:57.162 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:57.159+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:57.219 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-09-18T04:57:57.253 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:57.253 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:57.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:57.253 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:57.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.253 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72135 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 394240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:57.254 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:57:57.254 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.295914109000002, 2024-09-18T04:57:57.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048559739999999997, 2024-09-18T04:57:57.254 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.254 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.254 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.254 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.254 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.254 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72135 2024-09-18T04:57:57.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.254 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:57:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:57:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:57:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954005+0000", 2024-09-18T04:57:57.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5629999999999999e-06 2024-09-18T04:57:57.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954008+0000", 2024-09-18T04:57:57.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-18T04:57:57.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954012+0000", 2024-09-18T04:57:57.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4790000000000001e-06 2024-09-18T04:57:57.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002418+0000", 2024-09-18T04:57:57.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048406049 2024-09-18T04:57:57.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002460+0000", 2024-09-18T04:57:57.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2188999999999999e-05 2024-09-18T04:57:57.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002564+0000", 2024-09-18T04:57:57.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010377 2024-09-18T04:57:57.258 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.258 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.258 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.258 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72136 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 409600~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:57.258 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:57:57.258 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.295809161999998, 2024-09-18T04:57:57.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048670191000000002, 2024-09-18T04:57:57.258 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.258 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.258 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.258 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.259 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.259 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72136 2024-09-18T04:57:57.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.259 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:57:57.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:57:57.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954019+0000", 2024-09-18T04:57:57.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999099 2024-09-18T04:57:57.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954112+0000", 2024-09-18T04:57:57.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3273e-05 2024-09-18T04:57:57.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954121+0000", 2024-09-18T04:57:57.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0289999999999994e-06 2024-09-18T04:57:57.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954128+0000", 2024-09-18T04:57:57.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6189999999999996e-06 2024-09-18T04:57:57.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002604+0000", 2024-09-18T04:57:57.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048476048000000001 2024-09-18T04:57:57.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002682+0000", 2024-09-18T04:57:57.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7955999999999997e-05 2024-09-18T04:57:57.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.262 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.262 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002779+0000", 2024-09-18T04:57:57.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7372000000000002e-05 2024-09-18T04:57:57.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.262 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.262 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72138 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 440320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:57.262 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:57:57.263 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.295375309999997, 2024-09-18T04:57:57.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048549430999999997, 2024-09-18T04:57:57.263 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.263 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.263 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72138 2024-09-18T04:57:57.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.263 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:57:57.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:57:57.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954542+0000", 2024-09-18T04:57:57.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:57.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954545+0000", 2024-09-18T04:57:57.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4360000000000001e-06 2024-09-18T04:57:57.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954551+0000", 2024-09-18T04:57:57.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4949999999999996e-06 2024-09-18T04:57:57.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954557+0000", 2024-09-18T04:57:57.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5099999999999998e-06 2024-09-18T04:57:57.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002993+0000", 2024-09-18T04:57:57.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048436358999999998 2024-09-18T04:57:57.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003039+0000", 2024-09-18T04:57:57.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5884000000000003e-05 2024-09-18T04:57:57.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003092+0000", 2024-09-18T04:57:57.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3208000000000003e-05 2024-09-18T04:57:57.267 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.267 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.267 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.267 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72139 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 455680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:57.267 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:57:57.267 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.295356501999997, 2024-09-18T04:57:57.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048718773, 2024-09-18T04:57:57.267 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.267 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.268 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72139 2024-09-18T04:57:57.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.268 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:57:57.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:57:57.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954561+0000", 2024-09-18T04:57:57.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:57.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954564+0000", 2024-09-18T04:57:57.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4099999999999998e-06 2024-09-18T04:57:57.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954570+0000", 2024-09-18T04:57:57.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.178e-06 2024-09-18T04:57:57.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954575+0000", 2024-09-18T04:57:57.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0560000000000004e-06 2024-09-18T04:57:57.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003114+0000", 2024-09-18T04:57:57.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048539212999999998 2024-09-18T04:57:57.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003176+0000", 2024-09-18T04:57:57.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2154000000000002e-05 2024-09-18T04:57:57.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003280+0000", 2024-09-18T04:57:57.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010421600000000001 2024-09-18T04:57:57.271 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.271 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.271 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72140 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 471040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:57:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.295100560999998, 2024-09-18T04:57:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048670819999999997, 2024-09-18T04:57:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72140 2024-09-18T04:57:57.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:57:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:57:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954579+0000", 2024-09-18T04:57:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997611 2024-09-18T04:57:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954821+0000", 2024-09-18T04:57:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024241000000000001 2024-09-18T04:57:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954831+0000", 2024-09-18T04:57:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8770000000000005e-06 2024-09-18T04:57:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954838+0000", 2024-09-18T04:57:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0690000000000004e-06 2024-09-18T04:57:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003317+0000", 2024-09-18T04:57:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048478796999999997 2024-09-18T04:57:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003392+0000", 2024-09-18T04:57:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4411000000000003e-05 2024-09-18T04:57:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003488+0000", 2024-09-18T04:57:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6927000000000002e-05 2024-09-18T04:57:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72141 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 486400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:57:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.295073078000001, 2024-09-18T04:57:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048808114, 2024-09-18T04:57:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.276 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72141 2024-09-18T04:57:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:57:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:57:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:57:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954847+0000", 2024-09-18T04:57:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3800000000000001e-06 2024-09-18T04:57:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954854+0000", 2024-09-18T04:57:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6080000000000001e-06 2024-09-18T04:57:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954859+0000", 2024-09-18T04:57:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:57:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003526+0000", 2024-09-18T04:57:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048666433000000002 2024-09-18T04:57:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003599+0000", 2024-09-18T04:57:57.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2854000000000004e-05 2024-09-18T04:57:57.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003653+0000", 2024-09-18T04:57:57.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4511e-05 2024-09-18T04:57:57.280 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.280 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.280 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.280 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72142 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 501760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:57:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.295034276000003, 2024-09-18T04:57:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048885500999999998, 2024-09-18T04:57:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72142 2024-09-18T04:57:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:57:57.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:57:57.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954864+0000", 2024-09-18T04:57:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999804 2024-09-18T04:57:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954886+0000", 2024-09-18T04:57:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2042999999999999e-05 2024-09-18T04:57:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954893+0000", 2024-09-18T04:57:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.255e-06 2024-09-18T04:57:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954899+0000", 2024-09-18T04:57:57.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8780000000000003e-06 2024-09-18T04:57:57.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003678+0000", 2024-09-18T04:57:57.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048778834 2024-09-18T04:57:57.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003717+0000", 2024-09-18T04:57:57.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8860999999999999e-05 2024-09-18T04:57:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003769+0000", 2024-09-18T04:57:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2389000000000003e-05 2024-09-18T04:57:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72143 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 517120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:57:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.294908649999996, 2024-09-18T04:57:57.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048951738000000002, 2024-09-18T04:57:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72143 2024-09-18T04:57:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:57:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:57:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954904+0000", 2024-09-18T04:57:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998946 2024-09-18T04:57:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955013+0000", 2024-09-18T04:57:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108977 2024-09-18T04:57:57.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955021+0000", 2024-09-18T04:57:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7570000000000004e-06 2024-09-18T04:57:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955025+0000", 2024-09-18T04:57:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1219999999999997e-06 2024-09-18T04:57:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003792+0000", 2024-09-18T04:57:57.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048767379 2024-09-18T04:57:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003860+0000", 2024-09-18T04:57:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8033000000000004e-05 2024-09-18T04:57:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003961+0000", 2024-09-18T04:57:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010103300000000001 2024-09-18T04:57:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.290 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72161 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 13312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:57.290 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:57:57.290 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.294645013, 2024-09-18T04:57:57.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048891539999999997, 2024-09-18T04:57:57.290 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.290 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.290 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.290 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.290 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.291 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72161 2024-09-18T04:57:57.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.291 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:57:57.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:57:57.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955028+0000", 2024-09-18T04:57:57.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997549 2024-09-18T04:57:57.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955278+0000", 2024-09-18T04:57:57.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024973299999999998 2024-09-18T04:57:57.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955289+0000", 2024-09-18T04:57:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0812000000000001e-05 2024-09-18T04:57:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955296+0000", 2024-09-18T04:57:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2570000000000001e-06 2024-09-18T04:57:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004002+0000", 2024-09-18T04:57:57.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048705904000000001 2024-09-18T04:57:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004076+0000", 2024-09-18T04:57:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4086999999999997e-05 2024-09-18T04:57:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004165+0000", 2024-09-18T04:57:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8712e-05 2024-09-18T04:57:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72163 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 44032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:57:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.294019575, 2024-09-18T04:57:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048567011, 2024-09-18T04:57:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72163 2024-09-18T04:57:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:57:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:57:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955898+0000", 2024-09-18T04:57:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:57.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955901+0000", 2024-09-18T04:57:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.706e-06 2024-09-18T04:57:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955907+0000", 2024-09-18T04:57:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3559999999999998e-06 2024-09-18T04:57:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955912+0000", 2024-09-18T04:57:57.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9640000000000002e-06 2024-09-18T04:57:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004377+0000", 2024-09-18T04:57:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048465293999999999 2024-09-18T04:57:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004418+0000", 2024-09-18T04:57:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0163000000000002e-05 2024-09-18T04:57:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004466+0000", 2024-09-18T04:57:57.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8004000000000003e-05 2024-09-18T04:57:57.299 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.299 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.299 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.299 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72164 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:57.299 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:57:57.299 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.294001301999998, 2024-09-18T04:57:57.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048707253999999998, 2024-09-18T04:57:57.299 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.299 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.299 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.299 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72164 2024-09-18T04:57:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:57:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:57:57.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:57:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955919+0000", 2024-09-18T04:57:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2520000000000002e-06 2024-09-18T04:57:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955925+0000", 2024-09-18T04:57:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3740000000000003e-06 2024-09-18T04:57:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955930+0000", 2024-09-18T04:57:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:57:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004489+0000", 2024-09-18T04:57:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048558581000000003 2024-09-18T04:57:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004529+0000", 2024-09-18T04:57:57.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9722999999999999e-05 2024-09-18T04:57:57.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004624+0000", 2024-09-18T04:57:57.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5402999999999994e-05 2024-09-18T04:57:57.303 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.303 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.303 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.303 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72165 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 74752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:57.303 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:57:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.293726442999997, 2024-09-18T04:57:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048633223000000003, 2024-09-18T04:57:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72165 2024-09-18T04:57:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:57:57.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:57:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955935+0000", 2024-09-18T04:57:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999743 2024-09-18T04:57:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956199+0000", 2024-09-18T04:57:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000263906 2024-09-18T04:57:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956207+0000", 2024-09-18T04:57:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2519999999999999e-06 2024-09-18T04:57:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956214+0000", 2024-09-18T04:57:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7610000000000002e-06 2024-09-18T04:57:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004661+0000", 2024-09-18T04:57:57.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048447967000000002 2024-09-18T04:57:57.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004735+0000", 2024-09-18T04:57:57.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3789999999999997e-05 2024-09-18T04:57:57.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004825+0000", 2024-09-18T04:57:57.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9724999999999999e-05 2024-09-18T04:57:57.307 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.307 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72166 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 90112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:57:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.293472909000002, 2024-09-18T04:57:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048595737999999999, 2024-09-18T04:57:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72166 2024-09-18T04:57:57.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:57:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:57:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956221+0000", 2024-09-18T04:57:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997759 2024-09-18T04:57:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956448+0000", 2024-09-18T04:57:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022685500000000001 2024-09-18T04:57:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956456+0000", 2024-09-18T04:57:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0099999999999995e-06 2024-09-18T04:57:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956462+0000", 2024-09-18T04:57:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7200000000000003e-06 2024-09-18T04:57:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004862+0000", 2024-09-18T04:57:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048399795000000002 2024-09-18T04:57:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004941+0000", 2024-09-18T04:57:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9414999999999998e-05 2024-09-18T04:57:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005041+0000", 2024-09-18T04:57:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9942e-05 2024-09-18T04:57:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72167 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:57:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.293385870999998, 2024-09-18T04:57:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048622894999999999, 2024-09-18T04:57:57.312 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72167 2024-09-18T04:57:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:57:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:57:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956468+0000", 2024-09-18T04:57:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999352 2024-09-18T04:57:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956535+0000", 2024-09-18T04:57:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7440999999999993e-05 2024-09-18T04:57:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956544+0000", 2024-09-18T04:57:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1039999999999996e-06 2024-09-18T04:57:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956550+0000", 2024-09-18T04:57:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9850000000000003e-06 2024-09-18T04:57:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005066+0000", 2024-09-18T04:57:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048515614999999998 2024-09-18T04:57:57.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005107+0000", 2024-09-18T04:57:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0843999999999997e-05 2024-09-18T04:57:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005155+0000", 2024-09-18T04:57:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8686000000000001e-05 2024-09-18T04:57:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72168 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:57:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.293224807999998, 2024-09-18T04:57:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048585179999999999, 2024-09-18T04:57:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72168 2024-09-18T04:57:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:57:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:57:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956555+0000", 2024-09-18T04:57:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998612 2024-09-18T04:57:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956696+0000", 2024-09-18T04:57:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000140922 2024-09-18T04:57:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956707+0000", 2024-09-18T04:57:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1555e-05 2024-09-18T04:57:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956714+0000", 2024-09-18T04:57:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2709999999999998e-06 2024-09-18T04:57:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005177+0000", 2024-09-18T04:57:57.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048463421 2024-09-18T04:57:57.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005214+0000", 2024-09-18T04:57:57.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.676e-05 2024-09-18T04:57:57.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005279+0000", 2024-09-18T04:57:57.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4899000000000003e-05 2024-09-18T04:57:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72172 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 441344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.957556+0000", 2024-09-18T04:57:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.292362408000002, 2024-09-18T04:57:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048545056000000003, 2024-09-18T04:57:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.321 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72172 2024-09-18T04:57:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957556+0000", 2024-09-18T04:57:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957556+0000", 2024-09-18T04:57:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957555+0000", 2024-09-18T04:57:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957558+0000", 2024-09-18T04:57:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2900000000000001e-06 2024-09-18T04:57:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957564+0000", 2024-09-18T04:57:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0700000000000003e-06 2024-09-18T04:57:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957569+0000", 2024-09-18T04:57:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7870000000000001e-06 2024-09-18T04:57:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005922+0000", 2024-09-18T04:57:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048353259000000003 2024-09-18T04:57:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005999+0000", 2024-09-18T04:57:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6889000000000002e-05 2024-09-18T04:57:57.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.006101+0000", 2024-09-18T04:57:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102136 2024-09-18T04:57:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72199 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:57:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.290692915999998, 2024-09-18T04:57:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048626057, 2024-09-18T04:57:57.325 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72199 2024-09-18T04:57:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:57:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:57:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959195+0000", 2024-09-18T04:57:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999969 2024-09-18T04:57:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959228+0000", 2024-09-18T04:57:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3609000000000003e-05 2024-09-18T04:57:57.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959236+0000", 2024-09-18T04:57:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.157e-06 2024-09-18T04:57:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959243+0000", 2024-09-18T04:57:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5910000000000001e-06 2024-09-18T04:57:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007684+0000", 2024-09-18T04:57:57.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048440745 2024-09-18T04:57:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007762+0000", 2024-09-18T04:57:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8360000000000002e-05 2024-09-18T04:57:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007851+0000", 2024-09-18T04:57:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9388000000000005e-05 2024-09-18T04:57:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72240 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 428032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:57:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:57:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.286599606000003, 2024-09-18T04:57:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048643767999999997, 2024-09-18T04:57:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72240 2024-09-18T04:57:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:57:57.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:57:57.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963240+0000", 2024-09-18T04:57:57.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999218 2024-09-18T04:57:57.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963323+0000", 2024-09-18T04:57:57.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2375999999999999e-05 2024-09-18T04:57:57.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963332+0000", 2024-09-18T04:57:57.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1700000000000003e-06 2024-09-18T04:57:57.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963338+0000", 2024-09-18T04:57:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2149999999999998e-06 2024-09-18T04:57:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011785+0000", 2024-09-18T04:57:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048446493 2024-09-18T04:57:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011861+0000", 2024-09-18T04:57:57.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6472999999999998e-05 2024-09-18T04:57:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011962+0000", 2024-09-18T04:57:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101167 2024-09-18T04:57:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:117375 3.1 3:80b18d38:::rbd_header.11acae5d5365:head [watch reconnect cookie 140406307615440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T04:57:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:57:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.865525069, 2024-09-18T04:57:57.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.018908156, 2024-09-18T04:57:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117375 2024-09-18T04:57:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:57:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:57:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384391+0000", 2024-09-18T04:57:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:57:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384401+0000", 2024-09-18T04:57:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4299999999999995e-06 2024-09-18T04:57:57.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384418+0000", 2024-09-18T04:57:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6745e-05 2024-09-18T04:57:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384429+0000", 2024-09-18T04:57:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1122e-05 2024-09-18T04:57:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384496+0000", 2024-09-18T04:57:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6841000000000006e-05 2024-09-18T04:57:57.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-18T04:57:57.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384505+0000", 2024-09-18T04:57:57.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4080000000000004e-06 2024-09-18T04:57:57.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.388971+0000", 2024-09-18T04:57:57.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0044660000000000004 2024-09-18T04:57:57.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:57:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.388981+0000", 2024-09-18T04:57:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0118e-05 2024-09-18T04:57:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.399433+0000", 2024-09-18T04:57:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.010452055 2024-09-18T04:57:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.399594+0000", 2024-09-18T04:57:57.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000160669 2024-09-18T04:57:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.403301+0000", 2024-09-18T04:57:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037074370000000001 2024-09-18T04:57:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:117376 3.1 3:80b18d38:::rbd_header.11acae5d5365:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T04:57:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:57:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 33.863473419000002, 2024-09-18T04:57:57.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0196134539999999, 2024-09-18T04:57:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:57:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117376 2024-09-18T04:57:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:57:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:57:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386443+0000", 2024-09-18T04:57:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:57:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386457+0000", 2024-09-18T04:57:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4207e-05 2024-09-18T04:57:57.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386478+0000", 2024-09-18T04:57:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0349000000000001e-05 2024-09-18T04:57:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386503+0000", 2024-09-18T04:57:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5582e-05 2024-09-18T04:57:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386558+0000", 2024-09-18T04:57:57.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4892999999999998e-05 2024-09-18T04:57:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-18T04:57:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386562+0000", 2024-09-18T04:57:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.313e-06 2024-09-18T04:57:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.389010+0000", 2024-09-18T04:57:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024480890000000001 2024-09-18T04:57:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:57:57.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.389018+0000", 2024-09-18T04:57:57.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8469999999999996e-06 2024-09-18T04:57:57.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400073+0000", 2024-09-18T04:57:57.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.011054286 2024-09-18T04:57:57.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400124+0000", 2024-09-18T04:57:57.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1332999999999998e-05 2024-09-18T04:57:57.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:57:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400477+0000", 2024-09-18T04:57:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035329300000000002 2024-09-18T04:57:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:57:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.401353+0000", 2024-09-18T04:57:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00087584200000000005 2024-09-18T04:57:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:57:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406006+0000", 2024-09-18T04:57:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0046534250000000001 2024-09-18T04:57:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:57:57.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406037+0000", 2024-09-18T04:57:57.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0559000000000003e-05 2024-09-18T04:57:57.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406058+0000", 2024-09-18T04:57:57.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1260999999999999e-05 2024-09-18T04:57:57.347 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.347 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.347 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.347 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.347 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.347 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:57.349 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:57:57.349 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:57.346+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:57.450 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:57.446+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:57.484 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-09-18T04:57:57.493 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:57.493 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:57:57.494 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:57.502 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:57:57.552 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:57.550+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:57.560 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:57:57.604 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:57.604 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:57:57.604 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:57:57.604 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:57:57.604 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:57.612 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:57:57.653 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:57.650+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:57.691 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:57:57.691 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:57:57.708 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:57:57.708 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:57:57.724 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:57:57.725 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:57:57.741 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:57:57.741 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:57:57.754 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:57.751+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:57.756 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:57:57.756 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:57:57.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:57:57.756 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:57:57.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.757 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60344 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1456128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:57.757 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:57.757 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.641198963999997, 2024-09-18T04:57:57.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044026324999999998, 2024-09-18T04:57:57.757 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.757 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.757 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.757 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.757 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.757 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60344 2024-09-18T04:57:57.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.757 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:57.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:57.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:57:57.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:57.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112224+0000", 2024-09-18T04:57:57.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1320000000000001e-06 2024-09-18T04:57:57.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112230+0000", 2024-09-18T04:57:57.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9619999999999998e-06 2024-09-18T04:57:57.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112234+0000", 2024-09-18T04:57:57.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7890000000000002e-06 2024-09-18T04:57:57.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156105+0000", 2024-09-18T04:57:57.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043870531999999997 2024-09-18T04:57:57.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156170+0000", 2024-09-18T04:57:57.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5036000000000005e-05 2024-09-18T04:57:57.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156248+0000", 2024-09-18T04:57:57.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8295999999999997e-05 2024-09-18T04:57:57.761 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.761 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.761 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.761 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60345 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1471488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:57.761 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:57:57.761 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.641182198999999, 2024-09-18T04:57:57.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044180090999999998, 2024-09-18T04:57:57.761 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.762 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.762 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.762 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.762 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.762 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60345 2024-09-18T04:57:57.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.762 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:57:57.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:57:57.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112238+0000", 2024-09-18T04:57:57.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:57.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112240+0000", 2024-09-18T04:57:57.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9910000000000001e-06 2024-09-18T04:57:57.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112246+0000", 2024-09-18T04:57:57.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0859999999999998e-06 2024-09-18T04:57:57.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112251+0000", 2024-09-18T04:57:57.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5020000000000003e-06 2024-09-18T04:57:57.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156274+0000", 2024-09-18T04:57:57.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044023063000000001 2024-09-18T04:57:57.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156337+0000", 2024-09-18T04:57:57.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3077000000000006e-05 2024-09-18T04:57:57.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156419+0000", 2024-09-18T04:57:57.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1699000000000005e-05 2024-09-18T04:57:57.766 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.766 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.766 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.766 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60346 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1486848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:57.766 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:57:57.766 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.641164418000002, 2024-09-18T04:57:57.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044336331999999999, 2024-09-18T04:57:57.766 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.766 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.767 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.767 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.767 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.767 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60346 2024-09-18T04:57:57.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.767 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:57:57.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:57:57.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112256+0000", 2024-09-18T04:57:57.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:57.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112258+0000", 2024-09-18T04:57:57.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1720000000000001e-06 2024-09-18T04:57:57.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112265+0000", 2024-09-18T04:57:57.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4280000000000001e-06 2024-09-18T04:57:57.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112269+0000", 2024-09-18T04:57:57.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.741e-06 2024-09-18T04:57:57.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156445+0000", 2024-09-18T04:57:57.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044175341 2024-09-18T04:57:57.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156512+0000", 2024-09-18T04:57:57.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6723000000000005e-05 2024-09-18T04:57:57.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156593+0000", 2024-09-18T04:57:57.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1284000000000003e-05 2024-09-18T04:57:57.770 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.770 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.770 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.771 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60347 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1502208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:57.771 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:57:57.771 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.641136007, 2024-09-18T04:57:57.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044482560999999997, 2024-09-18T04:57:57.771 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.771 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.771 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.771 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.771 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.771 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60347 2024-09-18T04:57:57.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.771 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:57:57.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:57:57.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112284+0000", 2024-09-18T04:57:57.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:57.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112286+0000", 2024-09-18T04:57:57.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9709999999999998e-06 2024-09-18T04:57:57.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112290+0000", 2024-09-18T04:57:57.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9199999999999997e-06 2024-09-18T04:57:57.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112293+0000", 2024-09-18T04:57:57.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870000000000001e-06 2024-09-18T04:57:57.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156617+0000", 2024-09-18T04:57:57.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044323960000000003 2024-09-18T04:57:57.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156687+0000", 2024-09-18T04:57:57.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0141000000000005e-05 2024-09-18T04:57:57.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156767+0000", 2024-09-18T04:57:57.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0140000000000002e-05 2024-09-18T04:57:57.775 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.775 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.775 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.775 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60348 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1517568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:57.775 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:57.775 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.641125002999999, 2024-09-18T04:57:57.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044645562, 2024-09-18T04:57:57.775 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.776 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.776 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.776 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.776 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.776 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60348 2024-09-18T04:57:57.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.776 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:57.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:57.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:57:57.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:57.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112297+0000", 2024-09-18T04:57:57.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.384e-06 2024-09-18T04:57:57.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112301+0000", 2024-09-18T04:57:57.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0570000000000003e-06 2024-09-18T04:57:57.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112304+0000", 2024-09-18T04:57:57.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6000000000000001e-06 2024-09-18T04:57:57.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156793+0000", 2024-09-18T04:57:57.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044489494999999997 2024-09-18T04:57:57.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156859+0000", 2024-09-18T04:57:57.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5483999999999998e-05 2024-09-18T04:57:57.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156941+0000", 2024-09-18T04:57:57.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2856999999999996e-05 2024-09-18T04:57:57.779 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.779 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.779 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.780 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60349 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1532928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:57.780 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:57.780 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.641115026999998, 2024-09-18T04:57:57.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044832167999999999, 2024-09-18T04:57:57.780 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.780 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.780 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.780 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.780 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.780 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60349 2024-09-18T04:57:57.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.780 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:57.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:57.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:57:57.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:57.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112307+0000", 2024-09-18T04:57:57.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.308e-06 2024-09-18T04:57:57.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112310+0000", 2024-09-18T04:57:57.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-18T04:57:57.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112312+0000", 2024-09-18T04:57:57.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4889999999999998e-06 2024-09-18T04:57:57.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156977+0000", 2024-09-18T04:57:57.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044664598999999999 2024-09-18T04:57:57.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157053+0000", 2024-09-18T04:57:57.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6014999999999996e-05 2024-09-18T04:57:57.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157138+0000", 2024-09-18T04:57:57.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5162999999999998e-05 2024-09-18T04:57:57.784 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.784 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.784 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.784 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60350 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1548288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:57.784 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:57.784 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.641106113000006, 2024-09-18T04:57:57.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045011858000000002, 2024-09-18T04:57:57.784 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.784 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.785 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.785 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.785 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.785 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60350 2024-09-18T04:57:57.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.785 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:57.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:57.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:57:57.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:57.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112316+0000", 2024-09-18T04:57:57.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7579999999999999e-06 2024-09-18T04:57:57.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112321+0000", 2024-09-18T04:57:57.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1649999999999997e-06 2024-09-18T04:57:57.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112325+0000", 2024-09-18T04:57:57.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.726e-06 2024-09-18T04:57:57.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157172+0000", 2024-09-18T04:57:57.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.04484718 2024-09-18T04:57:57.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157242+0000", 2024-09-18T04:57:57.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9675e-05 2024-09-18T04:57:57.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157327+0000", 2024-09-18T04:57:57.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4614e-05 2024-09-18T04:57:57.788 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.788 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.788 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.789 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60351 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1563648~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:57.789 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:57.789 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.641092697000005, 2024-09-18T04:57:57.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045132113000000001, 2024-09-18T04:57:57.789 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.789 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.789 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.789 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.789 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.789 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60351 2024-09-18T04:57:57.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.789 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:57.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:57.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:57:57.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:57.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112330+0000", 2024-09-18T04:57:57.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1660000000000001e-06 2024-09-18T04:57:57.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112336+0000", 2024-09-18T04:57:57.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5940000000000001e-06 2024-09-18T04:57:57.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112340+0000", 2024-09-18T04:57:57.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5589999999999999e-06 2024-09-18T04:57:57.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157365+0000", 2024-09-18T04:57:57.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045024675 2024-09-18T04:57:57.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157409+0000", 2024-09-18T04:57:57.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4356e-05 2024-09-18T04:57:57.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157460+0000", 2024-09-18T04:57:57.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1072000000000002e-05 2024-09-18T04:57:57.793 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.793 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.793 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.794 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60501 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1494016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:57.794 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:57.794 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.635032000999999, 2024-09-18T04:57:57.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044007728000000003, 2024-09-18T04:57:57.794 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.794 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.794 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.794 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.794 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.794 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60501 2024-09-18T04:57:57.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.794 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:57.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:57.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:57:57.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:57.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118391+0000", 2024-09-18T04:57:57.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:57:57.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118396+0000", 2024-09-18T04:57:57.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2680000000000002e-06 2024-09-18T04:57:57.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118400+0000", 2024-09-18T04:57:57.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2080000000000002e-06 2024-09-18T04:57:57.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162317+0000", 2024-09-18T04:57:57.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043916753000000003 2024-09-18T04:57:57.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162351+0000", 2024-09-18T04:57:57.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4372000000000002e-05 2024-09-18T04:57:57.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162397+0000", 2024-09-18T04:57:57.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5207999999999998e-05 2024-09-18T04:57:57.798 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.798 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.798 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.798 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60502 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1509376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:57.798 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:57.798 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.635016523999994, 2024-09-18T04:57:57.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044092438999999997, 2024-09-18T04:57:57.798 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.798 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.799 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.799 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.799 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.799 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60502 2024-09-18T04:57:57.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.799 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:57.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:57.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:57:57.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:57.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118406+0000", 2024-09-18T04:57:57.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:57:57.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118412+0000", 2024-09-18T04:57:57.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.524e-06 2024-09-18T04:57:57.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118416+0000", 2024-09-18T04:57:57.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:57:57.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162418+0000", 2024-09-18T04:57:57.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001508000000002 2024-09-18T04:57:57.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162451+0000", 2024-09-18T04:57:57.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3662999999999999e-05 2024-09-18T04:57:57.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162497+0000", 2024-09-18T04:57:57.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5550000000000003e-05 2024-09-18T04:57:57.802 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.802 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.802 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.803 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60503 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1524736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:57.803 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:57:57.803 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.635000366, 2024-09-18T04:57:57.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044178780000000001, 2024-09-18T04:57:57.803 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.803 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.803 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.803 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.803 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.803 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60503 2024-09-18T04:57:57.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.803 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:57:57.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:57:57.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118420+0000", 2024-09-18T04:57:57.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:57.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118422+0000", 2024-09-18T04:57:57.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.08e-06 2024-09-18T04:57:57.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118428+0000", 2024-09-18T04:57:57.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.964e-06 2024-09-18T04:57:57.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118433+0000", 2024-09-18T04:57:57.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.425e-06 2024-09-18T04:57:57.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162519+0000", 2024-09-18T04:57:57.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044086082999999998 2024-09-18T04:57:57.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162552+0000", 2024-09-18T04:57:57.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3719e-05 2024-09-18T04:57:57.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162599+0000", 2024-09-18T04:57:57.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6873999999999999e-05 2024-09-18T04:57:57.807 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.807 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.807 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.807 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60504 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1540096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:57.807 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:57:57.807 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.634984261, 2024-09-18T04:57:57.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044261921000000003, 2024-09-18T04:57:57.807 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.807 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.807 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.808 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.808 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60504 2024-09-18T04:57:57.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.808 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:57:57.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:57:57.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118436+0000", 2024-09-18T04:57:57.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:57.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118438+0000", 2024-09-18T04:57:57.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0250000000000001e-06 2024-09-18T04:57:57.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118444+0000", 2024-09-18T04:57:57.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4360000000000001e-06 2024-09-18T04:57:57.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118448+0000", 2024-09-18T04:57:57.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.357e-06 2024-09-18T04:57:57.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162620+0000", 2024-09-18T04:57:57.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044171821 2024-09-18T04:57:57.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162653+0000", 2024-09-18T04:57:57.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3479999999999998e-05 2024-09-18T04:57:57.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162699+0000", 2024-09-18T04:57:57.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5113e-05 2024-09-18T04:57:57.811 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.811 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.811 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.811 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60584 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1196032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:57.811 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:57:57.812 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.633191244000002, 2024-09-18T04:57:57.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044048727000000003, 2024-09-18T04:57:57.812 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.812 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.812 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.812 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.812 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.812 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60584 2024-09-18T04:57:57.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.812 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:57:57.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:57:57.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120229+0000", 2024-09-18T04:57:57.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:57.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120231+0000", 2024-09-18T04:57:57.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:57:57.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120237+0000", 2024-09-18T04:57:57.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8379999999999998e-06 2024-09-18T04:57:57.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120242+0000", 2024-09-18T04:57:57.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3889999999999998e-06 2024-09-18T04:57:57.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164187+0000", 2024-09-18T04:57:57.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043945045000000002 2024-09-18T04:57:57.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164227+0000", 2024-09-18T04:57:57.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0627999999999998e-05 2024-09-18T04:57:57.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164278+0000", 2024-09-18T04:57:57.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1146000000000001e-05 2024-09-18T04:57:57.816 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.816 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.816 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.816 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60585 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1211392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:57.816 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:57:57.816 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.633175136000006, 2024-09-18T04:57:57.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044389814, 2024-09-18T04:57:57.817 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.817 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.817 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.817 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.817 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.817 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60585 2024-09-18T04:57:57.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.817 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:57:57.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:57:57.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120245+0000", 2024-09-18T04:57:57.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:57.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120247+0000", 2024-09-18T04:57:57.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0150000000000002e-06 2024-09-18T04:57:57.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120255+0000", 2024-09-18T04:57:57.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.858e-06 2024-09-18T04:57:57.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120260+0000", 2024-09-18T04:57:57.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6530000000000002e-06 2024-09-18T04:57:57.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164489+0000", 2024-09-18T04:57:57.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044229483999999999 2024-09-18T04:57:57.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164555+0000", 2024-09-18T04:57:57.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.533e-05 2024-09-18T04:57:57.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164636+0000", 2024-09-18T04:57:57.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0819e-05 2024-09-18T04:57:57.820 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.820 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.820 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.821 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60586 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1226752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:57.821 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:57.821 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.633156685000003, 2024-09-18T04:57:57.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044578214999999997, 2024-09-18T04:57:57.821 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.821 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.821 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.821 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.821 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.821 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60586 2024-09-18T04:57:57.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.821 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:57.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:57.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:57:57.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:57.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120266+0000", 2024-09-18T04:57:57.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0660000000000002e-06 2024-09-18T04:57:57.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120272+0000", 2024-09-18T04:57:57.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-18T04:57:57.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120276+0000", 2024-09-18T04:57:57.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.476e-06 2024-09-18T04:57:57.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164673+0000", 2024-09-18T04:57:57.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044396672999999998 2024-09-18T04:57:57.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164749+0000", 2024-09-18T04:57:57.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5811999999999999e-05 2024-09-18T04:57:57.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164842+0000", 2024-09-18T04:57:57.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3560000000000005e-05 2024-09-18T04:57:57.825 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.825 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.825 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.825 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60587 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1242112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:57.825 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:57:57.825 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.633107585999994, 2024-09-18T04:57:57.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044634794999999998, 2024-09-18T04:57:57.825 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.825 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.826 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.826 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.826 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.826 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60587 2024-09-18T04:57:57.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.826 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:57:57.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:57:57.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120280+0000", 2024-09-18T04:57:57.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999671 2024-09-18T04:57:57.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120316+0000", 2024-09-18T04:57:57.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5728000000000001e-05 2024-09-18T04:57:57.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120329+0000", 2024-09-18T04:57:57.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2914e-05 2024-09-18T04:57:57.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120335+0000", 2024-09-18T04:57:57.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8289999999999996e-06 2024-09-18T04:57:57.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164858+0000", 2024-09-18T04:57:57.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044523417000000003 2024-09-18T04:57:57.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164896+0000", 2024-09-18T04:57:57.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7969999999999997e-05 2024-09-18T04:57:57.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164948+0000", 2024-09-18T04:57:57.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1996000000000002e-05 2024-09-18T04:57:57.829 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.829 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.829 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.830 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60595 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1364992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:57.830 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:57:57.830 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.631264369999997, 2024-09-18T04:57:57.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001360000000003, 2024-09-18T04:57:57.830 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.830 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.830 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.830 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.830 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.830 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60595 2024-09-18T04:57:57.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.830 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:57:57.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:57:57.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122156+0000", 2024-09-18T04:57:57.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:57.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122158+0000", 2024-09-18T04:57:57.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0040000000000002e-06 2024-09-18T04:57:57.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122164+0000", 2024-09-18T04:57:57.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8719999999999999e-06 2024-09-18T04:57:57.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122169+0000", 2024-09-18T04:57:57.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5090000000000004e-06 2024-09-18T04:57:57.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166040+0000", 2024-09-18T04:57:57.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043871226999999999 2024-09-18T04:57:57.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166096+0000", 2024-09-18T04:57:57.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6495e-05 2024-09-18T04:57:57.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166158+0000", 2024-09-18T04:57:57.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1626999999999998e-05 2024-09-18T04:57:57.834 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.834 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.834 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.834 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60596 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1380352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:57.834 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:57.834 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.631248049000007, 2024-09-18T04:57:57.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044120800000000002, 2024-09-18T04:57:57.834 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.834 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.835 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.835 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.835 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.835 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60596 2024-09-18T04:57:57.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.835 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:57.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:57.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:57:57.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:57.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122175+0000", 2024-09-18T04:57:57.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:57:57.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122182+0000", 2024-09-18T04:57:57.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3509999999999996e-06 2024-09-18T04:57:57.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122187+0000", 2024-09-18T04:57:57.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7940000000000002e-06 2024-09-18T04:57:57.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166184+0000", 2024-09-18T04:57:57.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043997236000000002 2024-09-18T04:57:57.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166227+0000", 2024-09-18T04:57:57.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2561000000000001e-05 2024-09-18T04:57:57.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166294+0000", 2024-09-18T04:57:57.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7068000000000002e-05 2024-09-18T04:57:57.838 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.838 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.838 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.838 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60597 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1395712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:57.839 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:57.839 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.631230020999993, 2024-09-18T04:57:57.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044242166999999999, 2024-09-18T04:57:57.839 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.839 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.839 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.839 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.839 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.839 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60597 2024-09-18T04:57:57.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.839 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:57.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:57.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:57:57.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:57.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122193+0000", 2024-09-18T04:57:57.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:57:57.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122199+0000", 2024-09-18T04:57:57.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6449999999999999e-06 2024-09-18T04:57:57.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122204+0000", 2024-09-18T04:57:57.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4819999999999996e-06 2024-09-18T04:57:57.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166329+0000", 2024-09-18T04:57:57.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044125059000000001 2024-09-18T04:57:57.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166370+0000", 2024-09-18T04:57:57.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1012e-05 2024-09-18T04:57:57.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166433+0000", 2024-09-18T04:57:57.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3178999999999999e-05 2024-09-18T04:57:57.843 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.843 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.843 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.843 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60608 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1564672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:57:57.843 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:57:57.843 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.629232158999997, 2024-09-18T04:57:57.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044082723999999997, 2024-09-18T04:57:57.843 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:57:57.843 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:57:57.843 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:57:57.843 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:57:57.844 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:57:57.844 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60608 2024-09-18T04:57:57.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.844 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:57:57.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:57:57.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:57:57.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:57:57.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:57:57.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:57:57.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:57:57.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124188+0000", 2024-09-18T04:57:57.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:57:57.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:57:57.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124190+0000", 2024-09-18T04:57:57.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0329999999999998e-06 2024-09-18T04:57:57.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:57:57.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124196+0000", 2024-09-18T04:57:57.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.575e-06 2024-09-18T04:57:57.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:57:57.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124201+0000", 2024-09-18T04:57:57.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5000000000000001e-06 2024-09-18T04:57:57.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:57:57.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168166+0000", 2024-09-18T04:57:57.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043965763999999997 2024-09-18T04:57:57.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:57:57.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168218+0000", 2024-09-18T04:57:57.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1785e-05 2024-09-18T04:57:57.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:57:57.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:57:57.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:57:57.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168271+0000", 2024-09-18T04:57:57.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3429e-05 2024-09-18T04:57:57.847 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.847 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.847 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.847 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:57:57.847 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:57:57.847 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:57:57.848 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:57:57.848 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:57:57.849 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:57:57.849 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:57:57.850 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:57:57.850 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:57:57.850 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:57:57.851 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:57:57.851 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:57:57.864 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-09-18T04:57:57.865 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:57.861+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:57.953 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:57.953 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:57.956 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:57.954+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:57.962 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:57:58.048 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-09-18T04:57:58.088 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T04:57:58.091 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:58.088+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:58.113 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:58.113 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:58.123 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:57:58.167 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:58.165+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:58.197 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-09-18T04:57:58.228 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:58.228 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:58.237 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:57:58.259 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:58.256+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:58.374 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:58.372+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:58.380 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:58.380 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:58.389 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:57:58.402 INFO:teuthology.orchestra.run.smithi002.stderr:2024-09-18T04:57:58.399+0000 7fedf522b640 0 --2- 172.21.15.2:0/1699940263 >> [v2:172.21.15.2:3301/0,v1:172.21.15.2:6790/0] conn(0x7fedf0140370 0x7fedf014a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-09-18T04:57:58.461 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:58.459+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:58.491 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:58.491 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:58.500 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:57:58.583 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:58.581+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:58.641 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:58.642 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:58.650 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:57:58.692 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:58.689+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:58.757 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:58.758 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:58.764 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:58.761+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:58.766 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:57:58.865 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:58.863+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:58.912 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:58.913 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:58.921 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:57:58.967 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:57:58.963+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:57:59.028 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:59.028 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:59.037 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:57:59.078 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:57:59.075+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:57:59.083 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-18T04:57:59.168 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:59.166+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:59.197 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:59.197 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:59.207 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:57:59.270 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:57:59.267+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:57:59.314 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:59.314 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:59.323 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:57:59.370 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:57:59.367+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:57:59.429 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:59.429 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:59.438 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:57:59.471 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:59.468+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:59.572 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:57:59.571+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:57:59.585 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:59.585 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:59.594 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:57:59.673 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:57:59.670+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:57:59.701 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:59.702 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:59.709 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:57:59.774 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:59.772+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:59.816 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:59.816 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:59.824 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:57:59.852 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:57:59.876 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:57:59.874+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:57:59.933 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:57:59.933 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:57:59.943 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:57:59.988 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:57:59.985+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:57:59.994 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:57:59.994 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:58:00.011 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:58:00.011 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:58:00.027 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:58:00.027 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:58:00.043 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:58:00.043 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:58:00.053 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:00.053 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:00.054 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:58:00.054 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:58:00.054 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:00.060 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:58:00.060 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:58:00.062 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:58:00.077 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:58:00.077 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:58:00.078 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:00.075+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:00.099 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:58:00.100 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:58:00.121 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:58:00.121 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:58:00.178 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:00.175+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:00.198 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:00.198 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:58:00.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:58:00.198 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:58:00.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.198 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11154 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:00.199 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:58:00.199 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.305778497000006, 2024-09-18T04:58:00.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040591919999999997, 2024-09-18T04:58:00.199 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:00.199 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:00.199 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:00.199 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:00.199 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.199 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11154 2024-09-18T04:58:00.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.199 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:00.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:58:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:58:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889400+0000", 2024-09-18T04:58:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889403+0000", 2024-09-18T04:58:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3999999999999999e-06 2024-09-18T04:58:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889409+0000", 2024-09-18T04:58:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3380000000000001e-06 2024-09-18T04:58:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889414+0000", 2024-09-18T04:58:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5789999999999997e-06 2024-09-18T04:58:00.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929811+0000", 2024-09-18T04:58:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040397486000000003 2024-09-18T04:58:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929898+0000", 2024-09-18T04:58:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6370000000000001e-05 2024-09-18T04:58:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929993+0000", 2024-09-18T04:58:00.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5136000000000005e-05 2024-09-18T04:58:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11155 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2015232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:58:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.305761318999998, 2024-09-18T04:58:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040769156000000001, 2024-09-18T04:58:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:00.203 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:00.204 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:00.204 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.204 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11155 2024-09-18T04:58:00.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.204 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:00.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:00.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:58:00.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:58:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:58:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:00.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889420+0000", 2024-09-18T04:58:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.181e-06 2024-09-18T04:58:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889426+0000", 2024-09-18T04:58:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8599999999999998e-06 2024-09-18T04:58:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889430+0000", 2024-09-18T04:58:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4399999999999998e-06 2024-09-18T04:58:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:00.206 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930020+0000", 2024-09-18T04:58:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040589698 2024-09-18T04:58:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930105+0000", 2024-09-18T04:58:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4812e-05 2024-09-18T04:58:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930187+0000", 2024-09-18T04:58:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2540999999999995e-05 2024-09-18T04:58:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:00.207 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11156 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2064384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:58:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.305721024999997, 2024-09-18T04:58:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040934419, 2024-09-18T04:58:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11156 2024-09-18T04:58:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.208 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:58:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:58:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889457+0000", 2024-09-18T04:58:00.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:58:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889462+0000", 2024-09-18T04:58:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8350000000000003e-06 2024-09-18T04:58:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889481+0000", 2024-09-18T04:58:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8845999999999999e-05 2024-09-18T04:58:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889495+0000", 2024-09-18T04:58:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4229e-05 2024-09-18T04:58:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930216+0000", 2024-09-18T04:58:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040721081999999999 2024-09-18T04:58:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930299+0000", 2024-09-18T04:58:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2995000000000001e-05 2024-09-18T04:58:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:00.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930393+0000", 2024-09-18T04:58:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3185999999999999e-05 2024-09-18T04:58:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11404 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1609728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:58:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.298916863000002, 2024-09-18T04:58:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040669012999999997, 2024-09-18T04:58:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:00.212 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11404 2024-09-18T04:58:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:58:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:58:00.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:00.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:58:00.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:58:00.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:00.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896263+0000", 2024-09-18T04:58:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2729999999999999e-06 2024-09-18T04:58:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896266+0000", 2024-09-18T04:58:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8200000000000001e-06 2024-09-18T04:58:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896269+0000", 2024-09-18T04:58:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3199999999999998e-06 2024-09-18T04:58:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936538+0000", 2024-09-18T04:58:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040269394999999999 2024-09-18T04:58:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936657+0000", 2024-09-18T04:58:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119498 2024-09-18T04:58:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936931+0000", 2024-09-18T04:58:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027391900000000002 2024-09-18T04:58:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.216 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11405 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1658880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:58:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.298908604999994, 2024-09-18T04:58:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040891034999999999, 2024-09-18T04:58:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11405 2024-09-18T04:58:00.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:58:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:58:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:00.218 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896270+0000", 2024-09-18T04:58:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:58:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896272+0000", 2024-09-18T04:58:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.124e-06 2024-09-18T04:58:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896274+0000", 2024-09-18T04:58:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8660000000000002e-06 2024-09-18T04:58:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896285+0000", 2024-09-18T04:58:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0101e-05 2024-09-18T04:58:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936986+0000", 2024-09-18T04:58:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040701213999999999 2024-09-18T04:58:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937070+0000", 2024-09-18T04:58:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4211999999999999e-05 2024-09-18T04:58:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937162+0000", 2024-09-18T04:58:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1728e-05 2024-09-18T04:58:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11406 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1708032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:58:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.298892436000003, 2024-09-18T04:58:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041085251000000003, 2024-09-18T04:58:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:00.221 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11406 2024-09-18T04:58:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:58:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:00.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:58:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:58:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:58:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896288+0000", 2024-09-18T04:58:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2440000000000001e-06 2024-09-18T04:58:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896291+0000", 2024-09-18T04:58:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8289999999999999e-06 2024-09-18T04:58:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896293+0000", 2024-09-18T04:58:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5399999999999998e-06 2024-09-18T04:58:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937201+0000", 2024-09-18T04:58:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040907418000000001 2024-09-18T04:58:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937283+0000", 2024-09-18T04:58:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2460999999999995e-05 2024-09-18T04:58:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937372+0000", 2024-09-18T04:58:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8987999999999996e-05 2024-09-18T04:58:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.225 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11407 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:58:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.298882616, 2024-09-18T04:58:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041438580000000003, 2024-09-18T04:58:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11407 2024-09-18T04:58:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:58:00.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:58:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896296+0000", 2024-09-18T04:58:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:00.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896298+0000", 2024-09-18T04:58:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:58:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896304+0000", 2024-09-18T04:58:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6570000000000002e-06 2024-09-18T04:58:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896308+0000", 2024-09-18T04:58:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0940000000000001e-06 2024-09-18T04:58:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937540+0000", 2024-09-18T04:58:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041231469 2024-09-18T04:58:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937625+0000", 2024-09-18T04:58:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5767999999999996e-05 2024-09-18T04:58:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937735+0000", 2024-09-18T04:58:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010974 2024-09-18T04:58:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.229 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:00.230 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.230 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11408 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:00.230 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:58:00.230 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.298866977000003, 2024-09-18T04:58:00.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041653055000000001, 2024-09-18T04:58:00.230 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:00.230 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:00.230 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11408 2024-09-18T04:58:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:58:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:58:00.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:58:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896314+0000", 2024-09-18T04:58:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0140000000000001e-06 2024-09-18T04:58:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896320+0000", 2024-09-18T04:58:00.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-18T04:58:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896324+0000", 2024-09-18T04:58:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4410000000000003e-06 2024-09-18T04:58:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937769+0000", 2024-09-18T04:58:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041445243 2024-09-18T04:58:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:00.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937861+0000", 2024-09-18T04:58:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1471000000000006e-05 2024-09-18T04:58:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937965+0000", 2024-09-18T04:58:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104516 2024-09-18T04:58:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11409 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:58:00.234 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.298850920999996, 2024-09-18T04:58:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041840467999999999, 2024-09-18T04:58:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11409 2024-09-18T04:58:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:58:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:58:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:58:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896331+0000", 2024-09-18T04:58:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5950000000000001e-06 2024-09-18T04:58:00.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896336+0000", 2024-09-18T04:58:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7259999999999999e-06 2024-09-18T04:58:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896341+0000", 2024-09-18T04:58:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5619999999999997e-06 2024-09-18T04:58:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937995+0000", 2024-09-18T04:58:00.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041654060999999999 2024-09-18T04:58:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938078+0000", 2024-09-18T04:58:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2869999999999998e-05 2024-09-18T04:58:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938169+0000", 2024-09-18T04:58:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1040999999999998e-05 2024-09-18T04:58:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11410 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1921024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:58:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.298834438, 2024-09-18T04:58:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042060342000000001, 2024-09-18T04:58:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11410 2024-09-18T04:58:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:00.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:58:00.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:00.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:58:00.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.240 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:00.240 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896344+0000", 2024-09-18T04:58:00.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:00.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896360+0000", 2024-09-18T04:58:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5068e-05 2024-09-18T04:58:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896366+0000", 2024-09-18T04:58:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5119999999999997e-06 2024-09-18T04:58:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896371+0000", 2024-09-18T04:58:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:58:00.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938198+0000", 2024-09-18T04:58:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041826579000000003 2024-09-18T04:58:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938297+0000", 2024-09-18T04:58:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9067000000000005e-05 2024-09-18T04:58:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938405+0000", 2024-09-18T04:58:00.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108492 2024-09-18T04:58:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11411 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1970176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:58:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.298803595999999, 2024-09-18T04:58:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042259191000000002, 2024-09-18T04:58:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:00.243 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:00.244 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:00.244 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.244 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11411 2024-09-18T04:58:00.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.244 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:00.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:00.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:58:00.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:58:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896375+0000", 2024-09-18T04:58:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896377+0000", 2024-09-18T04:58:00.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2699999999999999e-06 2024-09-18T04:58:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896383+0000", 2024-09-18T04:58:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6740000000000002e-06 2024-09-18T04:58:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896388+0000", 2024-09-18T04:58:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:58:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:00.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938435+0000", 2024-09-18T04:58:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042047169000000002 2024-09-18T04:58:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938534+0000", 2024-09-18T04:58:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8939999999999998e-05 2024-09-18T04:58:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938635+0000", 2024-09-18T04:58:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101208 2024-09-18T04:58:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:00.247 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11412 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:58:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.298787614000005, 2024-09-18T04:58:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042828079999999998, 2024-09-18T04:58:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11412 2024-09-18T04:58:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.248 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:58:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:58:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896391+0000", 2024-09-18T04:58:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:00.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896393+0000", 2024-09-18T04:58:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0810000000000001e-06 2024-09-18T04:58:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896399+0000", 2024-09-18T04:58:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7230000000000001e-06 2024-09-18T04:58:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896403+0000", 2024-09-18T04:58:00.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3470000000000001e-06 2024-09-18T04:58:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939043+0000", 2024-09-18T04:58:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042639829999999997 2024-09-18T04:58:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939124+0000", 2024-09-18T04:58:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0809000000000005e-05 2024-09-18T04:58:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:00.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939220+0000", 2024-09-18T04:58:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5659e-05 2024-09-18T04:58:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11413 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2068480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:58:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.298771919000004, 2024-09-18T04:58:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043042108000000003, 2024-09-18T04:58:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:00.252 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:00.253 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.253 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11413 2024-09-18T04:58:00.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.253 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:00.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:00.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:58:00.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:58:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:58:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896409+0000", 2024-09-18T04:58:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.165e-06 2024-09-18T04:58:00.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896415+0000", 2024-09-18T04:58:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6149999999999996e-06 2024-09-18T04:58:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896419+0000", 2024-09-18T04:58:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2649999999999998e-06 2024-09-18T04:58:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939262+0000", 2024-09-18T04:58:00.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042843444000000001 2024-09-18T04:58:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939351+0000", 2024-09-18T04:58:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8944999999999996e-05 2024-09-18T04:58:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939449+0000", 2024-09-18T04:58:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8029000000000006e-05 2024-09-18T04:58:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.256 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145409 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:58:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.667352270999999, 2024-09-18T04:58:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040684845999999997, 2024-09-18T04:58:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.257 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145409 2024-09-18T04:58:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:58:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:58:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:00.258 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:58:00.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:00.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:00.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527829+0000", 2024-09-18T04:58:00.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.441e-06 2024-09-18T04:58:00.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:00.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527835+0000", 2024-09-18T04:58:00.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1650000000000003e-06 2024-09-18T04:58:00.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527840+0000", 2024-09-18T04:58:00.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8250000000000004e-06 2024-09-18T04:58:00.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:00.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568404+0000", 2024-09-18T04:58:00.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040563872000000001 2024-09-18T04:58:00.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:00.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568457+0000", 2024-09-18T04:58:00.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2713999999999997e-05 2024-09-18T04:58:00.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:00.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568512+0000", 2024-09-18T04:58:00.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5148000000000001e-05 2024-09-18T04:58:00.261 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.261 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:00.261 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.261 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145411 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:00.261 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:58:00.261 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.667334409999999, 2024-09-18T04:58:00.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040792108000000001, 2024-09-18T04:58:00.261 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:00.261 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:00.262 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:00.262 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:00.262 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.262 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145411 2024-09-18T04:58:00.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.262 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:00.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:00.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:58:00.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:00.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:58:00.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:00.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527844+0000", 2024-09-18T04:58:00.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:00.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:00.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527847+0000", 2024-09-18T04:58:00.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0260000000000002e-06 2024-09-18T04:58:00.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:00.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527852+0000", 2024-09-18T04:58:00.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9889999999999997e-06 2024-09-18T04:58:00.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527857+0000", 2024-09-18T04:58:00.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6410000000000001e-06 2024-09-18T04:58:00.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:00.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568537+0000", 2024-09-18T04:58:00.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040679743999999997 2024-09-18T04:58:00.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:00.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568583+0000", 2024-09-18T04:58:00.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6286999999999999e-05 2024-09-18T04:58:00.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:00.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568637+0000", 2024-09-18T04:58:00.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.376e-05 2024-09-18T04:58:00.265 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.265 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:00.265 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.265 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145413 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1859584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:00.265 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:58:00.266 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.667318030000001, 2024-09-18T04:58:00.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040919516000000003, 2024-09-18T04:58:00.266 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:00.266 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:00.266 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:00.266 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:00.266 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.266 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145413 2024-09-18T04:58:00.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.266 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:00.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:00.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:58:00.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:00.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:58:00.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:00.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:58:00.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:00.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.267 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:00.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527863+0000", 2024-09-18T04:58:00.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2630000000000002e-06 2024-09-18T04:58:00.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:00.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527869+0000", 2024-09-18T04:58:00.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9009999999999999e-06 2024-09-18T04:58:00.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527874+0000", 2024-09-18T04:58:00.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0590000000000002e-06 2024-09-18T04:58:00.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:00.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568663+0000", 2024-09-18T04:58:00.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040788805999999997 2024-09-18T04:58:00.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:00.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568716+0000", 2024-09-18T04:58:00.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3501999999999997e-05 2024-09-18T04:58:00.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:00.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568781+0000", 2024-09-18T04:58:00.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4338000000000006e-05 2024-09-18T04:58:00.269 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.269 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:00.269 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.270 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145414 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1908736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:00.270 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:58:00.270 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.667301072000001, 2024-09-18T04:58:00.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041030234999999998, 2024-09-18T04:58:00.270 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:00.270 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:00.270 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:00.270 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:00.270 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.270 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145414 2024-09-18T04:58:00.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.270 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:00.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:00.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:58:00.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:00.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:58:00.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:00.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:58:00.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:00.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:00.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527880+0000", 2024-09-18T04:58:00.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4770000000000002e-06 2024-09-18T04:58:00.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:00.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527886+0000", 2024-09-18T04:58:00.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5759999999999996e-06 2024-09-18T04:58:00.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527890+0000", 2024-09-18T04:58:00.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8180000000000003e-06 2024-09-18T04:58:00.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:00.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568807+0000", 2024-09-18T04:58:00.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040916984000000003 2024-09-18T04:58:00.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:00.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568854+0000", 2024-09-18T04:58:00.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6740999999999999e-05 2024-09-18T04:58:00.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:00.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568908+0000", 2024-09-18T04:58:00.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4228000000000003e-05 2024-09-18T04:58:00.274 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.274 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:00.274 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.274 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145416 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1957888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:00.274 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:58:00.274 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.66728458, 2024-09-18T04:58:00.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041147364999999998, 2024-09-18T04:58:00.274 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:00.274 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:00.274 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:00.275 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:00.275 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.275 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145416 2024-09-18T04:58:00.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.275 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:00.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:00.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:58:00.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:00.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:58:00.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:00.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527894+0000", 2024-09-18T04:58:00.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:00.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:00.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527896+0000", 2024-09-18T04:58:00.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1789999999999998e-06 2024-09-18T04:58:00.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:00.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527908+0000", 2024-09-18T04:58:00.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1780000000000001e-05 2024-09-18T04:58:00.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527914+0000", 2024-09-18T04:58:00.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2029999999999997e-06 2024-09-18T04:58:00.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:00.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568935+0000", 2024-09-18T04:58:00.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041020273000000003 2024-09-18T04:58:00.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:00.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568988+0000", 2024-09-18T04:58:00.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2911000000000002e-05 2024-09-18T04:58:00.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:00.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569042+0000", 2024-09-18T04:58:00.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4382000000000002e-05 2024-09-18T04:58:00.278 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.278 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:00.278 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.278 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145418 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 2007040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:00.278 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:58:00.278 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.667260183, 2024-09-18T04:58:00.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041247862000000003, 2024-09-18T04:58:00.279 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:00.279 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:00.279 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:00.279 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:00.279 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.279 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145418 2024-09-18T04:58:00.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.279 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:00.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:00.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:58:00.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:00.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:58:00.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:00.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:58:00.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:00.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:00.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527921+0000", 2024-09-18T04:58:00.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.667e-06 2024-09-18T04:58:00.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:00.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527929+0000", 2024-09-18T04:58:00.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.542e-06 2024-09-18T04:58:00.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527934+0000", 2024-09-18T04:58:00.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1440000000000002e-06 2024-09-18T04:58:00.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:00.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569068+0000", 2024-09-18T04:58:00.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041134190000000001 2024-09-18T04:58:00.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:00.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569114+0000", 2024-09-18T04:58:00.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6261999999999999e-05 2024-09-18T04:58:00.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:00.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569167+0000", 2024-09-18T04:58:00.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2506999999999997e-05 2024-09-18T04:58:00.282 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.282 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:00.282 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.283 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145420 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 2056192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:00.283 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:58:00.283 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 26.667241108999999, 2024-09-18T04:58:00.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041351472, 2024-09-18T04:58:00.283 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:00.283 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:00.283 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:00.283 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:00.283 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.283 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145420 2024-09-18T04:58:00.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.283 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:00.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:00.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:58:00.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:00.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:58:00.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:00.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:00.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:58:00.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:00.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:00.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527940+0000", 2024-09-18T04:58:00.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6570000000000001e-06 2024-09-18T04:58:00.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:00.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527948+0000", 2024-09-18T04:58:00.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8350000000000004e-06 2024-09-18T04:58:00.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527953+0000", 2024-09-18T04:58:00.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0690000000000001e-06 2024-09-18T04:58:00.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:00.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569182+0000", 2024-09-18T04:58:00.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041228734000000003 2024-09-18T04:58:00.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:00.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569234+0000", 2024-09-18T04:58:00.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1663999999999999e-05 2024-09-18T04:58:00.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:00.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:00.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:00.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569290+0000", 2024-09-18T04:58:00.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5937000000000003e-05 2024-09-18T04:58:00.287 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.287 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:00.287 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.287 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:00.287 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:00.287 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:00.288 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:58:00.289 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:00.286+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:00.389 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:00.386+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:00.391 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:00.391 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:00.391 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:58:00.391 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:00.400 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:58:00.490 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:00.488+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:00.542 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:58:00.542 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:58:00.542 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:00.550 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:58:00.594 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:00.592+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:00.654 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:00.654 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:58:00.654 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:58:00.654 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:00.654 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.654 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72135 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 394240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:00.654 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:58:00.654 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 63.696984047999997, 2024-09-18T04:58:00.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048559739999999997, 2024-09-18T04:58:00.655 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:00.655 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:00.655 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:00.655 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:00.655 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.655 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72135 2024-09-18T04:58:00.655 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.655 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:00.655 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.655 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:00.655 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:58:00.655 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:00.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:58:00.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.656 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.656 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.656 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:00.656 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:58:00.656 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:00.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:00.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954005+0000", 2024-09-18T04:58:00.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5629999999999999e-06 2024-09-18T04:58:00.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.657 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.657 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:00.657 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954008+0000", 2024-09-18T04:58:00.657 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-18T04:58:00.657 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954012+0000", 2024-09-18T04:58:00.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4790000000000001e-06 2024-09-18T04:58:00.658 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.658 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.658 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:00.658 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002418+0000", 2024-09-18T04:58:00.658 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048406049 2024-09-18T04:58:00.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:00.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002460+0000", 2024-09-18T04:58:00.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2188999999999999e-05 2024-09-18T04:58:00.659 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.659 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.659 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:00.659 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002564+0000", 2024-09-18T04:58:00.659 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010377 2024-09-18T04:58:00.659 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.659 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:00.659 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.660 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.660 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72136 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 409600~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:00.660 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:58:00.660 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 63.696879101, 2024-09-18T04:58:00.660 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048670191000000002, 2024-09-18T04:58:00.660 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:00.660 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:00.660 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:00.660 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:00.660 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.660 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72136 2024-09-18T04:58:00.660 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.661 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:00.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:00.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:58:00.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.661 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.661 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.661 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:00.661 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:58:00.661 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:00.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954019+0000", 2024-09-18T04:58:00.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999099 2024-09-18T04:58:00.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:00.662 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954112+0000", 2024-09-18T04:58:00.662 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3273e-05 2024-09-18T04:58:00.662 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.662 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.662 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:00.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954121+0000", 2024-09-18T04:58:00.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0289999999999994e-06 2024-09-18T04:58:00.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954128+0000", 2024-09-18T04:58:00.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6189999999999996e-06 2024-09-18T04:58:00.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.663 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:00.663 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002604+0000", 2024-09-18T04:58:00.663 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048476048000000001 2024-09-18T04:58:00.663 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.663 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:00.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002682+0000", 2024-09-18T04:58:00.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7955999999999997e-05 2024-09-18T04:58:00.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.664 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:00.664 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002779+0000", 2024-09-18T04:58:00.664 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7372000000000002e-05 2024-09-18T04:58:00.664 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.664 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:00.664 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.664 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.664 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.665 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72138 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 440320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:00.665 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:58:00.665 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 63.696445249, 2024-09-18T04:58:00.665 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048549430999999997, 2024-09-18T04:58:00.665 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:00.665 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:00.665 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:00.665 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:00.665 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.665 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72138 2024-09-18T04:58:00.665 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.665 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:00.665 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:00.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:58:00.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:00.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:58:00.666 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.666 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.666 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.666 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:00.666 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954542+0000", 2024-09-18T04:58:00.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:00.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:00.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954545+0000", 2024-09-18T04:58:00.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4360000000000001e-06 2024-09-18T04:58:00.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.667 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.667 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:00.667 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954551+0000", 2024-09-18T04:58:00.667 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4949999999999996e-06 2024-09-18T04:58:00.667 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954557+0000", 2024-09-18T04:58:00.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5099999999999998e-06 2024-09-18T04:58:00.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.668 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:00.668 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002993+0000", 2024-09-18T04:58:00.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048436358999999998 2024-09-18T04:58:00.668 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:00.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003039+0000", 2024-09-18T04:58:00.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5884000000000003e-05 2024-09-18T04:58:00.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:00.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003092+0000", 2024-09-18T04:58:00.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3208000000000003e-05 2024-09-18T04:58:00.669 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.669 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:00.669 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.669 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72139 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 455680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:00.670 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:58:00.670 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 63.696426441, 2024-09-18T04:58:00.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048718773, 2024-09-18T04:58:00.670 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:00.670 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:00.670 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:00.670 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:00.670 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.670 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72139 2024-09-18T04:58:00.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.671 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:00.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:00.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:58:00.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:00.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:58:00.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:00.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954561+0000", 2024-09-18T04:58:00.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:00.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:00.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954564+0000", 2024-09-18T04:58:00.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4099999999999998e-06 2024-09-18T04:58:00.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:00.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954570+0000", 2024-09-18T04:58:00.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.178e-06 2024-09-18T04:58:00.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954575+0000", 2024-09-18T04:58:00.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0560000000000004e-06 2024-09-18T04:58:00.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.673 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:00.673 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003114+0000", 2024-09-18T04:58:00.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048539212999999998 2024-09-18T04:58:00.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:00.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003176+0000", 2024-09-18T04:58:00.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2154000000000002e-05 2024-09-18T04:58:00.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:00.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003280+0000", 2024-09-18T04:58:00.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010421600000000001 2024-09-18T04:58:00.674 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.674 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:00.674 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.675 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72140 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 471040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:00.675 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:58:00.675 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 63.696170500000001, 2024-09-18T04:58:00.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048670819999999997, 2024-09-18T04:58:00.675 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:00.675 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:00.675 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:00.675 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:00.675 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.675 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72140 2024-09-18T04:58:00.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.675 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:00.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:00.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:58:00.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:00.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:58:00.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:00.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954579+0000", 2024-09-18T04:58:00.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997611 2024-09-18T04:58:00.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:00.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954821+0000", 2024-09-18T04:58:00.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024241000000000001 2024-09-18T04:58:00.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:00.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954831+0000", 2024-09-18T04:58:00.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8770000000000005e-06 2024-09-18T04:58:00.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954838+0000", 2024-09-18T04:58:00.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0690000000000004e-06 2024-09-18T04:58:00.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:00.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003317+0000", 2024-09-18T04:58:00.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048478796999999997 2024-09-18T04:58:00.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:00.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003392+0000", 2024-09-18T04:58:00.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4411000000000003e-05 2024-09-18T04:58:00.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:00.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003488+0000", 2024-09-18T04:58:00.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6927000000000002e-05 2024-09-18T04:58:00.679 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.680 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:00.680 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.680 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72141 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 486400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:00.680 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:58:00.680 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 63.696143016999997, 2024-09-18T04:58:00.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048808114, 2024-09-18T04:58:00.680 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:00.680 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:00.680 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:00.680 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:00.680 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.681 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72141 2024-09-18T04:58:00.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.681 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:00.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:00.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:58:00.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:00.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:58:00.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:00.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:58:00.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:00.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:00.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954847+0000", 2024-09-18T04:58:00.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3800000000000001e-06 2024-09-18T04:58:00.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:00.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954854+0000", 2024-09-18T04:58:00.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6080000000000001e-06 2024-09-18T04:58:00.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954859+0000", 2024-09-18T04:58:00.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:58:00.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:00.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003526+0000", 2024-09-18T04:58:00.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048666433000000002 2024-09-18T04:58:00.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:00.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003599+0000", 2024-09-18T04:58:00.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2854000000000004e-05 2024-09-18T04:58:00.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:00.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003653+0000", 2024-09-18T04:58:00.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4511e-05 2024-09-18T04:58:00.685 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.685 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:00.685 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.685 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72142 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 501760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:00.685 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:58:00.685 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 63.696104214999998, 2024-09-18T04:58:00.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048885500999999998, 2024-09-18T04:58:00.685 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:00.685 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:00.685 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:00.685 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:00.686 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.686 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72142 2024-09-18T04:58:00.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.686 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:00.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:00.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:58:00.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:00.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:58:00.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:00.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954864+0000", 2024-09-18T04:58:00.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999804 2024-09-18T04:58:00.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:00.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954886+0000", 2024-09-18T04:58:00.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2042999999999999e-05 2024-09-18T04:58:00.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:00.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954893+0000", 2024-09-18T04:58:00.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.255e-06 2024-09-18T04:58:00.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954899+0000", 2024-09-18T04:58:00.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8780000000000003e-06 2024-09-18T04:58:00.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:00.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003678+0000", 2024-09-18T04:58:00.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048778834 2024-09-18T04:58:00.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:00.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003717+0000", 2024-09-18T04:58:00.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8860999999999999e-05 2024-09-18T04:58:00.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:00.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003769+0000", 2024-09-18T04:58:00.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2389000000000003e-05 2024-09-18T04:58:00.690 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.690 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:00.690 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.690 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72143 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 517120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:00.690 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:58:00.690 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 63.695978588999999, 2024-09-18T04:58:00.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048951738000000002, 2024-09-18T04:58:00.690 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:00.690 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:00.690 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:00.690 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:00.691 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.691 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72143 2024-09-18T04:58:00.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.691 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:00.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:00.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:58:00.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:00.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:58:00.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:00.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954904+0000", 2024-09-18T04:58:00.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998946 2024-09-18T04:58:00.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:00.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955013+0000", 2024-09-18T04:58:00.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108977 2024-09-18T04:58:00.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:00.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955021+0000", 2024-09-18T04:58:00.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7570000000000004e-06 2024-09-18T04:58:00.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955025+0000", 2024-09-18T04:58:00.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1219999999999997e-06 2024-09-18T04:58:00.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:00.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003792+0000", 2024-09-18T04:58:00.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048767379 2024-09-18T04:58:00.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:00.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003860+0000", 2024-09-18T04:58:00.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8033000000000004e-05 2024-09-18T04:58:00.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:00.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003961+0000", 2024-09-18T04:58:00.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010103300000000001 2024-09-18T04:58:00.695 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.695 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:00.695 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.695 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72161 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 13312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:00.695 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:58:00.695 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 63.695714952000003, 2024-09-18T04:58:00.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048891539999999997, 2024-09-18T04:58:00.695 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:00.695 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:00.696 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:00.696 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:00.696 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.696 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72161 2024-09-18T04:58:00.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.696 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:00.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:00.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:58:00.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:00.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:58:00.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:00.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955028+0000", 2024-09-18T04:58:00.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997549 2024-09-18T04:58:00.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:00.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955278+0000", 2024-09-18T04:58:00.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024973299999999998 2024-09-18T04:58:00.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:00.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955289+0000", 2024-09-18T04:58:00.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0812000000000001e-05 2024-09-18T04:58:00.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955296+0000", 2024-09-18T04:58:00.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2570000000000001e-06 2024-09-18T04:58:00.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:00.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004002+0000", 2024-09-18T04:58:00.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048705904000000001 2024-09-18T04:58:00.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:00.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004076+0000", 2024-09-18T04:58:00.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4086999999999997e-05 2024-09-18T04:58:00.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.700 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:00.700 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004165+0000", 2024-09-18T04:58:00.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8712e-05 2024-09-18T04:58:00.700 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.700 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:00.700 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.700 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72163 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 44032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:00.700 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:58:00.700 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 63.695089514000003, 2024-09-18T04:58:00.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048567011, 2024-09-18T04:58:00.701 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:00.701 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:00.701 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:00.701 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:00.701 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.701 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72163 2024-09-18T04:58:00.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.701 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:00.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:00.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:58:00.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:00.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:58:00.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:00.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955898+0000", 2024-09-18T04:58:00.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:00.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:00.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955901+0000", 2024-09-18T04:58:00.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.706e-06 2024-09-18T04:58:00.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:00.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955907+0000", 2024-09-18T04:58:00.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3559999999999998e-06 2024-09-18T04:58:00.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955912+0000", 2024-09-18T04:58:00.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9640000000000002e-06 2024-09-18T04:58:00.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:00.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004377+0000", 2024-09-18T04:58:00.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048465293999999999 2024-09-18T04:58:00.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:00.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004418+0000", 2024-09-18T04:58:00.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0163000000000002e-05 2024-09-18T04:58:00.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:00.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004466+0000", 2024-09-18T04:58:00.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8004000000000003e-05 2024-09-18T04:58:00.705 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.705 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:00.705 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.705 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72164 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:00.705 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:58:00.705 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 63.695071241000001, 2024-09-18T04:58:00.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048707253999999998, 2024-09-18T04:58:00.706 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:00.706 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:00.706 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:00.706 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:00.706 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.706 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72164 2024-09-18T04:58:00.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.706 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:00.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:00.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:58:00.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:00.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:58:00.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:00.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:58:00.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:00.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:00.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955919+0000", 2024-09-18T04:58:00.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2520000000000002e-06 2024-09-18T04:58:00.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:00.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955925+0000", 2024-09-18T04:58:00.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3740000000000003e-06 2024-09-18T04:58:00.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955930+0000", 2024-09-18T04:58:00.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:58:00.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:00.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004489+0000", 2024-09-18T04:58:00.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048558581000000003 2024-09-18T04:58:00.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:00.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004529+0000", 2024-09-18T04:58:00.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9722999999999999e-05 2024-09-18T04:58:00.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:00.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004624+0000", 2024-09-18T04:58:00.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5402999999999994e-05 2024-09-18T04:58:00.710 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.710 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:00.710 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.710 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72165 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 74752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:00.710 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:58:00.710 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 63.694796382, 2024-09-18T04:58:00.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048633223000000003, 2024-09-18T04:58:00.710 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:00.710 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:00.711 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:00.711 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:00.711 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.711 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72165 2024-09-18T04:58:00.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.711 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:00.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:00.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:58:00.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:00.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:58:00.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:00.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955935+0000", 2024-09-18T04:58:00.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999743 2024-09-18T04:58:00.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:00.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956199+0000", 2024-09-18T04:58:00.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000263906 2024-09-18T04:58:00.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:00.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956207+0000", 2024-09-18T04:58:00.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2519999999999999e-06 2024-09-18T04:58:00.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956214+0000", 2024-09-18T04:58:00.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7610000000000002e-06 2024-09-18T04:58:00.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:00.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004661+0000", 2024-09-18T04:58:00.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048447967000000002 2024-09-18T04:58:00.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:00.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004735+0000", 2024-09-18T04:58:00.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3789999999999997e-05 2024-09-18T04:58:00.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:00.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004825+0000", 2024-09-18T04:58:00.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9724999999999999e-05 2024-09-18T04:58:00.715 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.715 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:00.715 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.715 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72166 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 90112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:00.715 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:58:00.715 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 63.694542847999998, 2024-09-18T04:58:00.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048595737999999999, 2024-09-18T04:58:00.716 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:00.716 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:00.716 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:00.716 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:00.716 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.716 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72166 2024-09-18T04:58:00.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.716 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:00.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:00.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:58:00.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:00.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:58:00.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:00.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956221+0000", 2024-09-18T04:58:00.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997759 2024-09-18T04:58:00.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:00.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956448+0000", 2024-09-18T04:58:00.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022685500000000001 2024-09-18T04:58:00.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:00.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956456+0000", 2024-09-18T04:58:00.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0099999999999995e-06 2024-09-18T04:58:00.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956462+0000", 2024-09-18T04:58:00.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7200000000000003e-06 2024-09-18T04:58:00.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:00.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004862+0000", 2024-09-18T04:58:00.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048399795000000002 2024-09-18T04:58:00.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:00.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004941+0000", 2024-09-18T04:58:00.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9414999999999998e-05 2024-09-18T04:58:00.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:00.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005041+0000", 2024-09-18T04:58:00.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9942e-05 2024-09-18T04:58:00.721 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.722 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:00.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.722 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72167 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:00.722 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:58:00.722 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 63.694455810000001, 2024-09-18T04:58:00.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048622894999999999, 2024-09-18T04:58:00.722 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:00.722 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:00.722 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:00.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:00.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.723 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72167 2024-09-18T04:58:00.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.723 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:00.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:00.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:58:00.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:00.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:58:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956468+0000", 2024-09-18T04:58:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999352 2024-09-18T04:58:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956535+0000", 2024-09-18T04:58:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7440999999999993e-05 2024-09-18T04:58:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956544+0000", 2024-09-18T04:58:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1039999999999996e-06 2024-09-18T04:58:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956550+0000", 2024-09-18T04:58:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9850000000000003e-06 2024-09-18T04:58:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005066+0000", 2024-09-18T04:58:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048515614999999998 2024-09-18T04:58:00.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005107+0000", 2024-09-18T04:58:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0843999999999997e-05 2024-09-18T04:58:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005155+0000", 2024-09-18T04:58:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8686000000000001e-05 2024-09-18T04:58:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.727 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72168 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:00.727 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:58:00.727 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 63.694294747000001, 2024-09-18T04:58:00.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048585179999999999, 2024-09-18T04:58:00.727 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:00.727 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:00.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:00.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:00.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.727 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72168 2024-09-18T04:58:00.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.728 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:00.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:00.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:58:00.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:00.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:58:00.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956555+0000", 2024-09-18T04:58:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998612 2024-09-18T04:58:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956696+0000", 2024-09-18T04:58:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000140922 2024-09-18T04:58:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:00.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956707+0000", 2024-09-18T04:58:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1555e-05 2024-09-18T04:58:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956714+0000", 2024-09-18T04:58:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2709999999999998e-06 2024-09-18T04:58:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005177+0000", 2024-09-18T04:58:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048463421 2024-09-18T04:58:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005214+0000", 2024-09-18T04:58:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.676e-05 2024-09-18T04:58:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005279+0000", 2024-09-18T04:58:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4899000000000003e-05 2024-09-18T04:58:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72172 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 441344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:00.731 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.957556+0000", 2024-09-18T04:58:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 63.693432346999998, 2024-09-18T04:58:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048545056000000003, 2024-09-18T04:58:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72172 2024-09-18T04:58:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957556+0000", 2024-09-18T04:58:00.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957556+0000", 2024-09-18T04:58:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957555+0000", 2024-09-18T04:58:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:00.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957558+0000", 2024-09-18T04:58:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2900000000000001e-06 2024-09-18T04:58:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957564+0000", 2024-09-18T04:58:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0700000000000003e-06 2024-09-18T04:58:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957569+0000", 2024-09-18T04:58:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7870000000000001e-06 2024-09-18T04:58:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005922+0000", 2024-09-18T04:58:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048353259000000003 2024-09-18T04:58:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005999+0000", 2024-09-18T04:58:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6889000000000002e-05 2024-09-18T04:58:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.006101+0000", 2024-09-18T04:58:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102136 2024-09-18T04:58:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.735 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72199 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:58:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 63.691762855, 2024-09-18T04:58:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048626057, 2024-09-18T04:58:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72199 2024-09-18T04:58:00.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:58:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:58:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959195+0000", 2024-09-18T04:58:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999969 2024-09-18T04:58:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959228+0000", 2024-09-18T04:58:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3609000000000003e-05 2024-09-18T04:58:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959236+0000", 2024-09-18T04:58:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.157e-06 2024-09-18T04:58:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959243+0000", 2024-09-18T04:58:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5910000000000001e-06 2024-09-18T04:58:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007684+0000", 2024-09-18T04:58:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048440745 2024-09-18T04:58:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007762+0000", 2024-09-18T04:58:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8360000000000002e-05 2024-09-18T04:58:00.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007851+0000", 2024-09-18T04:58:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9388000000000005e-05 2024-09-18T04:58:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72240 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 428032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:58:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 63.687669544999999, 2024-09-18T04:58:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048643767999999997, 2024-09-18T04:58:00.740 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72240 2024-09-18T04:58:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:58:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:58:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963240+0000", 2024-09-18T04:58:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999218 2024-09-18T04:58:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963323+0000", 2024-09-18T04:58:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2375999999999999e-05 2024-09-18T04:58:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963332+0000", 2024-09-18T04:58:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1700000000000003e-06 2024-09-18T04:58:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963338+0000", 2024-09-18T04:58:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2149999999999998e-06 2024-09-18T04:58:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011785+0000", 2024-09-18T04:58:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048446493 2024-09-18T04:58:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011861+0000", 2024-09-18T04:58:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6472999999999998e-05 2024-09-18T04:58:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011962+0000", 2024-09-18T04:58:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101167 2024-09-18T04:58:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.744 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:117375 3.1 3:80b18d38:::rbd_header.11acae5d5365:head [watch reconnect cookie 140406307615440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T04:58:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:58:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.266595008000003, 2024-09-18T04:58:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.018908156, 2024-09-18T04:58:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117375 2024-09-18T04:58:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:00.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:58:00.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:00.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:58:00.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:00.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384391+0000", 2024-09-18T04:58:00.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:58:00.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:00.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384401+0000", 2024-09-18T04:58:00.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4299999999999995e-06 2024-09-18T04:58:00.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384418+0000", 2024-09-18T04:58:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6745e-05 2024-09-18T04:58:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384429+0000", 2024-09-18T04:58:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1122e-05 2024-09-18T04:58:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384496+0000", 2024-09-18T04:58:00.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6841000000000006e-05 2024-09-18T04:58:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-18T04:58:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384505+0000", 2024-09-18T04:58:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4080000000000004e-06 2024-09-18T04:58:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.388971+0000", 2024-09-18T04:58:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0044660000000000004 2024-09-18T04:58:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:00.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.388981+0000", 2024-09-18T04:58:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0118e-05 2024-09-18T04:58:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.399433+0000", 2024-09-18T04:58:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.010452055 2024-09-18T04:58:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.399594+0000", 2024-09-18T04:58:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000160669 2024-09-18T04:58:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.403301+0000", 2024-09-18T04:58:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037074370000000001 2024-09-18T04:58:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:117376 3.1 3:80b18d38:::rbd_header.11acae5d5365:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T04:58:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:58:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 37.264543357999997, 2024-09-18T04:58:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0196134539999999, 2024-09-18T04:58:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:00.752 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117376 2024-09-18T04:58:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:58:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:58:00.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386443+0000", 2024-09-18T04:58:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:58:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386457+0000", 2024-09-18T04:58:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4207e-05 2024-09-18T04:58:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:00.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386478+0000", 2024-09-18T04:58:00.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0349000000000001e-05 2024-09-18T04:58:00.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:00.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386503+0000", 2024-09-18T04:58:00.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5582e-05 2024-09-18T04:58:00.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:00.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386558+0000", 2024-09-18T04:58:00.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4892999999999998e-05 2024-09-18T04:58:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-18T04:58:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386562+0000", 2024-09-18T04:58:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.313e-06 2024-09-18T04:58:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.389010+0000", 2024-09-18T04:58:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024480890000000001 2024-09-18T04:58:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:00.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.389018+0000", 2024-09-18T04:58:00.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8469999999999996e-06 2024-09-18T04:58:00.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:00.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400073+0000", 2024-09-18T04:58:00.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.011054286 2024-09-18T04:58:00.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400124+0000", 2024-09-18T04:58:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1332999999999998e-05 2024-09-18T04:58:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:58:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400477+0000", 2024-09-18T04:58:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035329300000000002 2024-09-18T04:58:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:00.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.401353+0000", 2024-09-18T04:58:00.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00087584200000000005 2024-09-18T04:58:00.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:00.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406006+0000", 2024-09-18T04:58:00.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0046534250000000001 2024-09-18T04:58:00.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:00.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406037+0000", 2024-09-18T04:58:00.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0559000000000003e-05 2024-09-18T04:58:00.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:00.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:00.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:00.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406058+0000", 2024-09-18T04:58:00.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1260999999999999e-05 2024-09-18T04:58:00.761 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.761 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:00.761 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.761 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:00.761 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:00.761 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:00.762 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:00.760+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:00.763 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:58:00.863 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:00.861+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:00.906 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:00.906 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:00.906 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:58:00.906 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:00.914 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:58:00.998 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:00.996+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:01.057 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:58:01.057 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:58:01.057 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:01.066 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:58:01.066 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:01.064+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:01.100 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T04:58:01.167 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:01.164+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:01.215 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:01.215 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:58:01.215 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:58:01.215 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:01.215 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.215 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60344 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1456128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:01.215 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:58:01.216 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.100045007999995, 2024-09-18T04:58:01.216 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044026324999999998, 2024-09-18T04:58:01.216 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.216 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.216 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.216 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.216 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.216 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60344 2024-09-18T04:58:01.216 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.216 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.216 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.216 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.216 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:58:01.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:58:01.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.217 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:58:01.217 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.217 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.217 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.217 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112224+0000", 2024-09-18T04:58:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1320000000000001e-06 2024-09-18T04:58:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112230+0000", 2024-09-18T04:58:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9619999999999998e-06 2024-09-18T04:58:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112234+0000", 2024-09-18T04:58:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7890000000000002e-06 2024-09-18T04:58:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.218 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156105+0000", 2024-09-18T04:58:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043870531999999997 2024-09-18T04:58:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156170+0000", 2024-09-18T04:58:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5036000000000005e-05 2024-09-18T04:58:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156248+0000", 2024-09-18T04:58:01.219 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8295999999999997e-05 2024-09-18T04:58:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60345 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1471488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:58:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.100028242999997, 2024-09-18T04:58:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044180090999999998, 2024-09-18T04:58:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.220 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60345 2024-09-18T04:58:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:58:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:58:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.221 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112238+0000", 2024-09-18T04:58:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112240+0000", 2024-09-18T04:58:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9910000000000001e-06 2024-09-18T04:58:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112246+0000", 2024-09-18T04:58:01.222 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0859999999999998e-06 2024-09-18T04:58:01.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112251+0000", 2024-09-18T04:58:01.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5020000000000003e-06 2024-09-18T04:58:01.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156274+0000", 2024-09-18T04:58:01.223 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044023063000000001 2024-09-18T04:58:01.223 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.223 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.223 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.223 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156337+0000", 2024-09-18T04:58:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3077000000000006e-05 2024-09-18T04:58:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156419+0000", 2024-09-18T04:58:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1699000000000005e-05 2024-09-18T04:58:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60346 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1486848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:58:01.224 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.100010462, 2024-09-18T04:58:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044336331999999999, 2024-09-18T04:58:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60346 2024-09-18T04:58:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:58:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.225 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:58:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112256+0000", 2024-09-18T04:58:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112258+0000", 2024-09-18T04:58:01.226 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1720000000000001e-06 2024-09-18T04:58:01.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112265+0000", 2024-09-18T04:58:01.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4280000000000001e-06 2024-09-18T04:58:01.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112269+0000", 2024-09-18T04:58:01.227 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.741e-06 2024-09-18T04:58:01.227 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.227 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.227 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.227 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156445+0000", 2024-09-18T04:58:01.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044175341 2024-09-18T04:58:01.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156512+0000", 2024-09-18T04:58:01.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6723000000000005e-05 2024-09-18T04:58:01.228 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.228 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.228 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.228 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156593+0000", 2024-09-18T04:58:01.228 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1284000000000003e-05 2024-09-18T04:58:01.228 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.228 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.228 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.229 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.229 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60347 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1502208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:01.229 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:58:01.229 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.099982050999998, 2024-09-18T04:58:01.229 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044482560999999997, 2024-09-18T04:58:01.229 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.229 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.229 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.229 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.229 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.229 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60347 2024-09-18T04:58:01.229 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.230 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:58:01.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:58:01.230 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.230 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.230 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.230 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.230 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112284+0000", 2024-09-18T04:58:01.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112286+0000", 2024-09-18T04:58:01.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9709999999999998e-06 2024-09-18T04:58:01.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.231 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.231 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112290+0000", 2024-09-18T04:58:01.231 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9199999999999997e-06 2024-09-18T04:58:01.231 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.231 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112293+0000", 2024-09-18T04:58:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870000000000001e-06 2024-09-18T04:58:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156617+0000", 2024-09-18T04:58:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044323960000000003 2024-09-18T04:58:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156687+0000", 2024-09-18T04:58:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0141000000000005e-05 2024-09-18T04:58:01.232 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.233 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.233 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156767+0000", 2024-09-18T04:58:01.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0140000000000002e-05 2024-09-18T04:58:01.233 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.233 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.233 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.233 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.233 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.233 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60348 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1517568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:01.233 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:58:01.233 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.099971046999997, 2024-09-18T04:58:01.233 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044645562, 2024-09-18T04:58:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60348 2024-09-18T04:58:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:58:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.234 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:58:01.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:58:01.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.235 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.235 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.235 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112297+0000", 2024-09-18T04:58:01.235 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.384e-06 2024-09-18T04:58:01.235 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112301+0000", 2024-09-18T04:58:01.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0570000000000003e-06 2024-09-18T04:58:01.236 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.236 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.236 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.236 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112304+0000", 2024-09-18T04:58:01.236 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6000000000000001e-06 2024-09-18T04:58:01.237 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156793+0000", 2024-09-18T04:58:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044489494999999997 2024-09-18T04:58:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156859+0000", 2024-09-18T04:58:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5483999999999998e-05 2024-09-18T04:58:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156941+0000", 2024-09-18T04:58:01.238 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2856999999999996e-05 2024-09-18T04:58:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60349 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1532928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:58:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.099961070999996, 2024-09-18T04:58:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044832167999999999, 2024-09-18T04:58:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.239 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60349 2024-09-18T04:58:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:58:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:58:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.240 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:58:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112307+0000", 2024-09-18T04:58:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.308e-06 2024-09-18T04:58:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112310+0000", 2024-09-18T04:58:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-18T04:58:01.241 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112312+0000", 2024-09-18T04:58:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4889999999999998e-06 2024-09-18T04:58:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156977+0000", 2024-09-18T04:58:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044664598999999999 2024-09-18T04:58:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157053+0000", 2024-09-18T04:58:01.242 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6014999999999996e-05 2024-09-18T04:58:01.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.243 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.243 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157138+0000", 2024-09-18T04:58:01.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5162999999999998e-05 2024-09-18T04:58:01.243 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.243 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.243 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.243 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.243 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.243 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60350 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1548288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:01.243 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:58:01.243 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.099952157000004, 2024-09-18T04:58:01.243 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045011858000000002, 2024-09-18T04:58:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60350 2024-09-18T04:58:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:58:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.244 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:58:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:58:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112316+0000", 2024-09-18T04:58:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7579999999999999e-06 2024-09-18T04:58:01.245 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112321+0000", 2024-09-18T04:58:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1649999999999997e-06 2024-09-18T04:58:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112325+0000", 2024-09-18T04:58:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.726e-06 2024-09-18T04:58:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157172+0000", 2024-09-18T04:58:01.246 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.04484718 2024-09-18T04:58:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157242+0000", 2024-09-18T04:58:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9675e-05 2024-09-18T04:58:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157327+0000", 2024-09-18T04:58:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4614e-05 2024-09-18T04:58:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.247 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.248 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.248 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60351 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1563648~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:01.248 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:58:01.248 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.099938741000003, 2024-09-18T04:58:01.248 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045132113000000001, 2024-09-18T04:58:01.248 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.248 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.248 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60351 2024-09-18T04:58:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:58:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:58:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.249 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:58:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112330+0000", 2024-09-18T04:58:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1660000000000001e-06 2024-09-18T04:58:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112336+0000", 2024-09-18T04:58:01.250 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5940000000000001e-06 2024-09-18T04:58:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112340+0000", 2024-09-18T04:58:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5589999999999999e-06 2024-09-18T04:58:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157365+0000", 2024-09-18T04:58:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045024675 2024-09-18T04:58:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.251 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157409+0000", 2024-09-18T04:58:01.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4356e-05 2024-09-18T04:58:01.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.252 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.252 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.252 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157460+0000", 2024-09-18T04:58:01.252 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1072000000000002e-05 2024-09-18T04:58:01.252 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.252 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.252 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.252 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.253 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.253 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60501 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1494016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:01.253 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:58:01.253 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.093878044999997, 2024-09-18T04:58:01.253 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044007728000000003, 2024-09-18T04:58:01.253 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.253 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.253 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.253 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.253 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.253 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60501 2024-09-18T04:58:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:58:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:58:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.254 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:58:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118391+0000", 2024-09-18T04:58:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:58:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118396+0000", 2024-09-18T04:58:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2680000000000002e-06 2024-09-18T04:58:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.255 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118400+0000", 2024-09-18T04:58:01.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2080000000000002e-06 2024-09-18T04:58:01.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162317+0000", 2024-09-18T04:58:01.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043916753000000003 2024-09-18T04:58:01.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.256 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.256 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.256 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162351+0000", 2024-09-18T04:58:01.256 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4372000000000002e-05 2024-09-18T04:58:01.256 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.257 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.257 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162397+0000", 2024-09-18T04:58:01.257 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5207999999999998e-05 2024-09-18T04:58:01.257 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.257 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.257 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.257 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.257 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.257 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60502 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1509376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:01.258 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:58:01.258 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.093862568000006, 2024-09-18T04:58:01.258 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044092438999999997, 2024-09-18T04:58:01.258 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.258 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.258 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.258 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.258 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.258 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60502 2024-09-18T04:58:01.258 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.258 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.258 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.258 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:58:01.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:58:01.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.259 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.259 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:58:01.259 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.259 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.259 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118406+0000", 2024-09-18T04:58:01.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:58:01.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118412+0000", 2024-09-18T04:58:01.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.524e-06 2024-09-18T04:58:01.260 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.260 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.260 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.260 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118416+0000", 2024-09-18T04:58:01.260 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:58:01.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162418+0000", 2024-09-18T04:58:01.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001508000000002 2024-09-18T04:58:01.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162451+0000", 2024-09-18T04:58:01.261 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3662999999999999e-05 2024-09-18T04:58:01.261 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.261 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.261 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.261 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162497+0000", 2024-09-18T04:58:01.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5550000000000003e-05 2024-09-18T04:58:01.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.262 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.262 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.262 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.262 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.262 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60503 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1524736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:01.262 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:58:01.262 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.093846409999998, 2024-09-18T04:58:01.262 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044178780000000001, 2024-09-18T04:58:01.262 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.262 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.262 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.262 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.263 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.263 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60503 2024-09-18T04:58:01.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.263 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:58:01.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.263 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.263 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.263 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:58:01.263 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.263 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118420+0000", 2024-09-18T04:58:01.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118422+0000", 2024-09-18T04:58:01.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.08e-06 2024-09-18T04:58:01.264 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.264 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.264 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.264 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118428+0000", 2024-09-18T04:58:01.264 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.964e-06 2024-09-18T04:58:01.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118433+0000", 2024-09-18T04:58:01.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.425e-06 2024-09-18T04:58:01.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162519+0000", 2024-09-18T04:58:01.265 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044086082999999998 2024-09-18T04:58:01.265 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.265 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.265 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.265 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162552+0000", 2024-09-18T04:58:01.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3719e-05 2024-09-18T04:58:01.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.266 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.266 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162599+0000", 2024-09-18T04:58:01.266 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6873999999999999e-05 2024-09-18T04:58:01.266 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.266 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.266 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.266 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.266 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.266 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60504 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1540096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:01.266 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:58:01.266 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.093830304999997, 2024-09-18T04:58:01.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044261921000000003, 2024-09-18T04:58:01.267 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.267 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.267 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60504 2024-09-18T04:58:01.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.267 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.267 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.267 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:58:01.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.267 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:58:01.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118436+0000", 2024-09-18T04:58:01.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118438+0000", 2024-09-18T04:58:01.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0250000000000001e-06 2024-09-18T04:58:01.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118444+0000", 2024-09-18T04:58:01.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4360000000000001e-06 2024-09-18T04:58:01.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118448+0000", 2024-09-18T04:58:01.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.357e-06 2024-09-18T04:58:01.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162620+0000", 2024-09-18T04:58:01.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044171821 2024-09-18T04:58:01.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162653+0000", 2024-09-18T04:58:01.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3479999999999998e-05 2024-09-18T04:58:01.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162699+0000", 2024-09-18T04:58:01.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5113e-05 2024-09-18T04:58:01.270 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.270 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.271 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.271 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60584 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1196032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:01.271 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:58:01.271 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.092037288, 2024-09-18T04:58:01.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044048727000000003, 2024-09-18T04:58:01.271 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.271 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.271 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.271 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60584 2024-09-18T04:58:01.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.272 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:58:01.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:58:01.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120229+0000", 2024-09-18T04:58:01.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120231+0000", 2024-09-18T04:58:01.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:58:01.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120237+0000", 2024-09-18T04:58:01.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8379999999999998e-06 2024-09-18T04:58:01.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120242+0000", 2024-09-18T04:58:01.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3889999999999998e-06 2024-09-18T04:58:01.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164187+0000", 2024-09-18T04:58:01.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043945045000000002 2024-09-18T04:58:01.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.274 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164227+0000", 2024-09-18T04:58:01.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0627999999999998e-05 2024-09-18T04:58:01.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164278+0000", 2024-09-18T04:58:01.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1146000000000001e-05 2024-09-18T04:58:01.275 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.275 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.275 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.275 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60585 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1211392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:01.275 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:58:01.275 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.092021180000003, 2024-09-18T04:58:01.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044389814, 2024-09-18T04:58:01.276 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.276 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.276 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.276 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.276 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.276 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60585 2024-09-18T04:58:01.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.276 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:58:01.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:58:01.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120245+0000", 2024-09-18T04:58:01.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120247+0000", 2024-09-18T04:58:01.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0150000000000002e-06 2024-09-18T04:58:01.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120255+0000", 2024-09-18T04:58:01.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.858e-06 2024-09-18T04:58:01.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120260+0000", 2024-09-18T04:58:01.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6530000000000002e-06 2024-09-18T04:58:01.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164489+0000", 2024-09-18T04:58:01.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044229483999999999 2024-09-18T04:58:01.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164555+0000", 2024-09-18T04:58:01.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.533e-05 2024-09-18T04:58:01.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164636+0000", 2024-09-18T04:58:01.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0819e-05 2024-09-18T04:58:01.279 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.279 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.279 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.280 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60586 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1226752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:01.280 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:58:01.280 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.092002729000001, 2024-09-18T04:58:01.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044578214999999997, 2024-09-18T04:58:01.280 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.280 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.280 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.280 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.280 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.280 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60586 2024-09-18T04:58:01.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.280 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:58:01.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:58:01.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:58:01.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120266+0000", 2024-09-18T04:58:01.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0660000000000002e-06 2024-09-18T04:58:01.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120272+0000", 2024-09-18T04:58:01.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-18T04:58:01.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120276+0000", 2024-09-18T04:58:01.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.476e-06 2024-09-18T04:58:01.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164673+0000", 2024-09-18T04:58:01.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044396672999999998 2024-09-18T04:58:01.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164749+0000", 2024-09-18T04:58:01.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5811999999999999e-05 2024-09-18T04:58:01.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164842+0000", 2024-09-18T04:58:01.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3560000000000005e-05 2024-09-18T04:58:01.284 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.284 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.284 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.284 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60587 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1242112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:01.284 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:58:01.284 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.091953630000006, 2024-09-18T04:58:01.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044634794999999998, 2024-09-18T04:58:01.284 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.284 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.284 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.285 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.285 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.285 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60587 2024-09-18T04:58:01.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.285 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:58:01.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:58:01.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120280+0000", 2024-09-18T04:58:01.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999671 2024-09-18T04:58:01.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120316+0000", 2024-09-18T04:58:01.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5728000000000001e-05 2024-09-18T04:58:01.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120329+0000", 2024-09-18T04:58:01.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2914e-05 2024-09-18T04:58:01.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120335+0000", 2024-09-18T04:58:01.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8289999999999996e-06 2024-09-18T04:58:01.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164858+0000", 2024-09-18T04:58:01.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044523417000000003 2024-09-18T04:58:01.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164896+0000", 2024-09-18T04:58:01.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7969999999999997e-05 2024-09-18T04:58:01.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164948+0000", 2024-09-18T04:58:01.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1996000000000002e-05 2024-09-18T04:58:01.288 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.288 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.288 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.288 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60595 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1364992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:01.288 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:58:01.288 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.090110413999994, 2024-09-18T04:58:01.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001360000000003, 2024-09-18T04:58:01.289 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.289 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.289 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.289 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.289 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.289 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60595 2024-09-18T04:58:01.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.289 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:58:01.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:58:01.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122156+0000", 2024-09-18T04:58:01.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122158+0000", 2024-09-18T04:58:01.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0040000000000002e-06 2024-09-18T04:58:01.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122164+0000", 2024-09-18T04:58:01.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8719999999999999e-06 2024-09-18T04:58:01.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122169+0000", 2024-09-18T04:58:01.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5090000000000004e-06 2024-09-18T04:58:01.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166040+0000", 2024-09-18T04:58:01.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043871226999999999 2024-09-18T04:58:01.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166096+0000", 2024-09-18T04:58:01.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6495e-05 2024-09-18T04:58:01.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166158+0000", 2024-09-18T04:58:01.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1626999999999998e-05 2024-09-18T04:58:01.292 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.292 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.292 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.293 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60596 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1380352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:01.293 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:58:01.293 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.090094093000005, 2024-09-18T04:58:01.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044120800000000002, 2024-09-18T04:58:01.293 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.293 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.293 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.293 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.293 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.293 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60596 2024-09-18T04:58:01.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.293 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:58:01.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:58:01.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:58:01.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122175+0000", 2024-09-18T04:58:01.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:58:01.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122182+0000", 2024-09-18T04:58:01.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3509999999999996e-06 2024-09-18T04:58:01.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122187+0000", 2024-09-18T04:58:01.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7940000000000002e-06 2024-09-18T04:58:01.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166184+0000", 2024-09-18T04:58:01.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043997236000000002 2024-09-18T04:58:01.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166227+0000", 2024-09-18T04:58:01.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2561000000000001e-05 2024-09-18T04:58:01.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166294+0000", 2024-09-18T04:58:01.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7068000000000002e-05 2024-09-18T04:58:01.297 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.297 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.297 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.297 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60597 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1395712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:01.297 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:58:01.297 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.090076065000005, 2024-09-18T04:58:01.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044242166999999999, 2024-09-18T04:58:01.297 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.297 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.297 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.298 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.298 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.298 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60597 2024-09-18T04:58:01.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.298 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:58:01.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:58:01.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:58:01.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122193+0000", 2024-09-18T04:58:01.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:58:01.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122199+0000", 2024-09-18T04:58:01.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6449999999999999e-06 2024-09-18T04:58:01.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122204+0000", 2024-09-18T04:58:01.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4819999999999996e-06 2024-09-18T04:58:01.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166329+0000", 2024-09-18T04:58:01.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044125059000000001 2024-09-18T04:58:01.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166370+0000", 2024-09-18T04:58:01.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1012e-05 2024-09-18T04:58:01.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166433+0000", 2024-09-18T04:58:01.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3178999999999999e-05 2024-09-18T04:58:01.301 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.301 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.301 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.301 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60608 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1564672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:01.301 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:58:01.302 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.088078202999995, 2024-09-18T04:58:01.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044082723999999997, 2024-09-18T04:58:01.302 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.302 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.302 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.302 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.302 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.302 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60608 2024-09-18T04:58:01.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.302 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:58:01.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:58:01.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124188+0000", 2024-09-18T04:58:01.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124190+0000", 2024-09-18T04:58:01.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0329999999999998e-06 2024-09-18T04:58:01.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124196+0000", 2024-09-18T04:58:01.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.575e-06 2024-09-18T04:58:01.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124201+0000", 2024-09-18T04:58:01.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5000000000000001e-06 2024-09-18T04:58:01.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168166+0000", 2024-09-18T04:58:01.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043965763999999997 2024-09-18T04:58:01.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168218+0000", 2024-09-18T04:58:01.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1785e-05 2024-09-18T04:58:01.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168271+0000", 2024-09-18T04:58:01.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3429e-05 2024-09-18T04:58:01.305 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.306 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.306 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.306 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.306 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.306 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:01.307 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:58:01.308 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:01.305+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:01.408 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:01.406+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:01.451 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:01.451 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:01.451 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:58:01.451 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:01.460 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:58:01.544 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:01.542+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:01.602 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:58:01.602 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:58:01.602 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:01.610 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:01.607+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:01.611 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:58:01.711 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:01.709+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:01.753 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:01.754 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:58:01.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:58:01.754 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:01.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.754 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26189 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 237568~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:01.754 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:58:01.754 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 81.110953136999996, 2024-09-18T04:58:01.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022263431, 2024-09-18T04:58:01.754 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.754 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.754 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.755 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.755 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.755 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26189 2024-09-18T04:58:01.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.755 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:58:01.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:58:01.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:58:01.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639787+0000", 2024-09-18T04:58:01.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9759999999999998e-06 2024-09-18T04:58:01.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639792+0000", 2024-09-18T04:58:01.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5829999999999997e-06 2024-09-18T04:58:01.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639796+0000", 2024-09-18T04:58:01.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0690000000000003e-06 2024-09-18T04:58:01.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.661944+0000", 2024-09-18T04:58:01.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022147463999999999 2024-09-18T04:58:01.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.661985+0000", 2024-09-18T04:58:01.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1360999999999999e-05 2024-09-18T04:58:01.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662048+0000", 2024-09-18T04:58:01.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3298000000000003e-05 2024-09-18T04:58:01.759 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.759 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.759 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.759 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26199 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 4096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:01.759 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:58:01.759 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 81.110937851000003, 2024-09-18T04:58:01.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022348776000000001, 2024-09-18T04:58:01.759 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.759 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.759 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.759 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.760 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.760 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26199 2024-09-18T04:58:01.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.760 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:58:01.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:58:01.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:58:01.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639806+0000", 2024-09-18T04:58:01.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1539999999999999e-06 2024-09-18T04:58:01.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639812+0000", 2024-09-18T04:58:01.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5520000000000004e-06 2024-09-18T04:58:01.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639816+0000", 2024-09-18T04:58:01.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1840000000000001e-06 2024-09-18T04:58:01.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662063+0000", 2024-09-18T04:58:01.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022247125999999999 2024-09-18T04:58:01.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662098+0000", 2024-09-18T04:58:01.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4968000000000001e-05 2024-09-18T04:58:01.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662149+0000", 2024-09-18T04:58:01.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1122000000000003e-05 2024-09-18T04:58:01.763 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.764 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.764 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.764 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26200 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 32768~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:01.764 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:58:01.764 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 81.110918326000004, 2024-09-18T04:58:01.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022501118000000001, 2024-09-18T04:58:01.764 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.765 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.765 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.765 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.765 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.765 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26200 2024-09-18T04:58:01.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.765 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:58:01.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:58:01.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:58:01.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639822+0000", 2024-09-18T04:58:01.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.007e-06 2024-09-18T04:58:01.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639827+0000", 2024-09-18T04:58:01.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5709999999999997e-06 2024-09-18T04:58:01.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639831+0000", 2024-09-18T04:58:01.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2819999999999998e-06 2024-09-18T04:58:01.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662181+0000", 2024-09-18T04:58:01.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022349878 2024-09-18T04:58:01.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662252+0000", 2024-09-18T04:58:01.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0542000000000003e-05 2024-09-18T04:58:01.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662321+0000", 2024-09-18T04:58:01.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9167999999999999e-05 2024-09-18T04:58:01.769 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.770 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.770 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.770 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26201 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 61440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:01.770 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:58:01.770 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 81.110903035000007, 2024-09-18T04:58:01.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022569783, 2024-09-18T04:58:01.772 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.772 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.772 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26201 2024-09-18T04:58:01.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.772 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:58:01.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:58:01.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:58:01.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639837+0000", 2024-09-18T04:58:01.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.928e-06 2024-09-18T04:58:01.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639843+0000", 2024-09-18T04:58:01.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8130000000000001e-06 2024-09-18T04:58:01.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639847+0000", 2024-09-18T04:58:01.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0999999999999997e-06 2024-09-18T04:58:01.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662335+0000", 2024-09-18T04:58:01.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022488207999999999 2024-09-18T04:58:01.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662370+0000", 2024-09-18T04:58:01.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5292e-05 2024-09-18T04:58:01.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662405+0000", 2024-09-18T04:58:01.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4765999999999999e-05 2024-09-18T04:58:01.775 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.775 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.776 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.776 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26202 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 90112~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:01.776 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:58:01.776 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 81.110887618000007, 2024-09-18T04:58:01.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022655937000000001, 2024-09-18T04:58:01.776 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.776 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.776 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.776 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.776 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.776 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26202 2024-09-18T04:58:01.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.777 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:58:01.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:58:01.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639850+0000", 2024-09-18T04:58:01.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639852+0000", 2024-09-18T04:58:01.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9700000000000002e-06 2024-09-18T04:58:01.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639858+0000", 2024-09-18T04:58:01.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.434e-06 2024-09-18T04:58:01.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639862+0000", 2024-09-18T04:58:01.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3499999999999999e-06 2024-09-18T04:58:01.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662429+0000", 2024-09-18T04:58:01.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022567409 2024-09-18T04:58:01.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662469+0000", 2024-09-18T04:58:01.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9801000000000001e-05 2024-09-18T04:58:01.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662507+0000", 2024-09-18T04:58:01.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7311000000000002e-05 2024-09-18T04:58:01.780 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.780 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.780 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.780 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26203 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 118784~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:01.780 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:58:01.780 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 81.110872463000007, 2024-09-18T04:58:01.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022730358999999999, 2024-09-18T04:58:01.780 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.781 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.781 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.781 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26203 2024-09-18T04:58:01.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.781 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:58:01.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:58:01.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639865+0000", 2024-09-18T04:58:01.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639867+0000", 2024-09-18T04:58:01.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0889999999999998e-06 2024-09-18T04:58:01.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639873+0000", 2024-09-18T04:58:01.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4979999999999997e-06 2024-09-18T04:58:01.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639877+0000", 2024-09-18T04:58:01.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4510000000000002e-06 2024-09-18T04:58:01.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662528+0000", 2024-09-18T04:58:01.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022650791999999999 2024-09-18T04:58:01.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662563+0000", 2024-09-18T04:58:01.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4530000000000003e-05 2024-09-18T04:58:01.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662596+0000", 2024-09-18T04:58:01.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3463000000000001e-05 2024-09-18T04:58:01.784 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.784 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.784 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.785 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26204 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 147456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:01.785 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:58:01.785 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 81.110857070999998, 2024-09-18T04:58:01.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022803233999999999, 2024-09-18T04:58:01.785 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.785 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.785 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.785 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.785 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.785 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26204 2024-09-18T04:58:01.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.785 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:58:01.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:58:01.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:58:01.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639883+0000", 2024-09-18T04:58:01.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6520000000000002e-06 2024-09-18T04:58:01.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639888+0000", 2024-09-18T04:58:01.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2870000000000004e-06 2024-09-18T04:58:01.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639893+0000", 2024-09-18T04:58:01.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3839999999999999e-06 2024-09-18T04:58:01.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662617+0000", 2024-09-18T04:58:01.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022723876 2024-09-18T04:58:01.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662651+0000", 2024-09-18T04:58:01.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3797000000000002e-05 2024-09-18T04:58:01.789 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.789 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.789 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.789 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662684+0000", 2024-09-18T04:58:01.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3828999999999997e-05 2024-09-18T04:58:01.789 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.789 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.789 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.790 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26205 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 176128~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:01.790 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:58:01.790 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 81.110834109999999, 2024-09-18T04:58:01.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022919108000000001, 2024-09-18T04:58:01.790 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.790 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.790 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.790 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.791 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.791 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26205 2024-09-18T04:58:01.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.791 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:58:01.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:58:01.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:58:01.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639906+0000", 2024-09-18T04:58:01.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.531e-06 2024-09-18T04:58:01.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639913+0000", 2024-09-18T04:58:01.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.331e-06 2024-09-18T04:58:01.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639920+0000", 2024-09-18T04:58:01.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5399999999999998e-06 2024-09-18T04:58:01.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662706+0000", 2024-09-18T04:58:01.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.02278587 2024-09-18T04:58:01.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662764+0000", 2024-09-18T04:58:01.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7630000000000002e-05 2024-09-18T04:58:01.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662823+0000", 2024-09-18T04:58:01.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9648000000000002e-05 2024-09-18T04:58:01.795 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.795 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.795 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.795 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26206 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 204800~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:01.795 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:58:01.795 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 81.110813711000006, 2024-09-18T04:58:01.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023047517, 2024-09-18T04:58:01.795 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.795 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.795 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.796 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.796 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.796 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26206 2024-09-18T04:58:01.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.796 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:58:01.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:58:01.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:58:01.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639926+0000", 2024-09-18T04:58:01.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3120000000000001e-06 2024-09-18T04:58:01.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639932+0000", 2024-09-18T04:58:01.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4890000000000003e-06 2024-09-18T04:58:01.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639936+0000", 2024-09-18T04:58:01.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.07e-06 2024-09-18T04:58:01.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662850+0000", 2024-09-18T04:58:01.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022913579999999999 2024-09-18T04:58:01.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662903+0000", 2024-09-18T04:58:01.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3176e-05 2024-09-18T04:58:01.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662972+0000", 2024-09-18T04:58:01.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9270999999999995e-05 2024-09-18T04:58:01.799 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.800 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.800 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.800 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26207 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 233472~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:01.800 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:58:01.800 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 81.110798567000003, 2024-09-18T04:58:01.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023139541999999999, 2024-09-18T04:58:01.800 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.800 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.800 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.801 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.801 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.801 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26207 2024-09-18T04:58:01.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.801 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:58:01.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:58:01.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639939+0000", 2024-09-18T04:58:01.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639941+0000", 2024-09-18T04:58:01.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0080000000000001e-06 2024-09-18T04:58:01.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639947+0000", 2024-09-18T04:58:01.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.485e-06 2024-09-18T04:58:01.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639951+0000", 2024-09-18T04:58:01.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4549999999999997e-06 2024-09-18T04:58:01.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663005+0000", 2024-09-18T04:58:01.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023053569999999999 2024-09-18T04:58:01.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663042+0000", 2024-09-18T04:58:01.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7245e-05 2024-09-18T04:58:01.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663079+0000", 2024-09-18T04:58:01.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7119000000000002e-05 2024-09-18T04:58:01.805 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.805 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.805 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.805 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26208 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 524288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:01.805 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:58:01.805 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 81.110781818999996, 2024-09-18T04:58:01.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023283564999999999, 2024-09-18T04:58:01.806 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.806 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.806 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.806 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.806 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.806 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26208 2024-09-18T04:58:01.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.806 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:58:01.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:58:01.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:58:01.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639958+0000", 2024-09-18T04:58:01.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0899999999999999e-06 2024-09-18T04:58:01.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639963+0000", 2024-09-18T04:58:01.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2209999999999996e-06 2024-09-18T04:58:01.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639967+0000", 2024-09-18T04:58:01.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1169999999999997e-06 2024-09-18T04:58:01.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663114+0000", 2024-09-18T04:58:01.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023146102000000002 2024-09-18T04:58:01.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663183+0000", 2024-09-18T04:58:01.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9686999999999999e-05 2024-09-18T04:58:01.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663240+0000", 2024-09-18T04:58:01.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6728000000000003e-05 2024-09-18T04:58:01.809 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.809 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.810 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.810 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26209 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 552960~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:01.810 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:58:01.810 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 81.110764043000003, 2024-09-18T04:58:01.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023440966000000001, 2024-09-18T04:58:01.810 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.810 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.810 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26209 2024-09-18T04:58:01.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.811 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:58:01.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:58:01.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:58:01.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639976+0000", 2024-09-18T04:58:01.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4940000000000002e-06 2024-09-18T04:58:01.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639982+0000", 2024-09-18T04:58:01.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0560000000000001e-06 2024-09-18T04:58:01.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639987+0000", 2024-09-18T04:58:01.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-18T04:58:01.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663278+0000", 2024-09-18T04:58:01.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023290789999999999 2024-09-18T04:58:01.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663348+0000", 2024-09-18T04:58:01.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0837e-05 2024-09-18T04:58:01.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663415+0000", 2024-09-18T04:58:01.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6736000000000006e-05 2024-09-18T04:58:01.814 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.814 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.814 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.814 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26210 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 581632~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:01.814 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:58:01.814 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 81.110747035000003, 2024-09-18T04:58:01.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023616437000000001, 2024-09-18T04:58:01.815 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.815 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.815 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.815 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.815 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.815 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26210 2024-09-18T04:58:01.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.815 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:58:01.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:58:01.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:58:01.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639993+0000", 2024-09-18T04:58:01.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.419e-06 2024-09-18T04:58:01.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640006+0000", 2024-09-18T04:58:01.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2918000000000001e-05 2024-09-18T04:58:01.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640011+0000", 2024-09-18T04:58:01.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9320000000000004e-06 2024-09-18T04:58:01.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663454+0000", 2024-09-18T04:58:01.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023443035000000001 2024-09-18T04:58:01.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663532+0000", 2024-09-18T04:58:01.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7443999999999999e-05 2024-09-18T04:58:01.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663608+0000", 2024-09-18T04:58:01.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6093000000000005e-05 2024-09-18T04:58:01.819 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.819 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.819 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.819 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26211 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 610304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:01.819 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:58:01.819 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 81.110722483000004, 2024-09-18T04:58:01.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023774337999999999, 2024-09-18T04:58:01.820 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.820 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.820 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.820 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.820 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.820 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26211 2024-09-18T04:58:01.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.820 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:58:01.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:58:01.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640015+0000", 2024-09-18T04:58:01.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640023+0000", 2024-09-18T04:58:01.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3900000000000004e-06 2024-09-18T04:58:01.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640036+0000", 2024-09-18T04:58:01.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3091999999999999e-05 2024-09-18T04:58:01.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640041+0000", 2024-09-18T04:58:01.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6219999999999997e-06 2024-09-18T04:58:01.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663644+0000", 2024-09-18T04:58:01.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023602643999999999 2024-09-18T04:58:01.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663718+0000", 2024-09-18T04:58:01.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4481999999999996e-05 2024-09-18T04:58:01.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663790+0000", 2024-09-18T04:58:01.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1589999999999997e-05 2024-09-18T04:58:01.823 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.823 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.823 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.824 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26212 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 638976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:01.824 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:58:01.824 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 81.110689442999998, 2024-09-18T04:58:01.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023924748999999999, 2024-09-18T04:58:01.824 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.824 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.824 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.824 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.824 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.824 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26212 2024-09-18T04:58:01.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.825 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:58:01.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:58:01.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640048+0000", 2024-09-18T04:58:01.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640051+0000", 2024-09-18T04:58:01.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.095e-06 2024-09-18T04:58:01.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640060+0000", 2024-09-18T04:58:01.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8929999999999994e-06 2024-09-18T04:58:01.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640065+0000", 2024-09-18T04:58:01.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0270000000000003e-06 2024-09-18T04:58:01.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663824+0000", 2024-09-18T04:58:01.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023758873999999999 2024-09-18T04:58:01.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663904+0000", 2024-09-18T04:58:01.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9856000000000003e-05 2024-09-18T04:58:01.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663974+0000", 2024-09-18T04:58:01.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9450999999999996e-05 2024-09-18T04:58:01.828 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.829 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.829 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.829 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26213 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 667648~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:01.829 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:58:01.829 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 81.110668271999998, 2024-09-18T04:58:01.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024059812999999999, 2024-09-18T04:58:01.829 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.829 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.829 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.830 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26213 2024-09-18T04:58:01.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.830 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:58:01.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:58:01.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:58:01.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640072+0000", 2024-09-18T04:58:01.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4200000000000001e-06 2024-09-18T04:58:01.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640078+0000", 2024-09-18T04:58:01.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0179999999999998e-06 2024-09-18T04:58:01.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640083+0000", 2024-09-18T04:58:01.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:58:01.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664008+0000", 2024-09-18T04:58:01.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023925287 2024-09-18T04:58:01.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664059+0000", 2024-09-18T04:58:01.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0730999999999999e-05 2024-09-18T04:58:01.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664130+0000", 2024-09-18T04:58:01.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0801000000000002e-05 2024-09-18T04:58:01.834 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.834 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.834 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.834 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26214 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 696320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:01.834 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:58:01.834 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 81.110651301999994, 2024-09-18T04:58:01.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024223738000000002, 2024-09-18T04:58:01.834 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.834 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.834 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.835 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.835 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.835 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26214 2024-09-18T04:58:01.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.835 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:58:01.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:58:01.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:58:01.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640089+0000", 2024-09-18T04:58:01.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0849999999999999e-06 2024-09-18T04:58:01.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640095+0000", 2024-09-18T04:58:01.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9279999999999998e-06 2024-09-18T04:58:01.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640099+0000", 2024-09-18T04:58:01.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:58:01.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664166+0000", 2024-09-18T04:58:01.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024066513000000001 2024-09-18T04:58:01.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664239+0000", 2024-09-18T04:58:01.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3201999999999995e-05 2024-09-18T04:58:01.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664311+0000", 2024-09-18T04:58:01.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1434999999999996e-05 2024-09-18T04:58:01.839 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.839 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.839 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.840 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26215 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 724992~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:01.840 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:58:01.840 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 81.110634540999996, 2024-09-18T04:58:01.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024302284, 2024-09-18T04:58:01.840 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.840 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.840 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.840 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.840 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.840 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26215 2024-09-18T04:58:01.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.840 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:58:01.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:58:01.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640103+0000", 2024-09-18T04:58:01.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640106+0000", 2024-09-18T04:58:01.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2120000000000002e-06 2024-09-18T04:58:01.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640111+0000", 2024-09-18T04:58:01.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7880000000000003e-06 2024-09-18T04:58:01.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640116+0000", 2024-09-18T04:58:01.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6099999999999999e-06 2024-09-18T04:58:01.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664327+0000", 2024-09-18T04:58:01.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024210941999999999 2024-09-18T04:58:01.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664367+0000", 2024-09-18T04:58:01.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9666000000000003e-05 2024-09-18T04:58:01.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664406+0000", 2024-09-18T04:58:01.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9402e-05 2024-09-18T04:58:01.844 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.844 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.844 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.844 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26216 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 753664~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:01.844 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:58:01.844 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 81.110618047000003, 2024-09-18T04:58:01.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024452854999999999, 2024-09-18T04:58:01.844 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.844 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.844 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.845 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.845 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.845 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26216 2024-09-18T04:58:01.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.845 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:58:01.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:58:01.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:58:01.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640122+0000", 2024-09-18T04:58:01.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3599999999999999e-06 2024-09-18T04:58:01.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640128+0000", 2024-09-18T04:58:01.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6339999999999998e-06 2024-09-18T04:58:01.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640132+0000", 2024-09-18T04:58:01.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:58:01.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664436+0000", 2024-09-18T04:58:01.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024303585999999999 2024-09-18T04:58:01.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664507+0000", 2024-09-18T04:58:01.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1046999999999999e-05 2024-09-18T04:58:01.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664573+0000", 2024-09-18T04:58:01.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6217000000000007e-05 2024-09-18T04:58:01.848 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.848 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.848 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.848 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26217 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 782336~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:01.849 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:58:01.849 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 81.110601813000002, 2024-09-18T04:58:01.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024526287000000001, 2024-09-18T04:58:01.849 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:01.849 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:01.849 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:01.849 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:01.849 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:01.849 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26217 2024-09-18T04:58:01.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.849 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:01.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:01.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:58:01.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:01.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:58:01.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:01.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:01.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:58:01.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:01.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:01.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640140+0000", 2024-09-18T04:58:01.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2989999999999998e-06 2024-09-18T04:58:01.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:01.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640146+0000", 2024-09-18T04:58:01.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9569999999999999e-06 2024-09-18T04:58:01.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:01.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640151+0000", 2024-09-18T04:58:01.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:58:01.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:01.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664590+0000", 2024-09-18T04:58:01.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0244389 2024-09-18T04:58:01.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:01.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664628+0000", 2024-09-18T04:58:01.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7910999999999997e-05 2024-09-18T04:58:01.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:01.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:01.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:01.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664663+0000", 2024-09-18T04:58:01.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5024999999999998e-05 2024-09-18T04:58:01.853 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.853 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.853 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.853 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:01.853 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:01.853 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:01.855 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:58:01.855 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:01.853+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:01.955 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:01.953+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:01.997 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:01.997 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:01.997 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:58:01.997 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:02.005 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:58:02.056 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:02.054+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:02.096 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-18T04:58:02.133 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:58:02.157 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:02.157 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:02.157 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:58:02.157 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:58:02.157 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:02.163 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:02.161+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:02.166 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:58:02.258 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:02.256+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:02.273 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:02.273 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:58:02.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:58:02.274 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:02.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.274 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:58:02.274 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:58:02.274 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 110.876849288, 2024-09-18T04:58:02.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037718511000000003, 2024-09-18T04:58:02.274 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:02.274 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:02.274 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:02.274 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:58:02.274 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:58:02.274 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-18T04:58:02.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.275 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:02.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:02.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:58:02.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:02.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:58:02.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:02.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393125+0000", 2024-09-18T04:58:02.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:58:02.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:02.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393133+0000", 2024-09-18T04:58:02.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.225e-06 2024-09-18T04:58:02.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:02.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393144+0000", 2024-09-18T04:58:02.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1279e-05 2024-09-18T04:58:02.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:02.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393153+0000", 2024-09-18T04:58:02.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4540000000000007e-06 2024-09-18T04:58:02.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:02.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393203+0000", 2024-09-18T04:58:02.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9963000000000003e-05 2024-09-18T04:58:02.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:02.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393210+0000", 2024-09-18T04:58:02.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5689999999999999e-06 2024-09-18T04:58:02.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.279 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:02.279 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427399+0000", 2024-09-18T04:58:02.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034188784999999999 2024-09-18T04:58:02.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:02.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427532+0000", 2024-09-18T04:58:02.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013264700000000001 2024-09-18T04:58:02.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:58:02.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427685+0000", 2024-09-18T04:58:02.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015300599999999999 2024-09-18T04:58:02.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:02.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.428520+0000", 2024-09-18T04:58:02.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00083502499999999998 2024-09-18T04:58:02.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:02.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430772+0000", 2024-09-18T04:58:02.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022518880000000001 2024-09-18T04:58:02.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:02.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430817+0000", 2024-09-18T04:58:02.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5077000000000002e-05 2024-09-18T04:58:02.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:02.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430845+0000", 2024-09-18T04:58:02.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8036999999999999e-05 2024-09-18T04:58:02.282 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.282 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:02.282 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.282 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:62520 3.13 3.84cdb893 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T04:58:02.282 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:02.282 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.972539405000006, 2024-09-18T04:58:02.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94452134799999998, 2024-09-18T04:58:02.283 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:02.283 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:58:02.283 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:02.283 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:02.283 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:02.283 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 62520 2024-09-18T04:58:02.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.283 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:02.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:02.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:02.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:02.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:02.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:02.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:02.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:02.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:02.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297441+0000", 2024-09-18T04:58:02.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9840000000000001e-06 2024-09-18T04:58:02.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:02.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297449+0000", 2024-09-18T04:58:02.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3999999999999992e-06 2024-09-18T04:58:02.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:02.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297455+0000", 2024-09-18T04:58:02.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3350000000000003e-06 2024-09-18T04:58:02.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:02.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297552+0000", 2024-09-18T04:58:02.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6602999999999996e-05 2024-09-18T04:58:02.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:02.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297559+0000", 2024-09-18T04:58:02.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.52e-06 2024-09-18T04:58:02.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:02.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241943+0000", 2024-09-18T04:58:02.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94438357100000003 2024-09-18T04:58:02.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:02.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241958+0000", 2024-09-18T04:58:02.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4598e-05 2024-09-18T04:58:02.288 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.288 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:02.288 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.288 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:113179 3.13 3:cf138017:::rbd_data.11ac5f86a3f9.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T04:58:02.288 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:58:02.288 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 43.582748993999999, 2024-09-18T04:58:02.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036508304999999998, 2024-09-18T04:58:02.288 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:02.288 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:02.288 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:02.288 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:02.289 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:02.289 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 113179 2024-09-18T04:58:02.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.289 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:02.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:02.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:58:02.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:02.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:58:02.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:02.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687226+0000", 2024-09-18T04:58:02.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:58:02.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:02.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688169+0000", 2024-09-18T04:58:02.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00094328200000000004 2024-09-18T04:58:02.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:02.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688182+0000", 2024-09-18T04:58:02.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2756e-05 2024-09-18T04:58:02.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:02.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688188+0000", 2024-09-18T04:58:02.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0290000000000002e-06 2024-09-18T04:58:02.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:02.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688225+0000", 2024-09-18T04:58:02.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7301000000000001e-05 2024-09-18T04:58:02.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:02.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688324+0000", 2024-09-18T04:58:02.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9419999999999993e-05 2024-09-18T04:58:02.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:58:02.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.700483+0000", 2024-09-18T04:58:02.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012158771000000001 2024-09-18T04:58:02.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:02.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723586+0000", 2024-09-18T04:58:02.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023102952 2024-09-18T04:58:02.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:02.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723634+0000", 2024-09-18T04:58:02.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7468e-05 2024-09-18T04:58:02.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:02.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723693+0000", 2024-09-18T04:58:02.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9539e-05 2024-09-18T04:58:02.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:02.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723735+0000", 2024-09-18T04:58:02.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1829000000000002e-05 2024-09-18T04:58:02.294 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.294 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:02.294 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.294 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146077 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1400832~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:02.294 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593265+0000", 2024-09-18T04:58:02.295 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.676710616000001, 2024-09-18T04:58:02.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035380397000000001, 2024-09-18T04:58:02.295 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:02.295 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:02.295 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:02.295 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:02.295 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:02.295 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146077 2024-09-18T04:58:02.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.295 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:02.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:02.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593265+0000", 2024-09-18T04:58:02.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:02.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593265+0000", 2024-09-18T04:58:02.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:02.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593265+0000", 2024-09-18T04:58:02.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:02.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:02.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593266+0000", 2024-09-18T04:58:02.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1039999999999999e-06 2024-09-18T04:58:02.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:02.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593269+0000", 2024-09-18T04:58:02.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8760000000000001e-06 2024-09-18T04:58:02.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:02.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593271+0000", 2024-09-18T04:58:02.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5100000000000001e-06 2024-09-18T04:58:02.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:02.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628545+0000", 2024-09-18T04:58:02.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035273977999999998 2024-09-18T04:58:02.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:02.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628589+0000", 2024-09-18T04:58:02.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4088999999999997e-05 2024-09-18T04:58:02.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:02.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628645+0000", 2024-09-18T04:58:02.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6059e-05 2024-09-18T04:58:02.300 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.300 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:02.300 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.300 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146078 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1449984~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:02.300 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:02.300 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.676671151000001, 2024-09-18T04:58:02.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035495325000000001, 2024-09-18T04:58:02.301 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:02.301 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:02.301 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:02.301 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:02.301 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:02.301 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146078 2024-09-18T04:58:02.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.301 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:02.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:02.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:02.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:02.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:02.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:02.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:02.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:02.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:02.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593306+0000", 2024-09-18T04:58:02.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.686e-06 2024-09-18T04:58:02.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:02.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593311+0000", 2024-09-18T04:58:02.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8500000000000002e-06 2024-09-18T04:58:02.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:02.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593314+0000", 2024-09-18T04:58:02.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0290000000000001e-06 2024-09-18T04:58:02.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:02.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628674+0000", 2024-09-18T04:58:02.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035360346000000001 2024-09-18T04:58:02.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:02.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628726+0000", 2024-09-18T04:58:02.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2219999999999998e-05 2024-09-18T04:58:02.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:02.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628800+0000", 2024-09-18T04:58:02.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3613000000000001e-05 2024-09-18T04:58:02.306 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.306 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:02.306 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.306 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146079 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1499136~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:02.306 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:02.306 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.676659745999999, 2024-09-18T04:58:02.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035604581000000003, 2024-09-18T04:58:02.306 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:02.306 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:02.306 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:02.306 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:02.307 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:02.307 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146079 2024-09-18T04:58:02.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.307 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:02.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:02.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:02.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:02.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:02.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:02.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:02.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:02.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:02.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593317+0000", 2024-09-18T04:58:02.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1599999999999999e-06 2024-09-18T04:58:02.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:02.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593320+0000", 2024-09-18T04:58:02.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0130000000000002e-06 2024-09-18T04:58:02.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:02.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593322+0000", 2024-09-18T04:58:02.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6079999999999998e-06 2024-09-18T04:58:02.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:02.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628825+0000", 2024-09-18T04:58:02.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035502747000000001 2024-09-18T04:58:02.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:02.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628868+0000", 2024-09-18T04:58:02.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2719999999999998e-05 2024-09-18T04:58:02.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:02.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628920+0000", 2024-09-18T04:58:02.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2533e-05 2024-09-18T04:58:02.312 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.312 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:02.312 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.313 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146080 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1548288~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:02.313 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:02.313 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.676644247999999, 2024-09-18T04:58:02.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035755146000000002, 2024-09-18T04:58:02.313 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:02.313 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:02.313 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:02.313 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:02.314 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:02.314 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146080 2024-09-18T04:58:02.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.314 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:02.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:02.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:02.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:02.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:02.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:02.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:02.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:02.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:02.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593333+0000", 2024-09-18T04:58:02.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.858e-06 2024-09-18T04:58:02.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:02.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593336+0000", 2024-09-18T04:58:02.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4910000000000001e-06 2024-09-18T04:58:02.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:02.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593340+0000", 2024-09-18T04:58:02.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2590000000000001e-06 2024-09-18T04:58:02.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:02.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628956+0000", 2024-09-18T04:58:02.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035616039000000002 2024-09-18T04:58:02.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:02.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629031+0000", 2024-09-18T04:58:02.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5504000000000001e-05 2024-09-18T04:58:02.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:02.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629086+0000", 2024-09-18T04:58:02.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5429000000000001e-05 2024-09-18T04:58:02.320 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.320 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:02.320 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.321 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146081 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1597440~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:02.321 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:02.321 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.676633676000002, 2024-09-18T04:58:02.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035873223000000003, 2024-09-18T04:58:02.321 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:02.321 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:02.322 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:02.322 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:02.322 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:02.323 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146081 2024-09-18T04:58:02.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.323 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:02.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:02.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:02.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:02.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:02.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:02.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:02.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:02.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:02.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593343+0000", 2024-09-18T04:58:02.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3319999999999999e-06 2024-09-18T04:58:02.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:02.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593346+0000", 2024-09-18T04:58:02.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9059999999999998e-06 2024-09-18T04:58:02.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:02.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593348+0000", 2024-09-18T04:58:02.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4530000000000001e-06 2024-09-18T04:58:02.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:02.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629110+0000", 2024-09-18T04:58:02.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035761814000000003 2024-09-18T04:58:02.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:02.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629159+0000", 2024-09-18T04:58:02.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8971000000000003e-05 2024-09-18T04:58:02.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:02.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629215+0000", 2024-09-18T04:58:02.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6107000000000003e-05 2024-09-18T04:58:02.328 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.328 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:02.328 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.328 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146082 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1646592~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:02.328 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:02.328 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.676617625999999, 2024-09-18T04:58:02.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035987007000000001, 2024-09-18T04:58:02.329 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:02.329 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:02.329 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:02.329 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:02.329 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:02.329 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146082 2024-09-18T04:58:02.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.329 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:02.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:02.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:02.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:02.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:02.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:02.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:02.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:02.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:02.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593360+0000", 2024-09-18T04:58:02.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8560000000000001e-06 2024-09-18T04:58:02.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:02.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593365+0000", 2024-09-18T04:58:02.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3410000000000002e-06 2024-09-18T04:58:02.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:02.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593369+0000", 2024-09-18T04:58:02.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4850000000000003e-06 2024-09-18T04:58:02.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:02.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629242+0000", 2024-09-18T04:58:02.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035872464999999999 2024-09-18T04:58:02.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:02.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629288+0000", 2024-09-18T04:58:02.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6592999999999999e-05 2024-09-18T04:58:02.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:02.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629345+0000", 2024-09-18T04:58:02.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6575e-05 2024-09-18T04:58:02.334 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.334 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:02.334 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.335 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146083 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1695744~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:02.335 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:58:02.335 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.676601778999999, 2024-09-18T04:58:02.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036100587000000003, 2024-09-18T04:58:02.335 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:02.335 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:02.335 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:02.335 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:02.335 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:02.335 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146083 2024-09-18T04:58:02.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.336 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:02.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:02.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:58:02.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:02.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:58:02.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:02.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593373+0000", 2024-09-18T04:58:02.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:02.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:02.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593376+0000", 2024-09-18T04:58:02.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.244e-06 2024-09-18T04:58:02.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:02.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593379+0000", 2024-09-18T04:58:02.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.619e-06 2024-09-18T04:58:02.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:02.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593382+0000", 2024-09-18T04:58:02.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7199999999999998e-06 2024-09-18T04:58:02.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:02.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629368+0000", 2024-09-18T04:58:02.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035986281000000002 2024-09-18T04:58:02.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:02.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629416+0000", 2024-09-18T04:58:02.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8055e-05 2024-09-18T04:58:02.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:02.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629474+0000", 2024-09-18T04:58:02.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8176e-05 2024-09-18T04:58:02.340 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.340 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:02.340 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.340 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146084 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1744896~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:02.340 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:02.340 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.676588648999999, 2024-09-18T04:58:02.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036214349, 2024-09-18T04:58:02.340 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:02.341 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:02.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:02.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:02.341 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:02.341 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146084 2024-09-18T04:58:02.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.341 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:02.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:02.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:02.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:02.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:02.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:02.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:02.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:02.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:02.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593388+0000", 2024-09-18T04:58:02.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.564e-06 2024-09-18T04:58:02.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:02.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593392+0000", 2024-09-18T04:58:02.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0579999999999999e-06 2024-09-18T04:58:02.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:02.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593395+0000", 2024-09-18T04:58:02.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8890000000000002e-06 2024-09-18T04:58:02.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:02.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629500+0000", 2024-09-18T04:58:02.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036105343999999998 2024-09-18T04:58:02.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:02.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629547+0000", 2024-09-18T04:58:02.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7036000000000002e-05 2024-09-18T04:58:02.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:02.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629601+0000", 2024-09-18T04:58:02.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3866999999999998e-05 2024-09-18T04:58:02.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.345 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:02.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.346 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146085 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1794048~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:02.346 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:02.346 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.676578262, 2024-09-18T04:58:02.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036331186000000001, 2024-09-18T04:58:02.346 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:02.346 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:02.347 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:02.347 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:02.347 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:02.347 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146085 2024-09-18T04:58:02.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.347 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:02.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:02.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:02.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:02.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:02.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:02.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:02.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:02.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:02.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593398+0000", 2024-09-18T04:58:02.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2270000000000001e-06 2024-09-18T04:58:02.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:02.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593401+0000", 2024-09-18T04:58:02.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.999e-06 2024-09-18T04:58:02.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:02.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593404+0000", 2024-09-18T04:58:02.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.486e-06 2024-09-18T04:58:02.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:02.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629624+0000", 2024-09-18T04:58:02.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036219983999999997 2024-09-18T04:58:02.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:02.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629672+0000", 2024-09-18T04:58:02.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8368000000000002e-05 2024-09-18T04:58:02.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:02.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629729+0000", 2024-09-18T04:58:02.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6359e-05 2024-09-18T04:58:02.351 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.351 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:02.351 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.352 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146086 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1843200~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:02.352 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:02.352 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.676569698000002, 2024-09-18T04:58:02.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036449098999999999, 2024-09-18T04:58:02.352 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:02.353 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:02.353 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:02.353 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:02.353 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:02.353 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146086 2024-09-18T04:58:02.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.353 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:02.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:02.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:02.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:02.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:02.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:02.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:02.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:02.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:02.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593407+0000", 2024-09-18T04:58:02.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1799999999999999e-06 2024-09-18T04:58:02.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:02.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593410+0000", 2024-09-18T04:58:02.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.926e-06 2024-09-18T04:58:02.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:02.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593413+0000", 2024-09-18T04:58:02.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8250000000000001e-06 2024-09-18T04:58:02.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:02.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629755+0000", 2024-09-18T04:58:02.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036342212999999998 2024-09-18T04:58:02.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:02.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629803+0000", 2024-09-18T04:58:02.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8619000000000003e-05 2024-09-18T04:58:02.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:02.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629855+0000", 2024-09-18T04:58:02.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1536000000000003e-05 2024-09-18T04:58:02.360 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.360 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:02.360 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.361 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146087 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1892352~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:02.361 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:02.361 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.676560772999999, 2024-09-18T04:58:02.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036564608999999998, 2024-09-18T04:58:02.361 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:02.361 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:02.361 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:02.361 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:02.361 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:02.362 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146087 2024-09-18T04:58:02.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.362 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:02.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:02.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:02.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:02.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:02.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:02.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:02.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:02.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:02.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593416+0000", 2024-09-18T04:58:02.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.218e-06 2024-09-18T04:58:02.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:02.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593419+0000", 2024-09-18T04:58:02.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-18T04:58:02.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:02.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593421+0000", 2024-09-18T04:58:02.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4329999999999998e-06 2024-09-18T04:58:02.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:02.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629878+0000", 2024-09-18T04:58:02.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036456578000000003 2024-09-18T04:58:02.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:02.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629925+0000", 2024-09-18T04:58:02.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7595000000000001e-05 2024-09-18T04:58:02.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:02.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629979+0000", 2024-09-18T04:58:02.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4117999999999999e-05 2024-09-18T04:58:02.366 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.366 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:02.366 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.367 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146088 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1941504~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:02.367 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:02.367 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.676538609000001, 2024-09-18T04:58:02.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036676121999999999, 2024-09-18T04:58:02.367 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:02.367 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:02.367 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:02.367 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:02.367 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:02.367 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146088 2024-09-18T04:58:02.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.367 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:02.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:02.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:02.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:02.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:02.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:02.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:02.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:02.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:02.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593438+0000", 2024-09-18T04:58:02.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7570000000000001e-06 2024-09-18T04:58:02.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:02.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593442+0000", 2024-09-18T04:58:02.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2880000000000001e-06 2024-09-18T04:58:02.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:02.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593444+0000", 2024-09-18T04:58:02.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.593e-06 2024-09-18T04:58:02.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:02.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630015+0000", 2024-09-18T04:58:02.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036571125000000003 2024-09-18T04:58:02.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:02.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630061+0000", 2024-09-18T04:58:02.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5445000000000003e-05 2024-09-18T04:58:02.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:02.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630113+0000", 2024-09-18T04:58:02.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2358000000000003e-05 2024-09-18T04:58:02.373 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.373 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:02.373 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.374 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146089 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1990656~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:02.374 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:58:02.374 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.676529028000001, 2024-09-18T04:58:02.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036791483999999999, 2024-09-18T04:58:02.374 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:02.374 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:02.374 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:02.374 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:02.375 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:02.375 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146089 2024-09-18T04:58:02.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.375 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:02.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:02.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:58:02.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:02.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:58:02.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:02.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593446+0000", 2024-09-18T04:58:02.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:02.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:02.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593448+0000", 2024-09-18T04:58:02.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1939999999999999e-06 2024-09-18T04:58:02.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:02.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593451+0000", 2024-09-18T04:58:02.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.766e-06 2024-09-18T04:58:02.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:02.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593458+0000", 2024-09-18T04:58:02.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8539999999999999e-06 2024-09-18T04:58:02.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:02.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630136+0000", 2024-09-18T04:58:02.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03667757 2024-09-18T04:58:02.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:02.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630183+0000", 2024-09-18T04:58:02.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7289000000000001e-05 2024-09-18T04:58:02.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:02.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630238+0000", 2024-09-18T04:58:02.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5013000000000003e-05 2024-09-18T04:58:02.379 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.379 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:02.379 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.379 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146090 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 2039808~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:02.380 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:02.380 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.676515148, 2024-09-18T04:58:02.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036904477999999998, 2024-09-18T04:58:02.380 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:02.380 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:02.381 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:02.381 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:02.381 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:02.381 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146090 2024-09-18T04:58:02.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.381 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:02.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:02.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:02.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:02.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:02.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:02.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:02.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:02.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:02.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593461+0000", 2024-09-18T04:58:02.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3680000000000001e-06 2024-09-18T04:58:02.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:02.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593465+0000", 2024-09-18T04:58:02.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8120000000000001e-06 2024-09-18T04:58:02.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:02.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593468+0000", 2024-09-18T04:58:02.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.582e-06 2024-09-18T04:58:02.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:02.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630268+0000", 2024-09-18T04:58:02.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036800449999999998 2024-09-18T04:58:02.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:02.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630313+0000", 2024-09-18T04:58:02.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4836000000000002e-05 2024-09-18T04:58:02.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:02.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630365+0000", 2024-09-18T04:58:02.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1839999999999998e-05 2024-09-18T04:58:02.389 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.389 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:02.389 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.390 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146091 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 2088960~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:02.390 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:02.390 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.676504686000001, 2024-09-18T04:58:02.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037011534999999998, 2024-09-18T04:58:02.390 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:02.390 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:02.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:02.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:02.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:02.390 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146091 2024-09-18T04:58:02.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.390 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:02.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:02.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:02.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:02.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:02.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:02.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:02.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:02.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:02.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593472+0000", 2024-09-18T04:58:02.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1620000000000001e-06 2024-09-18T04:58:02.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:02.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593475+0000", 2024-09-18T04:58:02.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0019999999999998e-06 2024-09-18T04:58:02.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:02.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593477+0000", 2024-09-18T04:58:02.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-18T04:58:02.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:02.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630381+0000", 2024-09-18T04:58:02.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036904014999999998 2024-09-18T04:58:02.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:02.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630428+0000", 2024-09-18T04:58:02.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6931000000000002e-05 2024-09-18T04:58:02.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:02.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630482+0000", 2024-09-18T04:58:02.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4138999999999998e-05 2024-09-18T04:58:02.395 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.395 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:02.395 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.395 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:162075 3.13 3:cf3f6048:::rbd_header.11ac6967455f:head [watch reconnect cookie 140406508629968 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T04:58:02.395 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:58:02.395 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.779531881, 2024-09-18T04:58:02.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.53184266599999996, 2024-09-18T04:58:02.395 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:02.395 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:02.395 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:02.396 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:02.396 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:02.396 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 162075 2024-09-18T04:58:02.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.396 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:02.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:02.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:58:02.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:02.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:58:02.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:02.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490442+0000", 2024-09-18T04:58:02.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:58:02.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:02.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490453+0000", 2024-09-18T04:58:02.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0613e-05 2024-09-18T04:58:02.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:02.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490466+0000", 2024-09-18T04:58:02.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3702e-05 2024-09-18T04:58:02.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:02.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490475+0000", 2024-09-18T04:58:02.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2690000000000005e-06 2024-09-18T04:58:02.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:02.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490612+0000", 2024-09-18T04:58:02.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013620900000000001 2024-09-18T04:58:02.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-18T04:58:02.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490620+0000", 2024-09-18T04:58:02.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3089999999999996e-06 2024-09-18T04:58:02.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:02.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503041+0000", 2024-09-18T04:58:02.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012420512 2024-09-18T04:58:02.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:02.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503048+0000", 2024-09-18T04:58:02.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0389999999999999e-06 2024-09-18T04:58:02.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:02.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.016122+0000", 2024-09-18T04:58:02.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.51307432900000005 2024-09-18T04:58:02.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:02.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.016289+0000", 2024-09-18T04:58:02.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000166875 2024-09-18T04:58:02.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:02.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.022286+0000", 2024-09-18T04:58:02.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059976049999999996 2024-09-18T04:58:02.401 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.401 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:02.401 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.401 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:162076 3.13 3:cf3f6048:::rbd_header.11ac6967455f:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T04:58:02.401 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:02.401 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 19.779409207, 2024-09-18T04:58:02.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.53464117600000005, 2024-09-18T04:58:02.401 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:02.401 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:02.402 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:02.402 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:02.402 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:02.402 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 162076 2024-09-18T04:58:02.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.402 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:02.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:02.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:02.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:02.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:02.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:02.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:02.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:02.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:58:02.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:02.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490571+0000", 2024-09-18T04:58:02.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8259999999999998e-06 2024-09-18T04:58:02.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:02.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490584+0000", 2024-09-18T04:58:02.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2391999999999999e-05 2024-09-18T04:58:02.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:02.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490591+0000", 2024-09-18T04:58:02.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5460000000000003e-06 2024-09-18T04:58:02.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:02.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490714+0000", 2024-09-18T04:58:02.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000122543 2024-09-18T04:58:02.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-18T04:58:02.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490731+0000", 2024-09-18T04:58:02.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7405e-05 2024-09-18T04:58:02.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:02.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503069+0000", 2024-09-18T04:58:02.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012337836 2024-09-18T04:58:02.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:02.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503078+0000", 2024-09-18T04:58:02.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8289999999999997e-06 2024-09-18T04:58:02.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:02.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019018+0000", 2024-09-18T04:58:02.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.51594035900000002 2024-09-18T04:58:02.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:02.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019091+0000", 2024-09-18T04:58:02.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2608999999999995e-05 2024-09-18T04:58:02.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-18T04:58:02.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019421+0000", 2024-09-18T04:58:02.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00033004300000000002 2024-09-18T04:58:02.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:02.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.021644+0000", 2024-09-18T04:58:02.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022229749999999999 2024-09-18T04:58:02.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:02.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025113+0000", 2024-09-18T04:58:02.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003469228 2024-09-18T04:58:02.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:02.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025167+0000", 2024-09-18T04:58:02.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3569999999999997e-05 2024-09-18T04:58:02.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:02.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:02.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:02.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025208+0000", 2024-09-18T04:58:02.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0821000000000001e-05 2024-09-18T04:58:02.411 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.411 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:02.412 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.412 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:02.412 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:02.412 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:02.413 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:58:02.414 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:58:02.415 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:58:02.415 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:58:02.416 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:02.412+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:02.416 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:58:02.417 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:58:02.417 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:58:02.417 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:58:02.418 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:58:02.418 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:58:02.419 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:58:02.419 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:58:02.419 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:58:02.420 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:58:02.420 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:58:02.421 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:58:02.421 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:58:02.514 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:02.512+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:02.527 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:02.527 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:02.536 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:58:02.615 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:02.613+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:02.643 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:02.643 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:02.652 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:58:02.736 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:02.734+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:02.799 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:02.799 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:02.808 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:58:02.849 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:02.847+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:02.918 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:02.915+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:02.955 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:02.955 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:02.964 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:58:03.048 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:03.046+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:03.119 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:03.117+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:03.120 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:03.121 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:03.130 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:58:03.220 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:03.218+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:03.278 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:03.278 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:03.287 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:58:03.331 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:03.329+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:03.334 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 5, 6, 7] out_osds: [1, 4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T04:58:03.334 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T04:58:03.334 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-18T04:58:03.334 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 3.0 2024-09-18T04:58:03.396 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:03.396 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:03.405 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:58:03.451 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:03.449+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:03.517 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:03.517 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:03.523 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:03.521+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:03.526 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:58:03.624 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:03.622+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:03.625 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:58:03.623+0000 7ff18d579640 -1 mgr.server reply reply (3) No such process no change 2024-09-18T04:58:03.626 INFO:teuthology.orchestra.run.smithi002.stdout:moved 7 / 82 (8.53659%) 2024-09-18T04:58:03.626 INFO:teuthology.orchestra.run.smithi002.stdout:avg 13.6667 2024-09-18T04:58:03.626 INFO:teuthology.orchestra.run.smithi002.stdout:stddev 3.63624 -> 3.03681 (expected baseline 3.37474) 2024-09-18T04:58:03.626 INFO:teuthology.orchestra.run.smithi002.stdout:min osd.2 with 7 -> 9 pgs (0.512195 -> 0.658537 * mean) 2024-09-18T04:58:03.626 INFO:teuthology.orchestra.run.smithi002.stdout:max osd.0 with 18 -> 15 pgs (1.31707 -> 1.09756 * mean) 2024-09-18T04:58:03.626 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:58:03.626 INFO:teuthology.orchestra.run.smithi002.stdout:oload 110 2024-09-18T04:58:03.626 INFO:teuthology.orchestra.run.smithi002.stdout:max_change 3 2024-09-18T04:58:03.626 INFO:teuthology.orchestra.run.smithi002.stdout:max_change_osds 4 2024-09-18T04:58:03.626 INFO:teuthology.orchestra.run.smithi002.stdout:average_utilization 0.0007 2024-09-18T04:58:03.626 INFO:teuthology.orchestra.run.smithi002.stdout:overload_utilization 0.0007 2024-09-18T04:58:03.627 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 weight 1.0000 -> 0.7581 2024-09-18T04:58:03.627 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 weight 1.0000 -> 0.8357 2024-09-18T04:58:03.627 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 weight 1.0000 -> 0.8951 2024-09-18T04:58:03.627 INFO:teuthology.orchestra.run.smithi002.stderr:no change 2024-09-18T04:58:03.639 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-18T04:58:03.639 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 1.0 2024-09-18T04:58:03.674 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:03.674 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:03.683 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:58:03.725 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:03.722+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:03.827 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:03.824+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:03.843 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:03.843 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:03.853 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:58:03.927 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:03.924+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:03.975 INFO:teuthology.orchestra.run.smithi002.stdout:moved 0 / 82 (0%) 2024-09-18T04:58:03.976 INFO:teuthology.orchestra.run.smithi002.stdout:avg 13.6667 2024-09-18T04:58:03.976 INFO:teuthology.orchestra.run.smithi002.stdout:stddev 3.63624 -> 3.63624 (expected baseline 3.37474) 2024-09-18T04:58:03.976 INFO:teuthology.orchestra.run.smithi002.stdout:min osd.2 with 7 -> 7 pgs (0.512195 -> 0.512195 * mean) 2024-09-18T04:58:03.976 INFO:teuthology.orchestra.run.smithi002.stdout:max osd.0 with 18 -> 18 pgs (1.31707 -> 1.31707 * mean) 2024-09-18T04:58:03.976 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:58:03.976 INFO:teuthology.orchestra.run.smithi002.stdout:oload 1000 2024-09-18T04:58:03.976 INFO:teuthology.orchestra.run.smithi002.stdout:max_change 1 2024-09-18T04:58:03.976 INFO:teuthology.orchestra.run.smithi002.stdout:max_change_osds 4 2024-09-18T04:58:03.976 INFO:teuthology.orchestra.run.smithi002.stdout:average_utilization 0.0007 2024-09-18T04:58:03.976 INFO:teuthology.orchestra.run.smithi002.stdout:overload_utilization 0.0067 2024-09-18T04:58:03.977 INFO:teuthology.orchestra.run.smithi002.stderr:no change 2024-09-18T04:58:03.989 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-18T04:58:03.989 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 1000 3.0 2024-09-18T04:58:04.001 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:04.001 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:04.010 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:58:04.051 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:04.049+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:04.123 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T04:58:04.128 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:04.126+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:04.163 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:04.163 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:04.173 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:58:04.230 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:04.228+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:04.288 INFO:teuthology.orchestra.run.smithi002.stdout:moved 0 / 82 (0%) 2024-09-18T04:58:04.288 INFO:teuthology.orchestra.run.smithi002.stdout:avg 13.6667 2024-09-18T04:58:04.288 INFO:teuthology.orchestra.run.smithi002.stdout:stddev 3.63624 -> 3.63624 (expected baseline 3.37474) 2024-09-18T04:58:04.288 INFO:teuthology.orchestra.run.smithi002.stdout:min osd.2 with 7 -> 7 pgs (0.512195 -> 0.512195 * mean) 2024-09-18T04:58:04.288 INFO:teuthology.orchestra.run.smithi002.stdout:max osd.0 with 18 -> 18 pgs (1.31707 -> 1.31707 * mean) 2024-09-18T04:58:04.288 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:58:04.288 INFO:teuthology.orchestra.run.smithi002.stdout:oload 1000 2024-09-18T04:58:04.288 INFO:teuthology.orchestra.run.smithi002.stdout:max_change 3 2024-09-18T04:58:04.289 INFO:teuthology.orchestra.run.smithi002.stdout:max_change_osds 4 2024-09-18T04:58:04.289 INFO:teuthology.orchestra.run.smithi002.stdout:average_utilization 0.0007 2024-09-18T04:58:04.289 INFO:teuthology.orchestra.run.smithi002.stdout:overload_utilization 0.0067 2024-09-18T04:58:04.289 INFO:teuthology.orchestra.run.smithi002.stderr:no change 2024-09-18T04:58:04.289 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:04.289 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:04.298 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:58:04.301 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-18T04:58:04.301 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-09-18T04:58:04.342 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:04.340+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:04.402 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:04.402 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:04.411 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:58:04.422 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:58:04.466 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:04.464+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:04.514 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:04.514 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:04.523 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:58:04.547 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:58:04.547 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:58:04.564 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:58:04.564 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:58:04.566 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:04.564+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:04.580 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:58:04.580 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:58:04.595 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:58:04.596 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:58:04.611 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:58:04.612 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:58:04.626 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:04.626 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:04.627 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:58:04.628 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:58:04.633 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:04.631+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:04.635 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:58:04.644 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:58:04.644 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:58:04.660 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:58:04.660 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:58:04.734 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:04.732+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:04.778 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:04.778 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:04.778 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:58:04.778 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:58:04.778 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:04.787 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:58:04.834 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:04.832+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:04.891 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:04.891 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:58:04.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:58:04.891 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:04.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.891 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60344 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1456128~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:04.891 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:58:04.891 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.775529458999998, 2024-09-18T04:58:04.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044026324999999998, 2024-09-18T04:58:04.891 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:04.891 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:04.892 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:04.892 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:04.892 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:04.892 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60344 2024-09-18T04:58:04.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.892 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:04.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:04.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:58:04.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:04.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:58:04.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:04.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112222+0000", 2024-09-18T04:58:04.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:04.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:04.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112224+0000", 2024-09-18T04:58:04.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1320000000000001e-06 2024-09-18T04:58:04.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:04.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112230+0000", 2024-09-18T04:58:04.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9619999999999998e-06 2024-09-18T04:58:04.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:04.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112234+0000", 2024-09-18T04:58:04.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7890000000000002e-06 2024-09-18T04:58:04.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:04.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156105+0000", 2024-09-18T04:58:04.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043870531999999997 2024-09-18T04:58:04.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:04.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156170+0000", 2024-09-18T04:58:04.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5036000000000005e-05 2024-09-18T04:58:04.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:04.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156248+0000", 2024-09-18T04:58:04.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8295999999999997e-05 2024-09-18T04:58:04.897 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.897 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:04.897 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.897 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60345 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1471488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:04.897 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:58:04.897 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.775512694, 2024-09-18T04:58:04.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044180090999999998, 2024-09-18T04:58:04.897 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:04.897 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:04.898 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:04.898 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:04.898 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:04.898 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60345 2024-09-18T04:58:04.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.898 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:04.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:04.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:58:04.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:04.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:58:04.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:04.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112238+0000", 2024-09-18T04:58:04.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:04.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:04.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112240+0000", 2024-09-18T04:58:04.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9910000000000001e-06 2024-09-18T04:58:04.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:04.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112246+0000", 2024-09-18T04:58:04.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0859999999999998e-06 2024-09-18T04:58:04.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:04.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112251+0000", 2024-09-18T04:58:04.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5020000000000003e-06 2024-09-18T04:58:04.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:04.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156274+0000", 2024-09-18T04:58:04.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044023063000000001 2024-09-18T04:58:04.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:04.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156337+0000", 2024-09-18T04:58:04.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3077000000000006e-05 2024-09-18T04:58:04.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:04.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156419+0000", 2024-09-18T04:58:04.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1699000000000005e-05 2024-09-18T04:58:04.903 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.903 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:04.903 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.903 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60346 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1486848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:04.903 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:58:04.903 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.775494913000003, 2024-09-18T04:58:04.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044336331999999999, 2024-09-18T04:58:04.903 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:04.903 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:04.903 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:04.904 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:04.904 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:04.904 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60346 2024-09-18T04:58:04.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.904 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:04.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:04.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:58:04.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:04.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:58:04.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:04.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112256+0000", 2024-09-18T04:58:04.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:04.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:04.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112258+0000", 2024-09-18T04:58:04.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1720000000000001e-06 2024-09-18T04:58:04.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:04.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112265+0000", 2024-09-18T04:58:04.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4280000000000001e-06 2024-09-18T04:58:04.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:04.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112269+0000", 2024-09-18T04:58:04.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.741e-06 2024-09-18T04:58:04.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:04.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156445+0000", 2024-09-18T04:58:04.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044175341 2024-09-18T04:58:04.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:04.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156512+0000", 2024-09-18T04:58:04.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6723000000000005e-05 2024-09-18T04:58:04.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:04.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156593+0000", 2024-09-18T04:58:04.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1284000000000003e-05 2024-09-18T04:58:04.908 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.908 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:04.908 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.908 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60347 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1502208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:04.909 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:58:04.909 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.775466502, 2024-09-18T04:58:04.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044482560999999997, 2024-09-18T04:58:04.909 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:04.909 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:04.909 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:04.909 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:04.909 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:04.909 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60347 2024-09-18T04:58:04.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.910 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:04.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:04.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:58:04.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:04.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:58:04.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:04.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112284+0000", 2024-09-18T04:58:04.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:04.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:04.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112286+0000", 2024-09-18T04:58:04.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9709999999999998e-06 2024-09-18T04:58:04.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:04.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112290+0000", 2024-09-18T04:58:04.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9199999999999997e-06 2024-09-18T04:58:04.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:04.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112293+0000", 2024-09-18T04:58:04.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870000000000001e-06 2024-09-18T04:58:04.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:04.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156617+0000", 2024-09-18T04:58:04.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044323960000000003 2024-09-18T04:58:04.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:04.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156687+0000", 2024-09-18T04:58:04.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0141000000000005e-05 2024-09-18T04:58:04.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:04.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156767+0000", 2024-09-18T04:58:04.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0140000000000002e-05 2024-09-18T04:58:04.914 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.914 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:04.914 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.914 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60348 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1517568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:04.914 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:58:04.914 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.775455497999999, 2024-09-18T04:58:04.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044645562, 2024-09-18T04:58:04.914 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:04.914 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:04.915 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:04.915 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:04.915 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:04.915 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60348 2024-09-18T04:58:04.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.915 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:04.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:04.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:58:04.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:04.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:58:04.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:04.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:58:04.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:04.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:04.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112297+0000", 2024-09-18T04:58:04.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.384e-06 2024-09-18T04:58:04.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:04.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112301+0000", 2024-09-18T04:58:04.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0570000000000003e-06 2024-09-18T04:58:04.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:04.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112304+0000", 2024-09-18T04:58:04.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6000000000000001e-06 2024-09-18T04:58:04.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:04.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156793+0000", 2024-09-18T04:58:04.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044489494999999997 2024-09-18T04:58:04.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:04.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156859+0000", 2024-09-18T04:58:04.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5483999999999998e-05 2024-09-18T04:58:04.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:04.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156941+0000", 2024-09-18T04:58:04.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2856999999999996e-05 2024-09-18T04:58:04.920 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.920 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:04.920 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.920 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60349 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1532928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:04.920 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:58:04.920 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.775445521999998, 2024-09-18T04:58:04.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044832167999999999, 2024-09-18T04:58:04.921 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:04.921 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:04.921 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:04.921 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:04.921 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:04.921 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60349 2024-09-18T04:58:04.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.921 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:04.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.921 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:04.921 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:58:04.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:04.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:58:04.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:04.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:58:04.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:04.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:04.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112307+0000", 2024-09-18T04:58:04.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.308e-06 2024-09-18T04:58:04.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:04.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112310+0000", 2024-09-18T04:58:04.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-18T04:58:04.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:04.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112312+0000", 2024-09-18T04:58:04.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4889999999999998e-06 2024-09-18T04:58:04.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:04.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156977+0000", 2024-09-18T04:58:04.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044664598999999999 2024-09-18T04:58:04.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:04.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157053+0000", 2024-09-18T04:58:04.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6014999999999996e-05 2024-09-18T04:58:04.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:04.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157138+0000", 2024-09-18T04:58:04.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5162999999999998e-05 2024-09-18T04:58:04.925 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.925 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:04.925 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.925 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60350 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1548288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:04.925 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:58:04.926 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.775436608000007, 2024-09-18T04:58:04.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045011858000000002, 2024-09-18T04:58:04.926 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:04.926 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:04.926 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:04.926 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:04.926 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:04.926 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60350 2024-09-18T04:58:04.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.926 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:04.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:04.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:58:04.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:04.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:58:04.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:04.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:58:04.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:04.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:04.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112316+0000", 2024-09-18T04:58:04.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7579999999999999e-06 2024-09-18T04:58:04.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:04.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112321+0000", 2024-09-18T04:58:04.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1649999999999997e-06 2024-09-18T04:58:04.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:04.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112325+0000", 2024-09-18T04:58:04.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.726e-06 2024-09-18T04:58:04.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:04.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157172+0000", 2024-09-18T04:58:04.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.04484718 2024-09-18T04:58:04.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:04.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157242+0000", 2024-09-18T04:58:04.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9675e-05 2024-09-18T04:58:04.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:04.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157327+0000", 2024-09-18T04:58:04.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4614e-05 2024-09-18T04:58:04.931 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.931 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:04.931 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.931 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60351 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1563648~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:04.931 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:58:04.931 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.775423192000005, 2024-09-18T04:58:04.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045132113000000001, 2024-09-18T04:58:04.932 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:04.932 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:04.932 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:04.932 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:04.932 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:04.932 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60351 2024-09-18T04:58:04.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.932 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:04.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:04.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:58:04.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:04.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:58:04.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:04.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:58:04.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:04.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:04.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112330+0000", 2024-09-18T04:58:04.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1660000000000001e-06 2024-09-18T04:58:04.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:04.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112336+0000", 2024-09-18T04:58:04.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5940000000000001e-06 2024-09-18T04:58:04.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:04.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112340+0000", 2024-09-18T04:58:04.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5589999999999999e-06 2024-09-18T04:58:04.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:04.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157365+0000", 2024-09-18T04:58:04.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045024675 2024-09-18T04:58:04.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:04.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157409+0000", 2024-09-18T04:58:04.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4356e-05 2024-09-18T04:58:04.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:04.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157460+0000", 2024-09-18T04:58:04.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1072000000000002e-05 2024-09-18T04:58:04.936 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.936 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:04.936 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.937 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60501 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1494016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:04.937 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:58:04.937 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.769362495999999, 2024-09-18T04:58:04.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044007728000000003, 2024-09-18T04:58:04.937 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:04.937 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:04.937 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:04.937 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:04.937 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:04.937 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60501 2024-09-18T04:58:04.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.938 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:04.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:04.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:58:04.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:04.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:58:04.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:04.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118389+0000", 2024-09-18T04:58:04.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:04.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:04.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118391+0000", 2024-09-18T04:58:04.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:58:04.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:04.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118396+0000", 2024-09-18T04:58:04.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2680000000000002e-06 2024-09-18T04:58:04.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:04.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118400+0000", 2024-09-18T04:58:04.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2080000000000002e-06 2024-09-18T04:58:04.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:04.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162317+0000", 2024-09-18T04:58:04.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043916753000000003 2024-09-18T04:58:04.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:04.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162351+0000", 2024-09-18T04:58:04.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4372000000000002e-05 2024-09-18T04:58:04.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:04.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162397+0000", 2024-09-18T04:58:04.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5207999999999998e-05 2024-09-18T04:58:04.941 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.941 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:04.942 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.942 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60502 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1509376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:04.942 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:58:04.942 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.769347018999994, 2024-09-18T04:58:04.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044092438999999997, 2024-09-18T04:58:04.942 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:04.942 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:04.942 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:04.942 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:04.943 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:04.943 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60502 2024-09-18T04:58:04.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.943 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:04.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:04.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:58:04.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:04.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:58:04.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:04.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:58:04.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:04.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:04.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118406+0000", 2024-09-18T04:58:04.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:58:04.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:04.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118412+0000", 2024-09-18T04:58:04.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.524e-06 2024-09-18T04:58:04.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:04.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118416+0000", 2024-09-18T04:58:04.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:58:04.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:04.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162418+0000", 2024-09-18T04:58:04.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001508000000002 2024-09-18T04:58:04.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:04.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162451+0000", 2024-09-18T04:58:04.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3662999999999999e-05 2024-09-18T04:58:04.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:04.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162497+0000", 2024-09-18T04:58:04.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5550000000000003e-05 2024-09-18T04:58:04.947 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.947 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:04.947 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.947 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60503 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1524736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:04.947 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:58:04.947 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.769330861, 2024-09-18T04:58:04.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044178780000000001, 2024-09-18T04:58:04.947 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:04.947 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:04.948 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:04.948 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:04.948 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:04.948 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60503 2024-09-18T04:58:04.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.948 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:04.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:04.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:58:04.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:04.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:58:04.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:04.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118420+0000", 2024-09-18T04:58:04.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:04.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:04.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118422+0000", 2024-09-18T04:58:04.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.08e-06 2024-09-18T04:58:04.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:04.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118428+0000", 2024-09-18T04:58:04.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.964e-06 2024-09-18T04:58:04.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:04.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118433+0000", 2024-09-18T04:58:04.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.425e-06 2024-09-18T04:58:04.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:04.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162519+0000", 2024-09-18T04:58:04.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044086082999999998 2024-09-18T04:58:04.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:04.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162552+0000", 2024-09-18T04:58:04.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3719e-05 2024-09-18T04:58:04.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:04.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162599+0000", 2024-09-18T04:58:04.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6873999999999999e-05 2024-09-18T04:58:04.952 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.952 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:04.952 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.952 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60504 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1540096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:04.952 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:58:04.953 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.769314756, 2024-09-18T04:58:04.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044261921000000003, 2024-09-18T04:58:04.953 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:04.953 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:04.953 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:04.953 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:04.953 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:04.953 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60504 2024-09-18T04:58:04.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.953 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:04.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:04.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:58:04.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:04.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:58:04.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:04.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118436+0000", 2024-09-18T04:58:04.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:04.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:04.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118438+0000", 2024-09-18T04:58:04.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0250000000000001e-06 2024-09-18T04:58:04.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:04.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118444+0000", 2024-09-18T04:58:04.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4360000000000001e-06 2024-09-18T04:58:04.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:04.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118448+0000", 2024-09-18T04:58:04.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.357e-06 2024-09-18T04:58:04.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:04.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162620+0000", 2024-09-18T04:58:04.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044171821 2024-09-18T04:58:04.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:04.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162653+0000", 2024-09-18T04:58:04.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3479999999999998e-05 2024-09-18T04:58:04.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:04.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162699+0000", 2024-09-18T04:58:04.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5113e-05 2024-09-18T04:58:04.957 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.957 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:04.957 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.958 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60584 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1196032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:04.958 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:58:04.958 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.767521739000003, 2024-09-18T04:58:04.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044048727000000003, 2024-09-18T04:58:04.958 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:04.958 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:04.958 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:04.958 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:04.958 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:04.958 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60584 2024-09-18T04:58:04.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.959 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:04.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:04.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:58:04.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:04.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120230+0000", 2024-09-18T04:58:04.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:04.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120229+0000", 2024-09-18T04:58:04.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:04.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:04.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120231+0000", 2024-09-18T04:58:04.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:58:04.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:04.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120237+0000", 2024-09-18T04:58:04.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8379999999999998e-06 2024-09-18T04:58:04.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:04.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120242+0000", 2024-09-18T04:58:04.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3889999999999998e-06 2024-09-18T04:58:04.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:04.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164187+0000", 2024-09-18T04:58:04.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043945045000000002 2024-09-18T04:58:04.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:04.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164227+0000", 2024-09-18T04:58:04.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0627999999999998e-05 2024-09-18T04:58:04.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:04.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164278+0000", 2024-09-18T04:58:04.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1146000000000001e-05 2024-09-18T04:58:04.963 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.963 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:04.963 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.963 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60585 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1211392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:04.963 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:58:04.963 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.767505631000006, 2024-09-18T04:58:04.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044389814, 2024-09-18T04:58:04.964 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:04.964 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:04.964 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:04.964 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:04.964 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:04.964 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60585 2024-09-18T04:58:04.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.964 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:04.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:04.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:58:04.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:04.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:58:04.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:04.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120245+0000", 2024-09-18T04:58:04.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:04.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:04.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120247+0000", 2024-09-18T04:58:04.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0150000000000002e-06 2024-09-18T04:58:04.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:04.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120255+0000", 2024-09-18T04:58:04.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.858e-06 2024-09-18T04:58:04.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:04.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120260+0000", 2024-09-18T04:58:04.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6530000000000002e-06 2024-09-18T04:58:04.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:04.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164489+0000", 2024-09-18T04:58:04.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044229483999999999 2024-09-18T04:58:04.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:04.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164555+0000", 2024-09-18T04:58:04.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.533e-05 2024-09-18T04:58:04.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:04.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164636+0000", 2024-09-18T04:58:04.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0819e-05 2024-09-18T04:58:04.968 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.968 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:04.968 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.968 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60586 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1226752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:04.969 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:58:04.969 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.767487180000003, 2024-09-18T04:58:04.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044578214999999997, 2024-09-18T04:58:04.969 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:04.969 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:04.969 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:04.969 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:04.970 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:04.970 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60586 2024-09-18T04:58:04.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.970 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:04.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:04.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:58:04.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:04.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:58:04.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:04.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:58:04.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:04.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:04.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120266+0000", 2024-09-18T04:58:04.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0660000000000002e-06 2024-09-18T04:58:04.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:04.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120272+0000", 2024-09-18T04:58:04.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-18T04:58:04.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:04.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120276+0000", 2024-09-18T04:58:04.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.476e-06 2024-09-18T04:58:04.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:04.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164673+0000", 2024-09-18T04:58:04.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044396672999999998 2024-09-18T04:58:04.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:04.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164749+0000", 2024-09-18T04:58:04.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5811999999999999e-05 2024-09-18T04:58:04.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:04.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164842+0000", 2024-09-18T04:58:04.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3560000000000005e-05 2024-09-18T04:58:04.974 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.974 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:04.974 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.975 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60587 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1242112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:04.975 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:58:04.975 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.767438080999995, 2024-09-18T04:58:04.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044634794999999998, 2024-09-18T04:58:04.975 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:04.975 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:04.975 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:04.975 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:04.975 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:04.975 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60587 2024-09-18T04:58:04.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.975 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:04.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:04.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:58:04.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:04.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:58:04.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:04.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120280+0000", 2024-09-18T04:58:04.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999671 2024-09-18T04:58:04.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:04.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120316+0000", 2024-09-18T04:58:04.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5728000000000001e-05 2024-09-18T04:58:04.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:04.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120329+0000", 2024-09-18T04:58:04.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2914e-05 2024-09-18T04:58:04.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:04.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120335+0000", 2024-09-18T04:58:04.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8289999999999996e-06 2024-09-18T04:58:04.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:04.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164858+0000", 2024-09-18T04:58:04.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044523417000000003 2024-09-18T04:58:04.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:04.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164896+0000", 2024-09-18T04:58:04.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7969999999999997e-05 2024-09-18T04:58:04.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:04.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164948+0000", 2024-09-18T04:58:04.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1996000000000002e-05 2024-09-18T04:58:04.979 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.979 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:04.979 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.980 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60595 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1364992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:04.980 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:58:04.980 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.765594864999997, 2024-09-18T04:58:04.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001360000000003, 2024-09-18T04:58:04.980 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:04.980 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:04.980 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:04.980 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:04.980 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:04.981 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60595 2024-09-18T04:58:04.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.981 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:04.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:04.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:58:04.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:04.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122157+0000", 2024-09-18T04:58:04.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:04.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122156+0000", 2024-09-18T04:58:04.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:04.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:04.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122158+0000", 2024-09-18T04:58:04.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0040000000000002e-06 2024-09-18T04:58:04.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:04.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122164+0000", 2024-09-18T04:58:04.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8719999999999999e-06 2024-09-18T04:58:04.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:04.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122169+0000", 2024-09-18T04:58:04.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5090000000000004e-06 2024-09-18T04:58:04.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:04.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166040+0000", 2024-09-18T04:58:04.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043871226999999999 2024-09-18T04:58:04.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:04.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166096+0000", 2024-09-18T04:58:04.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6495e-05 2024-09-18T04:58:04.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:04.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166158+0000", 2024-09-18T04:58:04.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1626999999999998e-05 2024-09-18T04:58:04.986 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.986 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:04.986 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.986 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60596 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1380352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:04.986 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:58:04.986 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.765578543999993, 2024-09-18T04:58:04.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044120800000000002, 2024-09-18T04:58:04.986 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:04.987 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:04.987 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:04.987 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:04.987 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:04.987 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60596 2024-09-18T04:58:04.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.987 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:04.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:04.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:58:04.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:04.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:58:04.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:04.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:58:04.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:04.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:04.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122175+0000", 2024-09-18T04:58:04.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:58:04.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:04.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122182+0000", 2024-09-18T04:58:04.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3509999999999996e-06 2024-09-18T04:58:04.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:04.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122187+0000", 2024-09-18T04:58:04.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7940000000000002e-06 2024-09-18T04:58:04.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:04.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166184+0000", 2024-09-18T04:58:04.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043997236000000002 2024-09-18T04:58:04.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:04.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166227+0000", 2024-09-18T04:58:04.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2561000000000001e-05 2024-09-18T04:58:04.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:04.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166294+0000", 2024-09-18T04:58:04.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7068000000000002e-05 2024-09-18T04:58:04.991 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.991 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:04.991 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.991 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60597 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1395712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:04.992 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:58:04.992 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.765560515999994, 2024-09-18T04:58:04.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044242166999999999, 2024-09-18T04:58:04.992 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:04.992 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:04.992 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:04.992 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:04.992 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:04.992 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60597 2024-09-18T04:58:04.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.992 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:04.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:04.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:58:04.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:04.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:58:04.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:04.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:58:04.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:04.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:04.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122193+0000", 2024-09-18T04:58:04.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:58:04.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:04.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122199+0000", 2024-09-18T04:58:04.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6449999999999999e-06 2024-09-18T04:58:04.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:04.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122204+0000", 2024-09-18T04:58:04.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4819999999999996e-06 2024-09-18T04:58:04.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:04.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166329+0000", 2024-09-18T04:58:04.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044125059000000001 2024-09-18T04:58:04.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:04.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166370+0000", 2024-09-18T04:58:04.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1012e-05 2024-09-18T04:58:04.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:04.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166433+0000", 2024-09-18T04:58:04.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3178999999999999e-05 2024-09-18T04:58:04.997 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.997 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:04.997 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:04.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.997 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60608 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1564672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:04.997 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:58:04.998 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.763562653999998, 2024-09-18T04:58:04.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044082723999999997, 2024-09-18T04:58:04.998 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:04.998 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:04.998 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:04.998 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:04.998 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:04.998 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60608 2024-09-18T04:58:04.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.998 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:04.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:04.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:58:04.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:04.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124189+0000", 2024-09-18T04:58:04.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:04.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:04.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:04.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:04.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124188+0000", 2024-09-18T04:58:04.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:04.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124190+0000", 2024-09-18T04:58:05.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0329999999999998e-06 2024-09-18T04:58:05.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124196+0000", 2024-09-18T04:58:05.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.575e-06 2024-09-18T04:58:05.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.124201+0000", 2024-09-18T04:58:05.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5000000000000001e-06 2024-09-18T04:58:05.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168166+0000", 2024-09-18T04:58:05.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043965763999999997 2024-09-18T04:58:05.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168218+0000", 2024-09-18T04:58:05.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1785e-05 2024-09-18T04:58:05.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.168271+0000", 2024-09-18T04:58:05.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3429e-05 2024-09-18T04:58:05.002 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.002 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.002 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.002 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.002 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.002 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:05.004 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:58:05.004 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:05.001+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:05.016 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-18T04:58:05.017 INFO:teuthology.orchestra.run.smithi002.stdout:moved 1 / 82 (1.21951%) 2024-09-18T04:58:05.017 INFO:teuthology.orchestra.run.smithi002.stdout:avg 13.6667 2024-09-18T04:58:05.017 INFO:teuthology.orchestra.run.smithi002.stdout:stddev 3.63624 -> 3.44803 (expected baseline 3.37474) 2024-09-18T04:58:05.017 INFO:teuthology.orchestra.run.smithi002.stdout:min osd.2 with 7 -> 7 pgs (0.512195 -> 0.512195 * mean) 2024-09-18T04:58:05.017 INFO:teuthology.orchestra.run.smithi002.stdout:max osd.0 with 18 -> 17 pgs (1.31707 -> 1.2439 * mean) 2024-09-18T04:58:05.017 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:58:05.017 INFO:teuthology.orchestra.run.smithi002.stdout:oload 110 2024-09-18T04:58:05.017 INFO:teuthology.orchestra.run.smithi002.stdout:max_change 0.05 2024-09-18T04:58:05.017 INFO:teuthology.orchestra.run.smithi002.stdout:max_change_osds 4 2024-09-18T04:58:05.017 INFO:teuthology.orchestra.run.smithi002.stdout:average_utilization 0.0007 2024-09-18T04:58:05.017 INFO:teuthology.orchestra.run.smithi002.stdout:overload_utilization 0.0007 2024-09-18T04:58:05.017 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 weight 1.0000 -> 0.9500 2024-09-18T04:58:05.017 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 weight 1.0000 -> 0.9500 2024-09-18T04:58:05.018 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 weight 1.0000 -> 0.9500 2024-09-18T04:58:05.035 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-18T04:58:05.035 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 1.0 2024-09-18T04:58:05.120 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:05.118+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:05.155 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:05.155 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:05.155 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:58:05.155 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:05.165 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:58:05.208 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:05.206+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:05.268 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:05.268 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:05.268 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:58:05.268 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:58:05.268 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:05.277 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:58:05.306 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:05.304+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:05.373 INFO:tasks.ceph.mgr.x.smithi099.stderr:2024-09-18T04:58:05.371+0000 7ff18d579640 -1 mgr.server reply reply (4) Interrupted system call no change 2024-09-18T04:58:05.374 INFO:teuthology.orchestra.run.smithi002.stdout:moved 10 / 82 (12.1951%) 2024-09-18T04:58:05.374 INFO:teuthology.orchestra.run.smithi002.stdout:avg 13.6667 2024-09-18T04:58:05.374 INFO:teuthology.orchestra.run.smithi002.stdout:stddev 3.44803 -> 2.68742 (expected baseline 3.37474) 2024-09-18T04:58:05.374 INFO:teuthology.orchestra.run.smithi002.stdout:min osd.2 with 7 -> 11 pgs (0.512195 -> 0.804878 * mean) 2024-09-18T04:58:05.374 INFO:teuthology.orchestra.run.smithi002.stdout:max osd.0 with 17 -> 16 pgs (1.2439 -> 1.17073 * mean) 2024-09-18T04:58:05.374 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:58:05.374 INFO:teuthology.orchestra.run.smithi002.stdout:oload 110 2024-09-18T04:58:05.375 INFO:teuthology.orchestra.run.smithi002.stdout:max_change 1 2024-09-18T04:58:05.375 INFO:teuthology.orchestra.run.smithi002.stdout:max_change_osds 4 2024-09-18T04:58:05.375 INFO:teuthology.orchestra.run.smithi002.stdout:average_utilization 0.0006 2024-09-18T04:58:05.375 INFO:teuthology.orchestra.run.smithi002.stdout:overload_utilization 0.0007 2024-09-18T04:58:05.375 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5 weight 0.9500 -> 0.6910 2024-09-18T04:58:05.375 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7 weight 1.0000 -> 0.9070 2024-09-18T04:58:05.375 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3 weight 0.9500 -> 0.7268 2024-09-18T04:58:05.375 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0 weight 0.9500 -> 0.8294 2024-09-18T04:58:05.375 INFO:teuthology.orchestra.run.smithi002.stderr:no change 2024-09-18T04:58:05.407 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:05.405+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:05.420 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:05.420 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:58:05.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:58:05.420 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:05.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.420 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26189 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 237568~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:05.421 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:58:05.421 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.777507108999998, 2024-09-18T04:58:05.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022263431, 2024-09-18T04:58:05.421 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.421 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.421 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.421 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.421 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.421 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26189 2024-09-18T04:58:05.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.421 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:58:05.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:58:05.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639785+0000", 2024-09-18T04:58:05.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639787+0000", 2024-09-18T04:58:05.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9759999999999998e-06 2024-09-18T04:58:05.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639792+0000", 2024-09-18T04:58:05.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5829999999999997e-06 2024-09-18T04:58:05.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639796+0000", 2024-09-18T04:58:05.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0690000000000003e-06 2024-09-18T04:58:05.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.661944+0000", 2024-09-18T04:58:05.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022147463999999999 2024-09-18T04:58:05.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.661985+0000", 2024-09-18T04:58:05.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1360999999999999e-05 2024-09-18T04:58:05.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662048+0000", 2024-09-18T04:58:05.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3298000000000003e-05 2024-09-18T04:58:05.425 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.425 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.425 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.425 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26199 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 4096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:05.425 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:58:05.425 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.777491823000005, 2024-09-18T04:58:05.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022348776000000001, 2024-09-18T04:58:05.425 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.425 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.426 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.426 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.426 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.426 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26199 2024-09-18T04:58:05.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.426 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:58:05.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:58:05.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639800+0000", 2024-09-18T04:58:05.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639806+0000", 2024-09-18T04:58:05.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.1539999999999999e-06 2024-09-18T04:58:05.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639812+0000", 2024-09-18T04:58:05.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5520000000000004e-06 2024-09-18T04:58:05.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639816+0000", 2024-09-18T04:58:05.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1840000000000001e-06 2024-09-18T04:58:05.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662063+0000", 2024-09-18T04:58:05.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022247125999999999 2024-09-18T04:58:05.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662098+0000", 2024-09-18T04:58:05.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4968000000000001e-05 2024-09-18T04:58:05.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662149+0000", 2024-09-18T04:58:05.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1122000000000003e-05 2024-09-18T04:58:05.429 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.429 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.429 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.430 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26200 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 32768~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:05.430 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:58:05.430 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.777472298000006, 2024-09-18T04:58:05.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022501118000000001, 2024-09-18T04:58:05.430 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.430 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.430 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.430 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.431 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.431 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26200 2024-09-18T04:58:05.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.431 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:58:05.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:58:05.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:58:05.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639822+0000", 2024-09-18T04:58:05.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.007e-06 2024-09-18T04:58:05.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639827+0000", 2024-09-18T04:58:05.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5709999999999997e-06 2024-09-18T04:58:05.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639831+0000", 2024-09-18T04:58:05.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2819999999999998e-06 2024-09-18T04:58:05.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662181+0000", 2024-09-18T04:58:05.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022349878 2024-09-18T04:58:05.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662252+0000", 2024-09-18T04:58:05.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0542000000000003e-05 2024-09-18T04:58:05.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662321+0000", 2024-09-18T04:58:05.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9167999999999999e-05 2024-09-18T04:58:05.434 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.434 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.434 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.434 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26201 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 61440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:05.434 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:58:05.434 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.777457006999995, 2024-09-18T04:58:05.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022569783, 2024-09-18T04:58:05.435 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.435 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.435 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.435 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.435 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.435 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26201 2024-09-18T04:58:05.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.435 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:58:05.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:58:05.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:58:05.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639837+0000", 2024-09-18T04:58:05.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.928e-06 2024-09-18T04:58:05.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639843+0000", 2024-09-18T04:58:05.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8130000000000001e-06 2024-09-18T04:58:05.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639847+0000", 2024-09-18T04:58:05.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0999999999999997e-06 2024-09-18T04:58:05.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662335+0000", 2024-09-18T04:58:05.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022488207999999999 2024-09-18T04:58:05.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662370+0000", 2024-09-18T04:58:05.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5292e-05 2024-09-18T04:58:05.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662405+0000", 2024-09-18T04:58:05.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4765999999999999e-05 2024-09-18T04:58:05.439 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.439 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.439 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.439 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26202 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 90112~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:05.439 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:58:05.439 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.777441589999995, 2024-09-18T04:58:05.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022655937000000001, 2024-09-18T04:58:05.439 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.439 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.439 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.439 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.440 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.440 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26202 2024-09-18T04:58:05.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.440 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:58:05.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:58:05.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639850+0000", 2024-09-18T04:58:05.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639852+0000", 2024-09-18T04:58:05.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9700000000000002e-06 2024-09-18T04:58:05.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639858+0000", 2024-09-18T04:58:05.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.434e-06 2024-09-18T04:58:05.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639862+0000", 2024-09-18T04:58:05.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3499999999999999e-06 2024-09-18T04:58:05.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662429+0000", 2024-09-18T04:58:05.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022567409 2024-09-18T04:58:05.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662469+0000", 2024-09-18T04:58:05.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9801000000000001e-05 2024-09-18T04:58:05.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662507+0000", 2024-09-18T04:58:05.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7311000000000002e-05 2024-09-18T04:58:05.444 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.444 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.444 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.444 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26203 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 118784~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:05.444 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:58:05.444 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.777426434999995, 2024-09-18T04:58:05.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022730358999999999, 2024-09-18T04:58:05.444 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.444 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.445 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.445 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.445 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26203 2024-09-18T04:58:05.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.445 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:58:05.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:58:05.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639865+0000", 2024-09-18T04:58:05.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639867+0000", 2024-09-18T04:58:05.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0889999999999998e-06 2024-09-18T04:58:05.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639873+0000", 2024-09-18T04:58:05.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4979999999999997e-06 2024-09-18T04:58:05.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639877+0000", 2024-09-18T04:58:05.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4510000000000002e-06 2024-09-18T04:58:05.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662528+0000", 2024-09-18T04:58:05.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022650791999999999 2024-09-18T04:58:05.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662563+0000", 2024-09-18T04:58:05.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4530000000000003e-05 2024-09-18T04:58:05.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662596+0000", 2024-09-18T04:58:05.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3463000000000001e-05 2024-09-18T04:58:05.449 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.449 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.449 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.449 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26204 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 147456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:05.449 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:58:05.449 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.777411043000001, 2024-09-18T04:58:05.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022803233999999999, 2024-09-18T04:58:05.449 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.449 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.449 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.450 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26204 2024-09-18T04:58:05.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.450 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:58:05.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:58:05.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:58:05.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639883+0000", 2024-09-18T04:58:05.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6520000000000002e-06 2024-09-18T04:58:05.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639888+0000", 2024-09-18T04:58:05.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2870000000000004e-06 2024-09-18T04:58:05.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639893+0000", 2024-09-18T04:58:05.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3839999999999999e-06 2024-09-18T04:58:05.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662617+0000", 2024-09-18T04:58:05.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022723876 2024-09-18T04:58:05.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662651+0000", 2024-09-18T04:58:05.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3797000000000002e-05 2024-09-18T04:58:05.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662684+0000", 2024-09-18T04:58:05.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3828999999999997e-05 2024-09-18T04:58:05.453 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.453 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.453 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.454 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26205 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 176128~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:05.454 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:58:05.454 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.777388082000002, 2024-09-18T04:58:05.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022919108000000001, 2024-09-18T04:58:05.454 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.454 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.454 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.454 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.454 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.454 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26205 2024-09-18T04:58:05.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.454 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:58:05.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:58:05.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:58:05.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639906+0000", 2024-09-18T04:58:05.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.531e-06 2024-09-18T04:58:05.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639913+0000", 2024-09-18T04:58:05.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.331e-06 2024-09-18T04:58:05.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639920+0000", 2024-09-18T04:58:05.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5399999999999998e-06 2024-09-18T04:58:05.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662706+0000", 2024-09-18T04:58:05.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.02278587 2024-09-18T04:58:05.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662764+0000", 2024-09-18T04:58:05.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7630000000000002e-05 2024-09-18T04:58:05.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662823+0000", 2024-09-18T04:58:05.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9648000000000002e-05 2024-09-18T04:58:05.458 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.458 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.458 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.458 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26206 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 204800~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:05.458 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:58:05.458 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.777367682999994, 2024-09-18T04:58:05.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023047517, 2024-09-18T04:58:05.458 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.458 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.458 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.459 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26206 2024-09-18T04:58:05.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.459 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:58:05.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:58:05.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:58:05.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639926+0000", 2024-09-18T04:58:05.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3120000000000001e-06 2024-09-18T04:58:05.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639932+0000", 2024-09-18T04:58:05.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4890000000000003e-06 2024-09-18T04:58:05.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639936+0000", 2024-09-18T04:58:05.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.07e-06 2024-09-18T04:58:05.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662850+0000", 2024-09-18T04:58:05.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022913579999999999 2024-09-18T04:58:05.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662903+0000", 2024-09-18T04:58:05.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3176e-05 2024-09-18T04:58:05.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662972+0000", 2024-09-18T04:58:05.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9270999999999995e-05 2024-09-18T04:58:05.462 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.462 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.462 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.462 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26207 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 233472~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:05.463 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:58:05.463 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.777352539000006, 2024-09-18T04:58:05.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023139541999999999, 2024-09-18T04:58:05.463 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.463 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.463 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26207 2024-09-18T04:58:05.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.463 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:58:05.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:58:05.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639939+0000", 2024-09-18T04:58:05.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639941+0000", 2024-09-18T04:58:05.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0080000000000001e-06 2024-09-18T04:58:05.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639947+0000", 2024-09-18T04:58:05.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.485e-06 2024-09-18T04:58:05.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639951+0000", 2024-09-18T04:58:05.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4549999999999997e-06 2024-09-18T04:58:05.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663005+0000", 2024-09-18T04:58:05.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023053569999999999 2024-09-18T04:58:05.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663042+0000", 2024-09-18T04:58:05.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7245e-05 2024-09-18T04:58:05.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663079+0000", 2024-09-18T04:58:05.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7119000000000002e-05 2024-09-18T04:58:05.469 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.469 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.469 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.469 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26208 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 524288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:05.469 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:58:05.469 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.777335790999999, 2024-09-18T04:58:05.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023283564999999999, 2024-09-18T04:58:05.469 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.470 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.470 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.470 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.470 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.470 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26208 2024-09-18T04:58:05.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.470 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:58:05.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:58:05.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:58:05.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639958+0000", 2024-09-18T04:58:05.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0899999999999999e-06 2024-09-18T04:58:05.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639963+0000", 2024-09-18T04:58:05.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2209999999999996e-06 2024-09-18T04:58:05.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639967+0000", 2024-09-18T04:58:05.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1169999999999997e-06 2024-09-18T04:58:05.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663114+0000", 2024-09-18T04:58:05.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023146102000000002 2024-09-18T04:58:05.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663183+0000", 2024-09-18T04:58:05.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9686999999999999e-05 2024-09-18T04:58:05.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663240+0000", 2024-09-18T04:58:05.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6728000000000003e-05 2024-09-18T04:58:05.473 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.473 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.473 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.474 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26209 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 552960~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:05.474 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:58:05.474 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.777318015000006, 2024-09-18T04:58:05.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023440966000000001, 2024-09-18T04:58:05.474 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.474 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.474 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.474 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.474 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.474 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26209 2024-09-18T04:58:05.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.474 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:58:05.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:58:05.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:58:05.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639976+0000", 2024-09-18T04:58:05.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4940000000000002e-06 2024-09-18T04:58:05.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639982+0000", 2024-09-18T04:58:05.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0560000000000001e-06 2024-09-18T04:58:05.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639987+0000", 2024-09-18T04:58:05.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-18T04:58:05.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663278+0000", 2024-09-18T04:58:05.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023290789999999999 2024-09-18T04:58:05.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663348+0000", 2024-09-18T04:58:05.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0837e-05 2024-09-18T04:58:05.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663415+0000", 2024-09-18T04:58:05.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6736000000000006e-05 2024-09-18T04:58:05.478 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.478 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.478 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.478 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26210 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 581632~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:05.478 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:58:05.478 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.777301007000005, 2024-09-18T04:58:05.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023616437000000001, 2024-09-18T04:58:05.478 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.479 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.479 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.479 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.479 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.479 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26210 2024-09-18T04:58:05.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.479 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:58:05.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:58:05.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:58:05.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639993+0000", 2024-09-18T04:58:05.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.419e-06 2024-09-18T04:58:05.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640006+0000", 2024-09-18T04:58:05.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2918000000000001e-05 2024-09-18T04:58:05.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640011+0000", 2024-09-18T04:58:05.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9320000000000004e-06 2024-09-18T04:58:05.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663454+0000", 2024-09-18T04:58:05.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023443035000000001 2024-09-18T04:58:05.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663532+0000", 2024-09-18T04:58:05.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7443999999999999e-05 2024-09-18T04:58:05.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663608+0000", 2024-09-18T04:58:05.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6093000000000005e-05 2024-09-18T04:58:05.482 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.482 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.482 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.483 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26211 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 610304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:05.483 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:58:05.483 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.777276455000006, 2024-09-18T04:58:05.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023774337999999999, 2024-09-18T04:58:05.483 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.483 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.483 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.483 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.483 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.483 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26211 2024-09-18T04:58:05.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.483 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:58:05.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:58:05.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640015+0000", 2024-09-18T04:58:05.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640023+0000", 2024-09-18T04:58:05.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3900000000000004e-06 2024-09-18T04:58:05.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640036+0000", 2024-09-18T04:58:05.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3091999999999999e-05 2024-09-18T04:58:05.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640041+0000", 2024-09-18T04:58:05.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6219999999999997e-06 2024-09-18T04:58:05.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663644+0000", 2024-09-18T04:58:05.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023602643999999999 2024-09-18T04:58:05.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663718+0000", 2024-09-18T04:58:05.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4481999999999996e-05 2024-09-18T04:58:05.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663790+0000", 2024-09-18T04:58:05.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1589999999999997e-05 2024-09-18T04:58:05.487 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.487 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.487 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.487 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26212 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 638976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:05.487 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:58:05.487 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.777243415000001, 2024-09-18T04:58:05.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023924748999999999, 2024-09-18T04:58:05.487 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.487 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.487 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.488 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.488 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.488 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26212 2024-09-18T04:58:05.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.488 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:58:05.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:58:05.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640048+0000", 2024-09-18T04:58:05.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640051+0000", 2024-09-18T04:58:05.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.095e-06 2024-09-18T04:58:05.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640060+0000", 2024-09-18T04:58:05.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8929999999999994e-06 2024-09-18T04:58:05.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640065+0000", 2024-09-18T04:58:05.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0270000000000003e-06 2024-09-18T04:58:05.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663824+0000", 2024-09-18T04:58:05.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023758873999999999 2024-09-18T04:58:05.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663904+0000", 2024-09-18T04:58:05.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9856000000000003e-05 2024-09-18T04:58:05.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663974+0000", 2024-09-18T04:58:05.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9450999999999996e-05 2024-09-18T04:58:05.491 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.491 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.491 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.491 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26213 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 667648~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:05.492 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:58:05.492 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.777222244000001, 2024-09-18T04:58:05.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024059812999999999, 2024-09-18T04:58:05.492 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.492 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.492 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.492 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.492 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.492 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26213 2024-09-18T04:58:05.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.492 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:58:05.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:58:05.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:58:05.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640072+0000", 2024-09-18T04:58:05.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4200000000000001e-06 2024-09-18T04:58:05.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640078+0000", 2024-09-18T04:58:05.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0179999999999998e-06 2024-09-18T04:58:05.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640083+0000", 2024-09-18T04:58:05.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:58:05.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664008+0000", 2024-09-18T04:58:05.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023925287 2024-09-18T04:58:05.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664059+0000", 2024-09-18T04:58:05.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0730999999999999e-05 2024-09-18T04:58:05.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664130+0000", 2024-09-18T04:58:05.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0801000000000002e-05 2024-09-18T04:58:05.496 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.496 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.496 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.496 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26214 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 696320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:05.496 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:58:05.496 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.777205273999996, 2024-09-18T04:58:05.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024223738000000002, 2024-09-18T04:58:05.496 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.496 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.497 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.497 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26214 2024-09-18T04:58:05.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.497 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:58:05.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:58:05.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:58:05.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640089+0000", 2024-09-18T04:58:05.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0849999999999999e-06 2024-09-18T04:58:05.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640095+0000", 2024-09-18T04:58:05.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9279999999999998e-06 2024-09-18T04:58:05.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640099+0000", 2024-09-18T04:58:05.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:58:05.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664166+0000", 2024-09-18T04:58:05.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024066513000000001 2024-09-18T04:58:05.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664239+0000", 2024-09-18T04:58:05.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3201999999999995e-05 2024-09-18T04:58:05.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664311+0000", 2024-09-18T04:58:05.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1434999999999996e-05 2024-09-18T04:58:05.500 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.501 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.501 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.501 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26215 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 724992~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:05.501 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:58:05.501 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.777188512999999, 2024-09-18T04:58:05.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024302284, 2024-09-18T04:58:05.501 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.501 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.501 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.501 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.501 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.501 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26215 2024-09-18T04:58:05.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.502 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:58:05.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:58:05.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640103+0000", 2024-09-18T04:58:05.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640106+0000", 2024-09-18T04:58:05.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2120000000000002e-06 2024-09-18T04:58:05.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640111+0000", 2024-09-18T04:58:05.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7880000000000003e-06 2024-09-18T04:58:05.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640116+0000", 2024-09-18T04:58:05.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6099999999999999e-06 2024-09-18T04:58:05.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664327+0000", 2024-09-18T04:58:05.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024210941999999999 2024-09-18T04:58:05.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664367+0000", 2024-09-18T04:58:05.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9666000000000003e-05 2024-09-18T04:58:05.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664406+0000", 2024-09-18T04:58:05.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9402e-05 2024-09-18T04:58:05.505 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.505 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.505 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.506 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26216 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 753664~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:05.506 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:58:05.506 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.777172019000005, 2024-09-18T04:58:05.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024452854999999999, 2024-09-18T04:58:05.506 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.506 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.506 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.507 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.507 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.507 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26216 2024-09-18T04:58:05.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.507 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:58:05.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:58:05.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:58:05.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640122+0000", 2024-09-18T04:58:05.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3599999999999999e-06 2024-09-18T04:58:05.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640128+0000", 2024-09-18T04:58:05.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6339999999999998e-06 2024-09-18T04:58:05.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640132+0000", 2024-09-18T04:58:05.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:58:05.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664436+0000", 2024-09-18T04:58:05.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024303585999999999 2024-09-18T04:58:05.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664507+0000", 2024-09-18T04:58:05.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1046999999999999e-05 2024-09-18T04:58:05.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664573+0000", 2024-09-18T04:58:05.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6217000000000007e-05 2024-09-18T04:58:05.510 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.510 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.510 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.510 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26217 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 782336~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:05.510 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:58:05.511 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 84.777155785000005, 2024-09-18T04:58:05.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024526287000000001, 2024-09-18T04:58:05.511 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.511 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.511 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.511 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.511 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.511 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26217 2024-09-18T04:58:05.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.511 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:58:05.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:58:05.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:58:05.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640140+0000", 2024-09-18T04:58:05.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2989999999999998e-06 2024-09-18T04:58:05.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640146+0000", 2024-09-18T04:58:05.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9569999999999999e-06 2024-09-18T04:58:05.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640151+0000", 2024-09-18T04:58:05.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:58:05.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664590+0000", 2024-09-18T04:58:05.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0244389 2024-09-18T04:58:05.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664628+0000", 2024-09-18T04:58:05.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7910999999999997e-05 2024-09-18T04:58:05.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664663+0000", 2024-09-18T04:58:05.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5024999999999998e-05 2024-09-18T04:58:05.515 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.515 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.515 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.515 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.515 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.515 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:05.516 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:58:05.517 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:05.514+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:05.617 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:05.615+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:05.660 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:05.660 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:05.661 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:58:05.661 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:05.669 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:58:05.718 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:05.715+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:05.771 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:05.772 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:05.772 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:58:05.772 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:58:05.772 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:05.780 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:58:05.820 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:05.817+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:05.922 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:05.919+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:05.923 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:05.923 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:58:05.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:58:05.924 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:05.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.924 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:58:05.924 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:58:05.925 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 114.52720305299999, 2024-09-18T04:58:05.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037718511000000003, 2024-09-18T04:58:05.925 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.925 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:05.925 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.925 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:58:05.925 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:58:05.926 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-18T04:58:05.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.927 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:58:05.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:58:05.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393125+0000", 2024-09-18T04:58:05.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:58:05.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393133+0000", 2024-09-18T04:58:05.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.225e-06 2024-09-18T04:58:05.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393144+0000", 2024-09-18T04:58:05.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1279e-05 2024-09-18T04:58:05.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393153+0000", 2024-09-18T04:58:05.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4540000000000007e-06 2024-09-18T04:58:05.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393203+0000", 2024-09-18T04:58:05.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9963000000000003e-05 2024-09-18T04:58:05.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:05.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393210+0000", 2024-09-18T04:58:05.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5689999999999999e-06 2024-09-18T04:58:05.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.930 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.930 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427399+0000", 2024-09-18T04:58:05.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034188784999999999 2024-09-18T04:58:05.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427532+0000", 2024-09-18T04:58:05.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013264700000000001 2024-09-18T04:58:05.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:58:05.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427685+0000", 2024-09-18T04:58:05.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015300599999999999 2024-09-18T04:58:05.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:05.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.428520+0000", 2024-09-18T04:58:05.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00083502499999999998 2024-09-18T04:58:05.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:05.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430772+0000", 2024-09-18T04:58:05.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022518880000000001 2024-09-18T04:58:05.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:05.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430817+0000", 2024-09-18T04:58:05.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5077000000000002e-05 2024-09-18T04:58:05.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430845+0000", 2024-09-18T04:58:05.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8036999999999999e-05 2024-09-18T04:58:05.933 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.933 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.933 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.933 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:62520 3.13 3.84cdb893 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T04:58:05.933 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:05.933 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 74.622893169999998, 2024-09-18T04:58:05.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94452134799999998, 2024-09-18T04:58:05.933 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.933 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:58:05.933 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.934 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.934 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.934 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 62520 2024-09-18T04:58:05.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.934 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:05.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:05.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:05.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.935 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297441+0000", 2024-09-18T04:58:05.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9840000000000001e-06 2024-09-18T04:58:05.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.935 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297449+0000", 2024-09-18T04:58:05.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3999999999999992e-06 2024-09-18T04:58:05.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297455+0000", 2024-09-18T04:58:05.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3350000000000003e-06 2024-09-18T04:58:05.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297552+0000", 2024-09-18T04:58:05.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6602999999999996e-05 2024-09-18T04:58:05.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:05.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297559+0000", 2024-09-18T04:58:05.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.52e-06 2024-09-18T04:58:05.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241943+0000", 2024-09-18T04:58:05.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94438357100000003 2024-09-18T04:58:05.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241958+0000", 2024-09-18T04:58:05.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4598e-05 2024-09-18T04:58:05.938 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.938 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.938 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.938 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:113179 3.13 3:cf138017:::rbd_data.11ac5f86a3f9.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T04:58:05.938 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:58:05.938 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 47.233102758999998, 2024-09-18T04:58:05.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036508304999999998, 2024-09-18T04:58:05.939 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.939 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:05.939 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.939 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.939 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.939 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 113179 2024-09-18T04:58:05.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.939 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:58:05.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:58:05.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687226+0000", 2024-09-18T04:58:05.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:58:05.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688169+0000", 2024-09-18T04:58:05.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00094328200000000004 2024-09-18T04:58:05.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688182+0000", 2024-09-18T04:58:05.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2756e-05 2024-09-18T04:58:05.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688188+0000", 2024-09-18T04:58:05.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0290000000000002e-06 2024-09-18T04:58:05.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688225+0000", 2024-09-18T04:58:05.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7301000000000001e-05 2024-09-18T04:58:05.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688324+0000", 2024-09-18T04:58:05.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9419999999999993e-05 2024-09-18T04:58:05.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:58:05.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.700483+0000", 2024-09-18T04:58:05.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012158771000000001 2024-09-18T04:58:05.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:05.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723586+0000", 2024-09-18T04:58:05.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023102952 2024-09-18T04:58:05.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:05.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723634+0000", 2024-09-18T04:58:05.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7468e-05 2024-09-18T04:58:05.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:05.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723693+0000", 2024-09-18T04:58:05.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9539e-05 2024-09-18T04:58:05.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723735+0000", 2024-09-18T04:58:05.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1829000000000002e-05 2024-09-18T04:58:05.945 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.945 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.945 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.945 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146077 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1400832~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:05.945 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593265+0000", 2024-09-18T04:58:05.945 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.327064381, 2024-09-18T04:58:05.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035380397000000001, 2024-09-18T04:58:05.945 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.945 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.945 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.946 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.946 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.946 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146077 2024-09-18T04:58:05.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.946 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593265+0000", 2024-09-18T04:58:05.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593265+0000", 2024-09-18T04:58:05.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593265+0000", 2024-09-18T04:58:05.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:05.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593266+0000", 2024-09-18T04:58:05.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1039999999999999e-06 2024-09-18T04:58:05.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593269+0000", 2024-09-18T04:58:05.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8760000000000001e-06 2024-09-18T04:58:05.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593271+0000", 2024-09-18T04:58:05.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5100000000000001e-06 2024-09-18T04:58:05.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628545+0000", 2024-09-18T04:58:05.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035273977999999998 2024-09-18T04:58:05.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628589+0000", 2024-09-18T04:58:05.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4088999999999997e-05 2024-09-18T04:58:05.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628645+0000", 2024-09-18T04:58:05.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6059e-05 2024-09-18T04:58:05.952 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.953 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.953 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.953 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146078 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1449984~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:05.953 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:05.953 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.327024915999999, 2024-09-18T04:58:05.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035495325000000001, 2024-09-18T04:58:05.953 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.953 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.953 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.954 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.954 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.954 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146078 2024-09-18T04:58:05.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.954 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:05.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:05.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:05.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593306+0000", 2024-09-18T04:58:05.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.686e-06 2024-09-18T04:58:05.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593311+0000", 2024-09-18T04:58:05.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8500000000000002e-06 2024-09-18T04:58:05.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593314+0000", 2024-09-18T04:58:05.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0290000000000001e-06 2024-09-18T04:58:05.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628674+0000", 2024-09-18T04:58:05.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035360346000000001 2024-09-18T04:58:05.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628726+0000", 2024-09-18T04:58:05.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2219999999999998e-05 2024-09-18T04:58:05.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628800+0000", 2024-09-18T04:58:05.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3613000000000001e-05 2024-09-18T04:58:05.959 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.959 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.959 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.960 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146079 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1499136~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:05.960 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:05.960 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.327013510999997, 2024-09-18T04:58:05.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035604581000000003, 2024-09-18T04:58:05.960 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.960 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.960 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.960 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.960 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.960 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146079 2024-09-18T04:58:05.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.961 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:05.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:05.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:05.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:05.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593317+0000", 2024-09-18T04:58:05.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1599999999999999e-06 2024-09-18T04:58:05.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593320+0000", 2024-09-18T04:58:05.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0130000000000002e-06 2024-09-18T04:58:05.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593322+0000", 2024-09-18T04:58:05.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6079999999999998e-06 2024-09-18T04:58:05.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628825+0000", 2024-09-18T04:58:05.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035502747000000001 2024-09-18T04:58:05.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628868+0000", 2024-09-18T04:58:05.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2719999999999998e-05 2024-09-18T04:58:05.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628920+0000", 2024-09-18T04:58:05.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2533e-05 2024-09-18T04:58:05.964 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.964 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.964 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.965 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146080 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1548288~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:05.965 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:05.965 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.326998013000001, 2024-09-18T04:58:05.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035755146000000002, 2024-09-18T04:58:05.965 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.965 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.965 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.965 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.965 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.965 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146080 2024-09-18T04:58:05.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.965 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:05.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:05.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:05.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593333+0000", 2024-09-18T04:58:05.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.858e-06 2024-09-18T04:58:05.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593336+0000", 2024-09-18T04:58:05.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4910000000000001e-06 2024-09-18T04:58:05.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593340+0000", 2024-09-18T04:58:05.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2590000000000001e-06 2024-09-18T04:58:05.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628956+0000", 2024-09-18T04:58:05.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035616039000000002 2024-09-18T04:58:05.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629031+0000", 2024-09-18T04:58:05.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5504000000000001e-05 2024-09-18T04:58:05.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629086+0000", 2024-09-18T04:58:05.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5429000000000001e-05 2024-09-18T04:58:05.969 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.969 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.969 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.969 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146081 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1597440~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:05.969 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:05.970 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.326987441, 2024-09-18T04:58:05.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035873223000000003, 2024-09-18T04:58:05.970 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.970 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.970 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.970 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.970 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.970 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146081 2024-09-18T04:58:05.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.971 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:05.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:05.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:05.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593343+0000", 2024-09-18T04:58:05.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3319999999999999e-06 2024-09-18T04:58:05.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593346+0000", 2024-09-18T04:58:05.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9059999999999998e-06 2024-09-18T04:58:05.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593348+0000", 2024-09-18T04:58:05.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4530000000000001e-06 2024-09-18T04:58:05.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629110+0000", 2024-09-18T04:58:05.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035761814000000003 2024-09-18T04:58:05.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629159+0000", 2024-09-18T04:58:05.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8971000000000003e-05 2024-09-18T04:58:05.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629215+0000", 2024-09-18T04:58:05.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6107000000000003e-05 2024-09-18T04:58:05.974 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.974 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.974 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.975 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146082 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1646592~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:05.975 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:05.975 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.326971391000001, 2024-09-18T04:58:05.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035987007000000001, 2024-09-18T04:58:05.975 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.975 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.975 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.975 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.975 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.975 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146082 2024-09-18T04:58:05.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.976 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:05.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:05.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:05.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593360+0000", 2024-09-18T04:58:05.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8560000000000001e-06 2024-09-18T04:58:05.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593365+0000", 2024-09-18T04:58:05.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3410000000000002e-06 2024-09-18T04:58:05.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593369+0000", 2024-09-18T04:58:05.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4850000000000003e-06 2024-09-18T04:58:05.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629242+0000", 2024-09-18T04:58:05.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035872464999999999 2024-09-18T04:58:05.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629288+0000", 2024-09-18T04:58:05.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6592999999999999e-05 2024-09-18T04:58:05.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629345+0000", 2024-09-18T04:58:05.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6575e-05 2024-09-18T04:58:05.980 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.980 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.981 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.981 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146083 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1695744~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:05.981 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:58:05.981 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.326955544, 2024-09-18T04:58:05.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036100587000000003, 2024-09-18T04:58:05.981 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.981 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.981 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.981 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.982 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.982 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146083 2024-09-18T04:58:05.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.982 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:58:05.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:58:05.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593373+0000", 2024-09-18T04:58:05.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593376+0000", 2024-09-18T04:58:05.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.244e-06 2024-09-18T04:58:05.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593379+0000", 2024-09-18T04:58:05.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.619e-06 2024-09-18T04:58:05.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593382+0000", 2024-09-18T04:58:05.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7199999999999998e-06 2024-09-18T04:58:05.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629368+0000", 2024-09-18T04:58:05.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035986281000000002 2024-09-18T04:58:05.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629416+0000", 2024-09-18T04:58:05.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8055e-05 2024-09-18T04:58:05.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629474+0000", 2024-09-18T04:58:05.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8176e-05 2024-09-18T04:58:05.987 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.987 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.987 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.988 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146084 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1744896~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:05.988 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:05.988 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.326942414000001, 2024-09-18T04:58:05.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036214349, 2024-09-18T04:58:05.988 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.988 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.988 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.988 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.988 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.989 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146084 2024-09-18T04:58:05.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.989 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:05.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:05.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:05.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:05.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593388+0000", 2024-09-18T04:58:05.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.564e-06 2024-09-18T04:58:05.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593392+0000", 2024-09-18T04:58:05.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0579999999999999e-06 2024-09-18T04:58:05.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.991 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593395+0000", 2024-09-18T04:58:05.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8890000000000002e-06 2024-09-18T04:58:05.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629500+0000", 2024-09-18T04:58:05.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036105343999999998 2024-09-18T04:58:05.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629547+0000", 2024-09-18T04:58:05.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7036000000000002e-05 2024-09-18T04:58:05.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629601+0000", 2024-09-18T04:58:05.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3866999999999998e-05 2024-09-18T04:58:05.992 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.993 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.993 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.993 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146085 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1794048~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:05.993 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:05.993 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.326932026999998, 2024-09-18T04:58:05.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036331186000000001, 2024-09-18T04:58:05.993 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.993 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.993 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.993 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.993 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.994 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146085 2024-09-18T04:58:05.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.994 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:05.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:05.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:05.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:05.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:05.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593398+0000", 2024-09-18T04:58:05.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2270000000000001e-06 2024-09-18T04:58:05.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:05.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593401+0000", 2024-09-18T04:58:05.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.999e-06 2024-09-18T04:58:05.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:05.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593404+0000", 2024-09-18T04:58:05.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.486e-06 2024-09-18T04:58:05.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:05.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629624+0000", 2024-09-18T04:58:05.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036219983999999997 2024-09-18T04:58:05.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:05.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629672+0000", 2024-09-18T04:58:05.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8368000000000002e-05 2024-09-18T04:58:05.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:05.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629729+0000", 2024-09-18T04:58:05.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6359e-05 2024-09-18T04:58:05.997 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.997 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:05.997 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:05.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.998 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146086 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1843200~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:05.998 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:05.998 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.326923463, 2024-09-18T04:58:05.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036449098999999999, 2024-09-18T04:58:05.998 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:05.998 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:05.998 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:05.998 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:05.998 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:05.998 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146086 2024-09-18T04:58:05.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.998 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:05.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:05.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:05.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:05.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:05.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:05.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:05.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:05.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:05.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:05.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:05.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:06.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593407+0000", 2024-09-18T04:58:06.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1799999999999999e-06 2024-09-18T04:58:06.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:06.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593410+0000", 2024-09-18T04:58:06.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.926e-06 2024-09-18T04:58:06.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593413+0000", 2024-09-18T04:58:06.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8250000000000001e-06 2024-09-18T04:58:06.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:06.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629755+0000", 2024-09-18T04:58:06.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036342212999999998 2024-09-18T04:58:06.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:06.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629803+0000", 2024-09-18T04:58:06.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8619000000000003e-05 2024-09-18T04:58:06.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:06.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629855+0000", 2024-09-18T04:58:06.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1536000000000003e-05 2024-09-18T04:58:06.002 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:06.002 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:06.002 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:06.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.002 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146087 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1892352~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:06.002 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:06.002 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.326914537999997, 2024-09-18T04:58:06.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036564608999999998, 2024-09-18T04:58:06.002 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:06.002 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:06.002 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:06.003 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:06.003 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.003 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146087 2024-09-18T04:58:06.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.003 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:06.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:06.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:06.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:06.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:06.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:06.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:06.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:06.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:06.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:06.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:06.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593416+0000", 2024-09-18T04:58:06.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.218e-06 2024-09-18T04:58:06.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:06.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593419+0000", 2024-09-18T04:58:06.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-18T04:58:06.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593421+0000", 2024-09-18T04:58:06.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4329999999999998e-06 2024-09-18T04:58:06.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:06.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629878+0000", 2024-09-18T04:58:06.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036456578000000003 2024-09-18T04:58:06.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:06.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629925+0000", 2024-09-18T04:58:06.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7595000000000001e-05 2024-09-18T04:58:06.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:06.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629979+0000", 2024-09-18T04:58:06.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4117999999999999e-05 2024-09-18T04:58:06.006 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:06.006 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:06.006 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146088 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1941504~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.326892374000003, 2024-09-18T04:58:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036676121999999999, 2024-09-18T04:58:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146088 2024-09-18T04:58:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.007 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:06.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593438+0000", 2024-09-18T04:58:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7570000000000001e-06 2024-09-18T04:58:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593442+0000", 2024-09-18T04:58:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2880000000000001e-06 2024-09-18T04:58:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593444+0000", 2024-09-18T04:58:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.593e-06 2024-09-18T04:58:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630015+0000", 2024-09-18T04:58:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036571125000000003 2024-09-18T04:58:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:06.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630061+0000", 2024-09-18T04:58:06.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5445000000000003e-05 2024-09-18T04:58:06.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:06.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630113+0000", 2024-09-18T04:58:06.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2358000000000003e-05 2024-09-18T04:58:06.011 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:06.012 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:06.012 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:06.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.012 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146089 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1990656~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:06.012 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:58:06.012 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.326882793000003, 2024-09-18T04:58:06.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036791483999999999, 2024-09-18T04:58:06.012 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:06.012 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:06.012 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:06.012 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:06.013 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.013 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146089 2024-09-18T04:58:06.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.013 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:06.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:58:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:58:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593446+0000", 2024-09-18T04:58:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:06.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593448+0000", 2024-09-18T04:58:06.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1939999999999999e-06 2024-09-18T04:58:06.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:06.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593451+0000", 2024-09-18T04:58:06.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.766e-06 2024-09-18T04:58:06.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593458+0000", 2024-09-18T04:58:06.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8539999999999999e-06 2024-09-18T04:58:06.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:06.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630136+0000", 2024-09-18T04:58:06.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03667757 2024-09-18T04:58:06.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:06.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630183+0000", 2024-09-18T04:58:06.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7289000000000001e-05 2024-09-18T04:58:06.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:06.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630238+0000", 2024-09-18T04:58:06.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5013000000000003e-05 2024-09-18T04:58:06.019 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:06.019 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:06.020 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:06.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.020 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146090 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 2039808~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:06.020 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:06.020 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.326868912999998, 2024-09-18T04:58:06.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036904477999999998, 2024-09-18T04:58:06.021 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:06.021 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:06.021 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:06.021 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:06.021 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.021 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146090 2024-09-18T04:58:06.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.021 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:06.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:06.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:06.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:06.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:06.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:06.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:06.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:06.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:06.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:06.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:06.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593461+0000", 2024-09-18T04:58:06.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3680000000000001e-06 2024-09-18T04:58:06.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:06.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593465+0000", 2024-09-18T04:58:06.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8120000000000001e-06 2024-09-18T04:58:06.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593468+0000", 2024-09-18T04:58:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.582e-06 2024-09-18T04:58:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630268+0000", 2024-09-18T04:58:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036800449999999998 2024-09-18T04:58:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:06.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630313+0000", 2024-09-18T04:58:06.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4836000000000002e-05 2024-09-18T04:58:06.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:06.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630365+0000", 2024-09-18T04:58:06.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1839999999999998e-05 2024-09-18T04:58:06.026 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:06.026 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:06.026 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:06.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.026 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146091 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 2088960~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:06.026 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 32.326858451, 2024-09-18T04:58:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037011534999999998, 2024-09-18T04:58:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146091 2024-09-18T04:58:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:06.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:06.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593472+0000", 2024-09-18T04:58:06.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1620000000000001e-06 2024-09-18T04:58:06.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:06.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593475+0000", 2024-09-18T04:58:06.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0019999999999998e-06 2024-09-18T04:58:06.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593477+0000", 2024-09-18T04:58:06.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-18T04:58:06.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630381+0000", 2024-09-18T04:58:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036904014999999998 2024-09-18T04:58:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630428+0000", 2024-09-18T04:58:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6931000000000002e-05 2024-09-18T04:58:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630482+0000", 2024-09-18T04:58:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4138999999999998e-05 2024-09-18T04:58:06.031 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:06.032 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:06.032 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:06.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.032 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:162075 3.13 3:cf3f6048:::rbd_header.11ac6967455f:head [watch reconnect cookie 140406508629968 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T04:58:06.032 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:58:06.032 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.429885645999999, 2024-09-18T04:58:06.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.53184266599999996, 2024-09-18T04:58:06.032 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:06.032 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:06.032 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:06.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:06.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.033 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 162075 2024-09-18T04:58:06.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.033 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:06.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:06.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:58:06.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:06.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:58:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490442+0000", 2024-09-18T04:58:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:58:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490453+0000", 2024-09-18T04:58:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0613e-05 2024-09-18T04:58:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490466+0000", 2024-09-18T04:58:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3702e-05 2024-09-18T04:58:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490475+0000", 2024-09-18T04:58:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2690000000000005e-06 2024-09-18T04:58:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490612+0000", 2024-09-18T04:58:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013620900000000001 2024-09-18T04:58:06.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-18T04:58:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490620+0000", 2024-09-18T04:58:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3089999999999996e-06 2024-09-18T04:58:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503041+0000", 2024-09-18T04:58:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012420512 2024-09-18T04:58:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503048+0000", 2024-09-18T04:58:06.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0389999999999999e-06 2024-09-18T04:58:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.016122+0000", 2024-09-18T04:58:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.51307432900000005 2024-09-18T04:58:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.016289+0000", 2024-09-18T04:58:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000166875 2024-09-18T04:58:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:06.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.022286+0000", 2024-09-18T04:58:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059976049999999996 2024-09-18T04:58:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:162076 3.13 3:cf3f6048:::rbd_header.11ac6967455f:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T04:58:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.429762971999999, 2024-09-18T04:58:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.53464117600000005, 2024-09-18T04:58:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:06.038 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:06.039 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.039 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 162076 2024-09-18T04:58:06.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.039 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:06.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.039 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:06.039 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:06.039 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:06.039 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.039 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:58:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490571+0000", 2024-09-18T04:58:06.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8259999999999998e-06 2024-09-18T04:58:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490584+0000", 2024-09-18T04:58:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2391999999999999e-05 2024-09-18T04:58:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490591+0000", 2024-09-18T04:58:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5460000000000003e-06 2024-09-18T04:58:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490714+0000", 2024-09-18T04:58:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000122543 2024-09-18T04:58:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-18T04:58:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490731+0000", 2024-09-18T04:58:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7405e-05 2024-09-18T04:58:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503069+0000", 2024-09-18T04:58:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012337836 2024-09-18T04:58:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503078+0000", 2024-09-18T04:58:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8289999999999997e-06 2024-09-18T04:58:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019018+0000", 2024-09-18T04:58:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.51594035900000002 2024-09-18T04:58:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019091+0000", 2024-09-18T04:58:06.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2608999999999995e-05 2024-09-18T04:58:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-18T04:58:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019421+0000", 2024-09-18T04:58:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00033004300000000002 2024-09-18T04:58:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.021644+0000", 2024-09-18T04:58:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022229749999999999 2024-09-18T04:58:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.044 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025113+0000", 2024-09-18T04:58:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003469228 2024-09-18T04:58:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025167+0000", 2024-09-18T04:58:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3569999999999997e-05 2024-09-18T04:58:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025208+0000", 2024-09-18T04:58:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0821000000000001e-05 2024-09-18T04:58:06.045 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:06.046 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:06.046 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:06.046 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:06.046 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:06.046 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:06.047 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:58:06.048 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:06.045+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:06.148 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:06.146+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:06.166 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:06.166 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:06.166 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:58:06.166 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:06.175 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:58:06.249 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:06.247+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:06.283 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:06.283 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:06.283 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:58:06.283 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:58:06.283 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:06.292 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:58:06.350 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:06.348+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:06.441 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:06.441 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:58:06.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:58:06.441 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:58:06.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.441 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9316 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:06.441 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:58:06.441 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 90.670653556999994, 2024-09-18T04:58:06.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044027208999999998, 2024-09-18T04:58:06.441 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:06.442 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:06.442 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:06.442 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:06.442 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.442 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9316 2024-09-18T04:58:06.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.442 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:06.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:06.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:58:06.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:06.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:58:06.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:06.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:58:06.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:06.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:06.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767144+0000", 2024-09-18T04:58:06.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3070000000000001e-06 2024-09-18T04:58:06.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:06.444 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767150+0000", 2024-09-18T04:58:06.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1700000000000002e-06 2024-09-18T04:58:06.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.444 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767163+0000", 2024-09-18T04:58:06.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2670999999999999e-05 2024-09-18T04:58:06.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:06.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810995+0000", 2024-09-18T04:58:06.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043832310999999999 2024-09-18T04:58:06.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:06.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811087+0000", 2024-09-18T04:58:06.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1345e-05 2024-09-18T04:58:06.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:06.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811169+0000", 2024-09-18T04:58:06.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2848000000000004e-05 2024-09-18T04:58:06.446 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.446 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:06.446 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.446 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9317 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1728512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:06.446 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:58:06.446 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 90.670627921999994, 2024-09-18T04:58:06.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044190170000000001, 2024-09-18T04:58:06.447 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:06.447 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:06.447 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:06.447 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:06.447 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.447 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9317 2024-09-18T04:58:06.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.447 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:06.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:06.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:58:06.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:06.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:58:06.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:06.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767167+0000", 2024-09-18T04:58:06.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:06.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:06.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767170+0000", 2024-09-18T04:58:06.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7259999999999998e-06 2024-09-18T04:58:06.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:06.449 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767176+0000", 2024-09-18T04:58:06.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1460000000000001e-06 2024-09-18T04:58:06.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767182+0000", 2024-09-18T04:58:06.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1020000000000002e-06 2024-09-18T04:58:06.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:06.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811204+0000", 2024-09-18T04:58:06.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044022039999999998 2024-09-18T04:58:06.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:06.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811286+0000", 2024-09-18T04:58:06.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1623000000000001e-05 2024-09-18T04:58:06.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:06.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811358+0000", 2024-09-18T04:58:06.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2125000000000005e-05 2024-09-18T04:58:06.451 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.451 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:06.451 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.451 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9318 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1777664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:06.452 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:58:06.452 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 90.670609377000005, 2024-09-18T04:58:06.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044380181999999997, 2024-09-18T04:58:06.452 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:06.452 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:06.452 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:06.452 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:06.452 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.452 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9318 2024-09-18T04:58:06.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.452 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:06.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:06.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:58:06.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:06.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:58:06.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:06.453 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:58:06.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:06.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.453 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:06.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767189+0000", 2024-09-18T04:58:06.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5909999999999998e-06 2024-09-18T04:58:06.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:06.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767194+0000", 2024-09-18T04:58:06.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7690000000000002e-06 2024-09-18T04:58:06.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767199+0000", 2024-09-18T04:58:06.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189999999999999e-06 2024-09-18T04:58:06.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:06.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811394+0000", 2024-09-18T04:58:06.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044195350000000001 2024-09-18T04:58:06.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:06.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811483+0000", 2024-09-18T04:58:06.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8282999999999995e-05 2024-09-18T04:58:06.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:06.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811567+0000", 2024-09-18T04:58:06.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3855000000000003e-05 2024-09-18T04:58:06.456 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.456 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:06.456 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.456 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9325 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:06.456 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:58:06.456 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 90.668878839000001, 2024-09-18T04:58:06.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044063989999999997, 2024-09-18T04:58:06.457 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:06.457 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:06.457 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:06.457 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:06.457 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.457 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9325 2024-09-18T04:58:06.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.457 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:06.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:06.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:58:06.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:06.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:58:06.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:06.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:58:06.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:06.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:06.458 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768919+0000", 2024-09-18T04:58:06.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:58:06.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:06.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768925+0000", 2024-09-18T04:58:06.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2310000000000001e-06 2024-09-18T04:58:06.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768929+0000", 2024-09-18T04:58:06.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4469999999999999e-06 2024-09-18T04:58:06.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:06.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812823+0000", 2024-09-18T04:58:06.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043893975000000002 2024-09-18T04:58:06.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:06.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812908+0000", 2024-09-18T04:58:06.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4862000000000002e-05 2024-09-18T04:58:06.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:06.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812981+0000", 2024-09-18T04:58:06.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2892000000000006e-05 2024-09-18T04:58:06.460 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.461 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:06.461 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.461 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9334 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:06.461 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:58:06.461 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 90.666860052999994, 2024-09-18T04:58:06.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044042853999999999, 2024-09-18T04:58:06.461 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:06.461 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:06.461 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:06.461 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:06.461 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.461 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9334 2024-09-18T04:58:06.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.462 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:06.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:06.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:58:06.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.462 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:06.462 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:58:06.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:06.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770935+0000", 2024-09-18T04:58:06.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:06.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:06.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770937+0000", 2024-09-18T04:58:06.463 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1489999999999999e-06 2024-09-18T04:58:06.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.463 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:06.463 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770940+0000", 2024-09-18T04:58:06.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9569999999999998e-06 2024-09-18T04:58:06.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770945+0000", 2024-09-18T04:58:06.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:58:06.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:06.464 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814838+0000", 2024-09-18T04:58:06.464 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043892635999999999 2024-09-18T04:58:06.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.464 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:06.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814920+0000", 2024-09-18T04:58:06.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2175000000000005e-05 2024-09-18T04:58:06.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.465 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:06.465 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814979+0000", 2024-09-18T04:58:06.465 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8372000000000003e-05 2024-09-18T04:58:06.465 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.465 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:06.465 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.465 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9352 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1159168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:06.466 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:58:06.466 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 90.664638875999998, 2024-09-18T04:58:06.466 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044011269999999998, 2024-09-18T04:58:06.466 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:06.466 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:06.466 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:06.466 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:06.466 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.466 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9352 2024-09-18T04:58:06.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.466 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:06.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.466 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:06.466 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:58:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:58:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773156+0000", 2024-09-18T04:58:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.467 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:06.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773159+0000", 2024-09-18T04:58:06.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2919999999999998e-06 2024-09-18T04:58:06.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:06.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773165+0000", 2024-09-18T04:58:06.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0580000000000003e-06 2024-09-18T04:58:06.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.468 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.468 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773177+0000", 2024-09-18T04:58:06.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.204e-05 2024-09-18T04:58:06.468 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:06.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817082+0000", 2024-09-18T04:58:06.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043905528999999999 2024-09-18T04:58:06.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:06.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817122+0000", 2024-09-18T04:58:06.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9258999999999998e-05 2024-09-18T04:58:06.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:06.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817168+0000", 2024-09-18T04:58:06.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6545000000000002e-05 2024-09-18T04:58:06.469 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.470 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:06.470 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.470 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9353 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1208320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:06.470 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:58:06.470 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 90.664614577999998, 2024-09-18T04:58:06.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044105537, 2024-09-18T04:58:06.470 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:06.470 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:06.470 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:06.470 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:06.470 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.470 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9353 2024-09-18T04:58:06.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.471 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:06.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:06.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:58:06.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:06.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:58:06.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:06.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:58:06.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:06.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:06.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773183+0000", 2024-09-18T04:58:06.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2929999999999999e-06 2024-09-18T04:58:06.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:06.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773190+0000", 2024-09-18T04:58:06.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4330000000000001e-06 2024-09-18T04:58:06.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773194+0000", 2024-09-18T04:58:06.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.865e-06 2024-09-18T04:58:06.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.473 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:06.473 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817196+0000", 2024-09-18T04:58:06.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044001708 2024-09-18T04:58:06.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:06.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817236+0000", 2024-09-18T04:58:06.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0043999999999998e-05 2024-09-18T04:58:06.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:06.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817287+0000", 2024-09-18T04:58:06.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0605000000000001e-05 2024-09-18T04:58:06.474 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.474 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:06.474 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.475 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9354 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:06.475 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:58:06.475 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 90.664597067000003, 2024-09-18T04:58:06.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044199796, 2024-09-18T04:58:06.475 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:06.475 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:06.475 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:06.475 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:06.475 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.475 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9354 2024-09-18T04:58:06.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.476 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:06.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:06.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:58:06.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:06.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:58:06.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:06.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773198+0000", 2024-09-18T04:58:06.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:06.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:06.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773200+0000", 2024-09-18T04:58:06.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4159999999999998e-06 2024-09-18T04:58:06.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.479 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:06.479 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773206+0000", 2024-09-18T04:58:06.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.789e-06 2024-09-18T04:58:06.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773210+0000", 2024-09-18T04:58:06.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2679999999999996e-06 2024-09-18T04:58:06.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:06.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817314+0000", 2024-09-18T04:58:06.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044103187000000002 2024-09-18T04:58:06.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:06.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817353+0000", 2024-09-18T04:58:06.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8973000000000001e-05 2024-09-18T04:58:06.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:06.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817399+0000", 2024-09-18T04:58:06.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5877999999999997e-05 2024-09-18T04:58:06.481 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.481 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:06.482 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.482 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46591 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2097152~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:06.485 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:58:06.485 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 76.739740650000002, 2024-09-18T04:58:06.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043987310000000002, 2024-09-18T04:58:06.485 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:06.485 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:06.485 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:06.485 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:06.485 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.485 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46591 2024-09-18T04:58:06.485 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.486 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:06.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:06.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:58:06.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:06.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:58:06.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:06.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:58:06.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:06.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:06.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698057+0000", 2024-09-18T04:58:06.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.531e-06 2024-09-18T04:58:06.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:06.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698064+0000", 2024-09-18T04:58:06.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-18T04:58:06.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698069+0000", 2024-09-18T04:58:06.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.091e-06 2024-09-18T04:58:06.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:06.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741843+0000", 2024-09-18T04:58:06.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043773470000000002 2024-09-18T04:58:06.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:06.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741928+0000", 2024-09-18T04:58:06.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5113999999999999e-05 2024-09-18T04:58:06.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:06.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742042+0000", 2024-09-18T04:58:06.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114527 2024-09-18T04:58:06.489 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.489 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:06.489 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.489 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46592 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2121728~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:06.490 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:58:06.490 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 76.739722223000001, 2024-09-18T04:58:06.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044077948999999998, 2024-09-18T04:58:06.490 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:06.490 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:06.490 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:06.490 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:06.490 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.490 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46592 2024-09-18T04:58:06.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.490 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:06.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:06.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:58:06.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:06.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:58:06.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:06.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698073+0000", 2024-09-18T04:58:06.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:06.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:06.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698076+0000", 2024-09-18T04:58:06.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.481e-06 2024-09-18T04:58:06.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:06.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698082+0000", 2024-09-18T04:58:06.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8730000000000004e-06 2024-09-18T04:58:06.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698086+0000", 2024-09-18T04:58:06.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:58:06.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:06.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742055+0000", 2024-09-18T04:58:06.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043969170000000002 2024-09-18T04:58:06.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:06.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742093+0000", 2024-09-18T04:58:06.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7707000000000003e-05 2024-09-18T04:58:06.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:06.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742152+0000", 2024-09-18T04:58:06.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8323999999999999e-05 2024-09-18T04:58:06.494 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.494 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:06.494 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.494 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46593 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2211840~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:06.494 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:58:06.494 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 76.739705319999999, 2024-09-18T04:58:06.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044259917000000003, 2024-09-18T04:58:06.494 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:06.494 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:06.494 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:06.495 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:06.495 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.495 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46593 2024-09-18T04:58:06.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.495 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:06.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:06.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:58:06.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:06.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:58:06.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:06.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:58:06.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:06.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:06.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698092+0000", 2024-09-18T04:58:06.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2249999999999999e-06 2024-09-18T04:58:06.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:06.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698098+0000", 2024-09-18T04:58:06.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:58:06.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698103+0000", 2024-09-18T04:58:06.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:58:06.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:06.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742182+0000", 2024-09-18T04:58:06.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044079508000000003 2024-09-18T04:58:06.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:06.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742257+0000", 2024-09-18T04:58:06.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4678000000000006e-05 2024-09-18T04:58:06.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:06.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742350+0000", 2024-09-18T04:58:06.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3531999999999998e-05 2024-09-18T04:58:06.498 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.498 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:06.499 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.499 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46594 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2236416~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:06.499 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:58:06.499 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 76.739689623000004, 2024-09-18T04:58:06.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044378984000000003, 2024-09-18T04:58:06.499 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:06.499 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:06.499 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:06.499 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:06.499 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.499 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46594 2024-09-18T04:58:06.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.500 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:06.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:06.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:58:06.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:06.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:58:06.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:06.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:58:06.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:06.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:06.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698108+0000", 2024-09-18T04:58:06.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0880000000000002e-06 2024-09-18T04:58:06.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:06.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698114+0000", 2024-09-18T04:58:06.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6269999999999997e-06 2024-09-18T04:58:06.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698118+0000", 2024-09-18T04:58:06.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4050000000000002e-06 2024-09-18T04:58:06.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:06.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742365+0000", 2024-09-18T04:58:06.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044246799000000003 2024-09-18T04:58:06.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:06.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742407+0000", 2024-09-18T04:58:06.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2008000000000002e-05 2024-09-18T04:58:06.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:06.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742485+0000", 2024-09-18T04:58:06.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8405999999999994e-05 2024-09-18T04:58:06.503 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.503 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:06.503 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.503 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46595 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:06.503 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:58:06.503 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 76.739673550000006, 2024-09-18T04:58:06.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.04440463, 2024-09-18T04:58:06.503 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:06.504 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:06.504 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:06.504 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:06.504 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.504 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46595 2024-09-18T04:58:06.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.504 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:06.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:06.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:58:06.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:06.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:58:06.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:06.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:58:06.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:06.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.505 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:06.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698124+0000", 2024-09-18T04:58:06.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1840000000000002e-06 2024-09-18T04:58:06.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:06.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698130+0000", 2024-09-18T04:58:06.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.947e-06 2024-09-18T04:58:06.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698136+0000", 2024-09-18T04:58:06.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6520000000000003e-06 2024-09-18T04:58:06.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:06.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742521+0000", 2024-09-18T04:58:06.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044385446000000002 2024-09-18T04:58:06.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:06.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742527+0000", 2024-09-18T04:58:06.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:58:06.507 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.507 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:06.507 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.508 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46596 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:06.508 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:58:06.508 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 76.739655306000003, 2024-09-18T04:58:06.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044418066999999999, 2024-09-18T04:58:06.508 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:06.508 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:06.508 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:06.508 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:06.508 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.508 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46596 2024-09-18T04:58:06.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.508 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:06.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:06.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:58:06.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:06.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:58:06.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:06.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:58:06.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:06.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:06.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698146+0000", 2024-09-18T04:58:06.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2779999999999999e-06 2024-09-18T04:58:06.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:06.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698153+0000", 2024-09-18T04:58:06.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3849999999999999e-06 2024-09-18T04:58:06.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698164+0000", 2024-09-18T04:58:06.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1375999999999999e-05 2024-09-18T04:58:06.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:06.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742554+0000", 2024-09-18T04:58:06.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044389482000000001 2024-09-18T04:58:06.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:06.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742559+0000", 2024-09-18T04:58:06.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9540000000000003e-06 2024-09-18T04:58:06.511 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.511 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:06.511 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.512 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46597 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:06.512 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:58:06.512 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 76.739626940999997, 2024-09-18T04:58:06.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044416513999999997, 2024-09-18T04:58:06.512 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:06.512 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:06.512 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:06.512 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:06.512 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.512 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46597 2024-09-18T04:58:06.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.512 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:06.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:06.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:58:06.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:06.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:58:06.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:06.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698168+0000", 2024-09-18T04:58:06.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:06.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:06.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698175+0000", 2024-09-18T04:58:06.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4520000000000002e-06 2024-09-18T04:58:06.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:06.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698181+0000", 2024-09-18T04:58:06.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4150000000000004e-06 2024-09-18T04:58:06.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698186+0000", 2024-09-18T04:58:06.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7489999999999998e-06 2024-09-18T04:58:06.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:06.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742580+0000", 2024-09-18T04:58:06.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044394325999999998 2024-09-18T04:58:06.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:06.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742585+0000", 2024-09-18T04:58:06.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9849999999999997e-06 2024-09-18T04:58:06.515 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.515 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:06.515 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.516 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46598 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:06.516 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:58:06.516 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 76.739519465000001, 2024-09-18T04:58:06.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044335316999999999, 2024-09-18T04:58:06.516 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:06.516 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:06.516 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:06.516 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:06.516 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.516 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46598 2024-09-18T04:58:06.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.516 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:06.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:06.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:58:06.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:06.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:58:06.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:06.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698190+0000", 2024-09-18T04:58:06.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999137 2024-09-18T04:58:06.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:06.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698280+0000", 2024-09-18T04:58:06.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0409999999999997e-05 2024-09-18T04:58:06.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:06.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698289+0000", 2024-09-18T04:58:06.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9490000000000002e-06 2024-09-18T04:58:06.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698296+0000", 2024-09-18T04:58:06.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2670000000000003e-06 2024-09-18T04:58:06.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:06.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742607+0000", 2024-09-18T04:58:06.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044311072999999999 2024-09-18T04:58:06.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:06.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742612+0000", 2024-09-18T04:58:06.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9400000000000001e-06 2024-09-18T04:58:06.519 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.519 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:06.519 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.520 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57986 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3086336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:06.520 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:58:06.520 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.752673741999999, 2024-09-18T04:58:06.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044047731999999999, 2024-09-18T04:58:06.520 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:06.520 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:06.520 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:06.520 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:06.520 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.520 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57986 2024-09-18T04:58:06.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.520 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:06.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:06.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:58:06.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:06.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:58:06.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:06.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:58:06.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:06.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:06.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685124+0000", 2024-09-18T04:58:06.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2859999999999998e-06 2024-09-18T04:58:06.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:06.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685130+0000", 2024-09-18T04:58:06.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5400000000000001e-06 2024-09-18T04:58:06.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685136+0000", 2024-09-18T04:58:06.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.152e-06 2024-09-18T04:58:06.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:06.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.728976+0000", 2024-09-18T04:58:06.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043840037999999998 2024-09-18T04:58:06.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:06.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729064+0000", 2024-09-18T04:58:06.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8709999999999996e-05 2024-09-18T04:58:06.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:06.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729170+0000", 2024-09-18T04:58:06.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000105386 2024-09-18T04:58:06.524 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.524 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:06.524 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.524 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57987 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3101696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:06.524 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:58:06.524 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.752624767, 2024-09-18T04:58:06.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044238123999999997, 2024-09-18T04:58:06.524 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:06.524 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:06.524 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:06.524 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:06.524 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.525 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57987 2024-09-18T04:58:06.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.525 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:06.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:06.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:58:06.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:06.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:58:06.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:06.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685170+0000", 2024-09-18T04:58:06.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:58:06.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:06.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685175+0000", 2024-09-18T04:58:06.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9069999999999998e-06 2024-09-18T04:58:06.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:06.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685187+0000", 2024-09-18T04:58:06.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.189e-05 2024-09-18T04:58:06.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685195+0000", 2024-09-18T04:58:06.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9459999999999998e-06 2024-09-18T04:58:06.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:06.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729212+0000", 2024-09-18T04:58:06.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044017239 2024-09-18T04:58:06.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:06.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729301+0000", 2024-09-18T04:58:06.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8887999999999993e-05 2024-09-18T04:58:06.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.528 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:06.528 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729409+0000", 2024-09-18T04:58:06.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108119 2024-09-18T04:58:06.528 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.528 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:06.528 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.528 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57988 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3117056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:06.528 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:58:06.529 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.752582648000001, 2024-09-18T04:58:06.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044429835000000001, 2024-09-18T04:58:06.529 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:06.529 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:06.529 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:06.529 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:06.529 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.529 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57988 2024-09-18T04:58:06.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.529 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:06.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:06.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:58:06.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:06.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:58:06.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:06.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685212+0000", 2024-09-18T04:58:06.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:06.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:06.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685216+0000", 2024-09-18T04:58:06.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7390000000000001e-06 2024-09-18T04:58:06.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:06.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685224+0000", 2024-09-18T04:58:06.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2260000000000005e-06 2024-09-18T04:58:06.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685230+0000", 2024-09-18T04:58:06.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0299999999999999e-06 2024-09-18T04:58:06.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:06.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729454+0000", 2024-09-18T04:58:06.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044223248999999999 2024-09-18T04:58:06.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:06.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729539+0000", 2024-09-18T04:58:06.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5029000000000002e-05 2024-09-18T04:58:06.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:06.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729643+0000", 2024-09-18T04:58:06.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104267 2024-09-18T04:58:06.532 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.533 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:06.533 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.533 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57989 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3132416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:06.533 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:58:06.533 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 75.752559895999994, 2024-09-18T04:58:06.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044642881000000002, 2024-09-18T04:58:06.533 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:06.533 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:06.533 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:06.533 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:06.533 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:06.533 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57989 2024-09-18T04:58:06.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.534 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:06.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:06.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:58:06.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.534 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:06.534 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:58:06.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:06.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:06.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685235+0000", 2024-09-18T04:58:06.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:06.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:06.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685238+0000", 2024-09-18T04:58:06.535 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.667e-06 2024-09-18T04:58:06.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:06.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685245+0000", 2024-09-18T04:58:06.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7909999999999999e-06 2024-09-18T04:58:06.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:06.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685250+0000", 2024-09-18T04:58:06.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3730000000000001e-06 2024-09-18T04:58:06.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:06.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729685+0000", 2024-09-18T04:58:06.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044434552000000002 2024-09-18T04:58:06.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:06.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729775+0000", 2024-09-18T04:58:06.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0395000000000004e-05 2024-09-18T04:58:06.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:06.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:06.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:06.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729879+0000", 2024-09-18T04:58:06.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010361099999999999 2024-09-18T04:58:06.537 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.537 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:06.537 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.537 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:06.537 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:06.538 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:06.539 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:58:06.539 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:06.537+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:06.640 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:06.637+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:06.672 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:58:06.689 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:06.689 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:06.689 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:58:06.689 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:06.696 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:58:06.741 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:06.738+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:06.812 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:58:06.813 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:58:06.830 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:58:06.830 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:58:06.841 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:06.839+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:06.846 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:58:06.846 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:58:06.857 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:06.857 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:06.863 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:58:06.863 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:58:06.866 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:58:06.879 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:58:06.879 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:58:06.898 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:58:06.898 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:58:06.913 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:58:06.913 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:58:06.929 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:58:06.930 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:58:06.982 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:06.980+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:06.986 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:06.986 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:06.995 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:58:07.036 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T04:58:07.043 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:07.041+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:07.144 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:07.142+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:07.150 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:07.150 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:07.161 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:58:07.245 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:07.243+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:07.284 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:07.284 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:07.294 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:58:07.346 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:07.344+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:07.406 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:07.406 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:07.415 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:58:07.448 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:07.445+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:07.521 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:07.521 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:07.530 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:58:07.550 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:07.547+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:07.637 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:07.637 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:07.646 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:58:07.689 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:07.687+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:07.751 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:07.748+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:07.752 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:07.752 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:07.761 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:58:07.852 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:07.850+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:07.870 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:07.870 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:07.879 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:58:07.964 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:07.962+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:08.015 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-18T04:58:08.028 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:08.029 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:08.038 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:58:08.054 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:08.052+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:08.155 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:08.153+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:08.184 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:08.184 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:08.192 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:58:08.256 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:08.254+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:08.336 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:08.336 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:08.345 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:58:08.390 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:08.388+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:08.448 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:08.448 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:08.457 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:58:08.458 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:08.456+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:08.559 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:08.556+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:08.600 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:08.600 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:08.608 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:58:08.659 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:08.657+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:08.750 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:08.750 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:08.758 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:58:08.759 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:08.757+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:08.860 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:08.861 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:08.861 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:08.858+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:08.869 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:58:08.942 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:58:08.962 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:08.959+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:09.010 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:09.010 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:09.019 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:58:09.062 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:09.060+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:09.083 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:58:09.084 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:58:09.100 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:58:09.100 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:58:09.116 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:58:09.117 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:58:09.133 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:58:09.133 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:58:09.149 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:58:09.150 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:58:09.163 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:09.160+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:09.163 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:09.163 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:09.166 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:58:09.167 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:58:09.173 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:58:09.183 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:58:09.183 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:58:09.200 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:58:09.201 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:58:09.264 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:09.262+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:09.317 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:09.318 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:09.318 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:58:09.318 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:58:09.318 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:09.326 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:58:09.365 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:09.363+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:09.430 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:09.430 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:58:09.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:58:09.430 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:09.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.430 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:58:09.430 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:58:09.431 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 118.03321520999999, 2024-09-18T04:58:09.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037718511000000003, 2024-09-18T04:58:09.431 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:09.431 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:09.431 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:09.431 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:58:09.431 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:58:09.431 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-18T04:58:09.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.431 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:09.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:09.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:58:09.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:09.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:58:09.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:09.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393125+0000", 2024-09-18T04:58:09.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:58:09.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:09.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393133+0000", 2024-09-18T04:58:09.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.225e-06 2024-09-18T04:58:09.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:09.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393144+0000", 2024-09-18T04:58:09.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1279e-05 2024-09-18T04:58:09.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393153+0000", 2024-09-18T04:58:09.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4540000000000007e-06 2024-09-18T04:58:09.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:09.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393203+0000", 2024-09-18T04:58:09.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9963000000000003e-05 2024-09-18T04:58:09.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:09.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393210+0000", 2024-09-18T04:58:09.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5689999999999999e-06 2024-09-18T04:58:09.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:09.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427399+0000", 2024-09-18T04:58:09.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034188784999999999 2024-09-18T04:58:09.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:09.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427532+0000", 2024-09-18T04:58:09.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013264700000000001 2024-09-18T04:58:09.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:58:09.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427685+0000", 2024-09-18T04:58:09.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015300599999999999 2024-09-18T04:58:09.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:09.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.428520+0000", 2024-09-18T04:58:09.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00083502499999999998 2024-09-18T04:58:09.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:09.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430772+0000", 2024-09-18T04:58:09.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022518880000000001 2024-09-18T04:58:09.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:09.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430817+0000", 2024-09-18T04:58:09.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5077000000000002e-05 2024-09-18T04:58:09.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:09.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430845+0000", 2024-09-18T04:58:09.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8036999999999999e-05 2024-09-18T04:58:09.438 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.438 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:09.438 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.438 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:62520 3.13 3.84cdb893 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T04:58:09.438 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:09.438 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.128905326999998, 2024-09-18T04:58:09.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94452134799999998, 2024-09-18T04:58:09.438 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:09.438 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:58:09.439 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:09.439 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:09.439 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.439 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 62520 2024-09-18T04:58:09.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.439 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:09.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:09.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:09.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:09.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:09.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:09.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:09.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:09.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:09.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297441+0000", 2024-09-18T04:58:09.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9840000000000001e-06 2024-09-18T04:58:09.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:09.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297449+0000", 2024-09-18T04:58:09.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3999999999999992e-06 2024-09-18T04:58:09.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297455+0000", 2024-09-18T04:58:09.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3350000000000003e-06 2024-09-18T04:58:09.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:09.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297552+0000", 2024-09-18T04:58:09.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6602999999999996e-05 2024-09-18T04:58:09.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:09.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297559+0000", 2024-09-18T04:58:09.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.52e-06 2024-09-18T04:58:09.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:09.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241943+0000", 2024-09-18T04:58:09.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94438357100000003 2024-09-18T04:58:09.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:09.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241958+0000", 2024-09-18T04:58:09.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4598e-05 2024-09-18T04:58:09.443 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.443 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:09.443 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.443 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:113179 3.13 3:cf138017:::rbd_data.11ac5f86a3f9.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T04:58:09.444 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:58:09.444 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 50.739114915999998, 2024-09-18T04:58:09.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036508304999999998, 2024-09-18T04:58:09.444 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:09.444 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:09.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:09.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:09.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.444 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 113179 2024-09-18T04:58:09.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.444 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:09.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:09.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:58:09.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:09.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:58:09.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:09.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687226+0000", 2024-09-18T04:58:09.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:58:09.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:09.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688169+0000", 2024-09-18T04:58:09.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00094328200000000004 2024-09-18T04:58:09.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:09.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688182+0000", 2024-09-18T04:58:09.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2756e-05 2024-09-18T04:58:09.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688188+0000", 2024-09-18T04:58:09.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0290000000000002e-06 2024-09-18T04:58:09.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:09.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688225+0000", 2024-09-18T04:58:09.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7301000000000001e-05 2024-09-18T04:58:09.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:09.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688324+0000", 2024-09-18T04:58:09.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9419999999999993e-05 2024-09-18T04:58:09.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:58:09.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.700483+0000", 2024-09-18T04:58:09.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012158771000000001 2024-09-18T04:58:09.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:09.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723586+0000", 2024-09-18T04:58:09.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023102952 2024-09-18T04:58:09.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:09.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723634+0000", 2024-09-18T04:58:09.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7468e-05 2024-09-18T04:58:09.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:09.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723693+0000", 2024-09-18T04:58:09.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9539e-05 2024-09-18T04:58:09.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:09.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723735+0000", 2024-09-18T04:58:09.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1829000000000002e-05 2024-09-18T04:58:09.450 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.450 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:09.450 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.450 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146077 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1400832~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:09.450 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593265+0000", 2024-09-18T04:58:09.450 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 35.833076538, 2024-09-18T04:58:09.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035380397000000001, 2024-09-18T04:58:09.451 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:09.451 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:09.451 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:09.451 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:09.451 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.451 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146077 2024-09-18T04:58:09.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.451 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:09.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:09.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593265+0000", 2024-09-18T04:58:09.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:09.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593265+0000", 2024-09-18T04:58:09.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:09.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593265+0000", 2024-09-18T04:58:09.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:09.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:09.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593266+0000", 2024-09-18T04:58:09.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1039999999999999e-06 2024-09-18T04:58:09.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:09.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593269+0000", 2024-09-18T04:58:09.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8760000000000001e-06 2024-09-18T04:58:09.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593271+0000", 2024-09-18T04:58:09.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5100000000000001e-06 2024-09-18T04:58:09.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:09.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628545+0000", 2024-09-18T04:58:09.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035273977999999998 2024-09-18T04:58:09.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:09.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628589+0000", 2024-09-18T04:58:09.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4088999999999997e-05 2024-09-18T04:58:09.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:09.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628645+0000", 2024-09-18T04:58:09.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6059e-05 2024-09-18T04:58:09.456 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.456 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:09.456 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.457 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146078 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1449984~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:09.457 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:09.457 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 35.833037073, 2024-09-18T04:58:09.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035495325000000001, 2024-09-18T04:58:09.457 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:09.457 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:09.457 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:09.457 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:09.457 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.457 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146078 2024-09-18T04:58:09.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.457 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:09.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:09.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:09.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:09.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:09.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:09.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:09.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:09.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:09.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593306+0000", 2024-09-18T04:58:09.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.686e-06 2024-09-18T04:58:09.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:09.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593311+0000", 2024-09-18T04:58:09.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8500000000000002e-06 2024-09-18T04:58:09.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593314+0000", 2024-09-18T04:58:09.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0290000000000001e-06 2024-09-18T04:58:09.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:09.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628674+0000", 2024-09-18T04:58:09.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035360346000000001 2024-09-18T04:58:09.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:09.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628726+0000", 2024-09-18T04:58:09.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2219999999999998e-05 2024-09-18T04:58:09.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:09.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628800+0000", 2024-09-18T04:58:09.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3613000000000001e-05 2024-09-18T04:58:09.461 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.461 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:09.461 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.461 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146079 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1499136~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:09.461 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:09.461 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 35.833025667999998, 2024-09-18T04:58:09.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035604581000000003, 2024-09-18T04:58:09.461 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:09.461 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:09.461 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:09.462 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:09.462 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.462 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146079 2024-09-18T04:58:09.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.462 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:09.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:09.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:09.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:09.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:09.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:09.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:09.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:09.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:09.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593317+0000", 2024-09-18T04:58:09.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1599999999999999e-06 2024-09-18T04:58:09.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:09.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593320+0000", 2024-09-18T04:58:09.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0130000000000002e-06 2024-09-18T04:58:09.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593322+0000", 2024-09-18T04:58:09.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6079999999999998e-06 2024-09-18T04:58:09.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:09.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628825+0000", 2024-09-18T04:58:09.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035502747000000001 2024-09-18T04:58:09.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:09.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628868+0000", 2024-09-18T04:58:09.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2719999999999998e-05 2024-09-18T04:58:09.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:09.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628920+0000", 2024-09-18T04:58:09.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2533e-05 2024-09-18T04:58:09.465 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.465 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:09.465 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.466 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146080 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1548288~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:09.466 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:09.466 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 35.833010170000001, 2024-09-18T04:58:09.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035755146000000002, 2024-09-18T04:58:09.466 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:09.466 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:09.466 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:09.466 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:09.466 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.466 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146080 2024-09-18T04:58:09.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.466 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:09.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:09.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:09.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:09.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:09.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:09.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:09.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:09.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:09.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593333+0000", 2024-09-18T04:58:09.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.858e-06 2024-09-18T04:58:09.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:09.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593336+0000", 2024-09-18T04:58:09.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4910000000000001e-06 2024-09-18T04:58:09.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593340+0000", 2024-09-18T04:58:09.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2590000000000001e-06 2024-09-18T04:58:09.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:09.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628956+0000", 2024-09-18T04:58:09.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035616039000000002 2024-09-18T04:58:09.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:09.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629031+0000", 2024-09-18T04:58:09.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5504000000000001e-05 2024-09-18T04:58:09.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:09.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629086+0000", 2024-09-18T04:58:09.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5429000000000001e-05 2024-09-18T04:58:09.470 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.470 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:09.470 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.470 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146081 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1597440~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:09.470 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:09.470 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 35.832999598000001, 2024-09-18T04:58:09.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035873223000000003, 2024-09-18T04:58:09.471 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:09.471 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:09.471 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:09.471 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:09.471 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.471 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146081 2024-09-18T04:58:09.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.471 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:09.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:09.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:09.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:09.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:09.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:09.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:09.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:09.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:09.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593343+0000", 2024-09-18T04:58:09.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3319999999999999e-06 2024-09-18T04:58:09.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:09.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593346+0000", 2024-09-18T04:58:09.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9059999999999998e-06 2024-09-18T04:58:09.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593348+0000", 2024-09-18T04:58:09.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4530000000000001e-06 2024-09-18T04:58:09.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:09.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629110+0000", 2024-09-18T04:58:09.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035761814000000003 2024-09-18T04:58:09.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:09.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629159+0000", 2024-09-18T04:58:09.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8971000000000003e-05 2024-09-18T04:58:09.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:09.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629215+0000", 2024-09-18T04:58:09.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6107000000000003e-05 2024-09-18T04:58:09.475 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.475 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:09.475 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.475 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146082 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1646592~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:09.475 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:09.475 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 35.832983548000001, 2024-09-18T04:58:09.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035987007000000001, 2024-09-18T04:58:09.475 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:09.475 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:09.475 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:09.476 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:09.476 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.476 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146082 2024-09-18T04:58:09.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.476 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:09.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:09.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:09.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:09.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:09.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:09.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:09.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:09.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:09.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593360+0000", 2024-09-18T04:58:09.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8560000000000001e-06 2024-09-18T04:58:09.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:09.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593365+0000", 2024-09-18T04:58:09.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3410000000000002e-06 2024-09-18T04:58:09.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593369+0000", 2024-09-18T04:58:09.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4850000000000003e-06 2024-09-18T04:58:09.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:09.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629242+0000", 2024-09-18T04:58:09.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035872464999999999 2024-09-18T04:58:09.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:09.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629288+0000", 2024-09-18T04:58:09.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6592999999999999e-05 2024-09-18T04:58:09.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:09.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629345+0000", 2024-09-18T04:58:09.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6575e-05 2024-09-18T04:58:09.479 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.479 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:09.480 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.480 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146083 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1695744~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:09.480 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:58:09.480 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 35.832967701000001, 2024-09-18T04:58:09.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036100587000000003, 2024-09-18T04:58:09.480 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:09.480 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:09.480 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:09.480 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:09.480 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.480 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146083 2024-09-18T04:58:09.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.481 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:09.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:09.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:58:09.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:09.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:58:09.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:09.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593373+0000", 2024-09-18T04:58:09.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:09.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:09.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593376+0000", 2024-09-18T04:58:09.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.244e-06 2024-09-18T04:58:09.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:09.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593379+0000", 2024-09-18T04:58:09.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.619e-06 2024-09-18T04:58:09.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593382+0000", 2024-09-18T04:58:09.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7199999999999998e-06 2024-09-18T04:58:09.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:09.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629368+0000", 2024-09-18T04:58:09.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035986281000000002 2024-09-18T04:58:09.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:09.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629416+0000", 2024-09-18T04:58:09.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8055e-05 2024-09-18T04:58:09.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:09.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629474+0000", 2024-09-18T04:58:09.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8176e-05 2024-09-18T04:58:09.484 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.484 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:09.484 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.484 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146084 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1744896~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:09.484 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:09.484 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 35.832954571000002, 2024-09-18T04:58:09.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036214349, 2024-09-18T04:58:09.485 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:09.485 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:09.485 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:09.485 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:09.485 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.485 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146084 2024-09-18T04:58:09.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.485 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:09.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:09.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:09.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:09.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:09.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:09.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:09.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:09.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:09.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593388+0000", 2024-09-18T04:58:09.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.564e-06 2024-09-18T04:58:09.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:09.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593392+0000", 2024-09-18T04:58:09.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0579999999999999e-06 2024-09-18T04:58:09.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593395+0000", 2024-09-18T04:58:09.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8890000000000002e-06 2024-09-18T04:58:09.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:09.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629500+0000", 2024-09-18T04:58:09.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036105343999999998 2024-09-18T04:58:09.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:09.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629547+0000", 2024-09-18T04:58:09.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7036000000000002e-05 2024-09-18T04:58:09.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:09.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629601+0000", 2024-09-18T04:58:09.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3866999999999998e-05 2024-09-18T04:58:09.489 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.489 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:09.489 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.489 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146085 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1794048~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:09.489 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:09.489 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 35.832944183999999, 2024-09-18T04:58:09.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036331186000000001, 2024-09-18T04:58:09.489 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:09.489 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:09.489 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:09.489 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:09.490 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.490 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146085 2024-09-18T04:58:09.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.490 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:09.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:09.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:09.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:09.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:09.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:09.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:09.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:09.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:09.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593398+0000", 2024-09-18T04:58:09.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2270000000000001e-06 2024-09-18T04:58:09.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:09.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593401+0000", 2024-09-18T04:58:09.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.999e-06 2024-09-18T04:58:09.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593404+0000", 2024-09-18T04:58:09.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.486e-06 2024-09-18T04:58:09.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:09.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629624+0000", 2024-09-18T04:58:09.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036219983999999997 2024-09-18T04:58:09.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:09.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629672+0000", 2024-09-18T04:58:09.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8368000000000002e-05 2024-09-18T04:58:09.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:09.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629729+0000", 2024-09-18T04:58:09.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6359e-05 2024-09-18T04:58:09.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.493 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:09.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.494 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146086 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1843200~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:09.494 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:09.494 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 35.832935620000001, 2024-09-18T04:58:09.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036449098999999999, 2024-09-18T04:58:09.494 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:09.494 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:09.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:09.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:09.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.494 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146086 2024-09-18T04:58:09.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.494 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:09.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:09.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:09.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:09.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:09.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:09.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:09.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:09.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:09.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593407+0000", 2024-09-18T04:58:09.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1799999999999999e-06 2024-09-18T04:58:09.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:09.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593410+0000", 2024-09-18T04:58:09.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.926e-06 2024-09-18T04:58:09.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593413+0000", 2024-09-18T04:58:09.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8250000000000001e-06 2024-09-18T04:58:09.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:09.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629755+0000", 2024-09-18T04:58:09.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036342212999999998 2024-09-18T04:58:09.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:09.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629803+0000", 2024-09-18T04:58:09.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8619000000000003e-05 2024-09-18T04:58:09.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:09.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629855+0000", 2024-09-18T04:58:09.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1536000000000003e-05 2024-09-18T04:58:09.498 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.498 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:09.498 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.498 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146087 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1892352~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:09.498 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:09.498 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 35.832926694999998, 2024-09-18T04:58:09.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036564608999999998, 2024-09-18T04:58:09.498 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:09.498 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:09.499 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:09.499 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:09.499 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.499 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146087 2024-09-18T04:58:09.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.499 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:09.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:09.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:09.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:09.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:09.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:09.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:09.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:09.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:09.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593416+0000", 2024-09-18T04:58:09.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.218e-06 2024-09-18T04:58:09.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:09.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593419+0000", 2024-09-18T04:58:09.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-18T04:58:09.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593421+0000", 2024-09-18T04:58:09.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4329999999999998e-06 2024-09-18T04:58:09.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:09.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629878+0000", 2024-09-18T04:58:09.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036456578000000003 2024-09-18T04:58:09.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:09.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629925+0000", 2024-09-18T04:58:09.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7595000000000001e-05 2024-09-18T04:58:09.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:09.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629979+0000", 2024-09-18T04:58:09.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4117999999999999e-05 2024-09-18T04:58:09.502 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.502 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:09.502 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.503 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146088 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1941504~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:09.503 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:09.503 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 35.832904530999997, 2024-09-18T04:58:09.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036676121999999999, 2024-09-18T04:58:09.503 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:09.503 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:09.503 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:09.503 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:09.503 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.503 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146088 2024-09-18T04:58:09.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.503 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:09.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:09.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:09.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:09.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:09.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:09.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:09.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:09.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:09.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593438+0000", 2024-09-18T04:58:09.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7570000000000001e-06 2024-09-18T04:58:09.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:09.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593442+0000", 2024-09-18T04:58:09.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2880000000000001e-06 2024-09-18T04:58:09.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593444+0000", 2024-09-18T04:58:09.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.593e-06 2024-09-18T04:58:09.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:09.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630015+0000", 2024-09-18T04:58:09.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036571125000000003 2024-09-18T04:58:09.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:09.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630061+0000", 2024-09-18T04:58:09.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5445000000000003e-05 2024-09-18T04:58:09.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:09.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630113+0000", 2024-09-18T04:58:09.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2358000000000003e-05 2024-09-18T04:58:09.507 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.507 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:09.507 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.507 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146089 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1990656~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:09.507 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:58:09.507 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 35.832894949999996, 2024-09-18T04:58:09.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036791483999999999, 2024-09-18T04:58:09.507 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:09.508 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:09.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:09.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:09.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.508 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146089 2024-09-18T04:58:09.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.508 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:09.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:09.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:58:09.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:09.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:58:09.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:09.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593446+0000", 2024-09-18T04:58:09.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:09.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:09.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593448+0000", 2024-09-18T04:58:09.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1939999999999999e-06 2024-09-18T04:58:09.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:09.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593451+0000", 2024-09-18T04:58:09.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.766e-06 2024-09-18T04:58:09.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593458+0000", 2024-09-18T04:58:09.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8539999999999999e-06 2024-09-18T04:58:09.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:09.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630136+0000", 2024-09-18T04:58:09.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03667757 2024-09-18T04:58:09.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:09.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630183+0000", 2024-09-18T04:58:09.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7289000000000001e-05 2024-09-18T04:58:09.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:09.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630238+0000", 2024-09-18T04:58:09.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5013000000000003e-05 2024-09-18T04:58:09.511 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.511 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:09.511 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.512 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146090 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 2039808~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:09.512 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:09.512 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 35.832881069999999, 2024-09-18T04:58:09.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036904477999999998, 2024-09-18T04:58:09.512 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:09.512 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:09.512 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:09.512 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:09.512 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.512 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146090 2024-09-18T04:58:09.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.513 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:09.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:09.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:09.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:09.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:09.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:09.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:09.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:09.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:09.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593461+0000", 2024-09-18T04:58:09.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3680000000000001e-06 2024-09-18T04:58:09.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:09.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593465+0000", 2024-09-18T04:58:09.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8120000000000001e-06 2024-09-18T04:58:09.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593468+0000", 2024-09-18T04:58:09.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.582e-06 2024-09-18T04:58:09.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:09.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630268+0000", 2024-09-18T04:58:09.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036800449999999998 2024-09-18T04:58:09.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:09.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630313+0000", 2024-09-18T04:58:09.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4836000000000002e-05 2024-09-18T04:58:09.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:09.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630365+0000", 2024-09-18T04:58:09.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1839999999999998e-05 2024-09-18T04:58:09.516 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.516 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:09.516 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.516 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146091 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 2088960~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:09.516 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:09.516 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 35.832870608, 2024-09-18T04:58:09.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037011534999999998, 2024-09-18T04:58:09.517 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:09.517 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:09.517 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:09.517 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:09.517 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.517 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146091 2024-09-18T04:58:09.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.517 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:09.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:09.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:09.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:09.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:09.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:09.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:09.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:09.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:09.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593472+0000", 2024-09-18T04:58:09.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1620000000000001e-06 2024-09-18T04:58:09.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:09.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593475+0000", 2024-09-18T04:58:09.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0019999999999998e-06 2024-09-18T04:58:09.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593477+0000", 2024-09-18T04:58:09.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-18T04:58:09.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:09.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630381+0000", 2024-09-18T04:58:09.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036904014999999998 2024-09-18T04:58:09.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:09.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630428+0000", 2024-09-18T04:58:09.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6931000000000002e-05 2024-09-18T04:58:09.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:09.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630482+0000", 2024-09-18T04:58:09.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4138999999999998e-05 2024-09-18T04:58:09.521 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.521 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:09.521 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.521 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:162075 3.13 3:cf3f6048:::rbd_header.11ac6967455f:head [watch reconnect cookie 140406508629968 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T04:58:09.521 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:58:09.521 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 26.935897803, 2024-09-18T04:58:09.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.53184266599999996, 2024-09-18T04:58:09.522 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:09.522 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:09.522 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:09.522 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:09.522 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.522 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 162075 2024-09-18T04:58:09.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.522 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:09.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:09.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:58:09.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:09.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:58:09.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:09.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490442+0000", 2024-09-18T04:58:09.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:58:09.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:09.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490453+0000", 2024-09-18T04:58:09.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0613e-05 2024-09-18T04:58:09.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:09.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490466+0000", 2024-09-18T04:58:09.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3702e-05 2024-09-18T04:58:09.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490475+0000", 2024-09-18T04:58:09.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2690000000000005e-06 2024-09-18T04:58:09.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:09.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490612+0000", 2024-09-18T04:58:09.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013620900000000001 2024-09-18T04:58:09.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-18T04:58:09.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490620+0000", 2024-09-18T04:58:09.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3089999999999996e-06 2024-09-18T04:58:09.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:09.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503041+0000", 2024-09-18T04:58:09.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012420512 2024-09-18T04:58:09.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:09.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503048+0000", 2024-09-18T04:58:09.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0389999999999999e-06 2024-09-18T04:58:09.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:09.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.016122+0000", 2024-09-18T04:58:09.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.51307432900000005 2024-09-18T04:58:09.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:09.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.016289+0000", 2024-09-18T04:58:09.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000166875 2024-09-18T04:58:09.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:09.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.022286+0000", 2024-09-18T04:58:09.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059976049999999996 2024-09-18T04:58:09.527 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.527 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:09.527 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.527 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:162076 3.13 3:cf3f6048:::rbd_header.11ac6967455f:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T04:58:09.528 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:09.528 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 26.935775129, 2024-09-18T04:58:09.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.53464117600000005, 2024-09-18T04:58:09.528 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:09.528 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:09.528 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:09.528 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:09.528 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.528 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 162076 2024-09-18T04:58:09.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.528 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:09.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:09.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:09.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:09.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:09.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:09.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:09.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:09.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:58:09.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:09.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490571+0000", 2024-09-18T04:58:09.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8259999999999998e-06 2024-09-18T04:58:09.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:09.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490584+0000", 2024-09-18T04:58:09.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2391999999999999e-05 2024-09-18T04:58:09.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490591+0000", 2024-09-18T04:58:09.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5460000000000003e-06 2024-09-18T04:58:09.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:09.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490714+0000", 2024-09-18T04:58:09.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000122543 2024-09-18T04:58:09.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.531 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-18T04:58:09.531 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490731+0000", 2024-09-18T04:58:09.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7405e-05 2024-09-18T04:58:09.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:09.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503069+0000", 2024-09-18T04:58:09.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012337836 2024-09-18T04:58:09.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:09.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503078+0000", 2024-09-18T04:58:09.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8289999999999997e-06 2024-09-18T04:58:09.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:09.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019018+0000", 2024-09-18T04:58:09.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.51594035900000002 2024-09-18T04:58:09.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:09.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019091+0000", 2024-09-18T04:58:09.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2608999999999995e-05 2024-09-18T04:58:09.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-18T04:58:09.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019421+0000", 2024-09-18T04:58:09.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00033004300000000002 2024-09-18T04:58:09.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:09.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.021644+0000", 2024-09-18T04:58:09.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022229749999999999 2024-09-18T04:58:09.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:09.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025113+0000", 2024-09-18T04:58:09.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003469228 2024-09-18T04:58:09.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:09.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025167+0000", 2024-09-18T04:58:09.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3569999999999997e-05 2024-09-18T04:58:09.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:09.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:09.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:09.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025208+0000", 2024-09-18T04:58:09.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0821000000000001e-05 2024-09-18T04:58:09.535 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.535 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:09.535 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.535 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:09.535 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:09.535 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:09.537 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:09.534+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:09.537 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:58:09.638 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:09.636+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:09.648 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:09.648 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:09.648 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:58:09.648 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:09.657 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:58:09.739 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:09.737+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:09.804 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:09.805 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:09.805 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:58:09.805 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:58:09.805 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:09.814 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:58:09.840 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:09.837+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:09.922 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:09.922 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:58:09.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:58:09.922 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:58:09.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.922 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9316 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:09.922 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:58:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.151677461999995, 2024-09-18T04:58:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044027208999999998, 2024-09-18T04:58:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9316 2024-09-18T04:58:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:58:09.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:09.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:58:09.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:09.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:58:09.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:09.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767144+0000", 2024-09-18T04:58:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3070000000000001e-06 2024-09-18T04:58:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767150+0000", 2024-09-18T04:58:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1700000000000002e-06 2024-09-18T04:58:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767163+0000", 2024-09-18T04:58:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2670999999999999e-05 2024-09-18T04:58:09.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:09.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810995+0000", 2024-09-18T04:58:09.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043832310999999999 2024-09-18T04:58:09.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:09.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811087+0000", 2024-09-18T04:58:09.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1345e-05 2024-09-18T04:58:09.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:09.927 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811169+0000", 2024-09-18T04:58:09.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2848000000000004e-05 2024-09-18T04:58:09.927 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.927 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:09.927 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.927 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9317 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1728512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:09.927 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:58:09.927 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.151651826999995, 2024-09-18T04:58:09.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044190170000000001, 2024-09-18T04:58:09.927 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:09.927 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9317 2024-09-18T04:58:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:58:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:09.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:58:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767167+0000", 2024-09-18T04:58:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767170+0000", 2024-09-18T04:58:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7259999999999998e-06 2024-09-18T04:58:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767176+0000", 2024-09-18T04:58:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1460000000000001e-06 2024-09-18T04:58:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767182+0000", 2024-09-18T04:58:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1020000000000002e-06 2024-09-18T04:58:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811204+0000", 2024-09-18T04:58:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044022039999999998 2024-09-18T04:58:09.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:09.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811286+0000", 2024-09-18T04:58:09.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1623000000000001e-05 2024-09-18T04:58:09.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:09.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811358+0000", 2024-09-18T04:58:09.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2125000000000005e-05 2024-09-18T04:58:09.931 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.931 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:09.931 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9318 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1777664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:58:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.151633282000006, 2024-09-18T04:58:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044380181999999997, 2024-09-18T04:58:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9318 2024-09-18T04:58:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.932 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:58:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:58:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:58:09.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:09.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:09.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767189+0000", 2024-09-18T04:58:09.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5909999999999998e-06 2024-09-18T04:58:09.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:09.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767194+0000", 2024-09-18T04:58:09.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7690000000000002e-06 2024-09-18T04:58:09.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767199+0000", 2024-09-18T04:58:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189999999999999e-06 2024-09-18T04:58:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811394+0000", 2024-09-18T04:58:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044195350000000001 2024-09-18T04:58:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811483+0000", 2024-09-18T04:58:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8282999999999995e-05 2024-09-18T04:58:09.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:09.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811567+0000", 2024-09-18T04:58:09.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3855000000000003e-05 2024-09-18T04:58:09.936 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.936 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:09.936 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.936 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9325 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:09.936 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:58:09.936 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.149902744000002, 2024-09-18T04:58:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044063989999999997, 2024-09-18T04:58:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9325 2024-09-18T04:58:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:58:09.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:09.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:58:09.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.938 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:09.938 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:58:09.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:09.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768919+0000", 2024-09-18T04:58:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:58:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768925+0000", 2024-09-18T04:58:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2310000000000001e-06 2024-09-18T04:58:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768929+0000", 2024-09-18T04:58:09.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4469999999999999e-06 2024-09-18T04:58:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812823+0000", 2024-09-18T04:58:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043893975000000002 2024-09-18T04:58:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812908+0000", 2024-09-18T04:58:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4862000000000002e-05 2024-09-18T04:58:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:09.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812981+0000", 2024-09-18T04:58:09.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2892000000000006e-05 2024-09-18T04:58:09.941 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.941 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:09.941 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.941 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9334 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:09.941 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:58:09.941 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.147883957999994, 2024-09-18T04:58:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044042853999999999, 2024-09-18T04:58:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9334 2024-09-18T04:58:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:58:09.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:58:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770935+0000", 2024-09-18T04:58:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:09.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770937+0000", 2024-09-18T04:58:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1489999999999999e-06 2024-09-18T04:58:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770940+0000", 2024-09-18T04:58:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9569999999999998e-06 2024-09-18T04:58:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770945+0000", 2024-09-18T04:58:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:58:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:09.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814838+0000", 2024-09-18T04:58:09.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043892635999999999 2024-09-18T04:58:09.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:09.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814920+0000", 2024-09-18T04:58:09.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2175000000000005e-05 2024-09-18T04:58:09.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:09.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814979+0000", 2024-09-18T04:58:09.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8372000000000003e-05 2024-09-18T04:58:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9352 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1159168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:58:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.145662780999999, 2024-09-18T04:58:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044011269999999998, 2024-09-18T04:58:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:09.946 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9352 2024-09-18T04:58:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:58:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:58:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773156+0000", 2024-09-18T04:58:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773159+0000", 2024-09-18T04:58:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2919999999999998e-06 2024-09-18T04:58:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773165+0000", 2024-09-18T04:58:09.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0580000000000003e-06 2024-09-18T04:58:09.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773177+0000", 2024-09-18T04:58:09.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.204e-05 2024-09-18T04:58:09.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:09.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817082+0000", 2024-09-18T04:58:09.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043905528999999999 2024-09-18T04:58:09.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817122+0000", 2024-09-18T04:58:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9258999999999998e-05 2024-09-18T04:58:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817168+0000", 2024-09-18T04:58:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6545000000000002e-05 2024-09-18T04:58:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.950 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9353 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1208320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:58:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.145638482999999, 2024-09-18T04:58:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044105537, 2024-09-18T04:58:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9353 2024-09-18T04:58:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:09.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:58:09.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:09.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:58:09.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:09.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:58:09.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773183+0000", 2024-09-18T04:58:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2929999999999999e-06 2024-09-18T04:58:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773190+0000", 2024-09-18T04:58:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4330000000000001e-06 2024-09-18T04:58:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773194+0000", 2024-09-18T04:58:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.865e-06 2024-09-18T04:58:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817196+0000", 2024-09-18T04:58:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044001708 2024-09-18T04:58:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817236+0000", 2024-09-18T04:58:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0043999999999998e-05 2024-09-18T04:58:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:09.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817287+0000", 2024-09-18T04:58:09.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0605000000000001e-05 2024-09-18T04:58:09.955 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.955 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:09.955 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.955 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9354 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:09.955 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:58:09.955 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.145620972000003, 2024-09-18T04:58:09.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044199796, 2024-09-18T04:58:09.955 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9354 2024-09-18T04:58:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:58:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:58:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773198+0000", 2024-09-18T04:58:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773200+0000", 2024-09-18T04:58:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4159999999999998e-06 2024-09-18T04:58:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:09.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773206+0000", 2024-09-18T04:58:09.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.789e-06 2024-09-18T04:58:09.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773210+0000", 2024-09-18T04:58:09.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2679999999999996e-06 2024-09-18T04:58:09.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:09.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817314+0000", 2024-09-18T04:58:09.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044103187000000002 2024-09-18T04:58:09.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:09.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817353+0000", 2024-09-18T04:58:09.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8973000000000001e-05 2024-09-18T04:58:09.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:09.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817399+0000", 2024-09-18T04:58:09.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5877999999999997e-05 2024-09-18T04:58:09.961 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.961 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46591 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2097152~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:58:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.220764555000002, 2024-09-18T04:58:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043987310000000002, 2024-09-18T04:58:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46591 2024-09-18T04:58:09.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:58:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:58:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:09.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:58:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698057+0000", 2024-09-18T04:58:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.531e-06 2024-09-18T04:58:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698064+0000", 2024-09-18T04:58:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-18T04:58:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698069+0000", 2024-09-18T04:58:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.091e-06 2024-09-18T04:58:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741843+0000", 2024-09-18T04:58:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043773470000000002 2024-09-18T04:58:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741928+0000", 2024-09-18T04:58:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5113999999999999e-05 2024-09-18T04:58:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.965 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.966 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:09.966 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742042+0000", 2024-09-18T04:58:09.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114527 2024-09-18T04:58:09.966 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.966 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:09.966 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.966 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.966 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.966 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46592 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2121728~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:09.966 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:58:09.966 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.220746128000002, 2024-09-18T04:58:09.966 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044077948999999998, 2024-09-18T04:58:09.966 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:09.966 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46592 2024-09-18T04:58:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:58:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.967 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:58:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698073+0000", 2024-09-18T04:58:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698076+0000", 2024-09-18T04:58:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.481e-06 2024-09-18T04:58:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698082+0000", 2024-09-18T04:58:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8730000000000004e-06 2024-09-18T04:58:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698086+0000", 2024-09-18T04:58:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:58:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742055+0000", 2024-09-18T04:58:09.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043969170000000002 2024-09-18T04:58:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742093+0000", 2024-09-18T04:58:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7707000000000003e-05 2024-09-18T04:58:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742152+0000", 2024-09-18T04:58:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8323999999999999e-05 2024-09-18T04:58:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:09.970 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46593 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2211840~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:58:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.220729224999999, 2024-09-18T04:58:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044259917000000003, 2024-09-18T04:58:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46593 2024-09-18T04:58:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.971 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:58:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:58:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:09.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:58:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698092+0000", 2024-09-18T04:58:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2249999999999999e-06 2024-09-18T04:58:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698098+0000", 2024-09-18T04:58:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:58:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698103+0000", 2024-09-18T04:58:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:58:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742182+0000", 2024-09-18T04:58:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044079508000000003 2024-09-18T04:58:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742257+0000", 2024-09-18T04:58:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4678000000000006e-05 2024-09-18T04:58:09.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742350+0000", 2024-09-18T04:58:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3531999999999998e-05 2024-09-18T04:58:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46594 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2236416~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:58:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.220713528000005, 2024-09-18T04:58:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044378984000000003, 2024-09-18T04:58:09.975 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46594 2024-09-18T04:58:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:58:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:58:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:58:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698108+0000", 2024-09-18T04:58:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0880000000000002e-06 2024-09-18T04:58:09.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698114+0000", 2024-09-18T04:58:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6269999999999997e-06 2024-09-18T04:58:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698118+0000", 2024-09-18T04:58:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4050000000000002e-06 2024-09-18T04:58:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742365+0000", 2024-09-18T04:58:09.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044246799000000003 2024-09-18T04:58:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742407+0000", 2024-09-18T04:58:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2008000000000002e-05 2024-09-18T04:58:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742485+0000", 2024-09-18T04:58:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8405999999999994e-05 2024-09-18T04:58:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.980 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46595 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:09.980 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:58:09.980 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.220697455000007, 2024-09-18T04:58:09.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.04440463, 2024-09-18T04:58:09.980 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:09.980 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:09.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:09.980 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46595 2024-09-18T04:58:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:58:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:58:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:58:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698124+0000", 2024-09-18T04:58:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1840000000000002e-06 2024-09-18T04:58:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698130+0000", 2024-09-18T04:58:09.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.947e-06 2024-09-18T04:58:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698136+0000", 2024-09-18T04:58:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6520000000000003e-06 2024-09-18T04:58:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742521+0000", 2024-09-18T04:58:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044385446000000002 2024-09-18T04:58:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742527+0000", 2024-09-18T04:58:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:58:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46596 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:58:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.220679211000004, 2024-09-18T04:58:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044418066999999999, 2024-09-18T04:58:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:09.984 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46596 2024-09-18T04:58:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:58:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:09.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:58:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:58:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698146+0000", 2024-09-18T04:58:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2779999999999999e-06 2024-09-18T04:58:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698153+0000", 2024-09-18T04:58:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3849999999999999e-06 2024-09-18T04:58:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698164+0000", 2024-09-18T04:58:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1375999999999999e-05 2024-09-18T04:58:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742554+0000", 2024-09-18T04:58:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044389482000000001 2024-09-18T04:58:09.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742559+0000", 2024-09-18T04:58:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9540000000000003e-06 2024-09-18T04:58:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46597 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:58:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.220650845999998, 2024-09-18T04:58:09.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044416513999999997, 2024-09-18T04:58:09.989 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:09.989 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:09.989 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:09.989 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:09.989 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.989 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46597 2024-09-18T04:58:09.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.989 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:09.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:09.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:58:09.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:09.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:58:09.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:09.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698168+0000", 2024-09-18T04:58:09.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:09.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:09.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698175+0000", 2024-09-18T04:58:09.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4520000000000002e-06 2024-09-18T04:58:09.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:09.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698181+0000", 2024-09-18T04:58:09.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4150000000000004e-06 2024-09-18T04:58:09.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698186+0000", 2024-09-18T04:58:09.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7489999999999998e-06 2024-09-18T04:58:09.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.991 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:09.991 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742580+0000", 2024-09-18T04:58:09.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044394325999999998 2024-09-18T04:58:09.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:09.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742585+0000", 2024-09-18T04:58:09.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9849999999999997e-06 2024-09-18T04:58:09.992 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.992 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:09.992 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.992 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46598 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:09.992 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:58:09.992 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 80.220543370000001, 2024-09-18T04:58:09.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044335316999999999, 2024-09-18T04:58:09.993 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:09.993 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:09.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:09.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:09.993 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.993 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46598 2024-09-18T04:58:09.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.993 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:09.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:09.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:58:09.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:09.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:58:09.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:09.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698190+0000", 2024-09-18T04:58:09.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999137 2024-09-18T04:58:09.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:09.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698280+0000", 2024-09-18T04:58:09.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0409999999999997e-05 2024-09-18T04:58:09.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:09.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698289+0000", 2024-09-18T04:58:09.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9490000000000002e-06 2024-09-18T04:58:09.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:09.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698296+0000", 2024-09-18T04:58:09.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2670000000000003e-06 2024-09-18T04:58:09.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:09.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742607+0000", 2024-09-18T04:58:09.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044311072999999999 2024-09-18T04:58:09.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:09.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742612+0000", 2024-09-18T04:58:09.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9400000000000001e-06 2024-09-18T04:58:09.996 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.996 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:09.996 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:09.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.996 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57986 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3086336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:09.996 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:58:09.997 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 79.233697647, 2024-09-18T04:58:09.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044047731999999999, 2024-09-18T04:58:09.997 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:09.997 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:09.997 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:09.997 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:09.997 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:09.997 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57986 2024-09-18T04:58:09.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.997 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:09.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:09.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:58:09.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:09.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:58:09.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:09.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:09.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:58:09.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:09.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:09.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685124+0000", 2024-09-18T04:58:09.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2859999999999998e-06 2024-09-18T04:58:09.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:09.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685130+0000", 2024-09-18T04:58:09.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5400000000000001e-06 2024-09-18T04:58:09.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:09.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:09.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685136+0000", 2024-09-18T04:58:10.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.152e-06 2024-09-18T04:58:10.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.728976+0000", 2024-09-18T04:58:10.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043840037999999998 2024-09-18T04:58:10.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:10.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729064+0000", 2024-09-18T04:58:10.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8709999999999996e-05 2024-09-18T04:58:10.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729170+0000", 2024-09-18T04:58:10.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000105386 2024-09-18T04:58:10.001 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.001 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.001 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.001 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57987 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3101696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:10.001 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:58:10.001 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 79.233648672000001, 2024-09-18T04:58:10.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044238123999999997, 2024-09-18T04:58:10.002 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.002 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:10.002 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.002 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.002 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.002 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57987 2024-09-18T04:58:10.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.002 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:58:10.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:58:10.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685170+0000", 2024-09-18T04:58:10.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:58:10.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685175+0000", 2024-09-18T04:58:10.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9069999999999998e-06 2024-09-18T04:58:10.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685187+0000", 2024-09-18T04:58:10.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.189e-05 2024-09-18T04:58:10.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685195+0000", 2024-09-18T04:58:10.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9459999999999998e-06 2024-09-18T04:58:10.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729212+0000", 2024-09-18T04:58:10.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044017239 2024-09-18T04:58:10.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:10.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729301+0000", 2024-09-18T04:58:10.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8887999999999993e-05 2024-09-18T04:58:10.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729409+0000", 2024-09-18T04:58:10.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108119 2024-09-18T04:58:10.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.005 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.006 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57988 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3117056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:10.006 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:58:10.006 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 79.233606553000001, 2024-09-18T04:58:10.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044429835000000001, 2024-09-18T04:58:10.006 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.006 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:10.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.006 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57988 2024-09-18T04:58:10.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.006 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:58:10.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:58:10.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685212+0000", 2024-09-18T04:58:10.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:10.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685216+0000", 2024-09-18T04:58:10.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7390000000000001e-06 2024-09-18T04:58:10.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685224+0000", 2024-09-18T04:58:10.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2260000000000005e-06 2024-09-18T04:58:10.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685230+0000", 2024-09-18T04:58:10.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0299999999999999e-06 2024-09-18T04:58:10.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729454+0000", 2024-09-18T04:58:10.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044223248999999999 2024-09-18T04:58:10.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:10.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729539+0000", 2024-09-18T04:58:10.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5029000000000002e-05 2024-09-18T04:58:10.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729643+0000", 2024-09-18T04:58:10.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104267 2024-09-18T04:58:10.010 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.010 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.010 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.010 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57989 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3132416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:10.010 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:58:10.010 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 79.233583800999995, 2024-09-18T04:58:10.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044642881000000002, 2024-09-18T04:58:10.010 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.010 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:10.011 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.011 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.011 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.011 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57989 2024-09-18T04:58:10.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.011 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:58:10.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:58:10.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685235+0000", 2024-09-18T04:58:10.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:10.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685238+0000", 2024-09-18T04:58:10.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.667e-06 2024-09-18T04:58:10.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685245+0000", 2024-09-18T04:58:10.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7909999999999999e-06 2024-09-18T04:58:10.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685250+0000", 2024-09-18T04:58:10.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3730000000000001e-06 2024-09-18T04:58:10.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729685+0000", 2024-09-18T04:58:10.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044434552000000002 2024-09-18T04:58:10.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:10.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729775+0000", 2024-09-18T04:58:10.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0395000000000004e-05 2024-09-18T04:58:10.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729879+0000", 2024-09-18T04:58:10.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010361099999999999 2024-09-18T04:58:10.014 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.014 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.014 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.014 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.014 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.015 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:10.016 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:58:10.017 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:10.013+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:10.034 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T04:58:10.118 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:10.116+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:10.148 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:10.148 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:10.148 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:58:10.148 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:10.158 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:58:10.242 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:10.240+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:10.307 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:10.307 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:10.308 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:58:10.308 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:58:10.308 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:10.316 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:58:10.320 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:10.316+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:10.388 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 5, 6, 7] out_osds: [1, 4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T04:58:10.388 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T04:58:10.388 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-09-18T04:58:10.389 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-18T04:58:10.389 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-18T04:58:10.389 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-09-18T04:58:10.419 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:10.416+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:10.467 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:10.467 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:58:10.467 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:58:10.467 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:58:10.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.468 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14125 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:10.468 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:58:10.468 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.336954720999998, 2024-09-18T04:58:10.468 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052956470999999998, 2024-09-18T04:58:10.468 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.468 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:10.468 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.468 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.468 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.469 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14125 2024-09-18T04:58:10.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.469 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:58:10.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.469 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.469 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:58:10.469 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126849+0000", 2024-09-18T04:58:10.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:10.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.470 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.470 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126852+0000", 2024-09-18T04:58:10.470 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3659999999999999e-06 2024-09-18T04:58:10.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126858+0000", 2024-09-18T04:58:10.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4540000000000004e-06 2024-09-18T04:58:10.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.471 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126863+0000", 2024-09-18T04:58:10.471 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1780000000000002e-06 2024-09-18T04:58:10.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.471 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179608+0000", 2024-09-18T04:58:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052745159999999999 2024-09-18T04:58:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179703+0000", 2024-09-18T04:58:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4773999999999998e-05 2024-09-18T04:58:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179806+0000", 2024-09-18T04:58:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010296799999999999 2024-09-18T04:58:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.472 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.473 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.473 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14136 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1732608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:10.473 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:58:10.473 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.334794169999995, 2024-09-18T04:58:10.473 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052973412999999997, 2024-09-18T04:58:10.473 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.474 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:10.474 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.474 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.474 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.474 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14136 2024-09-18T04:58:10.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.474 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.474 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.474 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:58:10.474 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.474 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:58:10.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.475 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:58:10.475 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:10.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.475 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129013+0000", 2024-09-18T04:58:10.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.96e-06 2024-09-18T04:58:10.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.476 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129020+0000", 2024-09-18T04:58:10.476 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6809999999999998e-06 2024-09-18T04:58:10.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.476 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129026+0000", 2024-09-18T04:58:10.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:58:10.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181873+0000", 2024-09-18T04:58:10.477 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052847238999999997 2024-09-18T04:58:10.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.477 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:10.477 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181932+0000", 2024-09-18T04:58:10.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9339000000000002e-05 2024-09-18T04:58:10.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.478 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.478 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181984+0000", 2024-09-18T04:58:10.478 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1348999999999999e-05 2024-09-18T04:58:10.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.478 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.479 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14137 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1781760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:10.479 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:58:10.479 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.334773764999994, 2024-09-18T04:58:10.479 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053128914999999999, 2024-09-18T04:58:10.479 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.479 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:10.479 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.479 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.479 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.479 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14137 2024-09-18T04:58:10.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.480 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:58:10.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.480 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.480 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:58:10.480 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.481 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.481 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129030+0000", 2024-09-18T04:58:10.481 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129033+0000", 2024-09-18T04:58:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7439999999999999e-06 2024-09-18T04:58:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129039+0000", 2024-09-18T04:58:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.314e-06 2024-09-18T04:58:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.482 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129044+0000", 2024-09-18T04:58:10.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9919999999999998e-06 2024-09-18T04:58:10.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182015+0000", 2024-09-18T04:58:10.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052970875000000001 2024-09-18T04:58:10.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.483 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:10.483 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182058+0000", 2024-09-18T04:58:10.483 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2997000000000002e-05 2024-09-18T04:58:10.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.484 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.484 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182160+0000", 2024-09-18T04:58:10.484 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101437 2024-09-18T04:58:10.484 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.484 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.484 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.484 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14138 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1830912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:10.485 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:58:10.485 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.334755731000001, 2024-09-18T04:58:10.485 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053256435999999997, 2024-09-18T04:58:10.485 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.485 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:10.485 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.485 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.485 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.485 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14138 2024-09-18T04:58:10.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.486 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.486 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:58:10.486 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.486 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.486 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:58:10.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.487 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.487 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129048+0000", 2024-09-18T04:58:10.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:10.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129056+0000", 2024-09-18T04:58:10.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.413e-06 2024-09-18T04:58:10.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129062+0000", 2024-09-18T04:58:10.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6810000000000001e-06 2024-09-18T04:58:10.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129068+0000", 2024-09-18T04:58:10.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0459999999999996e-06 2024-09-18T04:58:10.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182191+0000", 2024-09-18T04:58:10.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053123210999999997 2024-09-18T04:58:10.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:10.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182238+0000", 2024-09-18T04:58:10.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7404999999999998e-05 2024-09-18T04:58:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182305+0000", 2024-09-18T04:58:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7028999999999998e-05 2024-09-18T04:58:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15910 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2236416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:58:10.490 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.235668199000003, 2024-09-18T04:58:10.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053241054000000003, 2024-09-18T04:58:10.491 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.491 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:10.491 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.491 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.491 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.491 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15910 2024-09-18T04:58:10.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.491 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:58:10.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:58:10.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.492 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:58:10.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:10.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.492 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228138+0000", 2024-09-18T04:58:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:58:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228144+0000", 2024-09-18T04:58:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5369999999999997e-06 2024-09-18T04:58:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228149+0000", 2024-09-18T04:58:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1209999999999998e-06 2024-09-18T04:58:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280822+0000", 2024-09-18T04:58:10.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052673194999999999 2024-09-18T04:58:10.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:10.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280903+0000", 2024-09-18T04:58:10.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1179000000000003e-05 2024-09-18T04:58:10.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281377+0000", 2024-09-18T04:58:10.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047417899999999997 2024-09-18T04:58:10.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.495 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.495 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.495 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15911 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2285568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:10.495 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:58:10.495 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.235651466999997, 2024-09-18T04:58:10.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053415151000000001, 2024-09-18T04:58:10.495 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.496 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:10.496 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.496 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.496 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.496 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15911 2024-09-18T04:58:10.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.496 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:58:10.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:58:10.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:58:10.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:10.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228155+0000", 2024-09-18T04:58:10.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:58:10.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228161+0000", 2024-09-18T04:58:10.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:58:10.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228166+0000", 2024-09-18T04:58:10.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2320000000000001e-06 2024-09-18T04:58:10.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281405+0000", 2024-09-18T04:58:10.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053239336999999998 2024-09-18T04:58:10.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:10.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281482+0000", 2024-09-18T04:58:10.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6569999999999994e-05 2024-09-18T04:58:10.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281568+0000", 2024-09-18T04:58:10.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6328000000000004e-05 2024-09-18T04:58:10.500 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.500 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.500 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.500 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.500 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.500 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15918 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2662400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:10.500 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:58:10.500 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.233882485999999, 2024-09-18T04:58:10.500 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053091065, 2024-09-18T04:58:10.501 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.501 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:10.501 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.501 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.501 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.501 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15918 2024-09-18T04:58:10.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.501 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:58:10.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:58:10.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:58:10.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:10.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229924+0000", 2024-09-18T04:58:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:58:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229930+0000", 2024-09-18T04:58:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.066e-06 2024-09-18T04:58:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229935+0000", 2024-09-18T04:58:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9119999999999997e-06 2024-09-18T04:58:10.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282840+0000", 2024-09-18T04:58:10.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052905119 2024-09-18T04:58:10.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:10.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282922+0000", 2024-09-18T04:58:10.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2081000000000002e-05 2024-09-18T04:58:10.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.505 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283013+0000", 2024-09-18T04:58:10.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0828000000000005e-05 2024-09-18T04:58:10.505 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.505 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.505 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.505 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.505 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.505 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15919 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2711552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:10.505 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:58:10.505 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.233865207999997, 2024-09-18T04:58:10.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053293021000000003, 2024-09-18T04:58:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15919 2024-09-18T04:58:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:58:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:58:10.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:58:10.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:10.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229941+0000", 2024-09-18T04:58:10.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2639999999999998e-06 2024-09-18T04:58:10.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229947+0000", 2024-09-18T04:58:10.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9750000000000004e-06 2024-09-18T04:58:10.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229952+0000", 2024-09-18T04:58:10.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7929999999999997e-06 2024-09-18T04:58:10.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283062+0000", 2024-09-18T04:58:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05310952 2024-09-18T04:58:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283162+0000", 2024-09-18T04:58:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100098 2024-09-18T04:58:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283232+0000", 2024-09-18T04:58:10.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0693999999999998e-05 2024-09-18T04:58:10.510 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.510 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.510 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.510 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15920 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2760704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:10.510 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:58:10.510 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.233848447, 2024-09-18T04:58:10.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053502367000000002, 2024-09-18T04:58:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15920 2024-09-18T04:58:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:58:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:58:10.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:58:10.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229958+0000", 2024-09-18T04:58:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1770000000000001e-06 2024-09-18T04:58:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229964+0000", 2024-09-18T04:58:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8470000000000001e-06 2024-09-18T04:58:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229968+0000", 2024-09-18T04:58:10.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5680000000000001e-06 2024-09-18T04:58:10.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283269+0000", 2024-09-18T04:58:10.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053300620999999999 2024-09-18T04:58:10.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:10.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283359+0000", 2024-09-18T04:58:10.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9971999999999998e-05 2024-09-18T04:58:10.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283458+0000", 2024-09-18T04:58:10.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9483999999999998e-05 2024-09-18T04:58:10.515 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.515 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.515 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.516 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15921 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2809856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:10.516 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:58:10.516 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.233831897000002, 2024-09-18T04:58:10.516 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053800977, 2024-09-18T04:58:10.516 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.516 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:10.516 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.516 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.516 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.517 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15921 2024-09-18T04:58:10.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.517 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.517 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:58:10.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.517 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:58:10.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229972+0000", 2024-09-18T04:58:10.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:10.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229974+0000", 2024-09-18T04:58:10.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2910000000000002e-06 2024-09-18T04:58:10.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229982+0000", 2024-09-18T04:58:10.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7249999999999997e-06 2024-09-18T04:58:10.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229989+0000", 2024-09-18T04:58:10.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6429999999999998e-06 2024-09-18T04:58:10.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283601+0000", 2024-09-18T04:58:10.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053612211 2024-09-18T04:58:10.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:10.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283681+0000", 2024-09-18T04:58:10.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0125999999999998e-05 2024-09-18T04:58:10.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.522 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.522 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283774+0000", 2024-09-18T04:58:10.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2362999999999996e-05 2024-09-18T04:58:10.522 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.522 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.522 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.522 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15922 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2859008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:10.522 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:58:10.522 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.233786957999996, 2024-09-18T04:58:10.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053882270000000003, 2024-09-18T04:58:10.523 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.523 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:10.523 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.523 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.523 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.523 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15922 2024-09-18T04:58:10.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.523 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:58:10.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:58:10.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230017+0000", 2024-09-18T04:58:10.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:10.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230019+0000", 2024-09-18T04:58:10.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4210000000000002e-06 2024-09-18T04:58:10.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230027+0000", 2024-09-18T04:58:10.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000002e-06 2024-09-18T04:58:10.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230032+0000", 2024-09-18T04:58:10.529 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0089999999999998e-06 2024-09-18T04:58:10.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283800+0000", 2024-09-18T04:58:10.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053768418999999998 2024-09-18T04:58:10.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:10.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283849+0000", 2024-09-18T04:58:10.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9339000000000003e-05 2024-09-18T04:58:10.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283900+0000", 2024-09-18T04:58:10.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0421999999999999e-05 2024-09-18T04:58:10.532 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.532 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.533 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.533 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15923 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2924544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:10.533 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:58:10.533 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.233768088999994, 2024-09-18T04:58:10.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054073016000000002, 2024-09-18T04:58:10.534 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.534 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:10.534 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.534 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.534 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.535 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15923 2024-09-18T04:58:10.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.535 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:58:10.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:58:10.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:58:10.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:10.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230038+0000", 2024-09-18T04:58:10.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.463e-06 2024-09-18T04:58:10.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230045+0000", 2024-09-18T04:58:10.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3570000000000003e-06 2024-09-18T04:58:10.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230050+0000", 2024-09-18T04:58:10.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7299999999999996e-06 2024-09-18T04:58:10.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283928+0000", 2024-09-18T04:58:10.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053878218999999998 2024-09-18T04:58:10.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:10.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284010+0000", 2024-09-18T04:58:10.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2577000000000006e-05 2024-09-18T04:58:10.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284109+0000", 2024-09-18T04:58:10.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9102000000000001e-05 2024-09-18T04:58:10.541 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.541 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.541 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.542 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15924 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:10.542 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:58:10.542 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.233749618999994, 2024-09-18T04:58:10.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054108470999999998, 2024-09-18T04:58:10.542 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.542 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:10.542 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.543 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.543 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.543 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15924 2024-09-18T04:58:10.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.543 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:58:10.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:58:10.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:58:10.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:10.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230061+0000", 2024-09-18T04:58:10.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8240000000000002e-06 2024-09-18T04:58:10.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230068+0000", 2024-09-18T04:58:10.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4320000000000004e-06 2024-09-18T04:58:10.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230073+0000", 2024-09-18T04:58:10.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.921e-06 2024-09-18T04:58:10.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.545 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284153+0000", 2024-09-18T04:58:10.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054080793000000002 2024-09-18T04:58:10.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.546 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.546 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284163+0000", 2024-09-18T04:58:10.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.859e-06 2024-09-18T04:58:10.546 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.546 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.546 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.546 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15925 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:10.546 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:58:10.546 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.233725504000006, 2024-09-18T04:58:10.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054134632000000002, 2024-09-18T04:58:10.547 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.547 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:10.547 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.547 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.547 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.547 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15925 2024-09-18T04:58:10.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.547 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.547 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.547 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:58:10.547 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.547 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.547 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:58:10.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:58:10.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:10.548 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.548 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.548 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230081+0000", 2024-09-18T04:58:10.548 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4109999999999999e-06 2024-09-18T04:58:10.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230087+0000", 2024-09-18T04:58:10.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.161e-06 2024-09-18T04:58:10.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230092+0000", 2024-09-18T04:58:10.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9259999999999999e-06 2024-09-18T04:58:10.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284204+0000", 2024-09-18T04:58:10.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054111546000000003 2024-09-18T04:58:10.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284214+0000", 2024-09-18T04:58:10.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9639999999999998e-06 2024-09-18T04:58:10.550 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.550 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.550 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.550 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15926 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:10.550 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:58:10.550 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.233708179999994, 2024-09-18T04:58:10.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054143193999999999, 2024-09-18T04:58:10.551 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.551 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:10.551 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.551 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.551 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.551 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15926 2024-09-18T04:58:10.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.551 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:58:10.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:58:10.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:58:10.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:10.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.552 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230098+0000", 2024-09-18T04:58:10.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1399999999999998e-06 2024-09-18T04:58:10.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230104+0000", 2024-09-18T04:58:10.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-18T04:58:10.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230109+0000", 2024-09-18T04:58:10.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9250000000000003e-06 2024-09-18T04:58:10.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284231+0000", 2024-09-18T04:58:10.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054122297999999999 2024-09-18T04:58:10.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284239+0000", 2024-09-18T04:58:10.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1689999999999992e-06 2024-09-18T04:58:10.554 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.554 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.554 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.555 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15927 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:10.555 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:58:10.555 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.233691132999994, 2024-09-18T04:58:10.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054168435000000001, 2024-09-18T04:58:10.555 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.555 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:10.555 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.555 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.555 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.556 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15927 2024-09-18T04:58:10.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.556 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:58:10.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:58:10.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:58:10.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:10.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.557 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230116+0000", 2024-09-18T04:58:10.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5150000000000001e-06 2024-09-18T04:58:10.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230122+0000", 2024-09-18T04:58:10.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1190000000000002e-06 2024-09-18T04:58:10.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230126+0000", 2024-09-18T04:58:10.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.792e-06 2024-09-18T04:58:10.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284273+0000", 2024-09-18T04:58:10.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054146086000000003 2024-09-18T04:58:10.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284282+0000", 2024-09-18T04:58:10.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2590000000000006e-06 2024-09-18T04:58:10.559 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.559 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.559 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.560 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15928 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:10.560 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:58:10.560 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.233673749999994, 2024-09-18T04:58:10.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054189739000000001, 2024-09-18T04:58:10.560 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.560 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:10.560 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.560 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.560 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.560 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15928 2024-09-18T04:58:10.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.560 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:58:10.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:58:10.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230130+0000", 2024-09-18T04:58:10.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:10.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230133+0000", 2024-09-18T04:58:10.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.244e-06 2024-09-18T04:58:10.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.562 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.562 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230139+0000", 2024-09-18T04:58:10.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9830000000000001e-06 2024-09-18T04:58:10.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230143+0000", 2024-09-18T04:58:10.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7530000000000001e-06 2024-09-18T04:58:10.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284313+0000", 2024-09-18T04:58:10.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054169332000000001 2024-09-18T04:58:10.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284320+0000", 2024-09-18T04:58:10.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8699999999999992e-06 2024-09-18T04:58:10.565 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.565 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.565 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.565 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15929 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:10.565 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:58:10.565 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.233657309999998, 2024-09-18T04:58:10.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054218771999999998, 2024-09-18T04:58:10.565 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.566 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:10.566 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.566 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.566 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.566 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15929 2024-09-18T04:58:10.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.567 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:58:10.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:58:10.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:58:10.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:10.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230149+0000", 2024-09-18T04:58:10.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-18T04:58:10.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.568 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.568 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230155+0000", 2024-09-18T04:58:10.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2469999999999997e-06 2024-09-18T04:58:10.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230160+0000", 2024-09-18T04:58:10.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7129999999999996e-06 2024-09-18T04:58:10.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284355+0000", 2024-09-18T04:58:10.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054194900999999997 2024-09-18T04:58:10.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.570 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.570 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284366+0000", 2024-09-18T04:58:10.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0993e-05 2024-09-18T04:58:10.570 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.570 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.570 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.570 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.570 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15930 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:10.570 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:58:10.570 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.233640309999998, 2024-09-18T04:58:10.570 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054244713999999999, 2024-09-18T04:58:10.570 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.570 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:10.571 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.571 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.571 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.571 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15930 2024-09-18T04:58:10.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.571 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:58:10.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:58:10.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:58:10.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:10.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230166+0000", 2024-09-18T04:58:10.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1009999999999999e-06 2024-09-18T04:58:10.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230172+0000", 2024-09-18T04:58:10.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.981e-06 2024-09-18T04:58:10.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230184+0000", 2024-09-18T04:58:10.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2449000000000001e-05 2024-09-18T04:58:10.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284400+0000", 2024-09-18T04:58:10.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054215452999999997 2024-09-18T04:58:10.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284409+0000", 2024-09-18T04:58:10.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0839999999999998e-06 2024-09-18T04:58:10.579 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.579 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.580 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.580 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15931 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:10.580 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:58:10.580 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 94.233614521000007, 2024-09-18T04:58:10.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054261928000000001, 2024-09-18T04:58:10.581 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.581 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:10.581 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.581 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.581 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.582 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15931 2024-09-18T04:58:10.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.582 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:58:10.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:58:10.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230189+0000", 2024-09-18T04:58:10.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:10.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230192+0000", 2024-09-18T04:58:10.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6599999999999999e-06 2024-09-18T04:58:10.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.586 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.586 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230199+0000", 2024-09-18T04:58:10.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5570000000000001e-06 2024-09-18T04:58:10.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230204+0000", 2024-09-18T04:58:10.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2270000000000001e-06 2024-09-18T04:58:10.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284442+0000", 2024-09-18T04:58:10.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054238383000000001 2024-09-18T04:58:10.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284452+0000", 2024-09-18T04:58:10.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6069999999999997e-06 2024-09-18T04:58:10.588 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.588 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.588 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.588 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.588 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.588 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:10.590 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:58:10.591 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:10.591 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-18T04:58:10.591 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:10.591 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:10.587+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:10.691 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:10.688+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:10.739 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:10.739 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:10.739 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:58:10.739 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:10.748 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:58:10.791 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:10.789+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:10.854 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:10.854 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:10.854 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:58:10.854 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:58:10.854 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:10.863 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:58:10.892 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:10.890+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:10.971 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:10.971 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:58:10.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:58:10.971 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:58:10.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.971 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160713 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 645120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:10.971 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654655+0000", 2024-09-18T04:58:10.971 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.313090819999999, 2024-09-18T04:58:10.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039354912999999998, 2024-09-18T04:58:10.971 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.971 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:10.972 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.972 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.972 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.972 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160713 2024-09-18T04:58:10.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.972 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654655+0000", 2024-09-18T04:58:10.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.973 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654655+0000", 2024-09-18T04:58:10.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.973 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654654+0000", 2024-09-18T04:58:10.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:10.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654657+0000", 2024-09-18T04:58:10.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7049999999999999e-06 2024-09-18T04:58:10.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654669+0000", 2024-09-18T04:58:10.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1960000000000001e-05 2024-09-18T04:58:10.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654674+0000", 2024-09-18T04:58:10.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8729999999999998e-06 2024-09-18T04:58:10.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.693848+0000", 2024-09-18T04:58:10.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039174202999999998 2024-09-18T04:58:10.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:10.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.693926+0000", 2024-09-18T04:58:10.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8248e-05 2024-09-18T04:58:10.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694009+0000", 2024-09-18T04:58:10.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3363999999999997e-05 2024-09-18T04:58:10.976 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.976 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.976 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.976 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160714 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 665600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:10.976 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654678+0000", 2024-09-18T04:58:10.977 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.313067443999998, 2024-09-18T04:58:10.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039530155999999997, 2024-09-18T04:58:10.977 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.977 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:10.977 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.977 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.977 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.977 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160714 2024-09-18T04:58:10.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.977 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654678+0000", 2024-09-18T04:58:10.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654678+0000", 2024-09-18T04:58:10.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654678+0000", 2024-09-18T04:58:10.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:10.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654680+0000", 2024-09-18T04:58:10.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5229999999999998e-06 2024-09-18T04:58:10.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654686+0000", 2024-09-18T04:58:10.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9000000000000003e-06 2024-09-18T04:58:10.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654690+0000", 2024-09-18T04:58:10.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1060000000000002e-06 2024-09-18T04:58:10.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694042+0000", 2024-09-18T04:58:10.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039351490000000003 2024-09-18T04:58:10.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:10.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694142+0000", 2024-09-18T04:58:10.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100177 2024-09-18T04:58:10.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694208+0000", 2024-09-18T04:58:10.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6367999999999999e-05 2024-09-18T04:58:10.981 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.984 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.985 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.985 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160715 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:10.985 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:58:10.985 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.313051145000003, 2024-09-18T04:58:10.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039724306000000001, 2024-09-18T04:58:10.985 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.985 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:10.985 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.985 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.986 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.986 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160715 2024-09-18T04:58:10.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.986 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:58:10.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:58:10.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:58:10.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:10.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654696+0000", 2024-09-18T04:58:10.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1830000000000001e-06 2024-09-18T04:58:10.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654702+0000", 2024-09-18T04:58:10.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5110000000000003e-06 2024-09-18T04:58:10.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654706+0000", 2024-09-18T04:58:10.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2470000000000002e-06 2024-09-18T04:58:10.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694245+0000", 2024-09-18T04:58:10.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039539259 2024-09-18T04:58:10.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:10.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694327+0000", 2024-09-18T04:58:10.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1817000000000006e-05 2024-09-18T04:58:10.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694419+0000", 2024-09-18T04:58:10.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1639999999999997e-05 2024-09-18T04:58:10.990 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.990 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.990 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.990 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160716 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:10.991 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:58:10.991 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.313035960000001, 2024-09-18T04:58:10.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039842555000000002, 2024-09-18T04:58:10.991 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.991 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:10.991 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.991 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.991 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.991 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160716 2024-09-18T04:58:10.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.991 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:58:10.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:10.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:58:10.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:10.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:58:10.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:10.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:10.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654711+0000", 2024-09-18T04:58:10.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.99e-06 2024-09-18T04:58:10.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:10.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654717+0000", 2024-09-18T04:58:10.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6799999999999998e-06 2024-09-18T04:58:10.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:10.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654721+0000", 2024-09-18T04:58:10.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-18T04:58:10.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:10.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694449+0000", 2024-09-18T04:58:10.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039727476999999997 2024-09-18T04:58:10.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:10.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694497+0000", 2024-09-18T04:58:10.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8294000000000002e-05 2024-09-18T04:58:10.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:10.996 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694552+0000", 2024-09-18T04:58:10.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5024e-05 2024-09-18T04:58:10.996 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.996 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:10.996 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:10.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.997 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160717 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 727040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:10.997 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:58:10.997 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.312995374000003, 2024-09-18T04:58:10.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040030957999999998, 2024-09-18T04:58:10.997 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:10.998 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:10.998 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:10.998 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:10.998 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:10.998 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160717 2024-09-18T04:58:10.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.999 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:10.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:10.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:58:10.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:10.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:10.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:10.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:11.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:58:11.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:11.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:58:11.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:11.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:11.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654752+0000", 2024-09-18T04:58:11.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8219999999999998e-06 2024-09-18T04:58:11.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:11.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654769+0000", 2024-09-18T04:58:11.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6249e-05 2024-09-18T04:58:11.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:11.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654775+0000", 2024-09-18T04:58:11.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2090000000000002e-06 2024-09-18T04:58:11.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:11.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694589+0000", 2024-09-18T04:58:11.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039814651999999999 2024-09-18T04:58:11.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:11.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694674+0000", 2024-09-18T04:58:11.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5032999999999997e-05 2024-09-18T04:58:11.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:11.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694781+0000", 2024-09-18T04:58:11.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106452 2024-09-18T04:58:11.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.005 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:11.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.005 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160718 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 747520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:11.005 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:58:11.006 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.312965482999999, 2024-09-18T04:58:11.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040152520999999997, 2024-09-18T04:58:11.006 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:11.006 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:11.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:11.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:11.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:11.006 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160718 2024-09-18T04:58:11.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.006 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:11.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:11.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:58:11.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:11.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:58:11.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654779+0000", 2024-09-18T04:58:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654782+0000", 2024-09-18T04:58:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.633e-06 2024-09-18T04:58:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654789+0000", 2024-09-18T04:58:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8589999999999999e-06 2024-09-18T04:58:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:11.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654793+0000", 2024-09-18T04:58:11.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5449999999999997e-06 2024-09-18T04:58:11.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:11.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694817+0000", 2024-09-18T04:58:11.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040023940000000001 2024-09-18T04:58:11.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.010 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:11.010 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694861+0000", 2024-09-18T04:58:11.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3961999999999997e-05 2024-09-18T04:58:11.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694932+0000", 2024-09-18T04:58:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1033999999999998e-05 2024-09-18T04:58:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160719 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 768000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:58:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.312947885, 2024-09-18T04:58:11.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040275321000000003, 2024-09-18T04:58:11.012 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:11.012 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:11.012 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:11.012 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:11.012 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:11.012 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160719 2024-09-18T04:58:11.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.012 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:11.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:11.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:58:11.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:11.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:58:11.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:11.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:58:11.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:11.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:11.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654799+0000", 2024-09-18T04:58:11.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.204e-06 2024-09-18T04:58:11.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.014 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:11.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654806+0000", 2024-09-18T04:58:11.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0500000000000003e-06 2024-09-18T04:58:11.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:11.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654814+0000", 2024-09-18T04:58:11.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5549999999999997e-06 2024-09-18T04:58:11.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:11.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694967+0000", 2024-09-18T04:58:11.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040153393000000002 2024-09-18T04:58:11.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:11.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695014+0000", 2024-09-18T04:58:11.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7194999999999998e-05 2024-09-18T04:58:11.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:11.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695073+0000", 2024-09-18T04:58:11.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8315e-05 2024-09-18T04:58:11.017 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.017 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:11.017 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.017 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160720 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 788480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:11.018 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:58:11.018 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.312925591999999, 2024-09-18T04:58:11.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040546450999999997, 2024-09-18T04:58:11.018 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:11.018 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:11.018 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:11.018 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:11.018 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:11.018 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160720 2024-09-18T04:58:11.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.018 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:11.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:11.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:58:11.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:11.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:58:11.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:11.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654819+0000", 2024-09-18T04:58:11.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654822+0000", 2024-09-18T04:58:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:58:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654828+0000", 2024-09-18T04:58:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2450000000000003e-06 2024-09-18T04:58:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:11.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654833+0000", 2024-09-18T04:58:11.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6419999999999998e-06 2024-09-18T04:58:11.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:11.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695116+0000", 2024-09-18T04:58:11.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040283126000000002 2024-09-18T04:58:11.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:11.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695203+0000", 2024-09-18T04:58:11.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7693000000000002e-05 2024-09-18T04:58:11.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:11.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695366+0000", 2024-09-18T04:58:11.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016283700000000001 2024-09-18T04:58:11.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.023 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:11.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.023 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160721 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 808960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:11.023 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:58:11.023 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.312908256999997, 2024-09-18T04:58:11.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040737433000000003, 2024-09-18T04:58:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160721 2024-09-18T04:58:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:58:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:58:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:58:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654839+0000", 2024-09-18T04:58:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.43e-06 2024-09-18T04:58:11.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654847+0000", 2024-09-18T04:58:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7300000000000005e-06 2024-09-18T04:58:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654858+0000", 2024-09-18T04:58:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.099e-05 2024-09-18T04:58:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695391+0000", 2024-09-18T04:58:11.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040533476999999998 2024-09-18T04:58:11.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:11.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695477+0000", 2024-09-18T04:58:11.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5870000000000003e-05 2024-09-18T04:58:11.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:11.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695575+0000", 2024-09-18T04:58:11.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7264999999999998e-05 2024-09-18T04:58:11.027 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.027 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:11.027 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160722 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 829440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:58:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.312882727000002, 2024-09-18T04:58:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040875739000000001, 2024-09-18T04:58:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160722 2024-09-18T04:58:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:11.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:11.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:58:11.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:11.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:58:11.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654862+0000", 2024-09-18T04:58:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654883+0000", 2024-09-18T04:58:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0468000000000001e-05 2024-09-18T04:58:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654891+0000", 2024-09-18T04:58:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8320000000000006e-06 2024-09-18T04:58:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654896+0000", 2024-09-18T04:58:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.13e-06 2024-09-18T04:58:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695628+0000", 2024-09-18T04:58:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040732011999999998 2024-09-18T04:58:11.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:11.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695678+0000", 2024-09-18T04:58:11.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0453e-05 2024-09-18T04:58:11.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:11.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695738+0000", 2024-09-18T04:58:11.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0164999999999997e-05 2024-09-18T04:58:11.033 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.033 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:11.033 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.033 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160723 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 849920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:11.033 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:58:11.033 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.312790921000001, 2024-09-18T04:58:11.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041073576000000001, 2024-09-18T04:58:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160723 2024-09-18T04:58:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:58:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:58:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:58:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:11.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654958+0000", 2024-09-18T04:58:11.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7230000000000002e-06 2024-09-18T04:58:11.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:11.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654968+0000", 2024-09-18T04:58:11.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0665999999999999e-05 2024-09-18T04:58:11.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:11.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654974+0000", 2024-09-18T04:58:11.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9259999999999997e-06 2024-09-18T04:58:11.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:11.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695824+0000", 2024-09-18T04:58:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040850085000000001 2024-09-18T04:58:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695933+0000", 2024-09-18T04:58:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108738 2024-09-18T04:58:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696028+0000", 2024-09-18T04:58:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4988000000000006e-05 2024-09-18T04:58:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.038 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:11.039 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.039 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160724 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 870400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:11.039 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:58:11.039 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.312766578000002, 2024-09-18T04:58:11.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041279300999999997, 2024-09-18T04:58:11.039 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:11.039 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:11.039 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160724 2024-09-18T04:58:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:58:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:11.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:58:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654978+0000", 2024-09-18T04:58:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654981+0000", 2024-09-18T04:58:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4530000000000001e-06 2024-09-18T04:58:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654987+0000", 2024-09-18T04:58:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6359999999999997e-06 2024-09-18T04:58:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654995+0000", 2024-09-18T04:58:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9759999999999995e-06 2024-09-18T04:58:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696069+0000", 2024-09-18T04:58:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041073447999999999 2024-09-18T04:58:11.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:11.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696160+0000", 2024-09-18T04:58:11.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0835999999999996e-05 2024-09-18T04:58:11.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:11.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696258+0000", 2024-09-18T04:58:11.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8338999999999995e-05 2024-09-18T04:58:11.043 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.043 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:11.043 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.044 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160725 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 890880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:11.044 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:58:11.044 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.312745509000003, 2024-09-18T04:58:11.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041552253999999997, 2024-09-18T04:58:11.044 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:11.044 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:11.044 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:11.044 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:11.044 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:11.044 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160725 2024-09-18T04:58:11.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.044 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:11.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:58:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:58:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654999+0000", 2024-09-18T04:58:11.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:11.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:11.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655002+0000", 2024-09-18T04:58:11.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.419e-06 2024-09-18T04:58:11.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:11.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655008+0000", 2024-09-18T04:58:11.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2310000000000001e-06 2024-09-18T04:58:11.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:11.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655013+0000", 2024-09-18T04:58:11.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8439999999999997e-06 2024-09-18T04:58:11.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:11.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696353+0000", 2024-09-18T04:58:11.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041340322999999998 2024-09-18T04:58:11.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:11.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696455+0000", 2024-09-18T04:58:11.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000102105 2024-09-18T04:58:11.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:11.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696552+0000", 2024-09-18T04:58:11.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6738999999999997e-05 2024-09-18T04:58:11.048 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.048 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:11.048 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.048 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160726 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:11.048 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:58:11.048 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.312728276999998, 2024-09-18T04:58:11.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041786440000000001, 2024-09-18T04:58:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160726 2024-09-18T04:58:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:58:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:58:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:58:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655019+0000", 2024-09-18T04:58:11.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1100000000000001e-06 2024-09-18T04:58:11.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:11.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655025+0000", 2024-09-18T04:58:11.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2190000000000001e-06 2024-09-18T04:58:11.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:11.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655030+0000", 2024-09-18T04:58:11.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5340000000000001e-06 2024-09-18T04:58:11.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:11.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696607+0000", 2024-09-18T04:58:11.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041577309 2024-09-18T04:58:11.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:11.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696694+0000", 2024-09-18T04:58:11.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7230000000000003e-05 2024-09-18T04:58:11.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696804+0000", 2024-09-18T04:58:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0001094 2024-09-18T04:58:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160727 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 931840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:58:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.312711540000002, 2024-09-18T04:58:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042000527000000003, 2024-09-18T04:58:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:11.053 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160727 2024-09-18T04:58:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:58:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:58:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655033+0000", 2024-09-18T04:58:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655036+0000", 2024-09-18T04:58:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3489999999999999e-06 2024-09-18T04:58:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655042+0000", 2024-09-18T04:58:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1399999999999997e-06 2024-09-18T04:58:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655046+0000", 2024-09-18T04:58:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5480000000000003e-06 2024-09-18T04:58:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696843+0000", 2024-09-18T04:58:11.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041797103000000002 2024-09-18T04:58:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696931+0000", 2024-09-18T04:58:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7526000000000002e-05 2024-09-18T04:58:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697034+0000", 2024-09-18T04:58:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010336999999999999 2024-09-18T04:58:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160728 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 952320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:58:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.312694983, 2024-09-18T04:58:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042191293999999997, 2024-09-18T04:58:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160728 2024-09-18T04:58:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:11.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:58:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:58:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:58:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:11.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:11.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655052+0000", 2024-09-18T04:58:11.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.232e-06 2024-09-18T04:58:11.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:11.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655058+0000", 2024-09-18T04:58:11.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9869999999999996e-06 2024-09-18T04:58:11.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:11.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655063+0000", 2024-09-18T04:58:11.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8420000000000004e-06 2024-09-18T04:58:11.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:11.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697074+0000", 2024-09-18T04:58:11.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042010446 2024-09-18T04:58:11.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:11.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697157+0000", 2024-09-18T04:58:11.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3038e-05 2024-09-18T04:58:11.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697242+0000", 2024-09-18T04:58:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5115000000000001e-05 2024-09-18T04:58:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160729 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 972800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:58:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.312678224000003, 2024-09-18T04:58:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042367834, 2024-09-18T04:58:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:11.062 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160729 2024-09-18T04:58:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:58:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:11.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:58:11.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:11.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:58:11.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:11.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:11.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655069+0000", 2024-09-18T04:58:11.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2220000000000001e-06 2024-09-18T04:58:11.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:11.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655075+0000", 2024-09-18T04:58:11.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2129999999999996e-06 2024-09-18T04:58:11.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655080+0000", 2024-09-18T04:58:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6840000000000004e-06 2024-09-18T04:58:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697280+0000", 2024-09-18T04:58:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042199819999999999 2024-09-18T04:58:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697368+0000", 2024-09-18T04:58:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7863999999999998e-05 2024-09-18T04:58:11.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697435+0000", 2024-09-18T04:58:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7416000000000006e-05 2024-09-18T04:58:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160730 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 993280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:58:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.312661458999997, 2024-09-18T04:58:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042559582999999998, 2024-09-18T04:58:11.068 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:11.069 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:11.069 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:11.069 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:11.069 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:11.069 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160730 2024-09-18T04:58:11.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.069 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:11.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:11.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:58:11.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:58:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:58:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655086+0000", 2024-09-18T04:58:11.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1500000000000002e-06 2024-09-18T04:58:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655092+0000", 2024-09-18T04:58:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.896e-06 2024-09-18T04:58:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655096+0000", 2024-09-18T04:58:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7190000000000001e-06 2024-09-18T04:58:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:11.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697475+0000", 2024-09-18T04:58:11.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042378644 2024-09-18T04:58:11.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:11.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697553+0000", 2024-09-18T04:58:11.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7626999999999994e-05 2024-09-18T04:58:11.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:11.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697643+0000", 2024-09-18T04:58:11.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0921000000000006e-05 2024-09-18T04:58:11.072 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.072 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160731 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 1013760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:58:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.312628728999996, 2024-09-18T04:58:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042727764000000001, 2024-09-18T04:58:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160731 2024-09-18T04:58:11.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:58:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:58:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655116+0000", 2024-09-18T04:58:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655119+0000", 2024-09-18T04:58:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1240000000000001e-06 2024-09-18T04:58:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655131+0000", 2024-09-18T04:58:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2145e-05 2024-09-18T04:58:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:11.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655138+0000", 2024-09-18T04:58:11.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3369999999999996e-06 2024-09-18T04:58:11.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:11.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697688+0000", 2024-09-18T04:58:11.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042550604999999998 2024-09-18T04:58:11.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:11.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697781+0000", 2024-09-18T04:58:11.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2867000000000004e-05 2024-09-18T04:58:11.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:11.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697844+0000", 2024-09-18T04:58:11.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3236000000000002e-05 2024-09-18T04:58:11.077 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.077 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:11.077 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.077 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160732 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 1034240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:11.077 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:58:11.077 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 34.312602880999997, 2024-09-18T04:58:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042857977999999998, 2024-09-18T04:58:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160732 2024-09-18T04:58:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:58:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:58:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:58:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655145+0000", 2024-09-18T04:58:11.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7329999999999999e-06 2024-09-18T04:58:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655152+0000", 2024-09-18T04:58:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8249999999999999e-06 2024-09-18T04:58:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655158+0000", 2024-09-18T04:58:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5010000000000001e-06 2024-09-18T04:58:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697873+0000", 2024-09-18T04:58:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042714457999999997 2024-09-18T04:58:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697944+0000", 2024-09-18T04:58:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1486e-05 2024-09-18T04:58:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.698000+0000", 2024-09-18T04:58:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6332000000000002e-05 2024-09-18T04:58:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.081 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:11.082 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.082 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:11.082 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:11.082 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:11.083 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:58:11.084 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-18T04:58:11.084 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:11.081+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:11.184 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:11.181+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:11.211 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:58:11.233 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:11.233 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:11.233 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:58:11.234 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:11.243 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:58:11.285 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:11.282+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:11.336 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:58:11.336 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:58:11.353 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:58:11.353 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:58:11.369 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:58:11.370 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:58:11.385 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:58:11.386 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:58:11.386 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:11.383+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:11.392 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:11.393 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:11.402 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:58:11.402 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:58:11.402 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:58:11.418 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:58:11.418 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:58:11.435 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:58:11.435 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:58:11.451 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:58:11.452 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:58:11.504 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:11.502+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:11.510 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:11.510 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:11.519 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:58:11.588 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:11.585+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:11.623 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:11.623 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:11.631 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:58:11.689 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:11.687+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:11.763 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:11.763 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:11.764 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:58:11.791 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:11.789+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:11.882 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:12.438 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:12.441 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:58:12.441 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:12.437+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:12.540 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:12.538+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:12.594 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:12.594 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:12.604 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:58:12.645 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:12.643+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:12.742 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:12.740+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:12.748 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:12.748 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:12.757 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:58:12.843 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:12.841+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:12.898 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:12.898 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:12.907 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:58:12.948 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:12.945+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:13.045 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:13.042+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:13.058 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:13.058 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:13.068 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:58:13.085 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T04:58:13.155 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:13.150+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:13.211 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:13.211 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:13.220 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:58:13.262 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:13.259+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:13.348 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:13.346+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:13.364 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:13.364 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:13.372 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:58:13.458 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:13.455+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:13.464 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:58:13.515 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:13.516 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:13.524 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:58:13.551 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:13.548+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:13.587 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:58:13.587 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:58:13.604 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:58:13.604 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:58:13.620 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:58:13.620 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:58:13.636 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:58:13.636 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:58:13.651 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:13.649+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:13.652 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:58:13.652 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:58:13.667 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:13.667 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:13.668 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:58:13.669 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:58:13.677 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:58:13.686 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:58:13.686 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:58:13.703 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:58:13.703 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:58:13.755 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:13.753+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:13.781 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:13.781 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:58:13.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:58:13.781 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:13.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.781 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:58:13.781 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:58:13.781 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 122.38439309100001, 2024-09-18T04:58:13.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037718511000000003, 2024-09-18T04:58:13.782 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:13.782 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:13.782 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:13.782 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:58:13.782 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:58:13.782 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-18T04:58:13.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.782 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:13.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:13.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:58:13.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:13.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:58:13.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:13.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393125+0000", 2024-09-18T04:58:13.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:58:13.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:13.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393133+0000", 2024-09-18T04:58:13.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.225e-06 2024-09-18T04:58:13.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:13.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393144+0000", 2024-09-18T04:58:13.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1279e-05 2024-09-18T04:58:13.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:13.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393153+0000", 2024-09-18T04:58:13.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4540000000000007e-06 2024-09-18T04:58:13.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:13.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393203+0000", 2024-09-18T04:58:13.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9963000000000003e-05 2024-09-18T04:58:13.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:13.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393210+0000", 2024-09-18T04:58:13.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5689999999999999e-06 2024-09-18T04:58:13.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:13.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427399+0000", 2024-09-18T04:58:13.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034188784999999999 2024-09-18T04:58:13.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:13.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427532+0000", 2024-09-18T04:58:13.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013264700000000001 2024-09-18T04:58:13.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:58:13.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427685+0000", 2024-09-18T04:58:13.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015300599999999999 2024-09-18T04:58:13.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:13.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.428520+0000", 2024-09-18T04:58:13.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00083502499999999998 2024-09-18T04:58:13.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:13.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430772+0000", 2024-09-18T04:58:13.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022518880000000001 2024-09-18T04:58:13.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:13.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430817+0000", 2024-09-18T04:58:13.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5077000000000002e-05 2024-09-18T04:58:13.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:13.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430845+0000", 2024-09-18T04:58:13.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8036999999999999e-05 2024-09-18T04:58:13.790 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.790 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:13.790 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.790 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:62520 3.13 3.84cdb893 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T04:58:13.790 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:13.790 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 82.480083207999996, 2024-09-18T04:58:13.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94452134799999998, 2024-09-18T04:58:13.791 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:13.791 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:58:13.791 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:13.791 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:13.791 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:13.791 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 62520 2024-09-18T04:58:13.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.791 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:13.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:13.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:13.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:13.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:13.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:13.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:13.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:13.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:13.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297441+0000", 2024-09-18T04:58:13.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9840000000000001e-06 2024-09-18T04:58:13.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:13.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297449+0000", 2024-09-18T04:58:13.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3999999999999992e-06 2024-09-18T04:58:13.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:13.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297455+0000", 2024-09-18T04:58:13.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3350000000000003e-06 2024-09-18T04:58:13.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:13.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297552+0000", 2024-09-18T04:58:13.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6602999999999996e-05 2024-09-18T04:58:13.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.794 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:13.794 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297559+0000", 2024-09-18T04:58:13.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.52e-06 2024-09-18T04:58:13.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:13.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241943+0000", 2024-09-18T04:58:13.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94438357100000003 2024-09-18T04:58:13.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:13.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241958+0000", 2024-09-18T04:58:13.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4598e-05 2024-09-18T04:58:13.795 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.795 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:13.795 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.796 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:113179 3.13 3:cf138017:::rbd_data.11ac5f86a3f9.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T04:58:13.796 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:58:13.796 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.090292796999996, 2024-09-18T04:58:13.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036508304999999998, 2024-09-18T04:58:13.796 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:13.796 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:13.796 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:13.796 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:13.796 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:13.796 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 113179 2024-09-18T04:58:13.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.796 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:13.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:13.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:58:13.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:13.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:58:13.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:13.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687226+0000", 2024-09-18T04:58:13.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:58:13.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:13.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688169+0000", 2024-09-18T04:58:13.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00094328200000000004 2024-09-18T04:58:13.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:13.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688182+0000", 2024-09-18T04:58:13.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2756e-05 2024-09-18T04:58:13.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:13.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688188+0000", 2024-09-18T04:58:13.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0290000000000002e-06 2024-09-18T04:58:13.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:13.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688225+0000", 2024-09-18T04:58:13.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7301000000000001e-05 2024-09-18T04:58:13.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:13.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688324+0000", 2024-09-18T04:58:13.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9419999999999993e-05 2024-09-18T04:58:13.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:58:13.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.700483+0000", 2024-09-18T04:58:13.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012158771000000001 2024-09-18T04:58:13.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:13.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723586+0000", 2024-09-18T04:58:13.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023102952 2024-09-18T04:58:13.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:13.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723634+0000", 2024-09-18T04:58:13.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7468e-05 2024-09-18T04:58:13.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:13.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723693+0000", 2024-09-18T04:58:13.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9539e-05 2024-09-18T04:58:13.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:13.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723735+0000", 2024-09-18T04:58:13.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1829000000000002e-05 2024-09-18T04:58:13.801 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.801 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:13.801 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.802 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146077 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1400832~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:13.802 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593265+0000", 2024-09-18T04:58:13.802 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 40.184254418999998, 2024-09-18T04:58:13.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035380397000000001, 2024-09-18T04:58:13.802 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:13.802 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:13.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:13.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:13.802 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:13.802 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146077 2024-09-18T04:58:13.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.802 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:13.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:13.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593265+0000", 2024-09-18T04:58:13.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:13.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593265+0000", 2024-09-18T04:58:13.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:13.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593265+0000", 2024-09-18T04:58:13.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:13.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:13.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593266+0000", 2024-09-18T04:58:13.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1039999999999999e-06 2024-09-18T04:58:13.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:13.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593269+0000", 2024-09-18T04:58:13.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8760000000000001e-06 2024-09-18T04:58:13.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.804 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:13.804 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593271+0000", 2024-09-18T04:58:13.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5100000000000001e-06 2024-09-18T04:58:13.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:13.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628545+0000", 2024-09-18T04:58:13.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035273977999999998 2024-09-18T04:58:13.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:13.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628589+0000", 2024-09-18T04:58:13.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4088999999999997e-05 2024-09-18T04:58:13.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:13.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628645+0000", 2024-09-18T04:58:13.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6059e-05 2024-09-18T04:58:13.806 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.806 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:13.806 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.806 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146078 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1449984~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:13.806 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:13.806 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 40.184214953999998, 2024-09-18T04:58:13.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035495325000000001, 2024-09-18T04:58:13.806 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:13.806 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:13.806 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:13.806 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:13.806 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:13.807 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146078 2024-09-18T04:58:13.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.807 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:13.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:13.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:13.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:13.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:13.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:13.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:13.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:13.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:13.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593306+0000", 2024-09-18T04:58:13.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.686e-06 2024-09-18T04:58:13.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:13.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593311+0000", 2024-09-18T04:58:13.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8500000000000002e-06 2024-09-18T04:58:13.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:13.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593314+0000", 2024-09-18T04:58:13.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0290000000000001e-06 2024-09-18T04:58:13.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.809 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:13.809 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628674+0000", 2024-09-18T04:58:13.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035360346000000001 2024-09-18T04:58:13.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:13.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628726+0000", 2024-09-18T04:58:13.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2219999999999998e-05 2024-09-18T04:58:13.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:13.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628800+0000", 2024-09-18T04:58:13.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3613000000000001e-05 2024-09-18T04:58:13.810 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.810 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:13.810 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.811 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146079 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1499136~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:13.811 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:13.811 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 40.184203549000003, 2024-09-18T04:58:13.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035604581000000003, 2024-09-18T04:58:13.811 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:13.811 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:13.811 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:13.811 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:13.811 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:13.811 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146079 2024-09-18T04:58:13.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.811 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:13.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:13.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:13.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:13.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:13.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:13.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:13.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:13.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:13.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593317+0000", 2024-09-18T04:58:13.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1599999999999999e-06 2024-09-18T04:58:13.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:13.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593320+0000", 2024-09-18T04:58:13.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0130000000000002e-06 2024-09-18T04:58:13.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:13.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593322+0000", 2024-09-18T04:58:13.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6079999999999998e-06 2024-09-18T04:58:13.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:13.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628825+0000", 2024-09-18T04:58:13.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035502747000000001 2024-09-18T04:58:13.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:13.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628868+0000", 2024-09-18T04:58:13.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2719999999999998e-05 2024-09-18T04:58:13.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:13.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628920+0000", 2024-09-18T04:58:13.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2533e-05 2024-09-18T04:58:13.815 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.815 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:13.815 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.816 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146080 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1548288~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:13.816 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:13.816 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 40.184188051, 2024-09-18T04:58:13.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035755146000000002, 2024-09-18T04:58:13.816 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:13.816 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:13.816 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:13.817 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:13.817 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:13.817 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146080 2024-09-18T04:58:13.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.817 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:13.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:13.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:13.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:13.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:13.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:13.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:13.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:13.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:13.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593333+0000", 2024-09-18T04:58:13.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.858e-06 2024-09-18T04:58:13.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:13.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593336+0000", 2024-09-18T04:58:13.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4910000000000001e-06 2024-09-18T04:58:13.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:13.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593340+0000", 2024-09-18T04:58:13.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2590000000000001e-06 2024-09-18T04:58:13.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:13.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628956+0000", 2024-09-18T04:58:13.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035616039000000002 2024-09-18T04:58:13.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:13.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629031+0000", 2024-09-18T04:58:13.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5504000000000001e-05 2024-09-18T04:58:13.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:13.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629086+0000", 2024-09-18T04:58:13.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5429000000000001e-05 2024-09-18T04:58:13.820 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.820 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:13.820 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.821 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146081 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1597440~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:13.821 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:13.821 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 40.184177478999999, 2024-09-18T04:58:13.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035873223000000003, 2024-09-18T04:58:13.821 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:13.821 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:13.821 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:13.821 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:13.821 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:13.821 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146081 2024-09-18T04:58:13.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.821 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:13.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:13.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:13.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:13.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:13.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:13.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:13.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:13.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:13.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593343+0000", 2024-09-18T04:58:13.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3319999999999999e-06 2024-09-18T04:58:13.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:13.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593346+0000", 2024-09-18T04:58:13.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9059999999999998e-06 2024-09-18T04:58:13.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:13.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593348+0000", 2024-09-18T04:58:13.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4530000000000001e-06 2024-09-18T04:58:13.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:13.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629110+0000", 2024-09-18T04:58:13.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035761814000000003 2024-09-18T04:58:13.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:13.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629159+0000", 2024-09-18T04:58:13.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8971000000000003e-05 2024-09-18T04:58:13.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:13.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629215+0000", 2024-09-18T04:58:13.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6107000000000003e-05 2024-09-18T04:58:13.825 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.825 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:13.825 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.825 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146082 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1646592~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:13.825 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:13.825 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 40.184161429, 2024-09-18T04:58:13.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035987007000000001, 2024-09-18T04:58:13.825 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:13.825 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:13.825 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:13.825 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:13.826 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:13.826 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146082 2024-09-18T04:58:13.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.826 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:13.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:13.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:13.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:13.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:13.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:13.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:13.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:13.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:13.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593360+0000", 2024-09-18T04:58:13.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8560000000000001e-06 2024-09-18T04:58:13.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:13.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593365+0000", 2024-09-18T04:58:13.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3410000000000002e-06 2024-09-18T04:58:13.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:13.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593369+0000", 2024-09-18T04:58:13.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4850000000000003e-06 2024-09-18T04:58:13.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:13.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629242+0000", 2024-09-18T04:58:13.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035872464999999999 2024-09-18T04:58:13.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:13.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629288+0000", 2024-09-18T04:58:13.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6592999999999999e-05 2024-09-18T04:58:13.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:13.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629345+0000", 2024-09-18T04:58:13.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6575e-05 2024-09-18T04:58:13.829 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.830 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:13.830 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.830 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146083 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1695744~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:13.830 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:58:13.830 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 40.184145581999999, 2024-09-18T04:58:13.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036100587000000003, 2024-09-18T04:58:13.830 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:13.831 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:13.831 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:13.831 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:13.831 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:13.831 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146083 2024-09-18T04:58:13.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.831 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:13.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:13.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:58:13.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:13.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:58:13.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:13.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593373+0000", 2024-09-18T04:58:13.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:13.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:13.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593376+0000", 2024-09-18T04:58:13.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.244e-06 2024-09-18T04:58:13.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:13.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593379+0000", 2024-09-18T04:58:13.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.619e-06 2024-09-18T04:58:13.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:13.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593382+0000", 2024-09-18T04:58:13.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7199999999999998e-06 2024-09-18T04:58:13.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:13.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629368+0000", 2024-09-18T04:58:13.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035986281000000002 2024-09-18T04:58:13.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.834 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:13.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629416+0000", 2024-09-18T04:58:13.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8055e-05 2024-09-18T04:58:13.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:13.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629474+0000", 2024-09-18T04:58:13.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8176e-05 2024-09-18T04:58:13.835 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.835 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:13.835 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.836 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146084 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1744896~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:13.836 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:13.836 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 40.184132452, 2024-09-18T04:58:13.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036214349, 2024-09-18T04:58:13.836 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:13.836 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:13.836 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:13.836 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:13.836 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:13.836 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146084 2024-09-18T04:58:13.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.836 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:13.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:13.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:13.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:13.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:13.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:13.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:13.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:13.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:13.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593388+0000", 2024-09-18T04:58:13.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.564e-06 2024-09-18T04:58:13.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:13.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593392+0000", 2024-09-18T04:58:13.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0579999999999999e-06 2024-09-18T04:58:13.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:13.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593395+0000", 2024-09-18T04:58:13.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8890000000000002e-06 2024-09-18T04:58:13.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:13.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629500+0000", 2024-09-18T04:58:13.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036105343999999998 2024-09-18T04:58:13.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:13.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629547+0000", 2024-09-18T04:58:13.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7036000000000002e-05 2024-09-18T04:58:13.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:13.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629601+0000", 2024-09-18T04:58:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3866999999999998e-05 2024-09-18T04:58:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146085 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1794048~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 40.184122064999997, 2024-09-18T04:58:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036331186000000001, 2024-09-18T04:58:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:13.840 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:13.841 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:13.841 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146085 2024-09-18T04:58:13.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.841 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:13.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:13.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:13.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:13.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:13.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:13.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:13.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:13.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:13.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593398+0000", 2024-09-18T04:58:13.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2270000000000001e-06 2024-09-18T04:58:13.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:13.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593401+0000", 2024-09-18T04:58:13.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.999e-06 2024-09-18T04:58:13.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:13.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593404+0000", 2024-09-18T04:58:13.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.486e-06 2024-09-18T04:58:13.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:13.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629624+0000", 2024-09-18T04:58:13.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036219983999999997 2024-09-18T04:58:13.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:13.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629672+0000", 2024-09-18T04:58:13.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8368000000000002e-05 2024-09-18T04:58:13.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:13.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629729+0000", 2024-09-18T04:58:13.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6359e-05 2024-09-18T04:58:13.847 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.847 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:13.847 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.847 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146086 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1843200~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:13.847 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:13.847 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 40.184113500999999, 2024-09-18T04:58:13.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036449098999999999, 2024-09-18T04:58:13.848 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:13.848 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:13.848 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:13.848 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:13.848 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:13.848 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146086 2024-09-18T04:58:13.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.848 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:13.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:13.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:13.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:13.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:13.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:13.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:13.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:13.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:13.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593407+0000", 2024-09-18T04:58:13.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1799999999999999e-06 2024-09-18T04:58:13.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:13.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593410+0000", 2024-09-18T04:58:13.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.926e-06 2024-09-18T04:58:13.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:13.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593413+0000", 2024-09-18T04:58:13.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8250000000000001e-06 2024-09-18T04:58:13.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:13.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629755+0000", 2024-09-18T04:58:13.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036342212999999998 2024-09-18T04:58:13.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:13.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629803+0000", 2024-09-18T04:58:13.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8619000000000003e-05 2024-09-18T04:58:13.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:13.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629855+0000", 2024-09-18T04:58:13.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1536000000000003e-05 2024-09-18T04:58:13.852 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.852 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:13.852 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.852 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146087 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1892352~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:13.852 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:13.852 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 40.184104576000003, 2024-09-18T04:58:13.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036564608999999998, 2024-09-18T04:58:13.852 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:13.853 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:13.853 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:13.853 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:13.853 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:13.853 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146087 2024-09-18T04:58:13.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.853 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:13.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:13.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:13.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:13.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:13.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:13.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:13.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:13.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:13.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593416+0000", 2024-09-18T04:58:13.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.218e-06 2024-09-18T04:58:13.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:13.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593419+0000", 2024-09-18T04:58:13.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-18T04:58:13.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:13.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593421+0000", 2024-09-18T04:58:13.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4329999999999998e-06 2024-09-18T04:58:13.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.855 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:13.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629878+0000", 2024-09-18T04:58:13.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036456578000000003 2024-09-18T04:58:13.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:13.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629925+0000", 2024-09-18T04:58:13.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7595000000000001e-05 2024-09-18T04:58:13.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:13.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629979+0000", 2024-09-18T04:58:13.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4117999999999999e-05 2024-09-18T04:58:13.856 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.857 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:13.857 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.857 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146088 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1941504~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:13.857 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:13.857 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 40.184082412000002, 2024-09-18T04:58:13.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036676121999999999, 2024-09-18T04:58:13.857 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:13.857 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:13.857 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:13.857 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:13.857 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:13.858 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146088 2024-09-18T04:58:13.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.858 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:13.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:13.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:13.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.858 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.858 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:13.858 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:13.858 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.858 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:13.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:13.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:13.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:13.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593438+0000", 2024-09-18T04:58:13.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7570000000000001e-06 2024-09-18T04:58:13.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:13.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593442+0000", 2024-09-18T04:58:13.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2880000000000001e-06 2024-09-18T04:58:13.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:13.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593444+0000", 2024-09-18T04:58:13.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.593e-06 2024-09-18T04:58:13.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:13.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630015+0000", 2024-09-18T04:58:13.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036571125000000003 2024-09-18T04:58:13.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:13.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630061+0000", 2024-09-18T04:58:13.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5445000000000003e-05 2024-09-18T04:58:13.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.861 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:13.861 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630113+0000", 2024-09-18T04:58:13.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2358000000000003e-05 2024-09-18T04:58:13.861 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.861 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:13.861 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.861 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146089 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1990656~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:13.862 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:58:13.862 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 40.184072831000002, 2024-09-18T04:58:13.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036791483999999999, 2024-09-18T04:58:13.862 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:13.862 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:13.862 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:13.862 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:13.862 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:13.862 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146089 2024-09-18T04:58:13.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.862 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:13.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:13.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:58:13.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:13.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:58:13.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:13.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593446+0000", 2024-09-18T04:58:13.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:13.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:13.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593448+0000", 2024-09-18T04:58:13.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1939999999999999e-06 2024-09-18T04:58:13.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:13.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593451+0000", 2024-09-18T04:58:13.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.766e-06 2024-09-18T04:58:13.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:13.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593458+0000", 2024-09-18T04:58:13.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8539999999999999e-06 2024-09-18T04:58:13.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:13.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630136+0000", 2024-09-18T04:58:13.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03667757 2024-09-18T04:58:13.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:13.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630183+0000", 2024-09-18T04:58:13.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7289000000000001e-05 2024-09-18T04:58:13.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:13.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630238+0000", 2024-09-18T04:58:13.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5013000000000003e-05 2024-09-18T04:58:13.866 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.866 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:13.866 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.866 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146090 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 2039808~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:13.866 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:13.866 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 40.184058950999997, 2024-09-18T04:58:13.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036904477999999998, 2024-09-18T04:58:13.867 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:13.867 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:13.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:13.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:13.867 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:13.867 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146090 2024-09-18T04:58:13.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.867 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:13.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:13.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:13.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:13.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:13.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:13.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:13.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:13.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:13.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593461+0000", 2024-09-18T04:58:13.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3680000000000001e-06 2024-09-18T04:58:13.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:13.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593465+0000", 2024-09-18T04:58:13.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8120000000000001e-06 2024-09-18T04:58:13.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:13.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593468+0000", 2024-09-18T04:58:13.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.582e-06 2024-09-18T04:58:13.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:13.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630268+0000", 2024-09-18T04:58:13.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036800449999999998 2024-09-18T04:58:13.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:13.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630313+0000", 2024-09-18T04:58:13.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4836000000000002e-05 2024-09-18T04:58:13.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:13.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630365+0000", 2024-09-18T04:58:13.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1839999999999998e-05 2024-09-18T04:58:13.870 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.870 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:13.871 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.871 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146091 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 2088960~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:13.871 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:13.871 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 40.184048488999998, 2024-09-18T04:58:13.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037011534999999998, 2024-09-18T04:58:13.871 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:13.871 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:13.871 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:13.871 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:13.871 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:13.871 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146091 2024-09-18T04:58:13.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.872 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:13.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:13.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:13.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:13.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:13.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:13.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:13.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:13.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:13.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593472+0000", 2024-09-18T04:58:13.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1620000000000001e-06 2024-09-18T04:58:13.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:13.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593475+0000", 2024-09-18T04:58:13.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0019999999999998e-06 2024-09-18T04:58:13.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:13.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593477+0000", 2024-09-18T04:58:13.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-18T04:58:13.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:13.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630381+0000", 2024-09-18T04:58:13.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036904014999999998 2024-09-18T04:58:13.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:13.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630428+0000", 2024-09-18T04:58:13.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6931000000000002e-05 2024-09-18T04:58:13.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:13.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630482+0000", 2024-09-18T04:58:13.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4138999999999998e-05 2024-09-18T04:58:13.875 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.875 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:13.875 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.875 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:162075 3.13 3:cf3f6048:::rbd_header.11ac6967455f:head [watch reconnect cookie 140406508629968 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T04:58:13.875 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:58:13.875 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 31.287075684000001, 2024-09-18T04:58:13.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.53184266599999996, 2024-09-18T04:58:13.875 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:13.875 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:13.876 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:13.876 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:13.876 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:13.876 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 162075 2024-09-18T04:58:13.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.876 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:13.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:13.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:58:13.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:13.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:58:13.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:13.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490442+0000", 2024-09-18T04:58:13.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:58:13.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:13.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490453+0000", 2024-09-18T04:58:13.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0613e-05 2024-09-18T04:58:13.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:13.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490466+0000", 2024-09-18T04:58:13.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3702e-05 2024-09-18T04:58:13.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:13.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490475+0000", 2024-09-18T04:58:13.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2690000000000005e-06 2024-09-18T04:58:13.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:13.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490612+0000", 2024-09-18T04:58:13.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013620900000000001 2024-09-18T04:58:13.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-18T04:58:13.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490620+0000", 2024-09-18T04:58:13.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3089999999999996e-06 2024-09-18T04:58:13.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:13.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503041+0000", 2024-09-18T04:58:13.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012420512 2024-09-18T04:58:13.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:13.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503048+0000", 2024-09-18T04:58:13.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0389999999999999e-06 2024-09-18T04:58:13.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:13.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.016122+0000", 2024-09-18T04:58:13.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.51307432900000005 2024-09-18T04:58:13.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:13.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.016289+0000", 2024-09-18T04:58:13.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000166875 2024-09-18T04:58:13.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.880 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:13.880 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.022286+0000", 2024-09-18T04:58:13.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059976049999999996 2024-09-18T04:58:13.881 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.881 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:13.881 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.881 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:162076 3.13 3:cf3f6048:::rbd_header.11ac6967455f:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T04:58:13.881 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:13.881 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 31.286953010000001, 2024-09-18T04:58:13.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.53464117600000005, 2024-09-18T04:58:13.881 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:13.881 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:13.881 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:13.881 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:13.882 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:13.882 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 162076 2024-09-18T04:58:13.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.882 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:13.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:13.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:13.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:13.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:13.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:13.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:13.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:13.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:58:13.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:13.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490571+0000", 2024-09-18T04:58:13.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8259999999999998e-06 2024-09-18T04:58:13.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:13.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490584+0000", 2024-09-18T04:58:13.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2391999999999999e-05 2024-09-18T04:58:13.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:13.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490591+0000", 2024-09-18T04:58:13.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5460000000000003e-06 2024-09-18T04:58:13.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:13.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490714+0000", 2024-09-18T04:58:13.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000122543 2024-09-18T04:58:13.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-18T04:58:13.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490731+0000", 2024-09-18T04:58:13.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7405e-05 2024-09-18T04:58:13.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:13.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503069+0000", 2024-09-18T04:58:13.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012337836 2024-09-18T04:58:13.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:13.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503078+0000", 2024-09-18T04:58:13.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8289999999999997e-06 2024-09-18T04:58:13.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:13.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019018+0000", 2024-09-18T04:58:13.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.51594035900000002 2024-09-18T04:58:13.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:13.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019091+0000", 2024-09-18T04:58:13.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2608999999999995e-05 2024-09-18T04:58:13.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-18T04:58:13.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019421+0000", 2024-09-18T04:58:13.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00033004300000000002 2024-09-18T04:58:13.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:13.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.021644+0000", 2024-09-18T04:58:13.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022229749999999999 2024-09-18T04:58:13.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:13.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025113+0000", 2024-09-18T04:58:13.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003469228 2024-09-18T04:58:13.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:13.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025167+0000", 2024-09-18T04:58:13.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3569999999999997e-05 2024-09-18T04:58:13.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:13.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:13.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:13.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025208+0000", 2024-09-18T04:58:13.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0821000000000001e-05 2024-09-18T04:58:13.888 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.888 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:13.888 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.888 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:13.888 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:13.888 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:13.890 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:13.887+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:13.890 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:58:13.991 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:13.988+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:13.998 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:13.998 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:13.998 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:58:13.999 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:14.007 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:58:14.041 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-18T04:58:14.091 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:14.089+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:14.159 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:14.159 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:14.159 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:58:14.159 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:58:14.159 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:14.170 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:58:14.213 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:14.211+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:14.278 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:14.278 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:58:14.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:58:14.278 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:58:14.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.278 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9316 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.279 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:58:14.279 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.507899133999999, 2024-09-18T04:58:14.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044027208999999998, 2024-09-18T04:58:14.279 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.279 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.279 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.279 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.279 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.279 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9316 2024-09-18T04:58:14.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.279 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:58:14.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:58:14.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:58:14.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767144+0000", 2024-09-18T04:58:14.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3070000000000001e-06 2024-09-18T04:58:14.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767150+0000", 2024-09-18T04:58:14.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1700000000000002e-06 2024-09-18T04:58:14.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767163+0000", 2024-09-18T04:58:14.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2670999999999999e-05 2024-09-18T04:58:14.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810995+0000", 2024-09-18T04:58:14.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043832310999999999 2024-09-18T04:58:14.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811087+0000", 2024-09-18T04:58:14.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1345e-05 2024-09-18T04:58:14.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811169+0000", 2024-09-18T04:58:14.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2848000000000004e-05 2024-09-18T04:58:14.283 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.283 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.283 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.283 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9317 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1728512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.283 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:58:14.283 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.507873498999999, 2024-09-18T04:58:14.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044190170000000001, 2024-09-18T04:58:14.283 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.283 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.283 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.283 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.284 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.284 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9317 2024-09-18T04:58:14.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.284 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:58:14.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:58:14.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767167+0000", 2024-09-18T04:58:14.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767170+0000", 2024-09-18T04:58:14.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7259999999999998e-06 2024-09-18T04:58:14.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767176+0000", 2024-09-18T04:58:14.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1460000000000001e-06 2024-09-18T04:58:14.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767182+0000", 2024-09-18T04:58:14.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1020000000000002e-06 2024-09-18T04:58:14.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811204+0000", 2024-09-18T04:58:14.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044022039999999998 2024-09-18T04:58:14.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811286+0000", 2024-09-18T04:58:14.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1623000000000001e-05 2024-09-18T04:58:14.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811358+0000", 2024-09-18T04:58:14.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2125000000000005e-05 2024-09-18T04:58:14.288 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.288 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.288 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.288 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9318 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1777664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.288 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:58:14.288 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.507854953999995, 2024-09-18T04:58:14.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044380181999999997, 2024-09-18T04:58:14.288 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.289 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.289 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.289 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.289 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.289 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9318 2024-09-18T04:58:14.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.289 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:58:14.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:58:14.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:58:14.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767189+0000", 2024-09-18T04:58:14.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5909999999999998e-06 2024-09-18T04:58:14.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767194+0000", 2024-09-18T04:58:14.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7690000000000002e-06 2024-09-18T04:58:14.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767199+0000", 2024-09-18T04:58:14.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189999999999999e-06 2024-09-18T04:58:14.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811394+0000", 2024-09-18T04:58:14.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044195350000000001 2024-09-18T04:58:14.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811483+0000", 2024-09-18T04:58:14.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8282999999999995e-05 2024-09-18T04:58:14.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811567+0000", 2024-09-18T04:58:14.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3855000000000003e-05 2024-09-18T04:58:14.293 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.293 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.293 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.293 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9325 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.293 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:58:14.293 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.506124416000006, 2024-09-18T04:58:14.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044063989999999997, 2024-09-18T04:58:14.293 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.293 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.293 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.294 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.294 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.294 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9325 2024-09-18T04:58:14.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.294 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:58:14.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:58:14.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:58:14.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768919+0000", 2024-09-18T04:58:14.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:58:14.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768925+0000", 2024-09-18T04:58:14.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2310000000000001e-06 2024-09-18T04:58:14.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768929+0000", 2024-09-18T04:58:14.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4469999999999999e-06 2024-09-18T04:58:14.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812823+0000", 2024-09-18T04:58:14.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043893975000000002 2024-09-18T04:58:14.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812908+0000", 2024-09-18T04:58:14.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4862000000000002e-05 2024-09-18T04:58:14.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812981+0000", 2024-09-18T04:58:14.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2892000000000006e-05 2024-09-18T04:58:14.297 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.297 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.298 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.298 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9334 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.298 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:58:14.298 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.504105629999998, 2024-09-18T04:58:14.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044042853999999999, 2024-09-18T04:58:14.298 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.298 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.298 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.298 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.298 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.298 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9334 2024-09-18T04:58:14.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.299 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:58:14.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:58:14.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770935+0000", 2024-09-18T04:58:14.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770937+0000", 2024-09-18T04:58:14.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1489999999999999e-06 2024-09-18T04:58:14.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770940+0000", 2024-09-18T04:58:14.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9569999999999998e-06 2024-09-18T04:58:14.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770945+0000", 2024-09-18T04:58:14.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:58:14.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814838+0000", 2024-09-18T04:58:14.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043892635999999999 2024-09-18T04:58:14.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814920+0000", 2024-09-18T04:58:14.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2175000000000005e-05 2024-09-18T04:58:14.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814979+0000", 2024-09-18T04:58:14.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8372000000000003e-05 2024-09-18T04:58:14.303 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.303 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.303 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.303 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9352 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1159168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.303 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:58:14.303 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.501884453000002, 2024-09-18T04:58:14.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044011269999999998, 2024-09-18T04:58:14.304 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.304 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.304 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.304 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.304 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.304 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9352 2024-09-18T04:58:14.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.304 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:58:14.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:58:14.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773156+0000", 2024-09-18T04:58:14.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773159+0000", 2024-09-18T04:58:14.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2919999999999998e-06 2024-09-18T04:58:14.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773165+0000", 2024-09-18T04:58:14.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0580000000000003e-06 2024-09-18T04:58:14.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773177+0000", 2024-09-18T04:58:14.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.204e-05 2024-09-18T04:58:14.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817082+0000", 2024-09-18T04:58:14.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043905528999999999 2024-09-18T04:58:14.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817122+0000", 2024-09-18T04:58:14.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9258999999999998e-05 2024-09-18T04:58:14.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817168+0000", 2024-09-18T04:58:14.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6545000000000002e-05 2024-09-18T04:58:14.308 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.308 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.308 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.308 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9353 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1208320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.308 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:58:14.308 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.501860155000003, 2024-09-18T04:58:14.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044105537, 2024-09-18T04:58:14.308 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.308 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.308 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.309 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.309 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.309 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9353 2024-09-18T04:58:14.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.309 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:58:14.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:58:14.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:58:14.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773183+0000", 2024-09-18T04:58:14.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2929999999999999e-06 2024-09-18T04:58:14.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773190+0000", 2024-09-18T04:58:14.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4330000000000001e-06 2024-09-18T04:58:14.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773194+0000", 2024-09-18T04:58:14.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.865e-06 2024-09-18T04:58:14.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817196+0000", 2024-09-18T04:58:14.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044001708 2024-09-18T04:58:14.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817236+0000", 2024-09-18T04:58:14.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0043999999999998e-05 2024-09-18T04:58:14.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817287+0000", 2024-09-18T04:58:14.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0605000000000001e-05 2024-09-18T04:58:14.312 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.312 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.313 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.313 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9354 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.313 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:58:14.313 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.501842644000007, 2024-09-18T04:58:14.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044199796, 2024-09-18T04:58:14.313 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.313 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.313 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.313 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.313 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.313 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9354 2024-09-18T04:58:14.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.314 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:58:14.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:58:14.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773198+0000", 2024-09-18T04:58:14.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773200+0000", 2024-09-18T04:58:14.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4159999999999998e-06 2024-09-18T04:58:14.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.315 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773206+0000", 2024-09-18T04:58:14.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.789e-06 2024-09-18T04:58:14.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773210+0000", 2024-09-18T04:58:14.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2679999999999996e-06 2024-09-18T04:58:14.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817314+0000", 2024-09-18T04:58:14.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044103187000000002 2024-09-18T04:58:14.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817353+0000", 2024-09-18T04:58:14.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8973000000000001e-05 2024-09-18T04:58:14.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817399+0000", 2024-09-18T04:58:14.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5877999999999997e-05 2024-09-18T04:58:14.317 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.317 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.317 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.317 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46591 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2097152~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:14.317 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:58:14.317 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.576986227000006, 2024-09-18T04:58:14.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043987310000000002, 2024-09-18T04:58:14.318 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.318 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.318 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.318 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.318 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.318 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46591 2024-09-18T04:58:14.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.318 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:58:14.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:58:14.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:58:14.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698057+0000", 2024-09-18T04:58:14.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.531e-06 2024-09-18T04:58:14.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698064+0000", 2024-09-18T04:58:14.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-18T04:58:14.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698069+0000", 2024-09-18T04:58:14.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.091e-06 2024-09-18T04:58:14.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741843+0000", 2024-09-18T04:58:14.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043773470000000002 2024-09-18T04:58:14.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741928+0000", 2024-09-18T04:58:14.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5113999999999999e-05 2024-09-18T04:58:14.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742042+0000", 2024-09-18T04:58:14.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114527 2024-09-18T04:58:14.322 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.322 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.322 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.322 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46592 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2121728~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:14.322 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:58:14.322 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.576967800000006, 2024-09-18T04:58:14.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044077948999999998, 2024-09-18T04:58:14.322 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.322 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.322 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.322 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.322 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.323 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46592 2024-09-18T04:58:14.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.323 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:58:14.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:58:14.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698073+0000", 2024-09-18T04:58:14.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698076+0000", 2024-09-18T04:58:14.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.481e-06 2024-09-18T04:58:14.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698082+0000", 2024-09-18T04:58:14.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8730000000000004e-06 2024-09-18T04:58:14.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698086+0000", 2024-09-18T04:58:14.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:58:14.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742055+0000", 2024-09-18T04:58:14.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043969170000000002 2024-09-18T04:58:14.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742093+0000", 2024-09-18T04:58:14.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7707000000000003e-05 2024-09-18T04:58:14.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742152+0000", 2024-09-18T04:58:14.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8323999999999999e-05 2024-09-18T04:58:14.326 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.326 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.326 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.326 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46593 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2211840~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:14.326 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:58:14.326 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.576950897000003, 2024-09-18T04:58:14.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044259917000000003, 2024-09-18T04:58:14.327 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.327 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.327 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.327 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.327 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.327 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46593 2024-09-18T04:58:14.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.327 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:58:14.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:58:14.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:58:14.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698092+0000", 2024-09-18T04:58:14.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2249999999999999e-06 2024-09-18T04:58:14.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698098+0000", 2024-09-18T04:58:14.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:58:14.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698103+0000", 2024-09-18T04:58:14.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:58:14.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742182+0000", 2024-09-18T04:58:14.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044079508000000003 2024-09-18T04:58:14.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742257+0000", 2024-09-18T04:58:14.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4678000000000006e-05 2024-09-18T04:58:14.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742350+0000", 2024-09-18T04:58:14.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3531999999999998e-05 2024-09-18T04:58:14.330 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.330 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.331 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.331 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46594 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2236416~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:14.331 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:58:14.331 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.576935199999994, 2024-09-18T04:58:14.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044378984000000003, 2024-09-18T04:58:14.331 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.331 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.331 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.331 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.331 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.331 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46594 2024-09-18T04:58:14.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.332 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:58:14.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:58:14.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:58:14.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698108+0000", 2024-09-18T04:58:14.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0880000000000002e-06 2024-09-18T04:58:14.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698114+0000", 2024-09-18T04:58:14.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6269999999999997e-06 2024-09-18T04:58:14.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698118+0000", 2024-09-18T04:58:14.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4050000000000002e-06 2024-09-18T04:58:14.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742365+0000", 2024-09-18T04:58:14.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044246799000000003 2024-09-18T04:58:14.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742407+0000", 2024-09-18T04:58:14.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2008000000000002e-05 2024-09-18T04:58:14.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742485+0000", 2024-09-18T04:58:14.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8405999999999994e-05 2024-09-18T04:58:14.335 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.335 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.335 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.336 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46595 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:14.336 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:58:14.336 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.576919126999996, 2024-09-18T04:58:14.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.04440463, 2024-09-18T04:58:14.336 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.336 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:14.336 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.336 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.336 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.336 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46595 2024-09-18T04:58:14.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.336 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:58:14.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:58:14.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:58:14.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698124+0000", 2024-09-18T04:58:14.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1840000000000002e-06 2024-09-18T04:58:14.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698130+0000", 2024-09-18T04:58:14.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.947e-06 2024-09-18T04:58:14.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698136+0000", 2024-09-18T04:58:14.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6520000000000003e-06 2024-09-18T04:58:14.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742521+0000", 2024-09-18T04:58:14.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044385446000000002 2024-09-18T04:58:14.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742527+0000", 2024-09-18T04:58:14.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:58:14.340 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.340 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.340 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.341 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46596 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:14.341 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:58:14.341 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.576900882999993, 2024-09-18T04:58:14.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044418066999999999, 2024-09-18T04:58:14.341 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.341 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:14.341 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.341 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.341 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.341 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46596 2024-09-18T04:58:14.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.341 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:58:14.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.342 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.342 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:58:14.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:58:14.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698146+0000", 2024-09-18T04:58:14.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2779999999999999e-06 2024-09-18T04:58:14.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698153+0000", 2024-09-18T04:58:14.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3849999999999999e-06 2024-09-18T04:58:14.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698164+0000", 2024-09-18T04:58:14.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1375999999999999e-05 2024-09-18T04:58:14.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742554+0000", 2024-09-18T04:58:14.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044389482000000001 2024-09-18T04:58:14.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742559+0000", 2024-09-18T04:58:14.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9540000000000003e-06 2024-09-18T04:58:14.345 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.345 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.345 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.345 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46597 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:14.346 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:58:14.346 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.576872518000002, 2024-09-18T04:58:14.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044416513999999997, 2024-09-18T04:58:14.346 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.346 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:14.346 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.346 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.346 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.346 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46597 2024-09-18T04:58:14.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.346 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:58:14.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:58:14.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698168+0000", 2024-09-18T04:58:14.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698175+0000", 2024-09-18T04:58:14.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4520000000000002e-06 2024-09-18T04:58:14.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698181+0000", 2024-09-18T04:58:14.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4150000000000004e-06 2024-09-18T04:58:14.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698186+0000", 2024-09-18T04:58:14.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7489999999999998e-06 2024-09-18T04:58:14.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742580+0000", 2024-09-18T04:58:14.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044394325999999998 2024-09-18T04:58:14.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742585+0000", 2024-09-18T04:58:14.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9849999999999997e-06 2024-09-18T04:58:14.350 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.350 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.350 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.350 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46598 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:14.351 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:58:14.351 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 84.576765042000005, 2024-09-18T04:58:14.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044335316999999999, 2024-09-18T04:58:14.351 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.351 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:14.351 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.351 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.351 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.352 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46598 2024-09-18T04:58:14.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.352 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:58:14.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:58:14.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698190+0000", 2024-09-18T04:58:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999137 2024-09-18T04:58:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698280+0000", 2024-09-18T04:58:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0409999999999997e-05 2024-09-18T04:58:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698289+0000", 2024-09-18T04:58:14.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9490000000000002e-06 2024-09-18T04:58:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698296+0000", 2024-09-18T04:58:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2670000000000003e-06 2024-09-18T04:58:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742607+0000", 2024-09-18T04:58:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044311072999999999 2024-09-18T04:58:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742612+0000", 2024-09-18T04:58:14.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9400000000000001e-06 2024-09-18T04:58:14.355 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.355 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.355 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.355 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57986 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3086336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:14.355 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:58:14.355 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 83.589919319000003, 2024-09-18T04:58:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044047731999999999, 2024-09-18T04:58:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57986 2024-09-18T04:58:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:58:14.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:58:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:58:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685124+0000", 2024-09-18T04:58:14.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2859999999999998e-06 2024-09-18T04:58:14.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685130+0000", 2024-09-18T04:58:14.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5400000000000001e-06 2024-09-18T04:58:14.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685136+0000", 2024-09-18T04:58:14.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.152e-06 2024-09-18T04:58:14.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.728976+0000", 2024-09-18T04:58:14.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043840037999999998 2024-09-18T04:58:14.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729064+0000", 2024-09-18T04:58:14.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8709999999999996e-05 2024-09-18T04:58:14.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729170+0000", 2024-09-18T04:58:14.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000105386 2024-09-18T04:58:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57987 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3101696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:58:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 83.589870344000005, 2024-09-18T04:58:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044238123999999997, 2024-09-18T04:58:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.362 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57987 2024-09-18T04:58:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:58:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:58:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685170+0000", 2024-09-18T04:58:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:58:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685175+0000", 2024-09-18T04:58:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9069999999999998e-06 2024-09-18T04:58:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685187+0000", 2024-09-18T04:58:14.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.189e-05 2024-09-18T04:58:14.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685195+0000", 2024-09-18T04:58:14.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9459999999999998e-06 2024-09-18T04:58:14.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729212+0000", 2024-09-18T04:58:14.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044017239 2024-09-18T04:58:14.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729301+0000", 2024-09-18T04:58:14.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8887999999999993e-05 2024-09-18T04:58:14.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729409+0000", 2024-09-18T04:58:14.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108119 2024-09-18T04:58:14.367 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.367 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.367 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.367 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57988 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3117056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:14.367 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:58:14.367 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 83.589828225000005, 2024-09-18T04:58:14.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044429835000000001, 2024-09-18T04:58:14.367 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.367 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.367 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.368 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.368 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.368 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57988 2024-09-18T04:58:14.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.368 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:58:14.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:58:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685212+0000", 2024-09-18T04:58:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685216+0000", 2024-09-18T04:58:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7390000000000001e-06 2024-09-18T04:58:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685224+0000", 2024-09-18T04:58:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2260000000000005e-06 2024-09-18T04:58:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685230+0000", 2024-09-18T04:58:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0299999999999999e-06 2024-09-18T04:58:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729454+0000", 2024-09-18T04:58:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044223248999999999 2024-09-18T04:58:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729539+0000", 2024-09-18T04:58:14.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5029000000000002e-05 2024-09-18T04:58:14.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729643+0000", 2024-09-18T04:58:14.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104267 2024-09-18T04:58:14.371 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.372 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.372 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.372 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57989 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3132416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:14.372 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:58:14.372 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 83.589805472999998, 2024-09-18T04:58:14.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044642881000000002, 2024-09-18T04:58:14.372 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.372 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.372 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.373 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.373 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.373 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57989 2024-09-18T04:58:14.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.373 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:58:14.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:58:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685235+0000", 2024-09-18T04:58:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685238+0000", 2024-09-18T04:58:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.667e-06 2024-09-18T04:58:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685245+0000", 2024-09-18T04:58:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7909999999999999e-06 2024-09-18T04:58:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685250+0000", 2024-09-18T04:58:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3730000000000001e-06 2024-09-18T04:58:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729685+0000", 2024-09-18T04:58:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044434552000000002 2024-09-18T04:58:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729775+0000", 2024-09-18T04:58:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0395000000000004e-05 2024-09-18T04:58:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729879+0000", 2024-09-18T04:58:14.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010361099999999999 2024-09-18T04:58:14.377 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.377 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.377 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.377 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.377 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.377 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:14.378 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:58:14.379 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:14.376+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:14.479 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:14.477+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:14.525 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:14.525 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:14.525 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:58:14.525 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:14.534 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:58:14.580 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:14.578+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:14.641 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:14.642 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:14.642 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:58:14.642 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:58:14.642 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:14.650 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:58:14.691 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:14.689+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:14.782 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:14.780+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:14.798 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:14.799 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:58:14.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:58:14.799 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:58:14.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.799 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14125 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.799 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:58:14.799 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.668525174999999, 2024-09-18T04:58:14.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052956470999999998, 2024-09-18T04:58:14.799 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.799 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.799 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.799 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.800 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.800 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14125 2024-09-18T04:58:14.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.800 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:58:14.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:58:14.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126849+0000", 2024-09-18T04:58:14.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126852+0000", 2024-09-18T04:58:14.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3659999999999999e-06 2024-09-18T04:58:14.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126858+0000", 2024-09-18T04:58:14.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4540000000000004e-06 2024-09-18T04:58:14.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126863+0000", 2024-09-18T04:58:14.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1780000000000002e-06 2024-09-18T04:58:14.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179608+0000", 2024-09-18T04:58:14.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052745159999999999 2024-09-18T04:58:14.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179703+0000", 2024-09-18T04:58:14.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4773999999999998e-05 2024-09-18T04:58:14.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.805 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.805 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179806+0000", 2024-09-18T04:58:14.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010296799999999999 2024-09-18T04:58:14.805 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.805 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.805 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.805 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14136 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1732608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.805 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:58:14.806 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.666364623999996, 2024-09-18T04:58:14.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052973412999999997, 2024-09-18T04:58:14.806 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.806 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.806 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.806 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.807 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.808 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14136 2024-09-18T04:58:14.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.808 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:58:14.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:58:14.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:58:14.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129013+0000", 2024-09-18T04:58:14.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.96e-06 2024-09-18T04:58:14.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129020+0000", 2024-09-18T04:58:14.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6809999999999998e-06 2024-09-18T04:58:14.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129026+0000", 2024-09-18T04:58:14.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:58:14.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181873+0000", 2024-09-18T04:58:14.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052847238999999997 2024-09-18T04:58:14.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181932+0000", 2024-09-18T04:58:14.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9339000000000002e-05 2024-09-18T04:58:14.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181984+0000", 2024-09-18T04:58:14.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1348999999999999e-05 2024-09-18T04:58:14.811 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.812 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.812 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.812 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14137 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1781760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.812 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:58:14.812 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.666344218999996, 2024-09-18T04:58:14.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053128914999999999, 2024-09-18T04:58:14.812 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.812 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.812 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.812 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.812 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.813 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14137 2024-09-18T04:58:14.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.813 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:58:14.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:58:14.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129030+0000", 2024-09-18T04:58:14.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129033+0000", 2024-09-18T04:58:14.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7439999999999999e-06 2024-09-18T04:58:14.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129039+0000", 2024-09-18T04:58:14.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.314e-06 2024-09-18T04:58:14.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129044+0000", 2024-09-18T04:58:14.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9919999999999998e-06 2024-09-18T04:58:14.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182015+0000", 2024-09-18T04:58:14.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052970875000000001 2024-09-18T04:58:14.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182058+0000", 2024-09-18T04:58:14.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2997000000000002e-05 2024-09-18T04:58:14.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182160+0000", 2024-09-18T04:58:14.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101437 2024-09-18T04:58:14.816 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.816 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.816 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.816 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14138 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1830912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.816 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:58:14.817 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.666326185000003, 2024-09-18T04:58:14.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053256435999999997, 2024-09-18T04:58:14.817 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.817 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.817 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.817 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.817 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.817 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14138 2024-09-18T04:58:14.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.817 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:58:14.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:58:14.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129048+0000", 2024-09-18T04:58:14.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129056+0000", 2024-09-18T04:58:14.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.413e-06 2024-09-18T04:58:14.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129062+0000", 2024-09-18T04:58:14.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6810000000000001e-06 2024-09-18T04:58:14.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129068+0000", 2024-09-18T04:58:14.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0459999999999996e-06 2024-09-18T04:58:14.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182191+0000", 2024-09-18T04:58:14.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053123210999999997 2024-09-18T04:58:14.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182238+0000", 2024-09-18T04:58:14.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7404999999999998e-05 2024-09-18T04:58:14.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182305+0000", 2024-09-18T04:58:14.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7028999999999998e-05 2024-09-18T04:58:14.821 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.821 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.821 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.821 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15910 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2236416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.821 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:58:14.821 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.567238653000004, 2024-09-18T04:58:14.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053241054000000003, 2024-09-18T04:58:14.822 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.822 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.822 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.822 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.822 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.822 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15910 2024-09-18T04:58:14.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.822 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:58:14.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:58:14.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:58:14.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228138+0000", 2024-09-18T04:58:14.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:58:14.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228144+0000", 2024-09-18T04:58:14.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5369999999999997e-06 2024-09-18T04:58:14.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228149+0000", 2024-09-18T04:58:14.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1209999999999998e-06 2024-09-18T04:58:14.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280822+0000", 2024-09-18T04:58:14.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052673194999999999 2024-09-18T04:58:14.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280903+0000", 2024-09-18T04:58:14.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1179000000000003e-05 2024-09-18T04:58:14.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281377+0000", 2024-09-18T04:58:14.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047417899999999997 2024-09-18T04:58:14.826 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.826 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.826 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.826 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15911 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2285568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.826 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:58:14.826 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.567221920999998, 2024-09-18T04:58:14.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053415151000000001, 2024-09-18T04:58:14.827 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.827 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.827 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.827 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.827 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.827 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15911 2024-09-18T04:58:14.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.827 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:58:14.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:58:14.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:58:14.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228155+0000", 2024-09-18T04:58:14.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:58:14.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228161+0000", 2024-09-18T04:58:14.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:58:14.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228166+0000", 2024-09-18T04:58:14.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2320000000000001e-06 2024-09-18T04:58:14.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281405+0000", 2024-09-18T04:58:14.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053239336999999998 2024-09-18T04:58:14.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281482+0000", 2024-09-18T04:58:14.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6569999999999994e-05 2024-09-18T04:58:14.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281568+0000", 2024-09-18T04:58:14.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6328000000000004e-05 2024-09-18T04:58:14.831 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.831 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.831 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.831 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15918 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2662400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.831 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:58:14.831 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.56545294, 2024-09-18T04:58:14.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053091065, 2024-09-18T04:58:14.831 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.832 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.832 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.832 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.832 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.832 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15918 2024-09-18T04:58:14.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.832 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:58:14.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:58:14.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:58:14.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229924+0000", 2024-09-18T04:58:14.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:58:14.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229930+0000", 2024-09-18T04:58:14.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.066e-06 2024-09-18T04:58:14.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229935+0000", 2024-09-18T04:58:14.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9119999999999997e-06 2024-09-18T04:58:14.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282840+0000", 2024-09-18T04:58:14.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052905119 2024-09-18T04:58:14.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282922+0000", 2024-09-18T04:58:14.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2081000000000002e-05 2024-09-18T04:58:14.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283013+0000", 2024-09-18T04:58:14.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0828000000000005e-05 2024-09-18T04:58:14.836 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.836 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.836 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.836 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15919 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2711552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.836 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:58:14.836 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.565435661999999, 2024-09-18T04:58:14.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053293021000000003, 2024-09-18T04:58:14.836 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.836 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.837 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.837 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.837 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.837 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15919 2024-09-18T04:58:14.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.837 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:58:14.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:58:14.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:58:14.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229941+0000", 2024-09-18T04:58:14.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2639999999999998e-06 2024-09-18T04:58:14.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229947+0000", 2024-09-18T04:58:14.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9750000000000004e-06 2024-09-18T04:58:14.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229952+0000", 2024-09-18T04:58:14.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7929999999999997e-06 2024-09-18T04:58:14.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283062+0000", 2024-09-18T04:58:14.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05310952 2024-09-18T04:58:14.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283162+0000", 2024-09-18T04:58:14.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100098 2024-09-18T04:58:14.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283232+0000", 2024-09-18T04:58:14.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0693999999999998e-05 2024-09-18T04:58:14.840 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.840 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.840 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.841 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15920 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2760704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.841 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:58:14.841 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.565418901000001, 2024-09-18T04:58:14.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053502367000000002, 2024-09-18T04:58:14.841 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.841 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.841 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.842 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.842 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.842 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15920 2024-09-18T04:58:14.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.842 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:58:14.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:58:14.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:58:14.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229958+0000", 2024-09-18T04:58:14.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1770000000000001e-06 2024-09-18T04:58:14.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229964+0000", 2024-09-18T04:58:14.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8470000000000001e-06 2024-09-18T04:58:14.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229968+0000", 2024-09-18T04:58:14.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5680000000000001e-06 2024-09-18T04:58:14.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283269+0000", 2024-09-18T04:58:14.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053300620999999999 2024-09-18T04:58:14.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283359+0000", 2024-09-18T04:58:14.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9971999999999998e-05 2024-09-18T04:58:14.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283458+0000", 2024-09-18T04:58:14.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9483999999999998e-05 2024-09-18T04:58:14.845 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.845 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.845 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.846 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15921 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2809856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.846 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:58:14.846 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.565402351000003, 2024-09-18T04:58:14.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053800977, 2024-09-18T04:58:14.846 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.846 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.846 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.846 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.846 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.846 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15921 2024-09-18T04:58:14.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.846 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:58:14.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:58:14.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229972+0000", 2024-09-18T04:58:14.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229974+0000", 2024-09-18T04:58:14.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2910000000000002e-06 2024-09-18T04:58:14.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229982+0000", 2024-09-18T04:58:14.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7249999999999997e-06 2024-09-18T04:58:14.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229989+0000", 2024-09-18T04:58:14.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6429999999999998e-06 2024-09-18T04:58:14.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283601+0000", 2024-09-18T04:58:14.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053612211 2024-09-18T04:58:14.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283681+0000", 2024-09-18T04:58:14.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0125999999999998e-05 2024-09-18T04:58:14.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283774+0000", 2024-09-18T04:58:14.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2362999999999996e-05 2024-09-18T04:58:14.850 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.850 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.850 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.851 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15922 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2859008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.851 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:58:14.851 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.565357411999997, 2024-09-18T04:58:14.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053882270000000003, 2024-09-18T04:58:14.851 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.851 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.851 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.851 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.851 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.851 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15922 2024-09-18T04:58:14.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.851 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:58:14.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:58:14.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230017+0000", 2024-09-18T04:58:14.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230019+0000", 2024-09-18T04:58:14.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4210000000000002e-06 2024-09-18T04:58:14.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230027+0000", 2024-09-18T04:58:14.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000002e-06 2024-09-18T04:58:14.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230032+0000", 2024-09-18T04:58:14.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0089999999999998e-06 2024-09-18T04:58:14.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283800+0000", 2024-09-18T04:58:14.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053768418999999998 2024-09-18T04:58:14.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283849+0000", 2024-09-18T04:58:14.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9339000000000003e-05 2024-09-18T04:58:14.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283900+0000", 2024-09-18T04:58:14.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0421999999999999e-05 2024-09-18T04:58:14.855 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.855 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.855 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.855 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15923 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2924544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.855 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:58:14.855 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.565338542999996, 2024-09-18T04:58:14.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054073016000000002, 2024-09-18T04:58:14.855 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.856 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:14.856 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.856 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.856 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.856 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15923 2024-09-18T04:58:14.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.856 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:58:14.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:58:14.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:58:14.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230038+0000", 2024-09-18T04:58:14.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.463e-06 2024-09-18T04:58:14.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230045+0000", 2024-09-18T04:58:14.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3570000000000003e-06 2024-09-18T04:58:14.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230050+0000", 2024-09-18T04:58:14.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7299999999999996e-06 2024-09-18T04:58:14.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283928+0000", 2024-09-18T04:58:14.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053878218999999998 2024-09-18T04:58:14.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:14.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284010+0000", 2024-09-18T04:58:14.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2577000000000006e-05 2024-09-18T04:58:14.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284109+0000", 2024-09-18T04:58:14.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9102000000000001e-05 2024-09-18T04:58:14.859 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.860 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.860 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.860 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15924 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.860 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:58:14.860 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.565320072999995, 2024-09-18T04:58:14.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054108470999999998, 2024-09-18T04:58:14.860 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.860 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:14.860 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.860 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.860 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.861 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15924 2024-09-18T04:58:14.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.861 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:58:14.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:58:14.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:58:14.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230061+0000", 2024-09-18T04:58:14.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8240000000000002e-06 2024-09-18T04:58:14.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230068+0000", 2024-09-18T04:58:14.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4320000000000004e-06 2024-09-18T04:58:14.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230073+0000", 2024-09-18T04:58:14.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.921e-06 2024-09-18T04:58:14.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284153+0000", 2024-09-18T04:58:14.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054080793000000002 2024-09-18T04:58:14.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284163+0000", 2024-09-18T04:58:14.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.859e-06 2024-09-18T04:58:14.864 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.864 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.864 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.864 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15925 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.864 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:58:14.864 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.565295957999993, 2024-09-18T04:58:14.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054134632000000002, 2024-09-18T04:58:14.864 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.865 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:14.865 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.865 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.865 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.865 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15925 2024-09-18T04:58:14.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.865 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:58:14.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:58:14.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:58:14.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230081+0000", 2024-09-18T04:58:14.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4109999999999999e-06 2024-09-18T04:58:14.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230087+0000", 2024-09-18T04:58:14.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.161e-06 2024-09-18T04:58:14.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230092+0000", 2024-09-18T04:58:14.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9259999999999999e-06 2024-09-18T04:58:14.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284204+0000", 2024-09-18T04:58:14.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054111546000000003 2024-09-18T04:58:14.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284214+0000", 2024-09-18T04:58:14.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9639999999999998e-06 2024-09-18T04:58:14.870 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.870 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.870 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.871 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15926 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.871 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:58:14.871 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.565278633999995, 2024-09-18T04:58:14.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054143193999999999, 2024-09-18T04:58:14.871 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.871 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:14.871 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.871 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.871 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.871 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15926 2024-09-18T04:58:14.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.872 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:58:14.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:58:14.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:58:14.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230098+0000", 2024-09-18T04:58:14.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1399999999999998e-06 2024-09-18T04:58:14.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230104+0000", 2024-09-18T04:58:14.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-18T04:58:14.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230109+0000", 2024-09-18T04:58:14.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9250000000000003e-06 2024-09-18T04:58:14.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284231+0000", 2024-09-18T04:58:14.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054122297999999999 2024-09-18T04:58:14.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284239+0000", 2024-09-18T04:58:14.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1689999999999992e-06 2024-09-18T04:58:14.876 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.876 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.876 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.877 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15927 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.877 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:58:14.877 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.565261586999995, 2024-09-18T04:58:14.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054168435000000001, 2024-09-18T04:58:14.877 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.877 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:14.877 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.877 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.878 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.878 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15927 2024-09-18T04:58:14.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.878 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:58:14.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:58:14.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:58:14.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230116+0000", 2024-09-18T04:58:14.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5150000000000001e-06 2024-09-18T04:58:14.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230122+0000", 2024-09-18T04:58:14.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1190000000000002e-06 2024-09-18T04:58:14.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230126+0000", 2024-09-18T04:58:14.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.792e-06 2024-09-18T04:58:14.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284273+0000", 2024-09-18T04:58:14.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054146086000000003 2024-09-18T04:58:14.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284282+0000", 2024-09-18T04:58:14.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2590000000000006e-06 2024-09-18T04:58:14.881 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.881 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.882 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.882 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15928 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.882 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:58:14.882 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.565244203999995, 2024-09-18T04:58:14.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054189739000000001, 2024-09-18T04:58:14.882 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.882 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:14.882 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.882 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.882 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.882 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15928 2024-09-18T04:58:14.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.883 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:58:14.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:58:14.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230130+0000", 2024-09-18T04:58:14.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230133+0000", 2024-09-18T04:58:14.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.244e-06 2024-09-18T04:58:14.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230139+0000", 2024-09-18T04:58:14.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9830000000000001e-06 2024-09-18T04:58:14.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230143+0000", 2024-09-18T04:58:14.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7530000000000001e-06 2024-09-18T04:58:14.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284313+0000", 2024-09-18T04:58:14.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054169332000000001 2024-09-18T04:58:14.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284320+0000", 2024-09-18T04:58:14.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8699999999999992e-06 2024-09-18T04:58:14.886 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.886 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.886 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.886 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15929 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.886 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:58:14.887 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.565227763999999, 2024-09-18T04:58:14.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054218771999999998, 2024-09-18T04:58:14.887 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.887 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:14.887 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.887 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.887 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.887 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15929 2024-09-18T04:58:14.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.887 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:58:14.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:58:14.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:58:14.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230149+0000", 2024-09-18T04:58:14.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-18T04:58:14.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230155+0000", 2024-09-18T04:58:14.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2469999999999997e-06 2024-09-18T04:58:14.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230160+0000", 2024-09-18T04:58:14.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7129999999999996e-06 2024-09-18T04:58:14.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284355+0000", 2024-09-18T04:58:14.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054194900999999997 2024-09-18T04:58:14.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284366+0000", 2024-09-18T04:58:14.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0993e-05 2024-09-18T04:58:14.890 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.890 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.890 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.891 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15930 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.891 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:58:14.891 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.565210764, 2024-09-18T04:58:14.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054244713999999999, 2024-09-18T04:58:14.891 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.891 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:14.891 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.891 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.891 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.891 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15930 2024-09-18T04:58:14.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.891 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:58:14.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:58:14.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:58:14.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230166+0000", 2024-09-18T04:58:14.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1009999999999999e-06 2024-09-18T04:58:14.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230172+0000", 2024-09-18T04:58:14.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.981e-06 2024-09-18T04:58:14.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230184+0000", 2024-09-18T04:58:14.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2449000000000001e-05 2024-09-18T04:58:14.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284400+0000", 2024-09-18T04:58:14.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054215452999999997 2024-09-18T04:58:14.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284409+0000", 2024-09-18T04:58:14.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0839999999999998e-06 2024-09-18T04:58:14.894 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.895 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.895 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.895 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15931 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:14.895 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:58:14.895 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.565184974999994, 2024-09-18T04:58:14.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054261928000000001, 2024-09-18T04:58:14.895 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:14.895 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:14.895 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:14.895 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:14.896 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:14.896 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15931 2024-09-18T04:58:14.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.896 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:14.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:14.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:58:14.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:14.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:58:14.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230189+0000", 2024-09-18T04:58:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230192+0000", 2024-09-18T04:58:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6599999999999999e-06 2024-09-18T04:58:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:14.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230199+0000", 2024-09-18T04:58:14.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5570000000000001e-06 2024-09-18T04:58:14.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:14.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230204+0000", 2024-09-18T04:58:14.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2270000000000001e-06 2024-09-18T04:58:14.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:14.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284442+0000", 2024-09-18T04:58:14.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054238383000000001 2024-09-18T04:58:14.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:14.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:14.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:14.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284452+0000", 2024-09-18T04:58:14.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6069999999999997e-06 2024-09-18T04:58:14.899 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.899 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.899 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.899 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:14.899 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:14.899 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:14.901 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:58:14.901 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:14.898+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:15.001 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:14.999+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:15.055 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:15.055 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:58:15.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.056 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:166681 3.d 3:b07a8a0b:::rbd_header.11accf37a818:head [call rbd.snapshot_remove in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e125)", 2024-09-18T04:58:15.056 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:58:15.042495+0000", 2024-09-18T04:58:15.056 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 0.011192268, 2024-09-18T04:58:15.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.011209635000000001, 2024-09-18T04:58:15.056 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.056 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "waiting for sub ops", 2024-09-18T04:58:15.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.056 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 166681 2024-09-18T04:58:15.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.056 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:15.042495+0000", 2024-09-18T04:58:15.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:15.042495+0000", 2024-09-18T04:58:15.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:15.042493+0000", 2024-09-18T04:58:15.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999976 2024-09-18T04:58:15.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:15.042505+0000", 2024-09-18T04:58:15.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1664e-05 2024-09-18T04:58:15.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:15.042529+0000", 2024-09-18T04:58:15.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4161000000000001e-05 2024-09-18T04:58:15.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:15.042539+0000", 2024-09-18T04:58:15.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8870000000000004e-06 2024-09-18T04:58:15.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:15.042608+0000", 2024-09-18T04:58:15.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.9723999999999999e-05 2024-09-18T04:58:15.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op must wait for map", 2024-09-18T04:58:15.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:15.042616+0000", 2024-09-18T04:58:15.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.3119999999999996e-06 2024-09-18T04:58:15.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:15.046742+0000", 2024-09-18T04:58:15.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0041258220000000003 2024-09-18T04:58:15.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.060 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.060 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:15.046856+0000", 2024-09-18T04:58:15.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0001147 2024-09-18T04:58:15.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 5", 2024-09-18T04:58:15.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:15.047237+0000", 2024-09-18T04:58:15.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00038103900000000001 2024-09-18T04:58:15.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.061 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:58:15.061 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:15.049530+0000", 2024-09-18T04:58:15.061 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0022931380000000001 2024-09-18T04:58:15.061 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.061 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.061 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.061 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.061 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:58:15.062 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 1 2024-09-18T04:58:15.062 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:15.067 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:58:15.115 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:15.113+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:15.183 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:15.183 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:15.184 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:58:15.184 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:58:15.184 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:15.192 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:58:15.233 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:15.231+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:15.305 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:15.303+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:15.341 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:15.341 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:58:15.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:58:15.341 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:58:15.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.342 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160713 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 645120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:15.342 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654655+0000", 2024-09-18T04:58:15.342 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.682876254999996, 2024-09-18T04:58:15.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039354912999999998, 2024-09-18T04:58:15.342 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.342 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.342 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.342 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.342 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.342 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160713 2024-09-18T04:58:15.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.343 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654655+0000", 2024-09-18T04:58:15.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654655+0000", 2024-09-18T04:58:15.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654654+0000", 2024-09-18T04:58:15.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654657+0000", 2024-09-18T04:58:15.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7049999999999999e-06 2024-09-18T04:58:15.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654669+0000", 2024-09-18T04:58:15.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1960000000000001e-05 2024-09-18T04:58:15.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654674+0000", 2024-09-18T04:58:15.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8729999999999998e-06 2024-09-18T04:58:15.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.693848+0000", 2024-09-18T04:58:15.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039174202999999998 2024-09-18T04:58:15.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.693926+0000", 2024-09-18T04:58:15.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8248e-05 2024-09-18T04:58:15.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694009+0000", 2024-09-18T04:58:15.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3363999999999997e-05 2024-09-18T04:58:15.347 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.348 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.348 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.348 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160714 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 665600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:15.348 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654678+0000", 2024-09-18T04:58:15.348 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.682852879000002, 2024-09-18T04:58:15.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039530155999999997, 2024-09-18T04:58:15.348 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.348 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.349 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.349 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.349 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.349 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160714 2024-09-18T04:58:15.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.349 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654678+0000", 2024-09-18T04:58:15.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654678+0000", 2024-09-18T04:58:15.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654678+0000", 2024-09-18T04:58:15.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654680+0000", 2024-09-18T04:58:15.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5229999999999998e-06 2024-09-18T04:58:15.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654686+0000", 2024-09-18T04:58:15.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9000000000000003e-06 2024-09-18T04:58:15.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654690+0000", 2024-09-18T04:58:15.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1060000000000002e-06 2024-09-18T04:58:15.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694042+0000", 2024-09-18T04:58:15.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039351490000000003 2024-09-18T04:58:15.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694142+0000", 2024-09-18T04:58:15.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100177 2024-09-18T04:58:15.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694208+0000", 2024-09-18T04:58:15.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6367999999999999e-05 2024-09-18T04:58:15.354 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.354 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.355 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.355 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160715 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:15.355 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:58:15.355 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.68283658, 2024-09-18T04:58:15.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039724306000000001, 2024-09-18T04:58:15.355 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.355 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.355 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.355 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.355 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.356 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160715 2024-09-18T04:58:15.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.356 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:58:15.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:58:15.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:58:15.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654696+0000", 2024-09-18T04:58:15.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1830000000000001e-06 2024-09-18T04:58:15.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654702+0000", 2024-09-18T04:58:15.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5110000000000003e-06 2024-09-18T04:58:15.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654706+0000", 2024-09-18T04:58:15.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2470000000000002e-06 2024-09-18T04:58:15.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694245+0000", 2024-09-18T04:58:15.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039539259 2024-09-18T04:58:15.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694327+0000", 2024-09-18T04:58:15.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1817000000000006e-05 2024-09-18T04:58:15.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694419+0000", 2024-09-18T04:58:15.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1639999999999997e-05 2024-09-18T04:58:15.362 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.362 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.362 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.362 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160716 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:15.362 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:58:15.363 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.682821394999998, 2024-09-18T04:58:15.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039842555000000002, 2024-09-18T04:58:15.363 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.363 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.363 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.363 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.363 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.363 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160716 2024-09-18T04:58:15.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.363 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:58:15.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:58:15.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:58:15.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654711+0000", 2024-09-18T04:58:15.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.99e-06 2024-09-18T04:58:15.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654717+0000", 2024-09-18T04:58:15.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6799999999999998e-06 2024-09-18T04:58:15.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654721+0000", 2024-09-18T04:58:15.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-18T04:58:15.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694449+0000", 2024-09-18T04:58:15.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039727476999999997 2024-09-18T04:58:15.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694497+0000", 2024-09-18T04:58:15.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8294000000000002e-05 2024-09-18T04:58:15.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694552+0000", 2024-09-18T04:58:15.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5024e-05 2024-09-18T04:58:15.367 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.367 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.367 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.367 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160717 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 727040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:15.367 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:58:15.367 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.682780809, 2024-09-18T04:58:15.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040030957999999998, 2024-09-18T04:58:15.367 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.367 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.367 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.367 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.367 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.368 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160717 2024-09-18T04:58:15.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.368 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:58:15.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:58:15.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:58:15.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654752+0000", 2024-09-18T04:58:15.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8219999999999998e-06 2024-09-18T04:58:15.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.369 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.369 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654769+0000", 2024-09-18T04:58:15.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6249e-05 2024-09-18T04:58:15.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654775+0000", 2024-09-18T04:58:15.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2090000000000002e-06 2024-09-18T04:58:15.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694589+0000", 2024-09-18T04:58:15.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039814651999999999 2024-09-18T04:58:15.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694674+0000", 2024-09-18T04:58:15.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5032999999999997e-05 2024-09-18T04:58:15.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694781+0000", 2024-09-18T04:58:15.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106452 2024-09-18T04:58:15.371 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.371 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.371 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.371 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160718 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 747520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:15.372 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:58:15.372 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.682750917999996, 2024-09-18T04:58:15.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040152520999999997, 2024-09-18T04:58:15.372 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.372 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.372 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.372 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.372 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.372 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160718 2024-09-18T04:58:15.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.372 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:58:15.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:58:15.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654779+0000", 2024-09-18T04:58:15.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654782+0000", 2024-09-18T04:58:15.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.633e-06 2024-09-18T04:58:15.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654789+0000", 2024-09-18T04:58:15.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8589999999999999e-06 2024-09-18T04:58:15.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654793+0000", 2024-09-18T04:58:15.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5449999999999997e-06 2024-09-18T04:58:15.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694817+0000", 2024-09-18T04:58:15.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040023940000000001 2024-09-18T04:58:15.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694861+0000", 2024-09-18T04:58:15.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3961999999999997e-05 2024-09-18T04:58:15.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694932+0000", 2024-09-18T04:58:15.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1033999999999998e-05 2024-09-18T04:58:15.376 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.376 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.376 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.376 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160719 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 768000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:15.376 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:58:15.376 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.682733319999997, 2024-09-18T04:58:15.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040275321000000003, 2024-09-18T04:58:15.376 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.377 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.377 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.377 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.377 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.377 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160719 2024-09-18T04:58:15.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.377 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:58:15.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:58:15.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:58:15.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654799+0000", 2024-09-18T04:58:15.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.204e-06 2024-09-18T04:58:15.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654806+0000", 2024-09-18T04:58:15.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0500000000000003e-06 2024-09-18T04:58:15.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654814+0000", 2024-09-18T04:58:15.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5549999999999997e-06 2024-09-18T04:58:15.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694967+0000", 2024-09-18T04:58:15.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040153393000000002 2024-09-18T04:58:15.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695014+0000", 2024-09-18T04:58:15.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7194999999999998e-05 2024-09-18T04:58:15.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695073+0000", 2024-09-18T04:58:15.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8315e-05 2024-09-18T04:58:15.381 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.381 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.381 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.381 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160720 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 788480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:15.381 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:58:15.382 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.682711027000003, 2024-09-18T04:58:15.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040546450999999997, 2024-09-18T04:58:15.382 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.382 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.382 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.382 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.382 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.382 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160720 2024-09-18T04:58:15.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.382 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:58:15.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:58:15.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654819+0000", 2024-09-18T04:58:15.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654822+0000", 2024-09-18T04:58:15.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:58:15.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654828+0000", 2024-09-18T04:58:15.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2450000000000003e-06 2024-09-18T04:58:15.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654833+0000", 2024-09-18T04:58:15.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6419999999999998e-06 2024-09-18T04:58:15.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695116+0000", 2024-09-18T04:58:15.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040283126000000002 2024-09-18T04:58:15.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695203+0000", 2024-09-18T04:58:15.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7693000000000002e-05 2024-09-18T04:58:15.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695366+0000", 2024-09-18T04:58:15.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016283700000000001 2024-09-18T04:58:15.385 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.385 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.386 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.386 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160721 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 808960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:15.386 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:58:15.386 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.682693692000001, 2024-09-18T04:58:15.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040737433000000003, 2024-09-18T04:58:15.386 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.386 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.386 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.386 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.386 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.386 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160721 2024-09-18T04:58:15.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.387 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:58:15.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:58:15.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:58:15.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654839+0000", 2024-09-18T04:58:15.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.43e-06 2024-09-18T04:58:15.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654847+0000", 2024-09-18T04:58:15.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7300000000000005e-06 2024-09-18T04:58:15.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654858+0000", 2024-09-18T04:58:15.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.099e-05 2024-09-18T04:58:15.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695391+0000", 2024-09-18T04:58:15.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040533476999999998 2024-09-18T04:58:15.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695477+0000", 2024-09-18T04:58:15.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5870000000000003e-05 2024-09-18T04:58:15.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695575+0000", 2024-09-18T04:58:15.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7264999999999998e-05 2024-09-18T04:58:15.390 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.390 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.390 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.390 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160722 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 829440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:15.390 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:58:15.390 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.682668161999999, 2024-09-18T04:58:15.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040875739000000001, 2024-09-18T04:58:15.391 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.391 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.391 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.391 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.391 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.391 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160722 2024-09-18T04:58:15.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.391 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:58:15.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:58:15.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654862+0000", 2024-09-18T04:58:15.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654883+0000", 2024-09-18T04:58:15.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0468000000000001e-05 2024-09-18T04:58:15.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654891+0000", 2024-09-18T04:58:15.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8320000000000006e-06 2024-09-18T04:58:15.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654896+0000", 2024-09-18T04:58:15.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.13e-06 2024-09-18T04:58:15.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695628+0000", 2024-09-18T04:58:15.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040732011999999998 2024-09-18T04:58:15.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695678+0000", 2024-09-18T04:58:15.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0453e-05 2024-09-18T04:58:15.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695738+0000", 2024-09-18T04:58:15.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0164999999999997e-05 2024-09-18T04:58:15.394 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.395 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.395 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.395 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160723 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 849920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:15.395 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:58:15.395 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.682576355999998, 2024-09-18T04:58:15.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041073576000000001, 2024-09-18T04:58:15.395 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.395 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.395 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.395 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.395 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.395 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160723 2024-09-18T04:58:15.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.396 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:58:15.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:58:15.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:58:15.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654958+0000", 2024-09-18T04:58:15.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7230000000000002e-06 2024-09-18T04:58:15.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654968+0000", 2024-09-18T04:58:15.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0665999999999999e-05 2024-09-18T04:58:15.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654974+0000", 2024-09-18T04:58:15.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9259999999999997e-06 2024-09-18T04:58:15.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695824+0000", 2024-09-18T04:58:15.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040850085000000001 2024-09-18T04:58:15.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695933+0000", 2024-09-18T04:58:15.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108738 2024-09-18T04:58:15.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696028+0000", 2024-09-18T04:58:15.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4988000000000006e-05 2024-09-18T04:58:15.399 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.399 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.399 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.399 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160724 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 870400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:15.399 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:58:15.400 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.682552012999999, 2024-09-18T04:58:15.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041279300999999997, 2024-09-18T04:58:15.400 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.400 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.400 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.400 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.400 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.400 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160724 2024-09-18T04:58:15.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.400 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:58:15.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:58:15.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654978+0000", 2024-09-18T04:58:15.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654981+0000", 2024-09-18T04:58:15.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4530000000000001e-06 2024-09-18T04:58:15.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654987+0000", 2024-09-18T04:58:15.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6359999999999997e-06 2024-09-18T04:58:15.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654995+0000", 2024-09-18T04:58:15.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9759999999999995e-06 2024-09-18T04:58:15.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696069+0000", 2024-09-18T04:58:15.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041073447999999999 2024-09-18T04:58:15.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696160+0000", 2024-09-18T04:58:15.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0835999999999996e-05 2024-09-18T04:58:15.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696258+0000", 2024-09-18T04:58:15.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8338999999999995e-05 2024-09-18T04:58:15.404 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.404 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.404 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.404 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160725 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 890880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:15.404 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:58:15.404 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.682530944, 2024-09-18T04:58:15.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041552253999999997, 2024-09-18T04:58:15.404 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.405 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.405 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.405 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.405 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.405 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160725 2024-09-18T04:58:15.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.405 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:58:15.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:58:15.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654999+0000", 2024-09-18T04:58:15.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.406 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655002+0000", 2024-09-18T04:58:15.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.419e-06 2024-09-18T04:58:15.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.406 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655008+0000", 2024-09-18T04:58:15.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2310000000000001e-06 2024-09-18T04:58:15.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655013+0000", 2024-09-18T04:58:15.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8439999999999997e-06 2024-09-18T04:58:15.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696353+0000", 2024-09-18T04:58:15.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041340322999999998 2024-09-18T04:58:15.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696455+0000", 2024-09-18T04:58:15.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000102105 2024-09-18T04:58:15.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696552+0000", 2024-09-18T04:58:15.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6738999999999997e-05 2024-09-18T04:58:15.408 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.408 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.408 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.409 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160726 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:15.409 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:58:15.409 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.682513712000002, 2024-09-18T04:58:15.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041786440000000001, 2024-09-18T04:58:15.409 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.409 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.409 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.409 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.409 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.409 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160726 2024-09-18T04:58:15.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.409 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:58:15.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:58:15.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:58:15.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655019+0000", 2024-09-18T04:58:15.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1100000000000001e-06 2024-09-18T04:58:15.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655025+0000", 2024-09-18T04:58:15.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2190000000000001e-06 2024-09-18T04:58:15.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.411 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655030+0000", 2024-09-18T04:58:15.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5340000000000001e-06 2024-09-18T04:58:15.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696607+0000", 2024-09-18T04:58:15.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041577309 2024-09-18T04:58:15.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696694+0000", 2024-09-18T04:58:15.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7230000000000003e-05 2024-09-18T04:58:15.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696804+0000", 2024-09-18T04:58:15.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0001094 2024-09-18T04:58:15.413 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.413 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.413 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.413 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160727 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 931840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:15.413 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:58:15.413 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.682496974999999, 2024-09-18T04:58:15.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042000527000000003, 2024-09-18T04:58:15.413 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.413 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.413 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.414 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.414 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.414 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160727 2024-09-18T04:58:15.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.414 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:58:15.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:58:15.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655033+0000", 2024-09-18T04:58:15.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655036+0000", 2024-09-18T04:58:15.415 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3489999999999999e-06 2024-09-18T04:58:15.415 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.415 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.415 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.415 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655042+0000", 2024-09-18T04:58:15.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1399999999999997e-06 2024-09-18T04:58:15.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655046+0000", 2024-09-18T04:58:15.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5480000000000003e-06 2024-09-18T04:58:15.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.416 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.416 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696843+0000", 2024-09-18T04:58:15.416 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041797103000000002 2024-09-18T04:58:15.416 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.416 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696931+0000", 2024-09-18T04:58:15.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7526000000000002e-05 2024-09-18T04:58:15.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.417 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.417 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697034+0000", 2024-09-18T04:58:15.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010336999999999999 2024-09-18T04:58:15.417 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.417 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.417 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.417 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160728 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 952320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:15.418 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:58:15.418 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.682480417999997, 2024-09-18T04:58:15.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042191293999999997, 2024-09-18T04:58:15.418 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.418 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.418 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.418 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.418 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.418 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160728 2024-09-18T04:58:15.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.418 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:58:15.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:58:15.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:58:15.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655052+0000", 2024-09-18T04:58:15.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.232e-06 2024-09-18T04:58:15.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655058+0000", 2024-09-18T04:58:15.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9869999999999996e-06 2024-09-18T04:58:15.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655063+0000", 2024-09-18T04:58:15.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8420000000000004e-06 2024-09-18T04:58:15.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697074+0000", 2024-09-18T04:58:15.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042010446 2024-09-18T04:58:15.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697157+0000", 2024-09-18T04:58:15.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3038e-05 2024-09-18T04:58:15.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697242+0000", 2024-09-18T04:58:15.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5115000000000001e-05 2024-09-18T04:58:15.421 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.422 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.422 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.422 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160729 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 972800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:15.422 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:58:15.422 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.682463659, 2024-09-18T04:58:15.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042367834, 2024-09-18T04:58:15.422 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.422 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.422 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.422 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.422 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.422 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160729 2024-09-18T04:58:15.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.423 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:58:15.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:58:15.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:58:15.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655069+0000", 2024-09-18T04:58:15.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2220000000000001e-06 2024-09-18T04:58:15.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655075+0000", 2024-09-18T04:58:15.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2129999999999996e-06 2024-09-18T04:58:15.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655080+0000", 2024-09-18T04:58:15.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6840000000000004e-06 2024-09-18T04:58:15.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697280+0000", 2024-09-18T04:58:15.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042199819999999999 2024-09-18T04:58:15.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697368+0000", 2024-09-18T04:58:15.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7863999999999998e-05 2024-09-18T04:58:15.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.426 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.426 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697435+0000", 2024-09-18T04:58:15.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7416000000000006e-05 2024-09-18T04:58:15.426 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.426 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.426 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.426 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160730 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 993280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:15.426 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:58:15.426 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.682446894000002, 2024-09-18T04:58:15.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042559582999999998, 2024-09-18T04:58:15.427 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.427 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.427 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.427 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.427 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.427 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160730 2024-09-18T04:58:15.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.427 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:58:15.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:58:15.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:58:15.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.428 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655086+0000", 2024-09-18T04:58:15.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1500000000000002e-06 2024-09-18T04:58:15.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655092+0000", 2024-09-18T04:58:15.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.896e-06 2024-09-18T04:58:15.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655096+0000", 2024-09-18T04:58:15.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7190000000000001e-06 2024-09-18T04:58:15.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697475+0000", 2024-09-18T04:58:15.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042378644 2024-09-18T04:58:15.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697553+0000", 2024-09-18T04:58:15.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7626999999999994e-05 2024-09-18T04:58:15.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697643+0000", 2024-09-18T04:58:15.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0921000000000006e-05 2024-09-18T04:58:15.430 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.430 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.431 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.431 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160731 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 1013760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:15.431 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:58:15.431 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.682414164000001, 2024-09-18T04:58:15.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042727764000000001, 2024-09-18T04:58:15.431 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.431 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.431 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.431 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.431 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.431 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160731 2024-09-18T04:58:15.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.432 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:58:15.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:58:15.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655116+0000", 2024-09-18T04:58:15.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655119+0000", 2024-09-18T04:58:15.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1240000000000001e-06 2024-09-18T04:58:15.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655131+0000", 2024-09-18T04:58:15.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2145e-05 2024-09-18T04:58:15.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655138+0000", 2024-09-18T04:58:15.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3369999999999996e-06 2024-09-18T04:58:15.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697688+0000", 2024-09-18T04:58:15.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042550604999999998 2024-09-18T04:58:15.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697781+0000", 2024-09-18T04:58:15.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2867000000000004e-05 2024-09-18T04:58:15.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697844+0000", 2024-09-18T04:58:15.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3236000000000002e-05 2024-09-18T04:58:15.435 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.435 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.435 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.436 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160732 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 1034240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:15.436 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:58:15.436 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 38.682388316000001, 2024-09-18T04:58:15.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042857977999999998, 2024-09-18T04:58:15.436 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.436 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.436 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.436 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.436 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.436 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160732 2024-09-18T04:58:15.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.436 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:58:15.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:58:15.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:58:15.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655145+0000", 2024-09-18T04:58:15.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7329999999999999e-06 2024-09-18T04:58:15.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655152+0000", 2024-09-18T04:58:15.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8249999999999999e-06 2024-09-18T04:58:15.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655158+0000", 2024-09-18T04:58:15.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5010000000000001e-06 2024-09-18T04:58:15.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697873+0000", 2024-09-18T04:58:15.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042714457999999997 2024-09-18T04:58:15.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697944+0000", 2024-09-18T04:58:15.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1486e-05 2024-09-18T04:58:15.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.698000+0000", 2024-09-18T04:58:15.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6332000000000002e-05 2024-09-18T04:58:15.440 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.440 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.440 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.440 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.440 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.440 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:15.441 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:58:15.442 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:15.439+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:15.542 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:15.540+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:15.549 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:15.549 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:15.549 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:58:15.549 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:15.558 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:58:15.593 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 5, 6, 7] out_osds: [1, 4] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T04:58:15.593 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T04:58:15.593 INFO:tasks.thrashosds.thrasher:Removing osd 3, in_osds are: [0, 2, 3, 5, 6, 7] 2024-09-18T04:58:15.593 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 3 2024-09-18T04:58:15.643 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:15.641+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:15.665 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:15.665 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:15.665 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:58:15.665 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:58:15.665 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:15.676 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:58:15.715 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:58:15.744 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:15.741+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:15.789 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:15.789 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:58:15.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:58:15.789 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:58:15.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.789 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11154 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:15.789 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:58:15.789 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.895382257999998, 2024-09-18T04:58:15.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040591919999999997, 2024-09-18T04:58:15.789 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.790 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.790 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.790 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.790 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.790 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11154 2024-09-18T04:58:15.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.790 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:58:15.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:58:15.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889400+0000", 2024-09-18T04:58:15.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.791 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889403+0000", 2024-09-18T04:58:15.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3999999999999999e-06 2024-09-18T04:58:15.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889409+0000", 2024-09-18T04:58:15.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3380000000000001e-06 2024-09-18T04:58:15.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889414+0000", 2024-09-18T04:58:15.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5789999999999997e-06 2024-09-18T04:58:15.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929811+0000", 2024-09-18T04:58:15.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040397486000000003 2024-09-18T04:58:15.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929898+0000", 2024-09-18T04:58:15.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6370000000000001e-05 2024-09-18T04:58:15.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929993+0000", 2024-09-18T04:58:15.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5136000000000005e-05 2024-09-18T04:58:15.793 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.793 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.793 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.794 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11155 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2015232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:15.794 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:58:15.794 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.895365080000005, 2024-09-18T04:58:15.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040769156000000001, 2024-09-18T04:58:15.794 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.794 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.794 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.794 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.794 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.794 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11155 2024-09-18T04:58:15.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.795 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:58:15.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:58:15.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:58:15.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889420+0000", 2024-09-18T04:58:15.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.181e-06 2024-09-18T04:58:15.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.796 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.796 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889426+0000", 2024-09-18T04:58:15.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8599999999999998e-06 2024-09-18T04:58:15.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889430+0000", 2024-09-18T04:58:15.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4399999999999998e-06 2024-09-18T04:58:15.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930020+0000", 2024-09-18T04:58:15.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040589698 2024-09-18T04:58:15.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930105+0000", 2024-09-18T04:58:15.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4812e-05 2024-09-18T04:58:15.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930187+0000", 2024-09-18T04:58:15.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2540999999999995e-05 2024-09-18T04:58:15.798 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.798 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.798 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.798 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11156 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2064384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:15.799 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:58:15.799 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.895324786000003, 2024-09-18T04:58:15.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040934419, 2024-09-18T04:58:15.799 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.799 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.799 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.799 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.799 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.799 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11156 2024-09-18T04:58:15.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.800 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:58:15.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:58:15.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889457+0000", 2024-09-18T04:58:15.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:58:15.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889462+0000", 2024-09-18T04:58:15.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8350000000000003e-06 2024-09-18T04:58:15.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889481+0000", 2024-09-18T04:58:15.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8845999999999999e-05 2024-09-18T04:58:15.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889495+0000", 2024-09-18T04:58:15.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4229e-05 2024-09-18T04:58:15.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930216+0000", 2024-09-18T04:58:15.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040721081999999999 2024-09-18T04:58:15.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930299+0000", 2024-09-18T04:58:15.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2995000000000001e-05 2024-09-18T04:58:15.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930393+0000", 2024-09-18T04:58:15.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3185999999999999e-05 2024-09-18T04:58:15.803 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.803 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.803 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.804 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11404 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1609728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:15.804 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:58:15.804 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.888520623999995, 2024-09-18T04:58:15.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040669012999999997, 2024-09-18T04:58:15.805 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.805 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.805 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.805 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.805 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.805 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11404 2024-09-18T04:58:15.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.806 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:58:15.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:58:15.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:58:15.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:58:15.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896263+0000", 2024-09-18T04:58:15.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2729999999999999e-06 2024-09-18T04:58:15.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896266+0000", 2024-09-18T04:58:15.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8200000000000001e-06 2024-09-18T04:58:15.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896269+0000", 2024-09-18T04:58:15.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3199999999999998e-06 2024-09-18T04:58:15.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936538+0000", 2024-09-18T04:58:15.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040269394999999999 2024-09-18T04:58:15.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936657+0000", 2024-09-18T04:58:15.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119498 2024-09-18T04:58:15.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936931+0000", 2024-09-18T04:58:15.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027391900000000002 2024-09-18T04:58:15.810 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.810 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.810 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.810 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11405 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1658880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:15.810 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:58:15.810 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.888512366, 2024-09-18T04:58:15.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040891034999999999, 2024-09-18T04:58:15.811 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.811 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.811 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.811 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.811 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.811 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11405 2024-09-18T04:58:15.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.811 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:58:15.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:58:15.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896270+0000", 2024-09-18T04:58:15.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:58:15.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896272+0000", 2024-09-18T04:58:15.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.124e-06 2024-09-18T04:58:15.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896274+0000", 2024-09-18T04:58:15.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8660000000000002e-06 2024-09-18T04:58:15.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896285+0000", 2024-09-18T04:58:15.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0101e-05 2024-09-18T04:58:15.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936986+0000", 2024-09-18T04:58:15.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040701213999999999 2024-09-18T04:58:15.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937070+0000", 2024-09-18T04:58:15.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4211999999999999e-05 2024-09-18T04:58:15.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.814 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.814 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937162+0000", 2024-09-18T04:58:15.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1728e-05 2024-09-18T04:58:15.814 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.814 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.814 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.815 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11406 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1708032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:15.815 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:58:15.815 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.888496196999995, 2024-09-18T04:58:15.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041085251000000003, 2024-09-18T04:58:15.815 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.815 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.815 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.815 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.815 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.815 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11406 2024-09-18T04:58:15.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.815 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:58:15.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:58:15.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:58:15.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:58:15.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896288+0000", 2024-09-18T04:58:15.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2440000000000001e-06 2024-09-18T04:58:15.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896291+0000", 2024-09-18T04:58:15.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8289999999999999e-06 2024-09-18T04:58:15.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896293+0000", 2024-09-18T04:58:15.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5399999999999998e-06 2024-09-18T04:58:15.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937201+0000", 2024-09-18T04:58:15.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040907418000000001 2024-09-18T04:58:15.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937283+0000", 2024-09-18T04:58:15.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2460999999999995e-05 2024-09-18T04:58:15.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937372+0000", 2024-09-18T04:58:15.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8987999999999996e-05 2024-09-18T04:58:15.819 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.820 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.820 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.820 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11407 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:15.820 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:58:15.820 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.888486377000007, 2024-09-18T04:58:15.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041438580000000003, 2024-09-18T04:58:15.820 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.820 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.821 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.821 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.821 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.821 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11407 2024-09-18T04:58:15.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.821 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:58:15.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:58:15.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896296+0000", 2024-09-18T04:58:15.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896298+0000", 2024-09-18T04:58:15.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:58:15.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.823 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.823 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896304+0000", 2024-09-18T04:58:15.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6570000000000002e-06 2024-09-18T04:58:15.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896308+0000", 2024-09-18T04:58:15.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0940000000000001e-06 2024-09-18T04:58:15.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937540+0000", 2024-09-18T04:58:15.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041231469 2024-09-18T04:58:15.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937625+0000", 2024-09-18T04:58:15.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5767999999999996e-05 2024-09-18T04:58:15.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937735+0000", 2024-09-18T04:58:15.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010974 2024-09-18T04:58:15.825 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.825 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.826 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.826 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11408 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:15.826 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:58:15.826 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.888470737999995, 2024-09-18T04:58:15.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041653055000000001, 2024-09-18T04:58:15.826 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.826 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.826 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.826 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.827 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.827 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11408 2024-09-18T04:58:15.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.827 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:58:15.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:58:15.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:58:15.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896314+0000", 2024-09-18T04:58:15.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0140000000000001e-06 2024-09-18T04:58:15.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896320+0000", 2024-09-18T04:58:15.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-18T04:58:15.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896324+0000", 2024-09-18T04:58:15.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4410000000000003e-06 2024-09-18T04:58:15.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937769+0000", 2024-09-18T04:58:15.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041445243 2024-09-18T04:58:15.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937861+0000", 2024-09-18T04:58:15.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1471000000000006e-05 2024-09-18T04:58:15.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937965+0000", 2024-09-18T04:58:15.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104516 2024-09-18T04:58:15.831 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.831 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.831 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.831 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11409 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:15.832 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:58:15.832 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.888454682000003, 2024-09-18T04:58:15.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041840467999999999, 2024-09-18T04:58:15.832 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.832 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.832 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.832 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.832 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.832 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11409 2024-09-18T04:58:15.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.832 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:58:15.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:58:15.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:58:15.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896331+0000", 2024-09-18T04:58:15.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5950000000000001e-06 2024-09-18T04:58:15.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896336+0000", 2024-09-18T04:58:15.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7259999999999999e-06 2024-09-18T04:58:15.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896341+0000", 2024-09-18T04:58:15.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5619999999999997e-06 2024-09-18T04:58:15.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937995+0000", 2024-09-18T04:58:15.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041654060999999999 2024-09-18T04:58:15.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938078+0000", 2024-09-18T04:58:15.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2869999999999998e-05 2024-09-18T04:58:15.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.836 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.836 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938169+0000", 2024-09-18T04:58:15.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1040999999999998e-05 2024-09-18T04:58:15.836 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.836 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.836 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.836 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11410 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1921024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:15.836 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:58:15.836 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.888438199000007, 2024-09-18T04:58:15.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042060342000000001, 2024-09-18T04:58:15.837 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.837 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.837 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.837 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.837 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.837 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11410 2024-09-18T04:58:15.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.837 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:58:15.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:58:15.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896344+0000", 2024-09-18T04:58:15.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896360+0000", 2024-09-18T04:58:15.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5068e-05 2024-09-18T04:58:15.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896366+0000", 2024-09-18T04:58:15.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5119999999999997e-06 2024-09-18T04:58:15.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896371+0000", 2024-09-18T04:58:15.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:58:15.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938198+0000", 2024-09-18T04:58:15.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041826579000000003 2024-09-18T04:58:15.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938297+0000", 2024-09-18T04:58:15.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9067000000000005e-05 2024-09-18T04:58:15.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.840 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.840 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938405+0000", 2024-09-18T04:58:15.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108492 2024-09-18T04:58:15.840 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.840 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.841 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.841 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11411 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1970176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:15.841 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:58:15.841 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.888407357000006, 2024-09-18T04:58:15.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042259191000000002, 2024-09-18T04:58:15.841 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.841 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.841 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11411 2024-09-18T04:58:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:58:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:58:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896375+0000", 2024-09-18T04:58:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896377+0000", 2024-09-18T04:58:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2699999999999999e-06 2024-09-18T04:58:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896383+0000", 2024-09-18T04:58:15.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6740000000000002e-06 2024-09-18T04:58:15.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896388+0000", 2024-09-18T04:58:15.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:58:15.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938435+0000", 2024-09-18T04:58:15.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042047169000000002 2024-09-18T04:58:15.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938534+0000", 2024-09-18T04:58:15.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8939999999999998e-05 2024-09-18T04:58:15.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938635+0000", 2024-09-18T04:58:15.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101208 2024-09-18T04:58:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11412 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:58:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.888391374999998, 2024-09-18T04:58:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042828079999999998, 2024-09-18T04:58:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.846 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.847 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.847 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11412 2024-09-18T04:58:15.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.847 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:58:15.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:58:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896391+0000", 2024-09-18T04:58:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896393+0000", 2024-09-18T04:58:15.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0810000000000001e-06 2024-09-18T04:58:15.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896399+0000", 2024-09-18T04:58:15.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7230000000000001e-06 2024-09-18T04:58:15.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.849 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896403+0000", 2024-09-18T04:58:15.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3470000000000001e-06 2024-09-18T04:58:15.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939043+0000", 2024-09-18T04:58:15.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042639829999999997 2024-09-18T04:58:15.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939124+0000", 2024-09-18T04:58:15.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0809000000000005e-05 2024-09-18T04:58:15.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939220+0000", 2024-09-18T04:58:15.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5659e-05 2024-09-18T04:58:15.851 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.851 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.851 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.851 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11413 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2068480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:15.851 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:58:15.851 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.888375679999996, 2024-09-18T04:58:15.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043042108000000003, 2024-09-18T04:58:15.852 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.852 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.852 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.852 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.852 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.852 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11413 2024-09-18T04:58:15.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.852 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:58:15.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:58:15.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:58:15.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896409+0000", 2024-09-18T04:58:15.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.165e-06 2024-09-18T04:58:15.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896415+0000", 2024-09-18T04:58:15.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6149999999999996e-06 2024-09-18T04:58:15.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896419+0000", 2024-09-18T04:58:15.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2649999999999998e-06 2024-09-18T04:58:15.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939262+0000", 2024-09-18T04:58:15.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042843444000000001 2024-09-18T04:58:15.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939351+0000", 2024-09-18T04:58:15.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8944999999999996e-05 2024-09-18T04:58:15.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939449+0000", 2024-09-18T04:58:15.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8029000000000006e-05 2024-09-18T04:58:15.856 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.856 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.856 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.856 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145409 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:15.856 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:58:15.856 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.256956031999998, 2024-09-18T04:58:15.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040684845999999997, 2024-09-18T04:58:15.856 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.856 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.857 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.857 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.857 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.857 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145409 2024-09-18T04:58:15.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.857 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:58:15.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:58:15.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:58:15.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527829+0000", 2024-09-18T04:58:15.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.441e-06 2024-09-18T04:58:15.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527835+0000", 2024-09-18T04:58:15.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1650000000000003e-06 2024-09-18T04:58:15.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527840+0000", 2024-09-18T04:58:15.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8250000000000004e-06 2024-09-18T04:58:15.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568404+0000", 2024-09-18T04:58:15.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040563872000000001 2024-09-18T04:58:15.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568457+0000", 2024-09-18T04:58:15.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2713999999999997e-05 2024-09-18T04:58:15.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568512+0000", 2024-09-18T04:58:15.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5148000000000001e-05 2024-09-18T04:58:15.861 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.861 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.862 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.862 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145411 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:15.862 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:58:15.862 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.256938171000002, 2024-09-18T04:58:15.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040792108000000001, 2024-09-18T04:58:15.862 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.862 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.862 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.862 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.862 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.863 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145411 2024-09-18T04:58:15.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.863 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:58:15.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:58:15.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527844+0000", 2024-09-18T04:58:15.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527847+0000", 2024-09-18T04:58:15.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0260000000000002e-06 2024-09-18T04:58:15.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527852+0000", 2024-09-18T04:58:15.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9889999999999997e-06 2024-09-18T04:58:15.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527857+0000", 2024-09-18T04:58:15.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6410000000000001e-06 2024-09-18T04:58:15.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568537+0000", 2024-09-18T04:58:15.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040679743999999997 2024-09-18T04:58:15.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568583+0000", 2024-09-18T04:58:15.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6286999999999999e-05 2024-09-18T04:58:15.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568637+0000", 2024-09-18T04:58:15.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.376e-05 2024-09-18T04:58:15.867 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.867 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.867 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.867 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145413 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1859584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:15.867 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:58:15.867 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.256921791000003, 2024-09-18T04:58:15.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040919516000000003, 2024-09-18T04:58:15.868 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.868 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.868 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.868 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.868 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.868 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145413 2024-09-18T04:58:15.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.868 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:58:15.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:58:15.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:58:15.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527863+0000", 2024-09-18T04:58:15.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2630000000000002e-06 2024-09-18T04:58:15.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527869+0000", 2024-09-18T04:58:15.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9009999999999999e-06 2024-09-18T04:58:15.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527874+0000", 2024-09-18T04:58:15.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0590000000000002e-06 2024-09-18T04:58:15.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568663+0000", 2024-09-18T04:58:15.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040788805999999997 2024-09-18T04:58:15.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568716+0000", 2024-09-18T04:58:15.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3501999999999997e-05 2024-09-18T04:58:15.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.871 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.871 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568781+0000", 2024-09-18T04:58:15.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4338000000000006e-05 2024-09-18T04:58:15.871 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.872 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.872 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.872 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145414 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1908736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:15.872 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:58:15.872 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.256904833, 2024-09-18T04:58:15.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041030234999999998, 2024-09-18T04:58:15.872 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.872 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.872 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.872 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.872 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.873 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145414 2024-09-18T04:58:15.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.873 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:58:15.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:58:15.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:58:15.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527880+0000", 2024-09-18T04:58:15.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4770000000000002e-06 2024-09-18T04:58:15.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527886+0000", 2024-09-18T04:58:15.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5759999999999996e-06 2024-09-18T04:58:15.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527890+0000", 2024-09-18T04:58:15.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8180000000000003e-06 2024-09-18T04:58:15.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568807+0000", 2024-09-18T04:58:15.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040916984000000003 2024-09-18T04:58:15.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568854+0000", 2024-09-18T04:58:15.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6740999999999999e-05 2024-09-18T04:58:15.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.876 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568908+0000", 2024-09-18T04:58:15.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4228000000000003e-05 2024-09-18T04:58:15.876 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.876 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.876 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.876 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145416 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1957888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:15.876 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:58:15.877 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.256888341, 2024-09-18T04:58:15.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041147364999999998, 2024-09-18T04:58:15.877 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.877 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.877 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.877 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.877 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.877 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145416 2024-09-18T04:58:15.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.877 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:58:15.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:58:15.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527894+0000", 2024-09-18T04:58:15.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527896+0000", 2024-09-18T04:58:15.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1789999999999998e-06 2024-09-18T04:58:15.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527908+0000", 2024-09-18T04:58:15.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1780000000000001e-05 2024-09-18T04:58:15.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527914+0000", 2024-09-18T04:58:15.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2029999999999997e-06 2024-09-18T04:58:15.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568935+0000", 2024-09-18T04:58:15.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041020273000000003 2024-09-18T04:58:15.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568988+0000", 2024-09-18T04:58:15.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2911000000000002e-05 2024-09-18T04:58:15.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569042+0000", 2024-09-18T04:58:15.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4382000000000002e-05 2024-09-18T04:58:15.881 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.881 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.881 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.882 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145418 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 2007040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:15.882 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:58:15.882 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.256863944000003, 2024-09-18T04:58:15.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041247862000000003, 2024-09-18T04:58:15.882 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.882 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.882 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.882 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.882 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.882 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145418 2024-09-18T04:58:15.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.883 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:58:15.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:58:15.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:58:15.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527921+0000", 2024-09-18T04:58:15.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.667e-06 2024-09-18T04:58:15.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527929+0000", 2024-09-18T04:58:15.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.542e-06 2024-09-18T04:58:15.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527934+0000", 2024-09-18T04:58:15.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1440000000000002e-06 2024-09-18T04:58:15.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569068+0000", 2024-09-18T04:58:15.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041134190000000001 2024-09-18T04:58:15.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.886 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569114+0000", 2024-09-18T04:58:15.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6261999999999999e-05 2024-09-18T04:58:15.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569167+0000", 2024-09-18T04:58:15.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2506999999999997e-05 2024-09-18T04:58:15.887 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.887 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.887 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.888 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145420 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 2056192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:15.888 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:58:15.888 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.256844870000002, 2024-09-18T04:58:15.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041351472, 2024-09-18T04:58:15.888 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:15.888 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:15.888 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:15.888 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:15.888 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:15.888 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145420 2024-09-18T04:58:15.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.889 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:15.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:15.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:58:15.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:15.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:58:15.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:15.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:15.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:58:15.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:15.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:15.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527940+0000", 2024-09-18T04:58:15.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6570000000000001e-06 2024-09-18T04:58:15.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:15.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527948+0000", 2024-09-18T04:58:15.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8350000000000004e-06 2024-09-18T04:58:15.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:15.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527953+0000", 2024-09-18T04:58:15.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0690000000000001e-06 2024-09-18T04:58:15.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:15.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569182+0000", 2024-09-18T04:58:15.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041228734000000003 2024-09-18T04:58:15.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:15.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569234+0000", 2024-09-18T04:58:15.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1663999999999999e-05 2024-09-18T04:58:15.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:15.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:15.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:15.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569290+0000", 2024-09-18T04:58:15.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5937000000000003e-05 2024-09-18T04:58:15.895 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.895 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.895 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.895 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:15.895 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:15.895 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:15.897 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:58:15.897 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:58:15.898 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:58:15.898 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:58:15.898 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:58:15.898 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:58:15.899 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:58:15.899 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:58:15.899 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:58:15.900 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:15.894+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:15.904 INFO:tasks.ceph.mon.a.smithi002.stderr:2024-09-18T04:58:15.901+0000 7f7d3b87f640 -1 mon.a@0(leader).osd e125 definitely_dead 0 2024-09-18T04:58:15.908 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:58:15.908 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:58:15.926 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:58:15.926 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:58:15.942 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:58:15.942 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:58:15.958 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:58:15.959 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:58:16.005 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:16.005 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:16.011 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:16.009+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:16.013 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:58:16.050 INFO:teuthology.orchestra.run.smithi002.stderr:marked out osd.3. 2024-09-18T04:58:16.058 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T04:58:16.110 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:16.108+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:16.121 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:16.122 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:16.131 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:58:16.204 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:16.201+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:16.239 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:16.239 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:16.253 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:58:16.322 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:16.309+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:16.407 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:16.401+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:16.433 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:16.433 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:16.440 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:58:16.511 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:16.500+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:16.602 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:16.602 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:16.610 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:16.606+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:16.611 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:58:16.728 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:16.724+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:16.795 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:16.795 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:16.798 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:58:16.805 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:16.803+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:16.906 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:16.904+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:16.952 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:16.952 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:16.961 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:58:17.014 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:17.011+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:17.058 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-18T04:58:17.074 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:17.074 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:17.083 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:58:17.120 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:17.116+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:17.208 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:17.206+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:17.228 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:17.229 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:17.237 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:58:17.309 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:17.306+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:17.340 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:17.341 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:17.349 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:58:17.410 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:17.407+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:17.451 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:17.451 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:17.460 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:58:17.510 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:17.508+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:17.564 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:17.565 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:17.573 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:58:17.618 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:17.616+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:17.694 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:17.694 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:17.704 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:58:17.713 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:17.710+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:17.814 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:17.812+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:17.852 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:17.852 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:17.861 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:58:17.917 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:17.912+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:17.970 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:58:18.010 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:18.010 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:18.017 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:18.014+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:18.023 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:58:18.117 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:18.114+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:18.137 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:58:18.137 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:58:18.163 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:58:18.163 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:58:18.186 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:58:18.187 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:58:18.212 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:58:18.212 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:58:18.212 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:18.213 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:18.213 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:58:18.213 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:18.217 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:18.215+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:18.226 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:58:18.238 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:58:18.238 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:58:18.263 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:58:18.264 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:58:18.286 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:58:18.286 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:58:18.306 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:58:18.306 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:58:18.319 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:18.316+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:18.364 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:18.364 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:18.364 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:58:18.364 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:58:18.364 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:18.373 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:58:18.419 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:18.417+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:18.487 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:18.487 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:58:18.487 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:58:18.487 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:58:18.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.487 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14125 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:18.487 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:58:18.488 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 102.35667494899999, 2024-09-18T04:58:18.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052956470999999998, 2024-09-18T04:58:18.488 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:18.488 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:18.488 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:18.488 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:18.488 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:18.488 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14125 2024-09-18T04:58:18.488 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.488 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:18.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.488 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:18.488 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:58:18.488 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:18.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:58:18.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:18.489 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126849+0000", 2024-09-18T04:58:18.489 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:18.489 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.489 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:18.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126852+0000", 2024-09-18T04:58:18.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3659999999999999e-06 2024-09-18T04:58:18.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:18.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126858+0000", 2024-09-18T04:58:18.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4540000000000004e-06 2024-09-18T04:58:18.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.490 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.490 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:18.490 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126863+0000", 2024-09-18T04:58:18.490 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1780000000000002e-06 2024-09-18T04:58:18.490 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:18.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179608+0000", 2024-09-18T04:58:18.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052745159999999999 2024-09-18T04:58:18.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:18.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179703+0000", 2024-09-18T04:58:18.491 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4773999999999998e-05 2024-09-18T04:58:18.491 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.491 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.491 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:18.491 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179806+0000", 2024-09-18T04:58:18.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010296799999999999 2024-09-18T04:58:18.492 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.492 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:18.492 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.492 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.492 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.492 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14136 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1732608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:18.492 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:58:18.492 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 102.35451439800001, 2024-09-18T04:58:18.492 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052973412999999997, 2024-09-18T04:58:18.492 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:18.492 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:18.492 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:18.492 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:18.493 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:18.493 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14136 2024-09-18T04:58:18.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.493 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:18.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:18.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:58:18.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.493 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.493 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:18.493 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:58:18.493 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.493 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:18.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:58:18.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:18.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:18.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129013+0000", 2024-09-18T04:58:18.494 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.96e-06 2024-09-18T04:58:18.494 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.494 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.494 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:18.494 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129020+0000", 2024-09-18T04:58:18.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6809999999999998e-06 2024-09-18T04:58:18.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:18.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129026+0000", 2024-09-18T04:58:18.495 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:58:18.495 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.495 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.495 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:18.495 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181873+0000", 2024-09-18T04:58:18.496 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052847238999999997 2024-09-18T04:58:18.496 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.496 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.496 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:18.496 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181932+0000", 2024-09-18T04:58:18.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9339000000000002e-05 2024-09-18T04:58:18.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.497 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:18.497 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181984+0000", 2024-09-18T04:58:18.497 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1348999999999999e-05 2024-09-18T04:58:18.497 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.497 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:18.497 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.497 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.497 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.497 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14137 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1781760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:18.497 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:58:18.497 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 102.35449399300001, 2024-09-18T04:58:18.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053128914999999999, 2024-09-18T04:58:18.498 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:18.498 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:18.498 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:18.498 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:18.498 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:18.498 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14137 2024-09-18T04:58:18.498 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.498 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:18.498 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.498 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:18.498 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:58:18.498 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:18.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:58:18.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.499 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:18.499 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129030+0000", 2024-09-18T04:58:18.499 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:18.499 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.499 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.500 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:18.500 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129033+0000", 2024-09-18T04:58:18.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7439999999999999e-06 2024-09-18T04:58:18.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.501 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:18.501 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129039+0000", 2024-09-18T04:58:18.501 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.314e-06 2024-09-18T04:58:18.501 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.501 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:18.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129044+0000", 2024-09-18T04:58:18.502 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9919999999999998e-06 2024-09-18T04:58:18.502 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.502 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.502 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:18.502 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182015+0000", 2024-09-18T04:58:18.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052970875000000001 2024-09-18T04:58:18.503 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.503 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.503 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:18.503 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182058+0000", 2024-09-18T04:58:18.503 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2997000000000002e-05 2024-09-18T04:58:18.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.504 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:18.504 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182160+0000", 2024-09-18T04:58:18.504 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101437 2024-09-18T04:58:18.504 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.504 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:18.504 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.504 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.504 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.504 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14138 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1830912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:18.505 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:58:18.505 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 102.354475959, 2024-09-18T04:58:18.505 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053256435999999997, 2024-09-18T04:58:18.505 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:18.505 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:18.505 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:18.505 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:18.505 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:18.505 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14138 2024-09-18T04:58:18.506 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.506 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:18.506 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.506 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:18.506 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:58:18.506 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:18.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:58:18.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.507 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.507 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:18.507 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129048+0000", 2024-09-18T04:58:18.507 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:18.507 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:18.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129056+0000", 2024-09-18T04:58:18.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.413e-06 2024-09-18T04:58:18.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.508 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.508 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:18.508 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129062+0000", 2024-09-18T04:58:18.508 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6810000000000001e-06 2024-09-18T04:58:18.508 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:18.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129068+0000", 2024-09-18T04:58:18.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0459999999999996e-06 2024-09-18T04:58:18.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.509 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.509 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:18.509 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182191+0000", 2024-09-18T04:58:18.509 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053123210999999997 2024-09-18T04:58:18.509 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:18.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182238+0000", 2024-09-18T04:58:18.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7404999999999998e-05 2024-09-18T04:58:18.510 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.510 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.510 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:18.510 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182305+0000", 2024-09-18T04:58:18.510 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7028999999999998e-05 2024-09-18T04:58:18.510 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.510 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:18.510 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.511 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.511 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.511 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15910 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2236416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:18.511 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:58:18.511 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 102.255388427, 2024-09-18T04:58:18.511 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053241054000000003, 2024-09-18T04:58:18.511 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:18.511 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:18.511 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:18.511 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:18.511 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:18.512 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15910 2024-09-18T04:58:18.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.512 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:18.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:18.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:58:18.512 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.512 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.512 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.512 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:18.512 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:58:18.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:18.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:58:18.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:18.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.513 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.513 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:18.513 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228138+0000", 2024-09-18T04:58:18.513 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:58:18.513 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:18.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228144+0000", 2024-09-18T04:58:18.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5369999999999997e-06 2024-09-18T04:58:18.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:18.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228149+0000", 2024-09-18T04:58:18.514 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1209999999999998e-06 2024-09-18T04:58:18.514 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.514 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.514 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:18.514 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280822+0000", 2024-09-18T04:58:18.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052673194999999999 2024-09-18T04:58:18.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:18.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280903+0000", 2024-09-18T04:58:18.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1179000000000003e-05 2024-09-18T04:58:18.515 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.515 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.515 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:18.515 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281377+0000", 2024-09-18T04:58:18.515 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047417899999999997 2024-09-18T04:58:18.516 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.516 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:18.516 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.516 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.516 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.516 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15911 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2285568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:18.516 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:58:18.516 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 102.25537169499999, 2024-09-18T04:58:18.517 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053415151000000001, 2024-09-18T04:58:18.517 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:18.517 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:18.517 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:18.517 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:18.517 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:18.517 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15911 2024-09-18T04:58:18.517 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.517 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:18.517 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:18.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:58:18.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:18.518 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:58:18.518 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.518 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.518 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.518 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:18.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:58:18.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:18.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:18.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228155+0000", 2024-09-18T04:58:18.519 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:58:18.519 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.519 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.519 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:18.519 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228161+0000", 2024-09-18T04:58:18.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:58:18.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:18.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228166+0000", 2024-09-18T04:58:18.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2320000000000001e-06 2024-09-18T04:58:18.520 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.520 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.520 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:18.520 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281405+0000", 2024-09-18T04:58:18.520 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053239336999999998 2024-09-18T04:58:18.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:18.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281482+0000", 2024-09-18T04:58:18.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6569999999999994e-05 2024-09-18T04:58:18.521 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.521 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.521 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:18.521 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281568+0000", 2024-09-18T04:58:18.521 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6328000000000004e-05 2024-09-18T04:58:18.521 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.522 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:18.522 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.522 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.522 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.522 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15918 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2662400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:18.522 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:58:18.522 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 102.253602714, 2024-09-18T04:58:18.522 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053091065, 2024-09-18T04:58:18.522 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:18.523 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:18.523 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:18.523 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:18.523 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:18.523 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15918 2024-09-18T04:58:18.523 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.523 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:18.523 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.523 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:18.523 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:58:18.523 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:18.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:58:18.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.524 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.524 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.524 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:18.524 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:58:18.524 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:18.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:18.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229924+0000", 2024-09-18T04:58:18.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:58:18.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.525 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.525 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:18.525 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229930+0000", 2024-09-18T04:58:18.525 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.066e-06 2024-09-18T04:58:18.525 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:18.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229935+0000", 2024-09-18T04:58:18.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9119999999999997e-06 2024-09-18T04:58:18.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.526 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:18.526 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282840+0000", 2024-09-18T04:58:18.526 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052905119 2024-09-18T04:58:18.526 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.526 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:18.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282922+0000", 2024-09-18T04:58:18.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2081000000000002e-05 2024-09-18T04:58:18.527 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.527 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.527 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:18.527 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283013+0000", 2024-09-18T04:58:18.527 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0828000000000005e-05 2024-09-18T04:58:18.527 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.527 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:18.528 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.528 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.528 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.528 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15919 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2711552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:18.528 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:58:18.528 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 102.25358543599999, 2024-09-18T04:58:18.528 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053293021000000003, 2024-09-18T04:58:18.528 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:18.529 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:18.529 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:18.529 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:18.529 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:18.529 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15919 2024-09-18T04:58:18.529 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.529 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:18.529 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.529 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:18.529 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:58:18.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:18.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:58:18.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.530 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.530 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:18.530 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:58:18.530 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:18.530 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:18.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229941+0000", 2024-09-18T04:58:18.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2639999999999998e-06 2024-09-18T04:58:18.531 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.531 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.531 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:18.531 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229947+0000", 2024-09-18T04:58:18.531 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9750000000000004e-06 2024-09-18T04:58:18.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:18.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229952+0000", 2024-09-18T04:58:18.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7929999999999997e-06 2024-09-18T04:58:18.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.532 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.532 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:18.532 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283062+0000", 2024-09-18T04:58:18.532 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05310952 2024-09-18T04:58:18.532 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:18.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283162+0000", 2024-09-18T04:58:18.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100098 2024-09-18T04:58:18.533 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.533 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.533 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:18.533 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283232+0000", 2024-09-18T04:58:18.533 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0693999999999998e-05 2024-09-18T04:58:18.534 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.534 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:18.534 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.534 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.534 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.534 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15920 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2760704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:18.534 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:58:18.534 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 102.253568675, 2024-09-18T04:58:18.534 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053502367000000002, 2024-09-18T04:58:18.534 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:18.534 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:18.535 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:18.535 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:18.535 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:18.535 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15920 2024-09-18T04:58:18.535 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.535 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:18.535 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.535 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:18.535 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:58:18.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:18.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:58:18.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.536 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:18.536 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:58:18.536 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:18.536 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.536 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:18.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229958+0000", 2024-09-18T04:58:18.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1770000000000001e-06 2024-09-18T04:58:18.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.537 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.537 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:18.537 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229964+0000", 2024-09-18T04:58:18.537 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8470000000000001e-06 2024-09-18T04:58:18.537 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229968+0000", 2024-09-18T04:58:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5680000000000001e-06 2024-09-18T04:58:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283269+0000", 2024-09-18T04:58:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053300620999999999 2024-09-18T04:58:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283359+0000", 2024-09-18T04:58:18.538 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9971999999999998e-05 2024-09-18T04:58:18.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.539 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:18.539 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283458+0000", 2024-09-18T04:58:18.539 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9483999999999998e-05 2024-09-18T04:58:18.539 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.539 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:18.539 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.539 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.539 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.539 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15921 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2809856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:18.539 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:58:18.539 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 102.253552125, 2024-09-18T04:58:18.540 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053800977, 2024-09-18T04:58:18.540 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:18.540 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:18.540 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:18.540 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:18.540 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:18.540 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15921 2024-09-18T04:58:18.540 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.540 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:18.540 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.540 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:18.540 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:58:18.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:18.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:58:18.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.541 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:18.541 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229972+0000", 2024-09-18T04:58:18.541 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:18.541 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.541 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:18.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229974+0000", 2024-09-18T04:58:18.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2910000000000002e-06 2024-09-18T04:58:18.542 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.542 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.542 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:18.542 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229982+0000", 2024-09-18T04:58:18.542 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7249999999999997e-06 2024-09-18T04:58:18.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:18.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229989+0000", 2024-09-18T04:58:18.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6429999999999998e-06 2024-09-18T04:58:18.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.543 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.543 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:18.543 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283601+0000", 2024-09-18T04:58:18.543 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053612211 2024-09-18T04:58:18.543 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:18.544 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283681+0000", 2024-09-18T04:58:18.544 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0125999999999998e-05 2024-09-18T04:58:18.544 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.544 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.544 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:18.545 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283774+0000", 2024-09-18T04:58:18.545 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2362999999999996e-05 2024-09-18T04:58:18.545 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.545 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:18.545 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.545 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.545 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.545 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15922 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2859008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:18.545 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:58:18.546 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 102.25350718599999, 2024-09-18T04:58:18.546 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053882270000000003, 2024-09-18T04:58:18.546 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:18.546 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:18.546 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:18.546 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:18.546 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:18.546 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15922 2024-09-18T04:58:18.546 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.546 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:18.546 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.548 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:58:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:58:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230017+0000", 2024-09-18T04:58:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.549 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230019+0000", 2024-09-18T04:58:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4210000000000002e-06 2024-09-18T04:58:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230027+0000", 2024-09-18T04:58:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000002e-06 2024-09-18T04:58:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230032+0000", 2024-09-18T04:58:18.550 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0089999999999998e-06 2024-09-18T04:58:18.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:18.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283800+0000", 2024-09-18T04:58:18.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053768418999999998 2024-09-18T04:58:18.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:18.551 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283849+0000", 2024-09-18T04:58:18.551 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9339000000000003e-05 2024-09-18T04:58:18.551 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.551 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.551 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283900+0000", 2024-09-18T04:58:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0421999999999999e-05 2024-09-18T04:58:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15923 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2924544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:58:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 102.25348831700001, 2024-09-18T04:58:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054073016000000002, 2024-09-18T04:58:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:18.552 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15923 2024-09-18T04:58:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:58:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.553 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:58:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:58:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:18.554 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230038+0000", 2024-09-18T04:58:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.463e-06 2024-09-18T04:58:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230045+0000", 2024-09-18T04:58:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3570000000000003e-06 2024-09-18T04:58:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230050+0000", 2024-09-18T04:58:18.555 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7299999999999996e-06 2024-09-18T04:58:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283928+0000", 2024-09-18T04:58:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053878218999999998 2024-09-18T04:58:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284010+0000", 2024-09-18T04:58:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2577000000000006e-05 2024-09-18T04:58:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.556 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284109+0000", 2024-09-18T04:58:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9102000000000001e-05 2024-09-18T04:58:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15924 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:58:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 102.25346984700001, 2024-09-18T04:58:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054108470999999998, 2024-09-18T04:58:18.557 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15924 2024-09-18T04:58:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:58:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.558 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:58:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:58:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.559 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230061+0000", 2024-09-18T04:58:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8240000000000002e-06 2024-09-18T04:58:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230068+0000", 2024-09-18T04:58:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4320000000000004e-06 2024-09-18T04:58:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.560 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230073+0000", 2024-09-18T04:58:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.921e-06 2024-09-18T04:58:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284153+0000", 2024-09-18T04:58:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054080793000000002 2024-09-18T04:58:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284163+0000", 2024-09-18T04:58:18.561 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.859e-06 2024-09-18T04:58:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15925 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:58:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 102.253445732, 2024-09-18T04:58:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054134632000000002, 2024-09-18T04:58:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:18.562 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15925 2024-09-18T04:58:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:58:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.563 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:58:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:58:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:18.564 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230081+0000", 2024-09-18T04:58:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4109999999999999e-06 2024-09-18T04:58:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230087+0000", 2024-09-18T04:58:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.161e-06 2024-09-18T04:58:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230092+0000", 2024-09-18T04:58:18.565 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9259999999999999e-06 2024-09-18T04:58:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:18.566 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284204+0000", 2024-09-18T04:58:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054111546000000003 2024-09-18T04:58:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:18.567 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284214+0000", 2024-09-18T04:58:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9639999999999998e-06 2024-09-18T04:58:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15926 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:58:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 102.253428408, 2024-09-18T04:58:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054143193999999999, 2024-09-18T04:58:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:18.568 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15926 2024-09-18T04:58:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:58:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:58:18.569 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.570 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:58:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.571 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230098+0000", 2024-09-18T04:58:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1399999999999998e-06 2024-09-18T04:58:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230104+0000", 2024-09-18T04:58:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-18T04:58:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.572 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230109+0000", 2024-09-18T04:58:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9250000000000003e-06 2024-09-18T04:58:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284231+0000", 2024-09-18T04:58:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054122297999999999 2024-09-18T04:58:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.573 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284239+0000", 2024-09-18T04:58:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1689999999999992e-06 2024-09-18T04:58:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.574 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15927 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:58:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 102.253411361, 2024-09-18T04:58:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054168435000000001, 2024-09-18T04:58:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15927 2024-09-18T04:58:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:18.575 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:58:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:58:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.576 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:58:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230116+0000", 2024-09-18T04:58:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5150000000000001e-06 2024-09-18T04:58:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:18.577 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230122+0000", 2024-09-18T04:58:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1190000000000002e-06 2024-09-18T04:58:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230126+0000", 2024-09-18T04:58:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.792e-06 2024-09-18T04:58:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:18.578 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284273+0000", 2024-09-18T04:58:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054146086000000003 2024-09-18T04:58:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284282+0000", 2024-09-18T04:58:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2590000000000006e-06 2024-09-18T04:58:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.579 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15928 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:58:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 102.25339397800001, 2024-09-18T04:58:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054189739000000001, 2024-09-18T04:58:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:18.580 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15928 2024-09-18T04:58:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:58:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.581 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:58:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230130+0000", 2024-09-18T04:58:18.582 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230133+0000", 2024-09-18T04:58:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.244e-06 2024-09-18T04:58:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:18.583 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230139+0000", 2024-09-18T04:58:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9830000000000001e-06 2024-09-18T04:58:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230143+0000", 2024-09-18T04:58:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7530000000000001e-06 2024-09-18T04:58:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:18.584 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284313+0000", 2024-09-18T04:58:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054169332000000001 2024-09-18T04:58:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284320+0000", 2024-09-18T04:58:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8699999999999992e-06 2024-09-18T04:58:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.585 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15929 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:58:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 102.253377538, 2024-09-18T04:58:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054218771999999998, 2024-09-18T04:58:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:18.586 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15929 2024-09-18T04:58:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:58:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:58:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.587 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:58:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230149+0000", 2024-09-18T04:58:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-18T04:58:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:18.588 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230155+0000", 2024-09-18T04:58:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2469999999999997e-06 2024-09-18T04:58:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230160+0000", 2024-09-18T04:58:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7129999999999996e-06 2024-09-18T04:58:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284355+0000", 2024-09-18T04:58:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054194900999999997 2024-09-18T04:58:18.589 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284366+0000", 2024-09-18T04:58:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0993e-05 2024-09-18T04:58:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.590 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15930 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:58:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 102.253360538, 2024-09-18T04:58:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054244713999999999, 2024-09-18T04:58:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15930 2024-09-18T04:58:18.591 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:58:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:58:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.592 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:58:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230166+0000", 2024-09-18T04:58:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1009999999999999e-06 2024-09-18T04:58:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:18.593 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230172+0000", 2024-09-18T04:58:18.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.981e-06 2024-09-18T04:58:18.594 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.594 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.594 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:18.594 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230184+0000", 2024-09-18T04:58:18.594 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2449000000000001e-05 2024-09-18T04:58:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284400+0000", 2024-09-18T04:58:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054215452999999997 2024-09-18T04:58:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284409+0000", 2024-09-18T04:58:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0839999999999998e-06 2024-09-18T04:58:18.595 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15931 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:58:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 102.253334749, 2024-09-18T04:58:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054261928000000001, 2024-09-18T04:58:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:18.596 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15931 2024-09-18T04:58:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:58:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.597 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:18.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:58:18.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:18.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.598 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:18.598 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230189+0000", 2024-09-18T04:58:18.598 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:18.598 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.598 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:18.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230192+0000", 2024-09-18T04:58:18.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6599999999999999e-06 2024-09-18T04:58:18.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:18.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230199+0000", 2024-09-18T04:58:18.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5570000000000001e-06 2024-09-18T04:58:18.599 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.599 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.599 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:18.599 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230204+0000", 2024-09-18T04:58:18.599 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2270000000000001e-06 2024-09-18T04:58:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284442+0000", 2024-09-18T04:58:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054238383000000001 2024-09-18T04:58:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284452+0000", 2024-09-18T04:58:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6069999999999997e-06 2024-09-18T04:58:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.600 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:18.601 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:18.601 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:18.603 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:58:18.603 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:18.600+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:18.703 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:18.701+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:18.760 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:18.761 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:18.761 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:58:18.761 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:18.771 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:58:18.815 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:18.812+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:18.905 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:18.902+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:18.937 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:18.937 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:18.937 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:58:18.937 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:58:18.937 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:18.947 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:58:19.034 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:19.031+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:19.077 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T04:58:19.105 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:19.105 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:58:19.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:58:19.105 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:58:19.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.105 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160713 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 645120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:19.106 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654655+0000", 2024-09-18T04:58:19.106 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.447180328000002, 2024-09-18T04:58:19.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039354912999999998, 2024-09-18T04:58:19.106 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.106 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.106 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.106 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.106 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.106 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160713 2024-09-18T04:58:19.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.106 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654655+0000", 2024-09-18T04:58:19.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654655+0000", 2024-09-18T04:58:19.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654654+0000", 2024-09-18T04:58:19.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654657+0000", 2024-09-18T04:58:19.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7049999999999999e-06 2024-09-18T04:58:19.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654669+0000", 2024-09-18T04:58:19.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1960000000000001e-05 2024-09-18T04:58:19.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654674+0000", 2024-09-18T04:58:19.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8729999999999998e-06 2024-09-18T04:58:19.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.693848+0000", 2024-09-18T04:58:19.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039174202999999998 2024-09-18T04:58:19.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.693926+0000", 2024-09-18T04:58:19.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8248e-05 2024-09-18T04:58:19.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694009+0000", 2024-09-18T04:58:19.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3363999999999997e-05 2024-09-18T04:58:19.110 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.110 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.110 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.110 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160714 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 665600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:19.110 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654678+0000", 2024-09-18T04:58:19.110 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.447156952, 2024-09-18T04:58:19.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039530155999999997, 2024-09-18T04:58:19.110 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.110 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.110 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.110 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.110 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.111 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160714 2024-09-18T04:58:19.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.111 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654678+0000", 2024-09-18T04:58:19.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654678+0000", 2024-09-18T04:58:19.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654678+0000", 2024-09-18T04:58:19.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654680+0000", 2024-09-18T04:58:19.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5229999999999998e-06 2024-09-18T04:58:19.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.112 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.112 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654686+0000", 2024-09-18T04:58:19.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9000000000000003e-06 2024-09-18T04:58:19.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654690+0000", 2024-09-18T04:58:19.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.1060000000000002e-06 2024-09-18T04:58:19.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694042+0000", 2024-09-18T04:58:19.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039351490000000003 2024-09-18T04:58:19.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694142+0000", 2024-09-18T04:58:19.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100177 2024-09-18T04:58:19.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694208+0000", 2024-09-18T04:58:19.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6367999999999999e-05 2024-09-18T04:58:19.115 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.116 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.116 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.116 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160715 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:19.116 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:58:19.116 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.447140652999998, 2024-09-18T04:58:19.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039724306000000001, 2024-09-18T04:58:19.116 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.116 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.116 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.116 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.116 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.116 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160715 2024-09-18T04:58:19.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.117 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:58:19.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:58:19.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:58:19.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654696+0000", 2024-09-18T04:58:19.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1830000000000001e-06 2024-09-18T04:58:19.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654702+0000", 2024-09-18T04:58:19.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5110000000000003e-06 2024-09-18T04:58:19.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654706+0000", 2024-09-18T04:58:19.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2470000000000002e-06 2024-09-18T04:58:19.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694245+0000", 2024-09-18T04:58:19.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039539259 2024-09-18T04:58:19.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694327+0000", 2024-09-18T04:58:19.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1817000000000006e-05 2024-09-18T04:58:19.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694419+0000", 2024-09-18T04:58:19.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1639999999999997e-05 2024-09-18T04:58:19.120 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.120 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.120 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.120 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160716 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:19.120 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:58:19.120 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.447125468000003, 2024-09-18T04:58:19.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039842555000000002, 2024-09-18T04:58:19.120 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.121 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.121 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.121 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.121 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.121 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160716 2024-09-18T04:58:19.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.121 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:58:19.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:58:19.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:58:19.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.122 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.122 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654711+0000", 2024-09-18T04:58:19.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.99e-06 2024-09-18T04:58:19.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654717+0000", 2024-09-18T04:58:19.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6799999999999998e-06 2024-09-18T04:58:19.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654721+0000", 2024-09-18T04:58:19.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-18T04:58:19.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694449+0000", 2024-09-18T04:58:19.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039727476999999997 2024-09-18T04:58:19.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694497+0000", 2024-09-18T04:58:19.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8294000000000002e-05 2024-09-18T04:58:19.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694552+0000", 2024-09-18T04:58:19.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5024e-05 2024-09-18T04:58:19.124 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.124 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.124 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.125 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160717 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 727040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:19.125 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:58:19.125 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.447084881999999, 2024-09-18T04:58:19.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040030957999999998, 2024-09-18T04:58:19.125 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.125 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.125 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.125 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.125 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.125 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160717 2024-09-18T04:58:19.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.125 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:58:19.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:58:19.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:58:19.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654752+0000", 2024-09-18T04:58:19.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8219999999999998e-06 2024-09-18T04:58:19.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654769+0000", 2024-09-18T04:58:19.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6249e-05 2024-09-18T04:58:19.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654775+0000", 2024-09-18T04:58:19.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2090000000000002e-06 2024-09-18T04:58:19.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694589+0000", 2024-09-18T04:58:19.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039814651999999999 2024-09-18T04:58:19.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694674+0000", 2024-09-18T04:58:19.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5032999999999997e-05 2024-09-18T04:58:19.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694781+0000", 2024-09-18T04:58:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106452 2024-09-18T04:58:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160718 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 747520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:58:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.447054991000002, 2024-09-18T04:58:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040152520999999997, 2024-09-18T04:58:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.129 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160718 2024-09-18T04:58:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:58:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:58:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654779+0000", 2024-09-18T04:58:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654782+0000", 2024-09-18T04:58:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.633e-06 2024-09-18T04:58:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654789+0000", 2024-09-18T04:58:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8589999999999999e-06 2024-09-18T04:58:19.131 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654793+0000", 2024-09-18T04:58:19.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5449999999999997e-06 2024-09-18T04:58:19.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694817+0000", 2024-09-18T04:58:19.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040023940000000001 2024-09-18T04:58:19.132 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.132 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.132 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.132 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694861+0000", 2024-09-18T04:58:19.132 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3961999999999997e-05 2024-09-18T04:58:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694932+0000", 2024-09-18T04:58:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1033999999999998e-05 2024-09-18T04:58:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160719 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 768000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:58:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.447037393000002, 2024-09-18T04:58:19.133 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040275321000000003, 2024-09-18T04:58:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160719 2024-09-18T04:58:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:58:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.134 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:58:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:58:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654799+0000", 2024-09-18T04:58:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.204e-06 2024-09-18T04:58:19.135 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654806+0000", 2024-09-18T04:58:19.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0500000000000003e-06 2024-09-18T04:58:19.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.136 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654814+0000", 2024-09-18T04:58:19.136 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5549999999999997e-06 2024-09-18T04:58:19.136 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.136 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.136 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694967+0000", 2024-09-18T04:58:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040153393000000002 2024-09-18T04:58:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695014+0000", 2024-09-18T04:58:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7194999999999998e-05 2024-09-18T04:58:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695073+0000", 2024-09-18T04:58:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8315e-05 2024-09-18T04:58:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.137 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160720 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 788480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:58:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.447015100000002, 2024-09-18T04:58:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040546450999999997, 2024-09-18T04:58:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160720 2024-09-18T04:58:19.138 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:58:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:58:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.139 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654819+0000", 2024-09-18T04:58:19.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654822+0000", 2024-09-18T04:58:19.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:58:19.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.140 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.140 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654828+0000", 2024-09-18T04:58:19.140 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2450000000000003e-06 2024-09-18T04:58:19.140 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.140 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654833+0000", 2024-09-18T04:58:19.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6419999999999998e-06 2024-09-18T04:58:19.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695116+0000", 2024-09-18T04:58:19.141 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040283126000000002 2024-09-18T04:58:19.141 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.141 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.141 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.141 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695203+0000", 2024-09-18T04:58:19.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7693000000000002e-05 2024-09-18T04:58:19.142 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.142 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.142 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.142 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695366+0000", 2024-09-18T04:58:19.142 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016283700000000001 2024-09-18T04:58:19.142 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160721 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 808960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:58:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.446997764999999, 2024-09-18T04:58:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040737433000000003, 2024-09-18T04:58:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.143 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160721 2024-09-18T04:58:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:58:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:58:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:58:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654839+0000", 2024-09-18T04:58:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.43e-06 2024-09-18T04:58:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654847+0000", 2024-09-18T04:58:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7300000000000005e-06 2024-09-18T04:58:19.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654858+0000", 2024-09-18T04:58:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.099e-05 2024-09-18T04:58:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695391+0000", 2024-09-18T04:58:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040533476999999998 2024-09-18T04:58:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695477+0000", 2024-09-18T04:58:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5870000000000003e-05 2024-09-18T04:58:19.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695575+0000", 2024-09-18T04:58:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7264999999999998e-05 2024-09-18T04:58:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160722 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 829440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:58:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.446972234999997, 2024-09-18T04:58:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040875739000000001, 2024-09-18T04:58:19.147 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160722 2024-09-18T04:58:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:58:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:58:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654862+0000", 2024-09-18T04:58:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654883+0000", 2024-09-18T04:58:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0468000000000001e-05 2024-09-18T04:58:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.149 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654891+0000", 2024-09-18T04:58:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8320000000000006e-06 2024-09-18T04:58:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654896+0000", 2024-09-18T04:58:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.13e-06 2024-09-18T04:58:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695628+0000", 2024-09-18T04:58:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040732011999999998 2024-09-18T04:58:19.150 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695678+0000", 2024-09-18T04:58:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0453e-05 2024-09-18T04:58:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695738+0000", 2024-09-18T04:58:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0164999999999997e-05 2024-09-18T04:58:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160723 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 849920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:58:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.446880428999997, 2024-09-18T04:58:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041073576000000001, 2024-09-18T04:58:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160723 2024-09-18T04:58:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:58:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:58:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:58:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654958+0000", 2024-09-18T04:58:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7230000000000002e-06 2024-09-18T04:58:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654968+0000", 2024-09-18T04:58:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0665999999999999e-05 2024-09-18T04:58:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654974+0000", 2024-09-18T04:58:19.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9259999999999997e-06 2024-09-18T04:58:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695824+0000", 2024-09-18T04:58:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040850085000000001 2024-09-18T04:58:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695933+0000", 2024-09-18T04:58:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108738 2024-09-18T04:58:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696028+0000", 2024-09-18T04:58:19.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4988000000000006e-05 2024-09-18T04:58:19.156 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.156 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.156 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.156 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160724 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 870400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:19.156 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:58:19.156 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.446856085999997, 2024-09-18T04:58:19.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041279300999999997, 2024-09-18T04:58:19.156 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.156 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.156 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.157 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.157 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.157 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160724 2024-09-18T04:58:19.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.157 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.162 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:58:19.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:58:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654978+0000", 2024-09-18T04:58:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654981+0000", 2024-09-18T04:58:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4530000000000001e-06 2024-09-18T04:58:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654987+0000", 2024-09-18T04:58:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6359999999999997e-06 2024-09-18T04:58:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654995+0000", 2024-09-18T04:58:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9759999999999995e-06 2024-09-18T04:58:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696069+0000", 2024-09-18T04:58:19.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041073447999999999 2024-09-18T04:58:19.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696160+0000", 2024-09-18T04:58:19.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0835999999999996e-05 2024-09-18T04:58:19.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696258+0000", 2024-09-18T04:58:19.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8338999999999995e-05 2024-09-18T04:58:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160725 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 890880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:58:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.446835016999998, 2024-09-18T04:58:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041552253999999997, 2024-09-18T04:58:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.166 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160725 2024-09-18T04:58:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:58:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:58:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654999+0000", 2024-09-18T04:58:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655002+0000", 2024-09-18T04:58:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.419e-06 2024-09-18T04:58:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655008+0000", 2024-09-18T04:58:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2310000000000001e-06 2024-09-18T04:58:19.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655013+0000", 2024-09-18T04:58:19.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8439999999999997e-06 2024-09-18T04:58:19.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696353+0000", 2024-09-18T04:58:19.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041340322999999998 2024-09-18T04:58:19.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696455+0000", 2024-09-18T04:58:19.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000102105 2024-09-18T04:58:19.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696552+0000", 2024-09-18T04:58:19.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6738999999999997e-05 2024-09-18T04:58:19.170 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.170 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.170 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.170 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160726 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:19.170 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:58:19.170 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.446817785, 2024-09-18T04:58:19.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041786440000000001, 2024-09-18T04:58:19.171 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.171 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.171 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.171 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.171 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.171 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160726 2024-09-18T04:58:19.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.171 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:58:19.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:58:19.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:58:19.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655019+0000", 2024-09-18T04:58:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1100000000000001e-06 2024-09-18T04:58:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655025+0000", 2024-09-18T04:58:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2190000000000001e-06 2024-09-18T04:58:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655030+0000", 2024-09-18T04:58:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5340000000000001e-06 2024-09-18T04:58:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696607+0000", 2024-09-18T04:58:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041577309 2024-09-18T04:58:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696694+0000", 2024-09-18T04:58:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7230000000000003e-05 2024-09-18T04:58:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696804+0000", 2024-09-18T04:58:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0001094 2024-09-18T04:58:19.174 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160727 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 931840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:58:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.446801047999998, 2024-09-18T04:58:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042000527000000003, 2024-09-18T04:58:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.175 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160727 2024-09-18T04:58:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:58:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:58:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.176 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655033+0000", 2024-09-18T04:58:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655036+0000", 2024-09-18T04:58:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3489999999999999e-06 2024-09-18T04:58:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655042+0000", 2024-09-18T04:58:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1399999999999997e-06 2024-09-18T04:58:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655046+0000", 2024-09-18T04:58:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5480000000000003e-06 2024-09-18T04:58:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696843+0000", 2024-09-18T04:58:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041797103000000002 2024-09-18T04:58:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696931+0000", 2024-09-18T04:58:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7526000000000002e-05 2024-09-18T04:58:19.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697034+0000", 2024-09-18T04:58:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010336999999999999 2024-09-18T04:58:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160728 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 952320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:58:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.446784491000003, 2024-09-18T04:58:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042191293999999997, 2024-09-18T04:58:19.179 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160728 2024-09-18T04:58:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:58:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:58:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:58:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655052+0000", 2024-09-18T04:58:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.232e-06 2024-09-18T04:58:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.181 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655058+0000", 2024-09-18T04:58:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9869999999999996e-06 2024-09-18T04:58:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655063+0000", 2024-09-18T04:58:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8420000000000004e-06 2024-09-18T04:58:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697074+0000", 2024-09-18T04:58:19.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042010446 2024-09-18T04:58:19.183 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697157+0000", 2024-09-18T04:58:19.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3038e-05 2024-09-18T04:58:19.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697242+0000", 2024-09-18T04:58:19.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5115000000000001e-05 2024-09-18T04:58:19.184 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.184 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.184 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.185 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160729 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 972800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:19.185 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:58:19.185 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.446767731999998, 2024-09-18T04:58:19.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042367834, 2024-09-18T04:58:19.185 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.185 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.186 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.186 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.186 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.186 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160729 2024-09-18T04:58:19.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.186 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:58:19.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:58:19.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:58:19.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655069+0000", 2024-09-18T04:58:19.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2220000000000001e-06 2024-09-18T04:58:19.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655075+0000", 2024-09-18T04:58:19.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2129999999999996e-06 2024-09-18T04:58:19.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655080+0000", 2024-09-18T04:58:19.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6840000000000004e-06 2024-09-18T04:58:19.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697280+0000", 2024-09-18T04:58:19.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042199819999999999 2024-09-18T04:58:19.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697368+0000", 2024-09-18T04:58:19.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7863999999999998e-05 2024-09-18T04:58:19.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697435+0000", 2024-09-18T04:58:19.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7416000000000006e-05 2024-09-18T04:58:19.190 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.190 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160730 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 993280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:58:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.446750967, 2024-09-18T04:58:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042559582999999998, 2024-09-18T04:58:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160730 2024-09-18T04:58:19.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.192 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:58:19.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:58:19.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:58:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655086+0000", 2024-09-18T04:58:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1500000000000002e-06 2024-09-18T04:58:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655092+0000", 2024-09-18T04:58:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.896e-06 2024-09-18T04:58:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655096+0000", 2024-09-18T04:58:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7190000000000001e-06 2024-09-18T04:58:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697475+0000", 2024-09-18T04:58:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042378644 2024-09-18T04:58:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697553+0000", 2024-09-18T04:58:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7626999999999994e-05 2024-09-18T04:58:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697643+0000", 2024-09-18T04:58:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0921000000000006e-05 2024-09-18T04:58:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160731 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 1013760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:58:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.446718236999999, 2024-09-18T04:58:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042727764000000001, 2024-09-18T04:58:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.195 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160731 2024-09-18T04:58:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:58:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:58:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655116+0000", 2024-09-18T04:58:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655119+0000", 2024-09-18T04:58:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1240000000000001e-06 2024-09-18T04:58:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655131+0000", 2024-09-18T04:58:19.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2145e-05 2024-09-18T04:58:19.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655138+0000", 2024-09-18T04:58:19.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3369999999999996e-06 2024-09-18T04:58:19.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697688+0000", 2024-09-18T04:58:19.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042550604999999998 2024-09-18T04:58:19.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697781+0000", 2024-09-18T04:58:19.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2867000000000004e-05 2024-09-18T04:58:19.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697844+0000", 2024-09-18T04:58:19.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3236000000000002e-05 2024-09-18T04:58:19.200 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.200 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.200 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.200 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160732 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 1034240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:19.200 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:58:19.200 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 42.446692388999999, 2024-09-18T04:58:19.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042857977999999998, 2024-09-18T04:58:19.200 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.200 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.200 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160732 2024-09-18T04:58:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:58:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:58:19.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:58:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655145+0000", 2024-09-18T04:58:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7329999999999999e-06 2024-09-18T04:58:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.202 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655152+0000", 2024-09-18T04:58:19.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8249999999999999e-06 2024-09-18T04:58:19.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655158+0000", 2024-09-18T04:58:19.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5010000000000001e-06 2024-09-18T04:58:19.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697873+0000", 2024-09-18T04:58:19.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042714457999999997 2024-09-18T04:58:19.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697944+0000", 2024-09-18T04:58:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1486e-05 2024-09-18T04:58:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.698000+0000", 2024-09-18T04:58:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6332000000000002e-05 2024-09-18T04:58:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.204 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.205 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.205 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:19.206 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:58:19.207 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:19.203+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:19.307 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:19.304+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:19.353 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:19.354 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:19.354 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:58:19.354 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:19.366 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:58:19.410 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:19.408+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:19.472 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:19.472 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:19.472 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:58:19.473 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:58:19.473 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:19.481 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:58:19.522 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:19.520+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:19.609 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:19.607+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:19.629 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:19.629 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:58:19.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:58:19.630 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:58:19.630 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.630 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11154 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:19.630 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:58:19.630 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 103.737164976, 2024-09-18T04:58:19.630 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040591919999999997, 2024-09-18T04:58:19.630 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.630 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.630 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.630 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.630 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.630 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11154 2024-09-18T04:58:19.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.631 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:58:19.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.631 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.631 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:58:19.631 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.631 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.631 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889400+0000", 2024-09-18T04:58:19.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889403+0000", 2024-09-18T04:58:19.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3999999999999999e-06 2024-09-18T04:58:19.632 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.632 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.632 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.632 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889409+0000", 2024-09-18T04:58:19.632 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3380000000000001e-06 2024-09-18T04:58:19.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889414+0000", 2024-09-18T04:58:19.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5789999999999997e-06 2024-09-18T04:58:19.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.633 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929811+0000", 2024-09-18T04:58:19.633 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040397486000000003 2024-09-18T04:58:19.633 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.633 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.633 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929898+0000", 2024-09-18T04:58:19.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6370000000000001e-05 2024-09-18T04:58:19.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.634 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.634 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929993+0000", 2024-09-18T04:58:19.634 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5136000000000005e-05 2024-09-18T04:58:19.634 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.634 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.634 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.634 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.634 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.634 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11155 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2015232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:19.634 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:58:19.635 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 103.737147798, 2024-09-18T04:58:19.635 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040769156000000001, 2024-09-18T04:58:19.635 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.635 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.635 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.635 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.635 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.635 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11155 2024-09-18T04:58:19.635 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.635 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.635 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.635 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:58:19.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.636 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.636 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:58:19.636 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.636 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.636 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:58:19.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.637 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.637 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.637 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.637 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889420+0000", 2024-09-18T04:58:19.637 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.181e-06 2024-09-18T04:58:19.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889426+0000", 2024-09-18T04:58:19.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8599999999999998e-06 2024-09-18T04:58:19.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.638 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.638 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889430+0000", 2024-09-18T04:58:19.638 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4399999999999998e-06 2024-09-18T04:58:19.638 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.638 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930020+0000", 2024-09-18T04:58:19.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040589698 2024-09-18T04:58:19.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.639 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.639 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930105+0000", 2024-09-18T04:58:19.639 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4812e-05 2024-09-18T04:58:19.639 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.639 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.640 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.640 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930187+0000", 2024-09-18T04:58:19.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2540999999999995e-05 2024-09-18T04:58:19.640 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.640 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.640 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.640 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.640 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.640 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11156 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2064384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:19.640 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:58:19.640 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 103.73710750399999, 2024-09-18T04:58:19.640 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040934419, 2024-09-18T04:58:19.641 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.641 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.641 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.641 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.641 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.641 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11156 2024-09-18T04:58:19.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.641 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.641 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.641 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:58:19.641 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.641 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.641 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:58:19.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889457+0000", 2024-09-18T04:58:19.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:58:19.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.642 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.642 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.642 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889462+0000", 2024-09-18T04:58:19.642 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8350000000000003e-06 2024-09-18T04:58:19.642 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889481+0000", 2024-09-18T04:58:19.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8845999999999999e-05 2024-09-18T04:58:19.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889495+0000", 2024-09-18T04:58:19.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4229e-05 2024-09-18T04:58:19.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.643 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.643 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.643 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930216+0000", 2024-09-18T04:58:19.643 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040721081999999999 2024-09-18T04:58:19.643 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930299+0000", 2024-09-18T04:58:19.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2995000000000001e-05 2024-09-18T04:58:19.645 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.645 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.645 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.645 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930393+0000", 2024-09-18T04:58:19.645 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3185999999999999e-05 2024-09-18T04:58:19.645 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.645 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.646 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.646 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.646 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.646 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11404 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1609728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:19.646 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:58:19.646 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 103.730303342, 2024-09-18T04:58:19.646 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040669012999999997, 2024-09-18T04:58:19.646 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.646 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.646 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.646 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.647 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.647 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11404 2024-09-18T04:58:19.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.647 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:58:19.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.647 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.647 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.647 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:58:19.647 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.647 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:58:19.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:58:19.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896263+0000", 2024-09-18T04:58:19.648 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2729999999999999e-06 2024-09-18T04:58:19.648 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.648 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.648 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.648 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896266+0000", 2024-09-18T04:58:19.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8200000000000001e-06 2024-09-18T04:58:19.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896269+0000", 2024-09-18T04:58:19.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3199999999999998e-06 2024-09-18T04:58:19.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.649 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936538+0000", 2024-09-18T04:58:19.649 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040269394999999999 2024-09-18T04:58:19.649 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.649 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.649 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936657+0000", 2024-09-18T04:58:19.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119498 2024-09-18T04:58:19.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.650 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.650 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936931+0000", 2024-09-18T04:58:19.650 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027391900000000002 2024-09-18T04:58:19.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.650 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.650 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.650 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.650 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.650 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11405 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1658880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:19.650 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:58:19.651 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 103.73029508400001, 2024-09-18T04:58:19.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040891034999999999, 2024-09-18T04:58:19.651 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.651 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.651 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.651 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11405 2024-09-18T04:58:19.651 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.651 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.651 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.651 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.651 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:58:19.651 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:58:19.652 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.652 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.652 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.652 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.652 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896270+0000", 2024-09-18T04:58:19.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:58:19.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896272+0000", 2024-09-18T04:58:19.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.124e-06 2024-09-18T04:58:19.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.653 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896274+0000", 2024-09-18T04:58:19.653 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8660000000000002e-06 2024-09-18T04:58:19.653 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.653 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.653 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896285+0000", 2024-09-18T04:58:19.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0101e-05 2024-09-18T04:58:19.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936986+0000", 2024-09-18T04:58:19.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040701213999999999 2024-09-18T04:58:19.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.654 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.654 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937070+0000", 2024-09-18T04:58:19.654 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4211999999999999e-05 2024-09-18T04:58:19.654 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.654 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.655 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.655 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937162+0000", 2024-09-18T04:58:19.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1728e-05 2024-09-18T04:58:19.655 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.655 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.655 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.655 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.655 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.655 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11406 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1708032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:19.655 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:58:19.655 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 103.730278915, 2024-09-18T04:58:19.655 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041085251000000003, 2024-09-18T04:58:19.655 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.655 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.656 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.656 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.656 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.656 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11406 2024-09-18T04:58:19.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.656 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:58:19.656 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.656 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.656 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.656 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.656 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:58:19.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:58:19.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:58:19.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.657 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896288+0000", 2024-09-18T04:58:19.657 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2440000000000001e-06 2024-09-18T04:58:19.657 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.657 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.657 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896291+0000", 2024-09-18T04:58:19.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8289999999999999e-06 2024-09-18T04:58:19.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896293+0000", 2024-09-18T04:58:19.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5399999999999998e-06 2024-09-18T04:58:19.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.658 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.658 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937201+0000", 2024-09-18T04:58:19.658 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040907418000000001 2024-09-18T04:58:19.658 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.658 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937283+0000", 2024-09-18T04:58:19.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2460999999999995e-05 2024-09-18T04:58:19.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.659 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.659 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937372+0000", 2024-09-18T04:58:19.659 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8987999999999996e-05 2024-09-18T04:58:19.659 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.659 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.659 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.659 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.659 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.659 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11407 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:19.660 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:58:19.660 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 103.730269095, 2024-09-18T04:58:19.660 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041438580000000003, 2024-09-18T04:58:19.660 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.660 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.660 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.660 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.660 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.660 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11407 2024-09-18T04:58:19.660 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.660 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.660 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.660 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.660 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:58:19.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:58:19.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.661 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896296+0000", 2024-09-18T04:58:19.661 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.661 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.661 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.661 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896298+0000", 2024-09-18T04:58:19.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:58:19.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.662 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.662 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896304+0000", 2024-09-18T04:58:19.662 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6570000000000002e-06 2024-09-18T04:58:19.662 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.662 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896308+0000", 2024-09-18T04:58:19.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0940000000000001e-06 2024-09-18T04:58:19.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937540+0000", 2024-09-18T04:58:19.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041231469 2024-09-18T04:58:19.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.663 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.663 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.663 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937625+0000", 2024-09-18T04:58:19.663 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5767999999999996e-05 2024-09-18T04:58:19.663 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.664 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.664 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937735+0000", 2024-09-18T04:58:19.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010974 2024-09-18T04:58:19.664 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.664 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.664 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.664 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.664 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.664 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11408 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:19.664 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:58:19.664 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 103.730253456, 2024-09-18T04:58:19.664 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041653055000000001, 2024-09-18T04:58:19.665 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.665 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.665 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.665 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.665 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.665 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11408 2024-09-18T04:58:19.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.665 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.665 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:58:19.665 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.665 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.665 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.665 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:58:19.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:58:19.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.666 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.666 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896314+0000", 2024-09-18T04:58:19.666 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0140000000000001e-06 2024-09-18T04:58:19.666 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.666 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896320+0000", 2024-09-18T04:58:19.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-18T04:58:19.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896324+0000", 2024-09-18T04:58:19.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4410000000000003e-06 2024-09-18T04:58:19.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.667 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.667 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937769+0000", 2024-09-18T04:58:19.667 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041445243 2024-09-18T04:58:19.667 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.667 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937861+0000", 2024-09-18T04:58:19.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1471000000000006e-05 2024-09-18T04:58:19.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.668 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.668 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937965+0000", 2024-09-18T04:58:19.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104516 2024-09-18T04:58:19.668 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.668 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.668 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.668 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.668 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11409 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:19.669 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:58:19.669 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 103.73023739999999, 2024-09-18T04:58:19.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041840467999999999, 2024-09-18T04:58:19.669 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.669 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.669 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.669 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.670 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.670 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11409 2024-09-18T04:58:19.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.670 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:58:19.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:58:19.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:58:19.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896331+0000", 2024-09-18T04:58:19.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5950000000000001e-06 2024-09-18T04:58:19.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896336+0000", 2024-09-18T04:58:19.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7259999999999999e-06 2024-09-18T04:58:19.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896341+0000", 2024-09-18T04:58:19.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5619999999999997e-06 2024-09-18T04:58:19.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937995+0000", 2024-09-18T04:58:19.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041654060999999999 2024-09-18T04:58:19.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.673 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.673 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938078+0000", 2024-09-18T04:58:19.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2869999999999998e-05 2024-09-18T04:58:19.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938169+0000", 2024-09-18T04:58:19.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1040999999999998e-05 2024-09-18T04:58:19.674 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.674 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.674 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.674 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11410 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1921024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:19.674 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:58:19.674 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 103.730220917, 2024-09-18T04:58:19.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042060342000000001, 2024-09-18T04:58:19.674 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.674 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.675 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.675 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.675 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.675 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11410 2024-09-18T04:58:19.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.675 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:58:19.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:58:19.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896344+0000", 2024-09-18T04:58:19.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896360+0000", 2024-09-18T04:58:19.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5068e-05 2024-09-18T04:58:19.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896366+0000", 2024-09-18T04:58:19.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5119999999999997e-06 2024-09-18T04:58:19.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896371+0000", 2024-09-18T04:58:19.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:58:19.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938198+0000", 2024-09-18T04:58:19.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041826579000000003 2024-09-18T04:58:19.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938297+0000", 2024-09-18T04:58:19.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9067000000000005e-05 2024-09-18T04:58:19.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.678 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.678 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938405+0000", 2024-09-18T04:58:19.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108492 2024-09-18T04:58:19.678 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.678 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.678 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.678 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11411 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1970176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:19.679 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:58:19.679 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 103.730190075, 2024-09-18T04:58:19.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042259191000000002, 2024-09-18T04:58:19.679 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.679 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.679 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.679 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11411 2024-09-18T04:58:19.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.679 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:58:19.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:58:19.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896375+0000", 2024-09-18T04:58:19.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896377+0000", 2024-09-18T04:58:19.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2699999999999999e-06 2024-09-18T04:58:19.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896383+0000", 2024-09-18T04:58:19.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6740000000000002e-06 2024-09-18T04:58:19.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896388+0000", 2024-09-18T04:58:19.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:58:19.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938435+0000", 2024-09-18T04:58:19.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042047169000000002 2024-09-18T04:58:19.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938534+0000", 2024-09-18T04:58:19.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8939999999999998e-05 2024-09-18T04:58:19.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.682 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.682 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938635+0000", 2024-09-18T04:58:19.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101208 2024-09-18T04:58:19.682 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.683 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.683 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.683 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11412 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:19.683 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:58:19.683 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 103.730174093, 2024-09-18T04:58:19.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042828079999999998, 2024-09-18T04:58:19.683 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.683 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.683 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.683 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.683 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.683 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11412 2024-09-18T04:58:19.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.684 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:58:19.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:58:19.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896391+0000", 2024-09-18T04:58:19.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896393+0000", 2024-09-18T04:58:19.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0810000000000001e-06 2024-09-18T04:58:19.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896399+0000", 2024-09-18T04:58:19.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7230000000000001e-06 2024-09-18T04:58:19.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896403+0000", 2024-09-18T04:58:19.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3470000000000001e-06 2024-09-18T04:58:19.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939043+0000", 2024-09-18T04:58:19.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042639829999999997 2024-09-18T04:58:19.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939124+0000", 2024-09-18T04:58:19.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0809000000000005e-05 2024-09-18T04:58:19.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939220+0000", 2024-09-18T04:58:19.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5659e-05 2024-09-18T04:58:19.687 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.687 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.687 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.687 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11413 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2068480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:19.687 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:58:19.687 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 103.730158398, 2024-09-18T04:58:19.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043042108000000003, 2024-09-18T04:58:19.687 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.687 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.688 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.688 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11413 2024-09-18T04:58:19.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.688 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:58:19.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:58:19.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:58:19.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896409+0000", 2024-09-18T04:58:19.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.165e-06 2024-09-18T04:58:19.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896415+0000", 2024-09-18T04:58:19.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6149999999999996e-06 2024-09-18T04:58:19.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896419+0000", 2024-09-18T04:58:19.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2649999999999998e-06 2024-09-18T04:58:19.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939262+0000", 2024-09-18T04:58:19.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042843444000000001 2024-09-18T04:58:19.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939351+0000", 2024-09-18T04:58:19.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8944999999999996e-05 2024-09-18T04:58:19.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.691 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.691 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939449+0000", 2024-09-18T04:58:19.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8029000000000006e-05 2024-09-18T04:58:19.691 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.691 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.691 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.691 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145409 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:19.692 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:58:19.692 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 46.098738750000003, 2024-09-18T04:58:19.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040684845999999997, 2024-09-18T04:58:19.692 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.692 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.692 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.692 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.692 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.692 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145409 2024-09-18T04:58:19.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.692 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:58:19.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:58:19.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:58:19.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527829+0000", 2024-09-18T04:58:19.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.441e-06 2024-09-18T04:58:19.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527835+0000", 2024-09-18T04:58:19.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1650000000000003e-06 2024-09-18T04:58:19.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527840+0000", 2024-09-18T04:58:19.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8250000000000004e-06 2024-09-18T04:58:19.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568404+0000", 2024-09-18T04:58:19.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040563872000000001 2024-09-18T04:58:19.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568457+0000", 2024-09-18T04:58:19.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2713999999999997e-05 2024-09-18T04:58:19.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.695 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.695 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568512+0000", 2024-09-18T04:58:19.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5148000000000001e-05 2024-09-18T04:58:19.695 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.695 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.696 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.696 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145411 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:19.696 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:58:19.696 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 46.098720888999999, 2024-09-18T04:58:19.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040792108000000001, 2024-09-18T04:58:19.696 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.696 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.696 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.696 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.697 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.697 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145411 2024-09-18T04:58:19.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.697 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:58:19.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:58:19.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527844+0000", 2024-09-18T04:58:19.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527847+0000", 2024-09-18T04:58:19.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0260000000000002e-06 2024-09-18T04:58:19.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527852+0000", 2024-09-18T04:58:19.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9889999999999997e-06 2024-09-18T04:58:19.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527857+0000", 2024-09-18T04:58:19.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6410000000000001e-06 2024-09-18T04:58:19.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568537+0000", 2024-09-18T04:58:19.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040679743999999997 2024-09-18T04:58:19.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568583+0000", 2024-09-18T04:58:19.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6286999999999999e-05 2024-09-18T04:58:19.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.700 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.700 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568637+0000", 2024-09-18T04:58:19.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.376e-05 2024-09-18T04:58:19.700 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.700 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.700 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.701 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145413 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1859584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:19.701 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:58:19.701 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 46.098704509000001, 2024-09-18T04:58:19.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040919516000000003, 2024-09-18T04:58:19.701 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.701 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.701 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.701 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.701 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.701 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145413 2024-09-18T04:58:19.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.701 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:58:19.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:58:19.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:58:19.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527863+0000", 2024-09-18T04:58:19.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2630000000000002e-06 2024-09-18T04:58:19.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527869+0000", 2024-09-18T04:58:19.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9009999999999999e-06 2024-09-18T04:58:19.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527874+0000", 2024-09-18T04:58:19.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0590000000000002e-06 2024-09-18T04:58:19.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568663+0000", 2024-09-18T04:58:19.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040788805999999997 2024-09-18T04:58:19.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568716+0000", 2024-09-18T04:58:19.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3501999999999997e-05 2024-09-18T04:58:19.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568781+0000", 2024-09-18T04:58:19.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4338000000000006e-05 2024-09-18T04:58:19.704 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.705 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.705 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.705 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145414 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1908736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:19.705 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:58:19.705 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 46.098687550999998, 2024-09-18T04:58:19.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041030234999999998, 2024-09-18T04:58:19.705 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.705 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.705 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.705 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.705 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.705 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145414 2024-09-18T04:58:19.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.706 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:58:19.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:58:19.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:58:19.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527880+0000", 2024-09-18T04:58:19.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4770000000000002e-06 2024-09-18T04:58:19.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527886+0000", 2024-09-18T04:58:19.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5759999999999996e-06 2024-09-18T04:58:19.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527890+0000", 2024-09-18T04:58:19.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8180000000000003e-06 2024-09-18T04:58:19.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568807+0000", 2024-09-18T04:58:19.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040916984000000003 2024-09-18T04:58:19.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568854+0000", 2024-09-18T04:58:19.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6740999999999999e-05 2024-09-18T04:58:19.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.709 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.709 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568908+0000", 2024-09-18T04:58:19.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4228000000000003e-05 2024-09-18T04:58:19.709 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.709 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.709 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.709 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145416 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1957888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:19.709 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:58:19.709 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 46.098671058999997, 2024-09-18T04:58:19.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041147364999999998, 2024-09-18T04:58:19.709 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.710 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.710 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.710 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.710 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.710 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145416 2024-09-18T04:58:19.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.710 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:58:19.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:58:19.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527894+0000", 2024-09-18T04:58:19.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527896+0000", 2024-09-18T04:58:19.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1789999999999998e-06 2024-09-18T04:58:19.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527908+0000", 2024-09-18T04:58:19.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1780000000000001e-05 2024-09-18T04:58:19.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527914+0000", 2024-09-18T04:58:19.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2029999999999997e-06 2024-09-18T04:58:19.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568935+0000", 2024-09-18T04:58:19.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041020273000000003 2024-09-18T04:58:19.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568988+0000", 2024-09-18T04:58:19.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2911000000000002e-05 2024-09-18T04:58:19.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569042+0000", 2024-09-18T04:58:19.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4382000000000002e-05 2024-09-18T04:58:19.714 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.714 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.714 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.714 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145418 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 2007040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:19.714 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:58:19.714 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 46.098646662, 2024-09-18T04:58:19.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041247862000000003, 2024-09-18T04:58:19.714 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.714 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.714 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.715 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.715 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.715 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145418 2024-09-18T04:58:19.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.715 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:58:19.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:58:19.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:58:19.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527921+0000", 2024-09-18T04:58:19.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.667e-06 2024-09-18T04:58:19.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527929+0000", 2024-09-18T04:58:19.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.542e-06 2024-09-18T04:58:19.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527934+0000", 2024-09-18T04:58:19.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1440000000000002e-06 2024-09-18T04:58:19.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569068+0000", 2024-09-18T04:58:19.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041134190000000001 2024-09-18T04:58:19.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569114+0000", 2024-09-18T04:58:19.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6261999999999999e-05 2024-09-18T04:58:19.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.718 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.718 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569167+0000", 2024-09-18T04:58:19.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2506999999999997e-05 2024-09-18T04:58:19.718 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.718 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.718 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.719 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145420 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 2056192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:19.719 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:58:19.719 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 46.098627587999999, 2024-09-18T04:58:19.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041351472, 2024-09-18T04:58:19.719 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:19.719 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:19.719 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:19.719 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:19.719 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:19.719 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145420 2024-09-18T04:58:19.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.719 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:19.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:19.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:58:19.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:19.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:58:19.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:19.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:19.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:58:19.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:19.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:19.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527940+0000", 2024-09-18T04:58:19.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6570000000000001e-06 2024-09-18T04:58:19.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:19.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527948+0000", 2024-09-18T04:58:19.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8350000000000004e-06 2024-09-18T04:58:19.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:19.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527953+0000", 2024-09-18T04:58:19.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0690000000000001e-06 2024-09-18T04:58:19.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.722 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:19.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569182+0000", 2024-09-18T04:58:19.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041228734000000003 2024-09-18T04:58:19.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:19.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569234+0000", 2024-09-18T04:58:19.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1663999999999999e-05 2024-09-18T04:58:19.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:19.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:19.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:19.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569290+0000", 2024-09-18T04:58:19.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5937000000000003e-05 2024-09-18T04:58:19.724 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.724 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.724 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.724 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:19.724 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:19.724 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:19.726 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:58:19.726 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:19.723+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:19.826 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:19.824+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:19.828 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:19.828 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:19.828 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:58:19.828 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:19.837 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:58:19.928 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:19.926+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:19.938 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:19.938 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:19.938 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:58:19.939 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:58:19.939 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:19.947 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:58:20.029 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:20.027+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:20.049 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:20.050 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:58:20.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:58:20.050 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:20.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.050 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72135 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 394240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:20.050 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:58:20.050 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.092347516000004, 2024-09-18T04:58:20.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048559739999999997, 2024-09-18T04:58:20.050 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:20.050 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:20.050 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:20.050 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:20.051 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:20.051 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72135 2024-09-18T04:58:20.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.051 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:20.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:20.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:58:20.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:20.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:58:20.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:20.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:58:20.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:20.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:20.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954005+0000", 2024-09-18T04:58:20.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5629999999999999e-06 2024-09-18T04:58:20.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:20.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954008+0000", 2024-09-18T04:58:20.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-18T04:58:20.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:20.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954012+0000", 2024-09-18T04:58:20.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4790000000000001e-06 2024-09-18T04:58:20.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:20.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002418+0000", 2024-09-18T04:58:20.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048406049 2024-09-18T04:58:20.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:20.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002460+0000", 2024-09-18T04:58:20.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2188999999999999e-05 2024-09-18T04:58:20.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:20.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002564+0000", 2024-09-18T04:58:20.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010377 2024-09-18T04:58:20.054 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.054 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:20.054 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.055 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72136 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 409600~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:20.055 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:58:20.055 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.092242569000007, 2024-09-18T04:58:20.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048670191000000002, 2024-09-18T04:58:20.055 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:20.055 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:20.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:20.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:20.055 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:20.055 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72136 2024-09-18T04:58:20.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.055 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:20.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:20.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:58:20.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:20.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:58:20.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:20.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954019+0000", 2024-09-18T04:58:20.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999099 2024-09-18T04:58:20.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:20.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954112+0000", 2024-09-18T04:58:20.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3273e-05 2024-09-18T04:58:20.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:20.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954121+0000", 2024-09-18T04:58:20.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0289999999999994e-06 2024-09-18T04:58:20.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:20.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954128+0000", 2024-09-18T04:58:20.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6189999999999996e-06 2024-09-18T04:58:20.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:20.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002604+0000", 2024-09-18T04:58:20.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048476048000000001 2024-09-18T04:58:20.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:20.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002682+0000", 2024-09-18T04:58:20.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7955999999999997e-05 2024-09-18T04:58:20.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:20.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002779+0000", 2024-09-18T04:58:20.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7372000000000002e-05 2024-09-18T04:58:20.059 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.059 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:20.059 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.059 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72138 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 440320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:20.059 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:58:20.059 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.091808717000006, 2024-09-18T04:58:20.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048549430999999997, 2024-09-18T04:58:20.059 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:20.059 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:20.059 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:20.059 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:20.060 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:20.060 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72138 2024-09-18T04:58:20.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.060 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:20.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:20.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:58:20.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:20.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:58:20.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:20.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954542+0000", 2024-09-18T04:58:20.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:20.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:20.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954545+0000", 2024-09-18T04:58:20.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4360000000000001e-06 2024-09-18T04:58:20.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:20.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954551+0000", 2024-09-18T04:58:20.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4949999999999996e-06 2024-09-18T04:58:20.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:20.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954557+0000", 2024-09-18T04:58:20.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5099999999999998e-06 2024-09-18T04:58:20.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:20.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002993+0000", 2024-09-18T04:58:20.062 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048436358999999998 2024-09-18T04:58:20.062 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.062 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.062 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:20.062 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003039+0000", 2024-09-18T04:58:20.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5884000000000003e-05 2024-09-18T04:58:20.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.063 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:20.063 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003092+0000", 2024-09-18T04:58:20.063 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3208000000000003e-05 2024-09-18T04:58:20.063 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.063 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:20.063 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.063 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.063 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.063 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72139 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 455680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:20.063 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:58:20.063 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.091789908999999, 2024-09-18T04:58:20.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048718773, 2024-09-18T04:58:20.064 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:20.064 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:20.064 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:20.064 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:20.064 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:20.064 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72139 2024-09-18T04:58:20.064 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.064 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:20.064 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.064 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:20.064 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:58:20.064 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:20.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:58:20.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:20.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954561+0000", 2024-09-18T04:58:20.065 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:20.065 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.065 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.065 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:20.065 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954564+0000", 2024-09-18T04:58:20.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4099999999999998e-06 2024-09-18T04:58:20.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:20.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954570+0000", 2024-09-18T04:58:20.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.178e-06 2024-09-18T04:58:20.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.066 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:20.066 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954575+0000", 2024-09-18T04:58:20.066 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0560000000000004e-06 2024-09-18T04:58:20.066 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.066 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:20.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003114+0000", 2024-09-18T04:58:20.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048539212999999998 2024-09-18T04:58:20.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:20.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003176+0000", 2024-09-18T04:58:20.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2154000000000002e-05 2024-09-18T04:58:20.067 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.067 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.067 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:20.067 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003280+0000", 2024-09-18T04:58:20.067 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010421600000000001 2024-09-18T04:58:20.067 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.068 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:20.068 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.068 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.068 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.068 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72140 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 471040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:20.068 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:58:20.068 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.091533967999993, 2024-09-18T04:58:20.068 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048670819999999997, 2024-09-18T04:58:20.068 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:20.068 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:20.068 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:20.068 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:20.068 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:20.068 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72140 2024-09-18T04:58:20.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.069 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:20.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:20.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:58:20.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.069 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:20.069 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:58:20.069 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.069 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.069 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:20.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954579+0000", 2024-09-18T04:58:20.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997611 2024-09-18T04:58:20.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:20.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954821+0000", 2024-09-18T04:58:20.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024241000000000001 2024-09-18T04:58:20.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.070 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.070 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:20.070 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954831+0000", 2024-09-18T04:58:20.070 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8770000000000005e-06 2024-09-18T04:58:20.070 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:20.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954838+0000", 2024-09-18T04:58:20.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0690000000000004e-06 2024-09-18T04:58:20.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:20.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003317+0000", 2024-09-18T04:58:20.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048478796999999997 2024-09-18T04:58:20.071 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.071 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.071 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:20.071 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003392+0000", 2024-09-18T04:58:20.071 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4411000000000003e-05 2024-09-18T04:58:20.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.072 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:20.072 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003488+0000", 2024-09-18T04:58:20.072 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6927000000000002e-05 2024-09-18T04:58:20.072 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.072 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:20.072 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.072 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.072 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.073 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72141 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 486400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:20.073 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:58:20.073 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.091506484999996, 2024-09-18T04:58:20.073 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048808114, 2024-09-18T04:58:20.073 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:20.073 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:20.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:20.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:20.073 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:20.073 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72141 2024-09-18T04:58:20.073 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.073 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:20.073 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.073 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:20.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:58:20.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.074 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.074 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:20.074 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:58:20.074 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.074 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:20.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:58:20.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:20.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:20.075 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954847+0000", 2024-09-18T04:58:20.075 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3800000000000001e-06 2024-09-18T04:58:20.075 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.075 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.075 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:20.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954854+0000", 2024-09-18T04:58:20.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6080000000000001e-06 2024-09-18T04:58:20.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:20.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954859+0000", 2024-09-18T04:58:20.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:58:20.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.076 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.076 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:20.076 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003526+0000", 2024-09-18T04:58:20.076 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048666433000000002 2024-09-18T04:58:20.076 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:20.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003599+0000", 2024-09-18T04:58:20.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2854000000000004e-05 2024-09-18T04:58:20.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.077 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:20.077 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003653+0000", 2024-09-18T04:58:20.077 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4511e-05 2024-09-18T04:58:20.077 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.077 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:20.077 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.077 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.077 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.078 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72142 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 501760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:20.078 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:58:20.078 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.091467683000005, 2024-09-18T04:58:20.078 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048885500999999998, 2024-09-18T04:58:20.078 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:20.078 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:20.078 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:20.078 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:20.078 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:20.078 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72142 2024-09-18T04:58:20.078 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.078 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:20.078 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.078 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:20.078 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:58:20.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:20.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:58:20.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.079 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:20.079 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954864+0000", 2024-09-18T04:58:20.079 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999804 2024-09-18T04:58:20.079 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.079 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:20.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954886+0000", 2024-09-18T04:58:20.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2042999999999999e-05 2024-09-18T04:58:20.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:20.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954893+0000", 2024-09-18T04:58:20.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.255e-06 2024-09-18T04:58:20.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.080 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.080 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:20.080 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954899+0000", 2024-09-18T04:58:20.080 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8780000000000003e-06 2024-09-18T04:58:20.080 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:20.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003678+0000", 2024-09-18T04:58:20.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048778834 2024-09-18T04:58:20.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:20.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003717+0000", 2024-09-18T04:58:20.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8860999999999999e-05 2024-09-18T04:58:20.081 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.081 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.081 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:20.081 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003769+0000", 2024-09-18T04:58:20.081 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2389000000000003e-05 2024-09-18T04:58:20.082 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.082 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:20.082 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.082 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.082 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.082 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72143 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 517120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:20.082 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:58:20.082 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.091342057000006, 2024-09-18T04:58:20.082 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048951738000000002, 2024-09-18T04:58:20.082 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:20.082 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:20.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:20.082 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:20.083 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:20.083 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72143 2024-09-18T04:58:20.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.083 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:20.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:20.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:58:20.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.083 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:20.083 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:58:20.083 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.083 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.083 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:20.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954904+0000", 2024-09-18T04:58:20.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998946 2024-09-18T04:58:20.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:20.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955013+0000", 2024-09-18T04:58:20.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108977 2024-09-18T04:58:20.084 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.084 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.084 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:20.084 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955021+0000", 2024-09-18T04:58:20.084 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7570000000000004e-06 2024-09-18T04:58:20.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:20.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955025+0000", 2024-09-18T04:58:20.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1219999999999997e-06 2024-09-18T04:58:20.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:20.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003792+0000", 2024-09-18T04:58:20.085 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048767379 2024-09-18T04:58:20.085 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.085 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.085 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:20.085 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003860+0000", 2024-09-18T04:58:20.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8033000000000004e-05 2024-09-18T04:58:20.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.086 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:20.086 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003961+0000", 2024-09-18T04:58:20.086 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010103300000000001 2024-09-18T04:58:20.086 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.086 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:20.086 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.086 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.086 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.086 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72161 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 13312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:20.086 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:58:20.086 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.091078420000002, 2024-09-18T04:58:20.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048891539999999997, 2024-09-18T04:58:20.087 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:20.087 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:20.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:20.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:20.087 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:20.087 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72161 2024-09-18T04:58:20.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.087 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:20.087 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.087 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:20.087 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:58:20.087 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.087 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:20.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:58:20.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:20.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955028+0000", 2024-09-18T04:58:20.088 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997549 2024-09-18T04:58:20.088 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.088 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.088 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:20.088 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955278+0000", 2024-09-18T04:58:20.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024973299999999998 2024-09-18T04:58:20.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:20.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955289+0000", 2024-09-18T04:58:20.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0812000000000001e-05 2024-09-18T04:58:20.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:20.089 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955296+0000", 2024-09-18T04:58:20.089 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2570000000000001e-06 2024-09-18T04:58:20.089 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.089 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.089 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:20.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004002+0000", 2024-09-18T04:58:20.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048705904000000001 2024-09-18T04:58:20.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:20.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004076+0000", 2024-09-18T04:58:20.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4086999999999997e-05 2024-09-18T04:58:20.090 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.090 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.090 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:20.090 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004165+0000", 2024-09-18T04:58:20.090 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8712e-05 2024-09-18T04:58:20.090 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.090 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:20.091 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.091 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.091 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72163 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 44032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:20.091 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:58:20.091 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.090452982000002, 2024-09-18T04:58:20.091 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048567011, 2024-09-18T04:58:20.091 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:20.091 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:20.091 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:20.091 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:20.091 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:20.091 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72163 2024-09-18T04:58:20.091 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.091 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:20.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:20.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:58:20.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:20.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:58:20.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.092 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.092 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.092 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:20.092 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955898+0000", 2024-09-18T04:58:20.092 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:20.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:20.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955901+0000", 2024-09-18T04:58:20.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.706e-06 2024-09-18T04:58:20.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:20.093 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955907+0000", 2024-09-18T04:58:20.093 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3559999999999998e-06 2024-09-18T04:58:20.093 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.093 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.093 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:20.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955912+0000", 2024-09-18T04:58:20.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9640000000000002e-06 2024-09-18T04:58:20.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:20.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004377+0000", 2024-09-18T04:58:20.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048465293999999999 2024-09-18T04:58:20.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.094 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:20.094 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004418+0000", 2024-09-18T04:58:20.094 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0163000000000002e-05 2024-09-18T04:58:20.094 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.094 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.095 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:20.095 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004466+0000", 2024-09-18T04:58:20.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8004000000000003e-05 2024-09-18T04:58:20.095 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.095 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:20.095 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.095 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.095 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.095 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72164 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:20.095 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:58:20.095 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.090434708999993, 2024-09-18T04:58:20.095 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048707253999999998, 2024-09-18T04:58:20.095 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:20.095 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:20.096 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:20.096 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:20.096 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:20.096 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72164 2024-09-18T04:58:20.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.096 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:20.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:20.096 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:58:20.096 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.096 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.096 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.096 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:20.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:58:20.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:20.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:58:20.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:20.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.097 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.097 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:20.097 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955919+0000", 2024-09-18T04:58:20.097 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2520000000000002e-06 2024-09-18T04:58:20.097 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:20.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955925+0000", 2024-09-18T04:58:20.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3740000000000003e-06 2024-09-18T04:58:20.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:20.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955930+0000", 2024-09-18T04:58:20.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:58:20.098 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.098 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.098 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:20.098 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004489+0000", 2024-09-18T04:58:20.098 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048558581000000003 2024-09-18T04:58:20.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:20.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004529+0000", 2024-09-18T04:58:20.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9722999999999999e-05 2024-09-18T04:58:20.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.099 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.099 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:20.099 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004624+0000", 2024-09-18T04:58:20.099 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5402999999999994e-05 2024-09-18T04:58:20.099 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.099 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:20.099 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.099 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.100 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72165 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 74752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:20.100 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:58:20.100 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.090159850000006, 2024-09-18T04:58:20.100 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048633223000000003, 2024-09-18T04:58:20.100 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:20.100 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:20.100 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:20.100 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:20.100 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:20.100 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72165 2024-09-18T04:58:20.100 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.100 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:20.100 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:20.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:58:20.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:20.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:58:20.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.101 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.101 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:20.101 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955935+0000", 2024-09-18T04:58:20.101 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999743 2024-09-18T04:58:20.101 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:20.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956199+0000", 2024-09-18T04:58:20.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000263906 2024-09-18T04:58:20.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:20.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956207+0000", 2024-09-18T04:58:20.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2519999999999999e-06 2024-09-18T04:58:20.102 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.102 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.102 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:20.102 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956214+0000", 2024-09-18T04:58:20.102 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7610000000000002e-06 2024-09-18T04:58:20.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:20.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004661+0000", 2024-09-18T04:58:20.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048447967000000002 2024-09-18T04:58:20.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:20.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004735+0000", 2024-09-18T04:58:20.103 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3789999999999997e-05 2024-09-18T04:58:20.103 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.103 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.103 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:20.103 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004825+0000", 2024-09-18T04:58:20.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9724999999999999e-05 2024-09-18T04:58:20.104 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.104 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:20.104 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.104 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.104 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.104 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72166 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 90112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:20.104 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:58:20.104 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.089906315999997, 2024-09-18T04:58:20.104 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048595737999999999, 2024-09-18T04:58:20.104 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:20.104 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:20.104 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:20.104 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:20.105 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:20.105 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72166 2024-09-18T04:58:20.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.105 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:20.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:20.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:58:20.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.105 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.105 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:20.105 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:58:20.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.105 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:20.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956221+0000", 2024-09-18T04:58:20.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997759 2024-09-18T04:58:20.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:20.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956448+0000", 2024-09-18T04:58:20.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022685500000000001 2024-09-18T04:58:20.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.106 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:20.106 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956456+0000", 2024-09-18T04:58:20.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0099999999999995e-06 2024-09-18T04:58:20.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:20.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956462+0000", 2024-09-18T04:58:20.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7200000000000003e-06 2024-09-18T04:58:20.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:20.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004862+0000", 2024-09-18T04:58:20.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048399795000000002 2024-09-18T04:58:20.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:20.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004941+0000", 2024-09-18T04:58:20.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9414999999999998e-05 2024-09-18T04:58:20.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:20.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005041+0000", 2024-09-18T04:58:20.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9942e-05 2024-09-18T04:58:20.108 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.108 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:20.108 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.108 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72167 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:20.108 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:58:20.109 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.089819277999993, 2024-09-18T04:58:20.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048622894999999999, 2024-09-18T04:58:20.109 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:20.109 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:20.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:20.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:20.109 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:20.109 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72167 2024-09-18T04:58:20.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.109 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:20.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:20.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:58:20.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:20.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:58:20.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:20.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956468+0000", 2024-09-18T04:58:20.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999352 2024-09-18T04:58:20.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:20.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956535+0000", 2024-09-18T04:58:20.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7440999999999993e-05 2024-09-18T04:58:20.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:20.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956544+0000", 2024-09-18T04:58:20.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1039999999999996e-06 2024-09-18T04:58:20.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:20.111 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956550+0000", 2024-09-18T04:58:20.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9850000000000003e-06 2024-09-18T04:58:20.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.111 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:20.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005066+0000", 2024-09-18T04:58:20.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048515614999999998 2024-09-18T04:58:20.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:20.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005107+0000", 2024-09-18T04:58:20.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0843999999999997e-05 2024-09-18T04:58:20.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:20.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005155+0000", 2024-09-18T04:58:20.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8686000000000001e-05 2024-09-18T04:58:20.112 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.112 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:20.113 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.113 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72168 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:20.113 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:58:20.113 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.089658215, 2024-09-18T04:58:20.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048585179999999999, 2024-09-18T04:58:20.113 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:20.113 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:20.113 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:20.114 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:20.114 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:20.114 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72168 2024-09-18T04:58:20.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.114 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:20.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:20.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:58:20.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:20.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:58:20.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:20.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956555+0000", 2024-09-18T04:58:20.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998612 2024-09-18T04:58:20.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:20.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956696+0000", 2024-09-18T04:58:20.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000140922 2024-09-18T04:58:20.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:20.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956707+0000", 2024-09-18T04:58:20.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1555e-05 2024-09-18T04:58:20.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:20.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956714+0000", 2024-09-18T04:58:20.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2709999999999998e-06 2024-09-18T04:58:20.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.116 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:20.116 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005177+0000", 2024-09-18T04:58:20.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048463421 2024-09-18T04:58:20.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:20.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005214+0000", 2024-09-18T04:58:20.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.676e-05 2024-09-18T04:58:20.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:20.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005279+0000", 2024-09-18T04:58:20.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4899000000000003e-05 2024-09-18T04:58:20.117 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.117 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:20.117 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.118 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72172 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 441344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:20.118 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.957556+0000", 2024-09-18T04:58:20.118 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.088795814999997, 2024-09-18T04:58:20.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048545056000000003, 2024-09-18T04:58:20.118 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:20.118 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:20.118 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:20.118 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:20.118 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:20.118 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72172 2024-09-18T04:58:20.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.118 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:20.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:20.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957556+0000", 2024-09-18T04:58:20.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:20.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957556+0000", 2024-09-18T04:58:20.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:20.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957555+0000", 2024-09-18T04:58:20.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:20.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:20.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957558+0000", 2024-09-18T04:58:20.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2900000000000001e-06 2024-09-18T04:58:20.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:20.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957564+0000", 2024-09-18T04:58:20.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0700000000000003e-06 2024-09-18T04:58:20.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:20.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957569+0000", 2024-09-18T04:58:20.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7870000000000001e-06 2024-09-18T04:58:20.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:20.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005922+0000", 2024-09-18T04:58:20.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048353259000000003 2024-09-18T04:58:20.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:20.121 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005999+0000", 2024-09-18T04:58:20.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6889000000000002e-05 2024-09-18T04:58:20.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.121 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:20.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.006101+0000", 2024-09-18T04:58:20.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102136 2024-09-18T04:58:20.122 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.122 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:20.122 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.122 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72199 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:20.122 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:58:20.122 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.087126323000007, 2024-09-18T04:58:20.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048626057, 2024-09-18T04:58:20.122 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:20.122 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:20.122 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:20.123 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:20.123 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:20.123 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72199 2024-09-18T04:58:20.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.123 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:20.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:20.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:58:20.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:20.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:58:20.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:20.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959195+0000", 2024-09-18T04:58:20.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999969 2024-09-18T04:58:20.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:20.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959228+0000", 2024-09-18T04:58:20.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3609000000000003e-05 2024-09-18T04:58:20.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:20.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959236+0000", 2024-09-18T04:58:20.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.157e-06 2024-09-18T04:58:20.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:20.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959243+0000", 2024-09-18T04:58:20.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5910000000000001e-06 2024-09-18T04:58:20.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:20.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007684+0000", 2024-09-18T04:58:20.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048440745 2024-09-18T04:58:20.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:20.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007762+0000", 2024-09-18T04:58:20.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8360000000000002e-05 2024-09-18T04:58:20.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:20.126 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007851+0000", 2024-09-18T04:58:20.126 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9388000000000005e-05 2024-09-18T04:58:20.126 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.126 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:20.126 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.126 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72240 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 428032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:20.127 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:58:20.127 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 83.083033013000005, 2024-09-18T04:58:20.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048643767999999997, 2024-09-18T04:58:20.127 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:20.127 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:20.127 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:20.127 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:20.127 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:20.127 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72240 2024-09-18T04:58:20.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.127 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:20.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:20.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:58:20.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:20.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:58:20.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:20.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963240+0000", 2024-09-18T04:58:20.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999218 2024-09-18T04:58:20.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:20.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963323+0000", 2024-09-18T04:58:20.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2375999999999999e-05 2024-09-18T04:58:20.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:20.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963332+0000", 2024-09-18T04:58:20.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1700000000000003e-06 2024-09-18T04:58:20.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:20.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963338+0000", 2024-09-18T04:58:20.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2149999999999998e-06 2024-09-18T04:58:20.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:20.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011785+0000", 2024-09-18T04:58:20.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048446493 2024-09-18T04:58:20.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:20.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011861+0000", 2024-09-18T04:58:20.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6472999999999998e-05 2024-09-18T04:58:20.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:20.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011962+0000", 2024-09-18T04:58:20.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101167 2024-09-18T04:58:20.134 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.134 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:20.134 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.134 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:117375 3.1 3:80b18d38:::rbd_header.11acae5d5365:head [watch reconnect cookie 140406307615440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T04:58:20.134 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:58:20.134 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 56.661958476000002, 2024-09-18T04:58:20.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.018908156, 2024-09-18T04:58:20.135 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:20.135 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:20.135 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:20.135 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:20.135 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:20.135 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117375 2024-09-18T04:58:20.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.135 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:20.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:20.135 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:58:20.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:20.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:58:20.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:20.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384391+0000", 2024-09-18T04:58:20.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:58:20.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:20.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384401+0000", 2024-09-18T04:58:20.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4299999999999995e-06 2024-09-18T04:58:20.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:20.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384418+0000", 2024-09-18T04:58:20.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6745e-05 2024-09-18T04:58:20.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:20.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384429+0000", 2024-09-18T04:58:20.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1122e-05 2024-09-18T04:58:20.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:20.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384496+0000", 2024-09-18T04:58:20.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6841000000000006e-05 2024-09-18T04:58:20.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-18T04:58:20.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384505+0000", 2024-09-18T04:58:20.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4080000000000004e-06 2024-09-18T04:58:20.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:20.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.388971+0000", 2024-09-18T04:58:20.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0044660000000000004 2024-09-18T04:58:20.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:20.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.388981+0000", 2024-09-18T04:58:20.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0118e-05 2024-09-18T04:58:20.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:20.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.399433+0000", 2024-09-18T04:58:20.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.010452055 2024-09-18T04:58:20.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.139 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:20.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.399594+0000", 2024-09-18T04:58:20.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000160669 2024-09-18T04:58:20.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:20.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.403301+0000", 2024-09-18T04:58:20.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037074370000000001 2024-09-18T04:58:20.140 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.140 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:20.140 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.140 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:117376 3.1 3:80b18d38:::rbd_header.11acae5d5365:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T04:58:20.140 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:58:20.140 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 56.659906825999997, 2024-09-18T04:58:20.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0196134539999999, 2024-09-18T04:58:20.141 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:20.141 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:20.141 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:20.141 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:20.141 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:20.141 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117376 2024-09-18T04:58:20.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.141 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:20.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:20.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:58:20.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:20.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:58:20.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:20.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:20.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386443+0000", 2024-09-18T04:58:20.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:58:20.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:20.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386457+0000", 2024-09-18T04:58:20.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4207e-05 2024-09-18T04:58:20.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:20.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386478+0000", 2024-09-18T04:58:20.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0349000000000001e-05 2024-09-18T04:58:20.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:20.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386503+0000", 2024-09-18T04:58:20.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5582e-05 2024-09-18T04:58:20.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:20.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386558+0000", 2024-09-18T04:58:20.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4892999999999998e-05 2024-09-18T04:58:20.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-18T04:58:20.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386562+0000", 2024-09-18T04:58:20.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.313e-06 2024-09-18T04:58:20.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:20.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.389010+0000", 2024-09-18T04:58:20.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024480890000000001 2024-09-18T04:58:20.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:20.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.389018+0000", 2024-09-18T04:58:20.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8469999999999996e-06 2024-09-18T04:58:20.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:20.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400073+0000", 2024-09-18T04:58:20.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.011054286 2024-09-18T04:58:20.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:20.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400124+0000", 2024-09-18T04:58:20.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1332999999999998e-05 2024-09-18T04:58:20.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:58:20.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400477+0000", 2024-09-18T04:58:20.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035329300000000002 2024-09-18T04:58:20.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:20.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.401353+0000", 2024-09-18T04:58:20.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00087584200000000005 2024-09-18T04:58:20.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:20.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406006+0000", 2024-09-18T04:58:20.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0046534250000000001 2024-09-18T04:58:20.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:20.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406037+0000", 2024-09-18T04:58:20.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0559000000000003e-05 2024-09-18T04:58:20.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:20.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:20.148 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:20.148 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406058+0000", 2024-09-18T04:58:20.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1260999999999999e-05 2024-09-18T04:58:20.148 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.148 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:20.149 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.149 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:20.149 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:20.149 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:20.151 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-18T04:58:20.151 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:58:20.151 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:20.148+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:20.251 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:20.249+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:20.294 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:20.294 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:20.294 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:58:20.294 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:20.303 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:58:20.319 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:58:20.387 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:20.384+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:20.444 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:20.444 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:20.444 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:58:20.444 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:58:20.445 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:20.448 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:58:20.449 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:58:20.453 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:58:20.453 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:20.450+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:20.465 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:58:20.466 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:58:20.482 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:58:20.482 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:58:20.498 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:58:20.498 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:58:20.515 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:58:20.515 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:58:20.532 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:58:20.532 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:58:20.548 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:58:20.549 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:58:20.554 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:20.552+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:20.565 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:58:20.565 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:58:20.596 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:20.597 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:20.606 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:58:20.655 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:20.653+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:20.708 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:20.708 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:20.716 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:58:20.756 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:20.754+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:20.818 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:20.818 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:20.826 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:58:20.857 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:20.855+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:20.928 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:20.929 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:20.938 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:58:20.981 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:20.979+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:21.039 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:21.039 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:21.048 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:58:21.058 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:21.056+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:21.070 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 5, 6, 7] out_osds: [1, 4, 3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T04:58:21.070 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T04:58:21.070 INFO:tasks.thrashosds.thrasher:Adding osd 4 2024-09-18T04:58:21.070 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 4 2024-09-18T04:58:21.159 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:21.157+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:21.192 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:21.192 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:21.201 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:58:21.285 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:21.283+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:21.344 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:21.344 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:21.352 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:58:21.382 INFO:tasks.ceph.mon.a.smithi002.stderr:2024-09-18T04:58:21.379+0000 7f7d3b87f640 -1 mon.a@0(leader).osd e131 definitely_dead 0 2024-09-18T04:58:21.393 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:21.391+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:21.460 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:21.460 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:21.462 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:21.460+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:21.468 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:58:21.564 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:21.561+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:21.615 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:21.615 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:21.624 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:58:21.665 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:21.662+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:21.731 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:21.731 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:21.739 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:58:21.784 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:21.782+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:21.846 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:21.846 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:21.854 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:58:21.865 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:21.863+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:21.972 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:21.964+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:22.001 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:22.001 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:22.010 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:58:22.073 INFO:teuthology.orchestra.run.smithi002.stderr:marked in osd.4. 2024-09-18T04:58:22.074 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:22.071+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:22.090 INFO:tasks.thrashosds.thrasher:Added osd 4 2024-09-18T04:58:22.153 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T04:58:22.162 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:22.163 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:22.172 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:58:22.174 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:22.172+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:22.275 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:22.272+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:22.321 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:22.322 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:22.331 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:58:22.376 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:22.374+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:22.477 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:22.474+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:22.478 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:22.478 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:22.487 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:58:22.578 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:58:22.579 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:22.576+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:22.594 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:22.594 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:22.604 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:58:22.688 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:22.685+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:22.704 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:58:22.705 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:58:22.753 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:22.753 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:22.761 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:58:22.802 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:22.800+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:22.881 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:22.878+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:22.908 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:22.908 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:22.917 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:58:22.922 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:58:22.922 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:58:22.938 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:58:22.938 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:58:22.955 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:58:22.955 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:58:22.972 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:58:22.973 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:58:22.982 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:22.980+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:22.989 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:58:22.990 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:58:23.007 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:58:23.007 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:58:23.019 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-18T04:58:23.032 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:58:23.032 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:58:23.033 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:23.034 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:58:23.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:58:23.034 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:58:23.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.035 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11154 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:23.035 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:58:23.035 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 107.140504728, 2024-09-18T04:58:23.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040591919999999997, 2024-09-18T04:58:23.035 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:23.035 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:23.035 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:23.035 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:23.035 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.035 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11154 2024-09-18T04:58:23.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.035 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:23.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:23.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:58:23.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:23.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:58:23.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:23.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889400+0000", 2024-09-18T04:58:23.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:23.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:23.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889403+0000", 2024-09-18T04:58:23.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3999999999999999e-06 2024-09-18T04:58:23.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:23.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889409+0000", 2024-09-18T04:58:23.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3380000000000001e-06 2024-09-18T04:58:23.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889414+0000", 2024-09-18T04:58:23.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5789999999999997e-06 2024-09-18T04:58:23.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:23.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929811+0000", 2024-09-18T04:58:23.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040397486000000003 2024-09-18T04:58:23.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:23.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929898+0000", 2024-09-18T04:58:23.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6370000000000001e-05 2024-09-18T04:58:23.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:23.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929993+0000", 2024-09-18T04:58:23.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5136000000000005e-05 2024-09-18T04:58:23.039 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.039 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:23.039 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.039 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11155 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2015232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:23.039 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:58:23.039 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 107.14048755, 2024-09-18T04:58:23.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040769156000000001, 2024-09-18T04:58:23.039 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:23.040 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:23.040 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:23.040 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:23.040 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.040 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11155 2024-09-18T04:58:23.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.040 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:23.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:23.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:58:23.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:23.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:58:23.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:23.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:58:23.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:23.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.041 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:23.041 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889420+0000", 2024-09-18T04:58:23.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.181e-06 2024-09-18T04:58:23.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:23.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889426+0000", 2024-09-18T04:58:23.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8599999999999998e-06 2024-09-18T04:58:23.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889430+0000", 2024-09-18T04:58:23.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4399999999999998e-06 2024-09-18T04:58:23.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:23.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930020+0000", 2024-09-18T04:58:23.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040589698 2024-09-18T04:58:23.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:23.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930105+0000", 2024-09-18T04:58:23.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4812e-05 2024-09-18T04:58:23.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:23.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930187+0000", 2024-09-18T04:58:23.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2540999999999995e-05 2024-09-18T04:58:23.046 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.046 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:23.047 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.047 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11156 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2064384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:23.047 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:58:23.047 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 107.140447256, 2024-09-18T04:58:23.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040934419, 2024-09-18T04:58:23.047 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:23.047 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:23.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:23.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:23.047 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.047 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11156 2024-09-18T04:58:23.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.047 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:23.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:23.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:58:23.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:23.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:58:23.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:23.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889457+0000", 2024-09-18T04:58:23.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:58:23.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:23.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889462+0000", 2024-09-18T04:58:23.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8350000000000003e-06 2024-09-18T04:58:23.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:23.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889481+0000", 2024-09-18T04:58:23.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8845999999999999e-05 2024-09-18T04:58:23.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889495+0000", 2024-09-18T04:58:23.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4229e-05 2024-09-18T04:58:23.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:23.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930216+0000", 2024-09-18T04:58:23.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040721081999999999 2024-09-18T04:58:23.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:23.050 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930299+0000", 2024-09-18T04:58:23.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2995000000000001e-05 2024-09-18T04:58:23.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.050 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:23.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930393+0000", 2024-09-18T04:58:23.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3185999999999999e-05 2024-09-18T04:58:23.051 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.051 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:23.051 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.051 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11404 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1609728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:23.051 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:58:23.051 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 107.13364309400001, 2024-09-18T04:58:23.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040669012999999997, 2024-09-18T04:58:23.051 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:23.051 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:23.051 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:23.052 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:23.052 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.052 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11404 2024-09-18T04:58:23.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.052 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:23.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:23.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:58:23.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:23.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:58:23.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:23.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:58:23.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:58:23.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:23.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896263+0000", 2024-09-18T04:58:23.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2729999999999999e-06 2024-09-18T04:58:23.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:23.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896266+0000", 2024-09-18T04:58:23.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8200000000000001e-06 2024-09-18T04:58:23.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896269+0000", 2024-09-18T04:58:23.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3199999999999998e-06 2024-09-18T04:58:23.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:23.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936538+0000", 2024-09-18T04:58:23.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040269394999999999 2024-09-18T04:58:23.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:23.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936657+0000", 2024-09-18T04:58:23.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119498 2024-09-18T04:58:23.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:23.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936931+0000", 2024-09-18T04:58:23.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027391900000000002 2024-09-18T04:58:23.055 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.055 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:23.055 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.056 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11405 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1658880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:23.056 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:58:23.056 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 107.133634836, 2024-09-18T04:58:23.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040891034999999999, 2024-09-18T04:58:23.056 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:23.056 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:23.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:23.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:23.056 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.056 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11405 2024-09-18T04:58:23.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.056 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:23.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:23.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:58:23.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:23.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:58:23.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:23.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896270+0000", 2024-09-18T04:58:23.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:58:23.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:23.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896272+0000", 2024-09-18T04:58:23.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.124e-06 2024-09-18T04:58:23.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:23.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896274+0000", 2024-09-18T04:58:23.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8660000000000002e-06 2024-09-18T04:58:23.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896285+0000", 2024-09-18T04:58:23.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0101e-05 2024-09-18T04:58:23.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:23.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936986+0000", 2024-09-18T04:58:23.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040701213999999999 2024-09-18T04:58:23.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:23.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937070+0000", 2024-09-18T04:58:23.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4211999999999999e-05 2024-09-18T04:58:23.059 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.059 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.059 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:23.059 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937162+0000", 2024-09-18T04:58:23.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1728e-05 2024-09-18T04:58:23.059 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.060 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:23.060 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.060 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.060 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.060 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11406 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1708032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:23.060 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:58:23.060 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 107.13361866699999, 2024-09-18T04:58:23.060 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041085251000000003, 2024-09-18T04:58:23.060 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:23.060 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:23.060 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:23.060 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:23.060 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.060 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11406 2024-09-18T04:58:23.061 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.061 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:23.061 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:58:23.062 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:58:23.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:23.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896288+0000", 2024-09-18T04:58:23.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2440000000000001e-06 2024-09-18T04:58:23.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:23.063 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896291+0000", 2024-09-18T04:58:23.063 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8289999999999999e-06 2024-09-18T04:58:23.063 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.063 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.063 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896293+0000", 2024-09-18T04:58:23.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5399999999999998e-06 2024-09-18T04:58:23.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:23.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937201+0000", 2024-09-18T04:58:23.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040907418000000001 2024-09-18T04:58:23.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.064 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:23.064 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937283+0000", 2024-09-18T04:58:23.064 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2460999999999995e-05 2024-09-18T04:58:23.064 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.064 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.065 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:23.065 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937372+0000", 2024-09-18T04:58:23.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8987999999999996e-05 2024-09-18T04:58:23.065 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.065 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:23.065 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.065 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.065 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.065 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11407 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:23.065 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:58:23.065 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 107.133608847, 2024-09-18T04:58:23.065 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041438580000000003, 2024-09-18T04:58:23.065 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:23.066 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:23.066 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:23.066 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:23.066 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.066 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11407 2024-09-18T04:58:23.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.066 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:23.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:23.066 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:58:23.066 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.066 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.066 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.066 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:23.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:58:23.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:23.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896296+0000", 2024-09-18T04:58:23.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:23.067 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.067 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.067 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:23.067 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896298+0000", 2024-09-18T04:58:23.067 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:58:23.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:23.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896304+0000", 2024-09-18T04:58:23.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6570000000000002e-06 2024-09-18T04:58:23.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896308+0000", 2024-09-18T04:58:23.068 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0940000000000001e-06 2024-09-18T04:58:23.068 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.068 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.068 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:23.068 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937540+0000", 2024-09-18T04:58:23.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041231469 2024-09-18T04:58:23.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:23.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937625+0000", 2024-09-18T04:58:23.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5767999999999996e-05 2024-09-18T04:58:23.069 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.069 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.069 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:23.069 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937735+0000", 2024-09-18T04:58:23.069 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010974 2024-09-18T04:58:23.069 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.070 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:23.070 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.070 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.070 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.070 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11408 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:23.070 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:58:23.070 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 107.13359320799999, 2024-09-18T04:58:23.070 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041653055000000001, 2024-09-18T04:58:23.070 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:23.070 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:23.070 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:23.070 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:23.070 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.070 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11408 2024-09-18T04:58:23.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.071 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:23.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:23.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:58:23.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:23.071 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:58:23.071 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.071 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.071 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.071 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:23.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:58:23.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:23.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:23.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896314+0000", 2024-09-18T04:58:23.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0140000000000001e-06 2024-09-18T04:58:23.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.072 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.072 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:23.072 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896320+0000", 2024-09-18T04:58:23.072 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-18T04:58:23.072 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896324+0000", 2024-09-18T04:58:23.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4410000000000003e-06 2024-09-18T04:58:23.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:23.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937769+0000", 2024-09-18T04:58:23.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041445243 2024-09-18T04:58:23.073 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.073 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.073 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:23.073 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937861+0000", 2024-09-18T04:58:23.073 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1471000000000006e-05 2024-09-18T04:58:23.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.074 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:23.074 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937965+0000", 2024-09-18T04:58:23.074 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104516 2024-09-18T04:58:23.074 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.074 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:23.074 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.074 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.074 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.074 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11409 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:23.074 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:58:23.074 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 107.133577152, 2024-09-18T04:58:23.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041840467999999999, 2024-09-18T04:58:23.075 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:23.075 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:23.075 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:23.075 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:23.075 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.075 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11409 2024-09-18T04:58:23.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.075 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:23.075 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.075 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:23.075 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:58:23.075 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.075 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:23.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:58:23.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:23.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:58:23.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:23.076 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.076 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.076 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:23.076 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896331+0000", 2024-09-18T04:58:23.076 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5950000000000001e-06 2024-09-18T04:58:23.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:23.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896336+0000", 2024-09-18T04:58:23.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7259999999999999e-06 2024-09-18T04:58:23.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896341+0000", 2024-09-18T04:58:23.077 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5619999999999997e-06 2024-09-18T04:58:23.077 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.077 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.077 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:23.077 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937995+0000", 2024-09-18T04:58:23.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041654060999999999 2024-09-18T04:58:23.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:23.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938078+0000", 2024-09-18T04:58:23.078 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2869999999999998e-05 2024-09-18T04:58:23.078 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.078 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.078 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:23.078 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938169+0000", 2024-09-18T04:58:23.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1040999999999998e-05 2024-09-18T04:58:23.079 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.079 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:23.079 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.079 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.079 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.079 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11410 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1921024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:23.079 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:58:23.079 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 107.133560669, 2024-09-18T04:58:23.079 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042060342000000001, 2024-09-18T04:58:23.079 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:23.079 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:23.079 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:23.079 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:23.080 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.080 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11410 2024-09-18T04:58:23.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.080 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:23.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:23.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:58:23.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.080 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.080 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.080 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:23.080 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:58:23.080 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:23.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896344+0000", 2024-09-18T04:58:23.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:23.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:23.081 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896360+0000", 2024-09-18T04:58:23.081 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5068e-05 2024-09-18T04:58:23.081 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.081 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.081 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:23.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896366+0000", 2024-09-18T04:58:23.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5119999999999997e-06 2024-09-18T04:58:23.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896371+0000", 2024-09-18T04:58:23.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:58:23.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.082 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:23.082 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938198+0000", 2024-09-18T04:58:23.082 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041826579000000003 2024-09-18T04:58:23.082 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.082 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:23.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938297+0000", 2024-09-18T04:58:23.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9067000000000005e-05 2024-09-18T04:58:23.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.083 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:23.083 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938405+0000", 2024-09-18T04:58:23.083 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108492 2024-09-18T04:58:23.083 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.083 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:23.083 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.083 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.083 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.083 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11411 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1970176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:23.084 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:58:23.084 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 107.133529827, 2024-09-18T04:58:23.084 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042259191000000002, 2024-09-18T04:58:23.084 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:23.084 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:23.084 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:23.084 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:23.084 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.084 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11411 2024-09-18T04:58:23.084 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.084 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:23.084 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.084 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:23.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:58:23.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:23.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:58:23.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.085 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:23.085 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896375+0000", 2024-09-18T04:58:23.085 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:23.085 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.085 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:23.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896377+0000", 2024-09-18T04:58:23.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2699999999999999e-06 2024-09-18T04:58:23.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:23.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896383+0000", 2024-09-18T04:58:23.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6740000000000002e-06 2024-09-18T04:58:23.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.086 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.086 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.086 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896388+0000", 2024-09-18T04:58:23.086 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:58:23.086 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:23.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938435+0000", 2024-09-18T04:58:23.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042047169000000002 2024-09-18T04:58:23.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:23.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938534+0000", 2024-09-18T04:58:23.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8939999999999998e-05 2024-09-18T04:58:23.087 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.087 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.087 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:23.087 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938635+0000", 2024-09-18T04:58:23.087 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101208 2024-09-18T04:58:23.088 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.088 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:23.088 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.088 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.088 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.088 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11412 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:23.088 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:58:23.088 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 107.133513845, 2024-09-18T04:58:23.088 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042828079999999998, 2024-09-18T04:58:23.088 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:23.088 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:23.088 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:23.088 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:23.089 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.089 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11412 2024-09-18T04:58:23.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.089 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:23.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:23.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:58:23.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.089 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.089 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:23.089 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:58:23.089 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.089 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:23.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896391+0000", 2024-09-18T04:58:23.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:23.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:23.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896393+0000", 2024-09-18T04:58:23.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0810000000000001e-06 2024-09-18T04:58:23.090 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.090 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.090 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:23.090 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896399+0000", 2024-09-18T04:58:23.090 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7230000000000001e-06 2024-09-18T04:58:23.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896403+0000", 2024-09-18T04:58:23.091 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3470000000000001e-06 2024-09-18T04:58:23.091 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.091 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.091 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:23.091 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939043+0000", 2024-09-18T04:58:23.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042639829999999997 2024-09-18T04:58:23.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:23.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939124+0000", 2024-09-18T04:58:23.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0809000000000005e-05 2024-09-18T04:58:23.092 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.092 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.092 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:23.092 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939220+0000", 2024-09-18T04:58:23.092 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5659e-05 2024-09-18T04:58:23.092 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.092 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:23.092 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.093 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.093 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11413 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2068480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:23.093 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:58:23.093 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 107.13349814999999, 2024-09-18T04:58:23.093 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043042108000000003, 2024-09-18T04:58:23.093 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:23.093 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:23.093 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:23.093 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:23.093 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.093 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11413 2024-09-18T04:58:23.093 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.093 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:23.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:23.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:58:23.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:23.094 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:58:23.094 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.094 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.094 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.094 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:23.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:58:23.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:23.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:23.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896409+0000", 2024-09-18T04:58:23.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.165e-06 2024-09-18T04:58:23.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.095 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:23.095 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896415+0000", 2024-09-18T04:58:23.095 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6149999999999996e-06 2024-09-18T04:58:23.095 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.095 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896419+0000", 2024-09-18T04:58:23.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2649999999999998e-06 2024-09-18T04:58:23.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:23.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939262+0000", 2024-09-18T04:58:23.096 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042843444000000001 2024-09-18T04:58:23.096 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.096 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.096 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:23.096 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939351+0000", 2024-09-18T04:58:23.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8944999999999996e-05 2024-09-18T04:58:23.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.097 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:23.097 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939449+0000", 2024-09-18T04:58:23.097 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8029000000000006e-05 2024-09-18T04:58:23.097 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.097 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:23.097 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.097 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.097 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.097 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145409 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:23.097 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:58:23.097 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 49.502078502000003, 2024-09-18T04:58:23.098 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040684845999999997, 2024-09-18T04:58:23.098 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:23.098 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:23.098 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:23.098 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:23.098 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.098 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145409 2024-09-18T04:58:23.098 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.098 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:23.098 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.098 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:23.098 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:58:23.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:23.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:58:23.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.099 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:23.099 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:58:23.099 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:23.099 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.099 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:23.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527829+0000", 2024-09-18T04:58:23.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.441e-06 2024-09-18T04:58:23.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:23.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527835+0000", 2024-09-18T04:58:23.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1650000000000003e-06 2024-09-18T04:58:23.100 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.100 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.100 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.100 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527840+0000", 2024-09-18T04:58:23.100 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8250000000000004e-06 2024-09-18T04:58:23.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:23.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568404+0000", 2024-09-18T04:58:23.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040563872000000001 2024-09-18T04:58:23.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:23.101 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568457+0000", 2024-09-18T04:58:23.101 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2713999999999997e-05 2024-09-18T04:58:23.101 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.101 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.101 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:23.102 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568512+0000", 2024-09-18T04:58:23.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5148000000000001e-05 2024-09-18T04:58:23.102 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.102 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:23.102 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.102 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.102 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.102 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145411 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:23.102 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:58:23.102 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 49.502060641, 2024-09-18T04:58:23.102 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040792108000000001, 2024-09-18T04:58:23.102 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:23.102 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:23.102 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:23.103 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:23.103 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.103 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145411 2024-09-18T04:58:23.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.103 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:23.103 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.103 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:23.103 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:58:23.103 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.103 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:23.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:58:23.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:23.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527844+0000", 2024-09-18T04:58:23.104 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:23.104 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.104 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.104 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:23.104 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527847+0000", 2024-09-18T04:58:23.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0260000000000002e-06 2024-09-18T04:58:23.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.105 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:23.105 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527852+0000", 2024-09-18T04:58:23.105 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9889999999999997e-06 2024-09-18T04:58:23.105 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.105 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527857+0000", 2024-09-18T04:58:23.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6410000000000001e-06 2024-09-18T04:58:23.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:23.106 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568537+0000", 2024-09-18T04:58:23.106 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040679743999999997 2024-09-18T04:58:23.106 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.106 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.106 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:23.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568583+0000", 2024-09-18T04:58:23.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6286999999999999e-05 2024-09-18T04:58:23.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.107 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:23.107 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568637+0000", 2024-09-18T04:58:23.107 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.376e-05 2024-09-18T04:58:23.107 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.107 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:23.107 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.107 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.107 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.107 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145413 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1859584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:23.107 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:58:23.108 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 49.502044261000002, 2024-09-18T04:58:23.108 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040919516000000003, 2024-09-18T04:58:23.108 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:23.108 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:23.108 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:23.108 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:23.108 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.108 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145413 2024-09-18T04:58:23.108 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.108 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:23.108 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.108 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:23.108 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:58:23.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:23.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:58:23.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:23.109 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:58:23.109 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:23.109 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.109 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.109 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:23.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527863+0000", 2024-09-18T04:58:23.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2630000000000002e-06 2024-09-18T04:58:23.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:23.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527869+0000", 2024-09-18T04:58:23.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9009999999999999e-06 2024-09-18T04:58:23.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.110 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.110 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527874+0000", 2024-09-18T04:58:23.110 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0590000000000002e-06 2024-09-18T04:58:23.110 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.110 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:23.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568663+0000", 2024-09-18T04:58:23.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040788805999999997 2024-09-18T04:58:23.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:23.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568716+0000", 2024-09-18T04:58:23.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3501999999999997e-05 2024-09-18T04:58:23.111 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.111 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.111 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:23.111 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568781+0000", 2024-09-18T04:58:23.111 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4338000000000006e-05 2024-09-18T04:58:23.112 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.112 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:23.112 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.112 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.112 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.112 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145414 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1908736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:23.112 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:58:23.112 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 49.502027302999998, 2024-09-18T04:58:23.112 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041030234999999998, 2024-09-18T04:58:23.112 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:23.112 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:23.112 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:23.112 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:23.112 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.113 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145414 2024-09-18T04:58:23.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.113 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:23.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:23.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:58:23.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.113 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.113 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:23.113 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:58:23.113 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.113 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:23.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:58:23.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:23.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:23.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527880+0000", 2024-09-18T04:58:23.114 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4770000000000002e-06 2024-09-18T04:58:23.114 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.114 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.114 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:23.114 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527886+0000", 2024-09-18T04:58:23.115 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5759999999999996e-06 2024-09-18T04:58:23.115 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.115 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.115 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.115 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527890+0000", 2024-09-18T04:58:23.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8180000000000003e-06 2024-09-18T04:58:23.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:23.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568807+0000", 2024-09-18T04:58:23.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040916984000000003 2024-09-18T04:58:23.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.116 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:23.116 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568854+0000", 2024-09-18T04:58:23.116 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6740999999999999e-05 2024-09-18T04:58:23.116 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.116 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.117 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:23.117 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568908+0000", 2024-09-18T04:58:23.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4228000000000003e-05 2024-09-18T04:58:23.117 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.117 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:23.117 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.117 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.117 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.117 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145416 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1957888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:23.117 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:58:23.117 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 49.502010810999998, 2024-09-18T04:58:23.117 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041147364999999998, 2024-09-18T04:58:23.118 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:23.118 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:23.118 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:23.118 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:23.118 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.118 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145416 2024-09-18T04:58:23.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.118 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:23.118 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.118 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:23.118 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:58:23.118 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.118 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:23.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:58:23.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:23.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527894+0000", 2024-09-18T04:58:23.119 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:23.119 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.119 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.119 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:23.119 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527896+0000", 2024-09-18T04:58:23.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1789999999999998e-06 2024-09-18T04:58:23.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:23.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527908+0000", 2024-09-18T04:58:23.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1780000000000001e-05 2024-09-18T04:58:23.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.120 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527914+0000", 2024-09-18T04:58:23.120 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2029999999999997e-06 2024-09-18T04:58:23.120 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.120 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.120 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:23.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568935+0000", 2024-09-18T04:58:23.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041020273000000003 2024-09-18T04:58:23.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:23.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568988+0000", 2024-09-18T04:58:23.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2911000000000002e-05 2024-09-18T04:58:23.121 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.121 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.121 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:23.121 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569042+0000", 2024-09-18T04:58:23.121 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4382000000000002e-05 2024-09-18T04:58:23.121 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.122 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:23.122 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.122 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.122 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.122 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145418 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 2007040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:23.122 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:58:23.122 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 49.501986414000001, 2024-09-18T04:58:23.122 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041247862000000003, 2024-09-18T04:58:23.122 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:23.122 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:23.122 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:23.122 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:23.122 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.122 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145418 2024-09-18T04:58:23.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.123 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:23.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:23.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:58:23.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.123 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:23.123 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:58:23.123 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.123 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.123 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:23.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:58:23.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:23.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:23.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527921+0000", 2024-09-18T04:58:23.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.667e-06 2024-09-18T04:58:23.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.124 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.124 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:23.124 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527929+0000", 2024-09-18T04:58:23.124 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.542e-06 2024-09-18T04:58:23.124 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527934+0000", 2024-09-18T04:58:23.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1440000000000002e-06 2024-09-18T04:58:23.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:23.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569068+0000", 2024-09-18T04:58:23.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041134190000000001 2024-09-18T04:58:23.125 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.125 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.125 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:23.125 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569114+0000", 2024-09-18T04:58:23.125 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6261999999999999e-05 2024-09-18T04:58:23.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.126 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:23.126 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569167+0000", 2024-09-18T04:58:23.126 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2506999999999997e-05 2024-09-18T04:58:23.126 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.126 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:23.126 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.126 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.126 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.126 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145420 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 2056192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:23.126 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:58:23.126 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 49.50196734, 2024-09-18T04:58:23.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041351472, 2024-09-18T04:58:23.127 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:23.127 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:23.127 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:23.127 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:23.127 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.127 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145420 2024-09-18T04:58:23.127 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.127 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:23.127 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.127 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:23.127 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:58:23.127 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:23.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:58:23.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:23.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:23.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:58:23.128 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:23.128 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.128 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.128 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:23.128 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527940+0000", 2024-09-18T04:58:23.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6570000000000001e-06 2024-09-18T04:58:23.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:23.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527948+0000", 2024-09-18T04:58:23.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8350000000000004e-06 2024-09-18T04:58:23.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.129 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527953+0000", 2024-09-18T04:58:23.129 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0690000000000001e-06 2024-09-18T04:58:23.129 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.129 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.129 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:23.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569182+0000", 2024-09-18T04:58:23.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041228734000000003 2024-09-18T04:58:23.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:23.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569234+0000", 2024-09-18T04:58:23.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1663999999999999e-05 2024-09-18T04:58:23.130 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:23.130 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:23.130 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:23.130 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569290+0000", 2024-09-18T04:58:23.130 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5937000000000003e-05 2024-09-18T04:58:23.130 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.130 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:23.131 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.131 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:23.131 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:23.131 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:23.133 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:58:23.134 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:23.130+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:23.233 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:23.231+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:23.242 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:23.242 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:58:23.242 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:23.251 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:58:23.334 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:23.332+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:23.393 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:23.393 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:23.393 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:58:23.394 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:58:23.394 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:23.402 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:58:23.446 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:23.444+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:23.506 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:23.506 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:58:23.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:58:23.507 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:23.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.507 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72135 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 394240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:23.507 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:58:23.507 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 86.548877906000001, 2024-09-18T04:58:23.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048559739999999997, 2024-09-18T04:58:23.507 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:23.507 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:23.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:23.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:23.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.508 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72135 2024-09-18T04:58:23.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.509 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:23.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:23.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:58:23.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:23.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:58:23.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:23.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:58:23.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:23.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:23.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954005+0000", 2024-09-18T04:58:23.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5629999999999999e-06 2024-09-18T04:58:23.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:23.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954008+0000", 2024-09-18T04:58:23.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-18T04:58:23.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954012+0000", 2024-09-18T04:58:23.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4790000000000001e-06 2024-09-18T04:58:23.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:23.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002418+0000", 2024-09-18T04:58:23.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048406049 2024-09-18T04:58:23.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:23.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002460+0000", 2024-09-18T04:58:23.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2188999999999999e-05 2024-09-18T04:58:23.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:23.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002564+0000", 2024-09-18T04:58:23.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010377 2024-09-18T04:58:23.514 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.514 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:23.514 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.514 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72136 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 409600~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:23.514 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:58:23.514 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 86.548772959000004, 2024-09-18T04:58:23.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048670191000000002, 2024-09-18T04:58:23.514 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:23.514 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:23.515 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:23.515 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:23.515 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.515 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72136 2024-09-18T04:58:23.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.516 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:23.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:23.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:58:23.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:23.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:58:23.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:23.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954019+0000", 2024-09-18T04:58:23.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999099 2024-09-18T04:58:23.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:23.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954112+0000", 2024-09-18T04:58:23.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3273e-05 2024-09-18T04:58:23.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:23.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954121+0000", 2024-09-18T04:58:23.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0289999999999994e-06 2024-09-18T04:58:23.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954128+0000", 2024-09-18T04:58:23.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6189999999999996e-06 2024-09-18T04:58:23.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:23.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002604+0000", 2024-09-18T04:58:23.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048476048000000001 2024-09-18T04:58:23.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:23.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002682+0000", 2024-09-18T04:58:23.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7955999999999997e-05 2024-09-18T04:58:23.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:23.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002779+0000", 2024-09-18T04:58:23.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7372000000000002e-05 2024-09-18T04:58:23.520 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.520 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:23.520 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.520 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72138 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 440320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:23.520 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:58:23.520 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 86.548339107000004, 2024-09-18T04:58:23.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048549430999999997, 2024-09-18T04:58:23.520 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:23.521 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:23.521 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:23.521 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:23.521 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.521 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72138 2024-09-18T04:58:23.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.521 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:23.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:23.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:58:23.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:23.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:58:23.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:23.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954542+0000", 2024-09-18T04:58:23.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:23.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:23.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954545+0000", 2024-09-18T04:58:23.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4360000000000001e-06 2024-09-18T04:58:23.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:23.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954551+0000", 2024-09-18T04:58:23.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4949999999999996e-06 2024-09-18T04:58:23.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954557+0000", 2024-09-18T04:58:23.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5099999999999998e-06 2024-09-18T04:58:23.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:23.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002993+0000", 2024-09-18T04:58:23.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048436358999999998 2024-09-18T04:58:23.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:23.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003039+0000", 2024-09-18T04:58:23.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5884000000000003e-05 2024-09-18T04:58:23.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:23.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003092+0000", 2024-09-18T04:58:23.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3208000000000003e-05 2024-09-18T04:58:23.525 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.525 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:23.525 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.525 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72139 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 455680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:23.525 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:58:23.525 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 86.548320298999997, 2024-09-18T04:58:23.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048718773, 2024-09-18T04:58:23.526 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:23.526 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:23.526 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:23.526 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:23.526 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.526 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72139 2024-09-18T04:58:23.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.526 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:23.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:23.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:58:23.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:23.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:58:23.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:23.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954561+0000", 2024-09-18T04:58:23.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:23.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:23.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954564+0000", 2024-09-18T04:58:23.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4099999999999998e-06 2024-09-18T04:58:23.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:23.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954570+0000", 2024-09-18T04:58:23.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.178e-06 2024-09-18T04:58:23.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954575+0000", 2024-09-18T04:58:23.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0560000000000004e-06 2024-09-18T04:58:23.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:23.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003114+0000", 2024-09-18T04:58:23.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048539212999999998 2024-09-18T04:58:23.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:23.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003176+0000", 2024-09-18T04:58:23.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2154000000000002e-05 2024-09-18T04:58:23.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:23.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003280+0000", 2024-09-18T04:58:23.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010421600000000001 2024-09-18T04:58:23.530 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.530 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:23.530 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.531 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72140 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 471040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:23.531 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:58:23.531 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 86.548064358000005, 2024-09-18T04:58:23.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048670819999999997, 2024-09-18T04:58:23.531 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:23.531 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:23.531 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:23.531 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:23.531 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.531 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72140 2024-09-18T04:58:23.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.532 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:23.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:23.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:58:23.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:23.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:58:23.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:23.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954579+0000", 2024-09-18T04:58:23.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997611 2024-09-18T04:58:23.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:23.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954821+0000", 2024-09-18T04:58:23.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024241000000000001 2024-09-18T04:58:23.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:23.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954831+0000", 2024-09-18T04:58:23.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8770000000000005e-06 2024-09-18T04:58:23.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954838+0000", 2024-09-18T04:58:23.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0690000000000004e-06 2024-09-18T04:58:23.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:23.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003317+0000", 2024-09-18T04:58:23.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048478796999999997 2024-09-18T04:58:23.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:23.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003392+0000", 2024-09-18T04:58:23.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4411000000000003e-05 2024-09-18T04:58:23.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:23.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003488+0000", 2024-09-18T04:58:23.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6927000000000002e-05 2024-09-18T04:58:23.535 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.535 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:23.536 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.536 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72141 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 486400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:23.536 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:58:23.536 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 86.548036874999994, 2024-09-18T04:58:23.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048808114, 2024-09-18T04:58:23.536 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:23.536 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:23.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:23.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:23.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.537 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72141 2024-09-18T04:58:23.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.537 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:23.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:23.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:58:23.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:23.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:58:23.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:23.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:58:23.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:23.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:23.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954847+0000", 2024-09-18T04:58:23.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3800000000000001e-06 2024-09-18T04:58:23.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:23.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954854+0000", 2024-09-18T04:58:23.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6080000000000001e-06 2024-09-18T04:58:23.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954859+0000", 2024-09-18T04:58:23.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:58:23.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:23.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003526+0000", 2024-09-18T04:58:23.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048666433000000002 2024-09-18T04:58:23.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:23.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003599+0000", 2024-09-18T04:58:23.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2854000000000004e-05 2024-09-18T04:58:23.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.540 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:23.540 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003653+0000", 2024-09-18T04:58:23.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4511e-05 2024-09-18T04:58:23.541 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.541 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:23.541 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.541 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72142 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 501760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:23.541 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:58:23.541 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 86.547998073000002, 2024-09-18T04:58:23.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048885500999999998, 2024-09-18T04:58:23.541 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:23.541 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:23.542 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:23.542 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:23.542 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.542 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72142 2024-09-18T04:58:23.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.542 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:23.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:23.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:58:23.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:23.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:58:23.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:23.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954864+0000", 2024-09-18T04:58:23.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999804 2024-09-18T04:58:23.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:23.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954886+0000", 2024-09-18T04:58:23.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2042999999999999e-05 2024-09-18T04:58:23.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:23.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954893+0000", 2024-09-18T04:58:23.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.255e-06 2024-09-18T04:58:23.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954899+0000", 2024-09-18T04:58:23.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8780000000000003e-06 2024-09-18T04:58:23.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:23.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003678+0000", 2024-09-18T04:58:23.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048778834 2024-09-18T04:58:23.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:23.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003717+0000", 2024-09-18T04:58:23.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8860999999999999e-05 2024-09-18T04:58:23.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:23.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003769+0000", 2024-09-18T04:58:23.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2389000000000003e-05 2024-09-18T04:58:23.546 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.546 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:23.546 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.547 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72143 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 517120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:23.547 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:58:23.547 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 86.547872447000003, 2024-09-18T04:58:23.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048951738000000002, 2024-09-18T04:58:23.547 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:23.547 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:23.547 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:23.547 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:23.547 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.547 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72143 2024-09-18T04:58:23.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.548 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:23.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:23.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:58:23.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:23.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:58:23.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:23.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954904+0000", 2024-09-18T04:58:23.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998946 2024-09-18T04:58:23.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:23.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955013+0000", 2024-09-18T04:58:23.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108977 2024-09-18T04:58:23.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:23.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955021+0000", 2024-09-18T04:58:23.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7570000000000004e-06 2024-09-18T04:58:23.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955025+0000", 2024-09-18T04:58:23.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1219999999999997e-06 2024-09-18T04:58:23.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:23.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003792+0000", 2024-09-18T04:58:23.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048767379 2024-09-18T04:58:23.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:23.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003860+0000", 2024-09-18T04:58:23.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8033000000000004e-05 2024-09-18T04:58:23.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:23.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003961+0000", 2024-09-18T04:58:23.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010103300000000001 2024-09-18T04:58:23.551 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.552 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:23.552 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.552 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72161 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 13312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:23.552 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:58:23.552 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 86.54760881, 2024-09-18T04:58:23.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048891539999999997, 2024-09-18T04:58:23.552 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:23.552 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:23.552 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:23.553 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:23.553 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.553 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72161 2024-09-18T04:58:23.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.553 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:23.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.553 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:23.553 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:58:23.553 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.553 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.553 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:23.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:58:23.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:23.554 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955028+0000", 2024-09-18T04:58:23.554 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997549 2024-09-18T04:58:23.554 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.554 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.554 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:23.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955278+0000", 2024-09-18T04:58:23.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024973299999999998 2024-09-18T04:58:23.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.555 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.555 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:23.555 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955289+0000", 2024-09-18T04:58:23.555 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0812000000000001e-05 2024-09-18T04:58:23.555 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955296+0000", 2024-09-18T04:58:23.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2570000000000001e-06 2024-09-18T04:58:23.556 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.556 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.556 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:23.556 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004002+0000", 2024-09-18T04:58:23.556 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048705904000000001 2024-09-18T04:58:23.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:23.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004076+0000", 2024-09-18T04:58:23.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4086999999999997e-05 2024-09-18T04:58:23.557 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.557 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.557 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:23.557 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004165+0000", 2024-09-18T04:58:23.557 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8712e-05 2024-09-18T04:58:23.557 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.558 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:23.558 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.558 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.558 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.558 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72163 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 44032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:23.558 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:58:23.558 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 86.546983372, 2024-09-18T04:58:23.558 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048567011, 2024-09-18T04:58:23.558 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:23.558 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:23.558 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:23.558 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:23.559 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.559 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72163 2024-09-18T04:58:23.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.559 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:23.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:23.559 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:58:23.559 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.559 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.559 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.559 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:23.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:58:23.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:23.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955898+0000", 2024-09-18T04:58:23.560 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:23.560 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.560 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.560 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:23.560 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955901+0000", 2024-09-18T04:58:23.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.706e-06 2024-09-18T04:58:23.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:23.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955907+0000", 2024-09-18T04:58:23.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3559999999999998e-06 2024-09-18T04:58:23.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.561 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.561 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.561 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955912+0000", 2024-09-18T04:58:23.561 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9640000000000002e-06 2024-09-18T04:58:23.561 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:23.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004377+0000", 2024-09-18T04:58:23.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048465293999999999 2024-09-18T04:58:23.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:23.562 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004418+0000", 2024-09-18T04:58:23.562 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0163000000000002e-05 2024-09-18T04:58:23.562 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.562 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.562 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:23.563 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004466+0000", 2024-09-18T04:58:23.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8004000000000003e-05 2024-09-18T04:58:23.563 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.563 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:23.563 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.563 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.563 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.563 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72164 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:23.563 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:58:23.563 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 86.546965099000005, 2024-09-18T04:58:23.563 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048707253999999998, 2024-09-18T04:58:23.563 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:23.564 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:23.564 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:23.564 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:23.564 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.564 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72164 2024-09-18T04:58:23.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.564 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:23.564 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.564 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:23.564 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:58:23.564 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.564 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:23.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:58:23.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:23.565 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:58:23.565 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:23.565 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.565 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.565 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:23.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955919+0000", 2024-09-18T04:58:23.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2520000000000002e-06 2024-09-18T04:58:23.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:23.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955925+0000", 2024-09-18T04:58:23.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3740000000000003e-06 2024-09-18T04:58:23.566 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.566 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.566 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.566 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955930+0000", 2024-09-18T04:58:23.566 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:58:23.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:23.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004489+0000", 2024-09-18T04:58:23.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048558581000000003 2024-09-18T04:58:23.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.567 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:23.567 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004529+0000", 2024-09-18T04:58:23.567 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9722999999999999e-05 2024-09-18T04:58:23.567 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.567 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.568 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:23.568 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004624+0000", 2024-09-18T04:58:23.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5402999999999994e-05 2024-09-18T04:58:23.568 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.568 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:23.568 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.568 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.568 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.568 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72165 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 74752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:23.568 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:58:23.568 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 86.546690240000004, 2024-09-18T04:58:23.568 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048633223000000003, 2024-09-18T04:58:23.569 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:23.569 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:23.569 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:23.569 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:23.569 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.569 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72165 2024-09-18T04:58:23.569 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.569 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:23.569 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.569 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:23.569 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:58:23.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:23.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:58:23.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.570 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.570 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.570 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:23.570 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955935+0000", 2024-09-18T04:58:23.570 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999743 2024-09-18T04:58:23.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:23.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956199+0000", 2024-09-18T04:58:23.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000263906 2024-09-18T04:58:23.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.571 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:23.571 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956207+0000", 2024-09-18T04:58:23.571 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2519999999999999e-06 2024-09-18T04:58:23.571 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.571 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956214+0000", 2024-09-18T04:58:23.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7610000000000002e-06 2024-09-18T04:58:23.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:23.572 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004661+0000", 2024-09-18T04:58:23.572 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048447967000000002 2024-09-18T04:58:23.572 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.572 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.572 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:23.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004735+0000", 2024-09-18T04:58:23.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3789999999999997e-05 2024-09-18T04:58:23.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.573 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.573 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:23.573 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004825+0000", 2024-09-18T04:58:23.573 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9724999999999999e-05 2024-09-18T04:58:23.573 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.573 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:23.573 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.573 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.574 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.574 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72166 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 90112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:23.574 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:58:23.574 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 86.546436705999994, 2024-09-18T04:58:23.574 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048595737999999999, 2024-09-18T04:58:23.574 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:23.574 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:23.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:23.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:23.574 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.574 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72166 2024-09-18T04:58:23.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.575 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:23.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:23.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:58:23.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.575 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.575 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.575 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:23.575 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:58:23.575 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:23.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956221+0000", 2024-09-18T04:58:23.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997759 2024-09-18T04:58:23.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.576 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.576 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:23.576 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956448+0000", 2024-09-18T04:58:23.576 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022685500000000001 2024-09-18T04:58:23.576 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:23.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956456+0000", 2024-09-18T04:58:23.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0099999999999995e-06 2024-09-18T04:58:23.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.577 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.577 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.577 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956462+0000", 2024-09-18T04:58:23.577 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7200000000000003e-06 2024-09-18T04:58:23.577 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:23.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004862+0000", 2024-09-18T04:58:23.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048399795000000002 2024-09-18T04:58:23.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.578 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.578 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:23.578 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004941+0000", 2024-09-18T04:58:23.578 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9414999999999998e-05 2024-09-18T04:58:23.578 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.579 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:23.579 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005041+0000", 2024-09-18T04:58:23.579 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9942e-05 2024-09-18T04:58:23.579 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.579 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:23.579 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.579 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.579 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.579 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72167 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:23.579 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:58:23.580 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 86.546349668000005, 2024-09-18T04:58:23.580 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048622894999999999, 2024-09-18T04:58:23.580 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:23.580 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:23.580 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:23.580 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:23.580 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.580 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72167 2024-09-18T04:58:23.580 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.580 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:23.580 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:23.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:58:23.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:23.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:58:23.581 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.581 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.581 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.581 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:23.581 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956468+0000", 2024-09-18T04:58:23.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999352 2024-09-18T04:58:23.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:23.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956535+0000", 2024-09-18T04:58:23.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7440999999999993e-05 2024-09-18T04:58:23.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.582 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.582 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:23.582 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956544+0000", 2024-09-18T04:58:23.582 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1039999999999996e-06 2024-09-18T04:58:23.582 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956550+0000", 2024-09-18T04:58:23.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9850000000000003e-06 2024-09-18T04:58:23.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:23.583 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005066+0000", 2024-09-18T04:58:23.583 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048515614999999998 2024-09-18T04:58:23.583 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.583 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.583 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:23.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005107+0000", 2024-09-18T04:58:23.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0843999999999997e-05 2024-09-18T04:58:23.584 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.584 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.584 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:23.584 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005155+0000", 2024-09-18T04:58:23.584 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8686000000000001e-05 2024-09-18T04:58:23.584 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.584 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:23.584 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.585 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.585 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.585 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72168 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:23.585 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:58:23.585 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 86.546188604999998, 2024-09-18T04:58:23.585 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048585179999999999, 2024-09-18T04:58:23.585 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:23.585 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:23.585 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:23.585 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:23.585 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.585 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72168 2024-09-18T04:58:23.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.586 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:23.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:23.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:58:23.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.586 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.586 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.586 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:23.586 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:58:23.586 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:23.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956555+0000", 2024-09-18T04:58:23.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998612 2024-09-18T04:58:23.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.587 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.587 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:23.587 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956696+0000", 2024-09-18T04:58:23.587 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000140922 2024-09-18T04:58:23.587 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:23.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956707+0000", 2024-09-18T04:58:23.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1555e-05 2024-09-18T04:58:23.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.588 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.588 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956714+0000", 2024-09-18T04:58:23.588 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2709999999999998e-06 2024-09-18T04:58:23.588 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.588 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:23.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005177+0000", 2024-09-18T04:58:23.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048463421 2024-09-18T04:58:23.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:23.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005214+0000", 2024-09-18T04:58:23.589 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.676e-05 2024-09-18T04:58:23.589 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.589 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.589 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:23.589 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005279+0000", 2024-09-18T04:58:23.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4899000000000003e-05 2024-09-18T04:58:23.590 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.590 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:23.590 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.590 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.590 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.590 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72172 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 441344~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:23.590 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.957556+0000", 2024-09-18T04:58:23.590 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 86.545326204999995, 2024-09-18T04:58:23.590 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048545056000000003, 2024-09-18T04:58:23.590 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:23.591 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:23.591 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:23.591 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:23.591 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.591 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72172 2024-09-18T04:58:23.591 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.591 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:23.591 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.591 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:23.591 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957556+0000", 2024-09-18T04:58:23.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:23.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957556+0000", 2024-09-18T04:58:23.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.592 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.592 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.592 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:23.592 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957555+0000", 2024-09-18T04:58:23.592 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:23.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:23.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957558+0000", 2024-09-18T04:58:23.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2900000000000001e-06 2024-09-18T04:58:23.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.593 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.593 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:23.593 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957564+0000", 2024-09-18T04:58:23.593 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0700000000000003e-06 2024-09-18T04:58:23.593 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.957569+0000", 2024-09-18T04:58:23.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7870000000000001e-06 2024-09-18T04:58:23.594 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.594 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.594 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:23.594 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005922+0000", 2024-09-18T04:58:23.594 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048353259000000003 2024-09-18T04:58:23.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:23.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005999+0000", 2024-09-18T04:58:23.595 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6889000000000002e-05 2024-09-18T04:58:23.595 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.595 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.595 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:23.595 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.006101+0000", 2024-09-18T04:58:23.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000102136 2024-09-18T04:58:23.596 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.596 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:23.596 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.596 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.596 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.596 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72199 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:23.596 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:58:23.596 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 86.543656713000004, 2024-09-18T04:58:23.596 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048626057, 2024-09-18T04:58:23.596 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:23.597 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:23.597 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:23.597 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:23.597 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.597 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72199 2024-09-18T04:58:23.597 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.597 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:23.597 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.597 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:23.597 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:58:23.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:23.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:58:23.598 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.598 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.598 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.598 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:23.598 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959195+0000", 2024-09-18T04:58:23.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999969 2024-09-18T04:58:23.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:23.599 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959228+0000", 2024-09-18T04:58:23.599 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3609000000000003e-05 2024-09-18T04:58:23.599 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.599 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.599 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:23.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959236+0000", 2024-09-18T04:58:23.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.157e-06 2024-09-18T04:58:23.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.600 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959243+0000", 2024-09-18T04:58:23.600 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5910000000000001e-06 2024-09-18T04:58:23.600 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.600 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.600 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:23.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007684+0000", 2024-09-18T04:58:23.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048440745 2024-09-18T04:58:23.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:23.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007762+0000", 2024-09-18T04:58:23.601 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8360000000000002e-05 2024-09-18T04:58:23.601 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.601 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.601 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:23.601 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007851+0000", 2024-09-18T04:58:23.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9388000000000005e-05 2024-09-18T04:58:23.602 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.602 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:23.602 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.602 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.602 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.602 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72240 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 428032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:23.602 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:58:23.602 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 86.539563403000002, 2024-09-18T04:58:23.602 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048643767999999997, 2024-09-18T04:58:23.602 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:23.602 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:23.603 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:23.603 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:23.603 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.603 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72240 2024-09-18T04:58:23.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.603 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:23.603 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.603 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:23.603 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:58:23.603 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.603 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:23.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:58:23.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.604 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:23.604 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963240+0000", 2024-09-18T04:58:23.604 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999218 2024-09-18T04:58:23.604 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.604 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:23.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963323+0000", 2024-09-18T04:58:23.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2375999999999999e-05 2024-09-18T04:58:23.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:23.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963332+0000", 2024-09-18T04:58:23.605 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1700000000000003e-06 2024-09-18T04:58:23.605 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.605 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.605 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.605 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963338+0000", 2024-09-18T04:58:23.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2149999999999998e-06 2024-09-18T04:58:23.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:23.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011785+0000", 2024-09-18T04:58:23.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048446493 2024-09-18T04:58:23.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.606 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.606 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:23.606 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011861+0000", 2024-09-18T04:58:23.606 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6472999999999998e-05 2024-09-18T04:58:23.606 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.607 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:23.607 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011962+0000", 2024-09-18T04:58:23.607 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101167 2024-09-18T04:58:23.607 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.607 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:23.607 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.607 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.607 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.607 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:117375 3.1 3:80b18d38:::rbd_header.11acae5d5365:head [watch reconnect cookie 140406307615440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T04:58:23.607 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:58:23.607 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.118488866, 2024-09-18T04:58:23.608 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.018908156, 2024-09-18T04:58:23.608 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:23.608 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:23.608 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:23.608 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:23.608 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.608 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117375 2024-09-18T04:58:23.608 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.608 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:23.608 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.608 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:23.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:58:23.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:23.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:58:23.609 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.609 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.609 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.609 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:23.609 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384391+0000", 2024-09-18T04:58:23.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:58:23.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:23.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384401+0000", 2024-09-18T04:58:23.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4299999999999995e-06 2024-09-18T04:58:23.610 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.610 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.610 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:23.610 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384418+0000", 2024-09-18T04:58:23.610 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6745e-05 2024-09-18T04:58:23.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384429+0000", 2024-09-18T04:58:23.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1122e-05 2024-09-18T04:58:23.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.611 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.611 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:23.611 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384496+0000", 2024-09-18T04:58:23.611 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6841000000000006e-05 2024-09-18T04:58:23.611 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-18T04:58:23.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384505+0000", 2024-09-18T04:58:23.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4080000000000004e-06 2024-09-18T04:58:23.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.612 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:23.612 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.388971+0000", 2024-09-18T04:58:23.612 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0044660000000000004 2024-09-18T04:58:23.612 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.612 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:23.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.388981+0000", 2024-09-18T04:58:23.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0118e-05 2024-09-18T04:58:23.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:23.613 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.399433+0000", 2024-09-18T04:58:23.613 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.010452055 2024-09-18T04:58:23.613 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.613 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.613 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:23.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.399594+0000", 2024-09-18T04:58:23.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000160669 2024-09-18T04:58:23.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.614 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:23.614 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.403301+0000", 2024-09-18T04:58:23.614 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037074370000000001 2024-09-18T04:58:23.614 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.614 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:23.614 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.614 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.614 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.615 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:117376 3.1 3:80b18d38:::rbd_header.11acae5d5365:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T04:58:23.615 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:58:23.615 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 60.116437216000001, 2024-09-18T04:58:23.615 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0196134539999999, 2024-09-18T04:58:23.615 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:23.615 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:23.615 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:23.615 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:23.615 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:23.615 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117376 2024-09-18T04:58:23.615 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.616 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:23.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:23.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:58:23.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.616 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.616 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:23.616 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:58:23.616 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:23.616 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:23.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386443+0000", 2024-09-18T04:58:23.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:58:23.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.617 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:23.617 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386457+0000", 2024-09-18T04:58:23.617 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4207e-05 2024-09-18T04:58:23.617 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.617 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:23.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386478+0000", 2024-09-18T04:58:23.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0349000000000001e-05 2024-09-18T04:58:23.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:23.618 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386503+0000", 2024-09-18T04:58:23.618 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5582e-05 2024-09-18T04:58:23.618 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.618 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.618 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:23.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386558+0000", 2024-09-18T04:58:23.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4892999999999998e-05 2024-09-18T04:58:23.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-18T04:58:23.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386562+0000", 2024-09-18T04:58:23.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.313e-06 2024-09-18T04:58:23.619 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.619 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.619 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:23.619 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.389010+0000", 2024-09-18T04:58:23.619 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024480890000000001 2024-09-18T04:58:23.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:23.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.389018+0000", 2024-09-18T04:58:23.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8469999999999996e-06 2024-09-18T04:58:23.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.620 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.620 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:23.620 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400073+0000", 2024-09-18T04:58:23.620 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.011054286 2024-09-18T04:58:23.620 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:23.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400124+0000", 2024-09-18T04:58:23.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1332999999999998e-05 2024-09-18T04:58:23.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:58:23.621 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400477+0000", 2024-09-18T04:58:23.621 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035329300000000002 2024-09-18T04:58:23.621 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.621 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.621 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:23.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.401353+0000", 2024-09-18T04:58:23.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00087584200000000005 2024-09-18T04:58:23.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:23.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406006+0000", 2024-09-18T04:58:23.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0046534250000000001 2024-09-18T04:58:23.622 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.622 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.622 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:23.622 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406037+0000", 2024-09-18T04:58:23.622 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0559000000000003e-05 2024-09-18T04:58:23.623 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:23.623 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:23.623 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:23.623 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406058+0000", 2024-09-18T04:58:23.623 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1260999999999999e-05 2024-09-18T04:58:23.623 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.623 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:23.623 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.623 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:23.623 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:23.623 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:23.625 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:58:23.626 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:23.623+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:23.725 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:23.723+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:23.769 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:23.770 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:23.770 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:58:23.770 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:23.778 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:58:23.826 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:23.824+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:23.919 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:23.920 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:23.920 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:58:23.920 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:58:23.920 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:23.927 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:58:23.928 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:23.924+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:24.032 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:24.029+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:24.105 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:24.105 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:58:24.105 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:58:24.106 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:24.106 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.106 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60345 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1471488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:24.106 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:58:24.106 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 92.989700698000007, 2024-09-18T04:58:24.106 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044180090999999998, 2024-09-18T04:58:24.106 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.106 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.106 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.106 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.106 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.106 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60345 2024-09-18T04:58:24.106 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.107 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:58:24.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.107 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:58:24.107 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.107 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.107 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.107 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112238+0000", 2024-09-18T04:58:24.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.108 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.108 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112240+0000", 2024-09-18T04:58:24.108 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9910000000000001e-06 2024-09-18T04:58:24.108 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.108 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112246+0000", 2024-09-18T04:58:24.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0859999999999998e-06 2024-09-18T04:58:24.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.109 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112251+0000", 2024-09-18T04:58:24.109 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5020000000000003e-06 2024-09-18T04:58:24.109 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.109 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.109 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156274+0000", 2024-09-18T04:58:24.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044023063000000001 2024-09-18T04:58:24.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156337+0000", 2024-09-18T04:58:24.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3077000000000006e-05 2024-09-18T04:58:24.110 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.110 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.110 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.110 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156419+0000", 2024-09-18T04:58:24.110 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1699000000000005e-05 2024-09-18T04:58:24.110 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.111 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.111 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.111 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.111 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.111 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60346 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1486848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:24.111 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:58:24.111 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 92.989682916999996, 2024-09-18T04:58:24.111 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044336331999999999, 2024-09-18T04:58:24.111 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.111 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.111 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.111 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.111 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.112 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60346 2024-09-18T04:58:24.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.112 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:58:24.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.112 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.112 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.112 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:58:24.112 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.112 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112256+0000", 2024-09-18T04:58:24.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112258+0000", 2024-09-18T04:58:24.113 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1720000000000001e-06 2024-09-18T04:58:24.113 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.113 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.113 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.113 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112265+0000", 2024-09-18T04:58:24.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4280000000000001e-06 2024-09-18T04:58:24.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112269+0000", 2024-09-18T04:58:24.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.741e-06 2024-09-18T04:58:24.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.114 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.114 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156445+0000", 2024-09-18T04:58:24.114 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044175341 2024-09-18T04:58:24.114 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.114 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156512+0000", 2024-09-18T04:58:24.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6723000000000005e-05 2024-09-18T04:58:24.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.115 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.115 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156593+0000", 2024-09-18T04:58:24.115 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1284000000000003e-05 2024-09-18T04:58:24.115 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.115 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.115 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.115 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.115 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.116 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60347 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1502208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:24.116 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:58:24.116 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 92.989654505999994, 2024-09-18T04:58:24.116 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044482560999999997, 2024-09-18T04:58:24.116 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.116 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.116 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.116 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.116 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.116 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60347 2024-09-18T04:58:24.116 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.116 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.116 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:58:24.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:58:24.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.117 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.117 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.117 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112284+0000", 2024-09-18T04:58:24.117 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.117 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112286+0000", 2024-09-18T04:58:24.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9709999999999998e-06 2024-09-18T04:58:24.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112290+0000", 2024-09-18T04:58:24.118 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9199999999999997e-06 2024-09-18T04:58:24.118 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.118 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.118 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.118 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112293+0000", 2024-09-18T04:58:24.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870000000000001e-06 2024-09-18T04:58:24.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.119 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.119 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.119 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156617+0000", 2024-09-18T04:58:24.119 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044323960000000003 2024-09-18T04:58:24.119 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156687+0000", 2024-09-18T04:58:24.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0141000000000005e-05 2024-09-18T04:58:24.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.120 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.120 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.120 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156767+0000", 2024-09-18T04:58:24.120 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0140000000000002e-05 2024-09-18T04:58:24.120 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.120 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.120 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.120 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.121 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.121 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60348 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1517568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:24.121 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:58:24.121 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 92.989643502000007, 2024-09-18T04:58:24.121 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044645562, 2024-09-18T04:58:24.121 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.121 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.121 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.121 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.121 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.121 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60348 2024-09-18T04:58:24.121 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.122 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:58:24.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:58:24.122 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.122 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.122 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.122 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.122 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:58:24.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112297+0000", 2024-09-18T04:58:24.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.384e-06 2024-09-18T04:58:24.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.123 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112301+0000", 2024-09-18T04:58:24.123 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0570000000000003e-06 2024-09-18T04:58:24.123 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.123 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.123 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112304+0000", 2024-09-18T04:58:24.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6000000000000001e-06 2024-09-18T04:58:24.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156793+0000", 2024-09-18T04:58:24.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044489494999999997 2024-09-18T04:58:24.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.124 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.124 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.124 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156859+0000", 2024-09-18T04:58:24.124 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5483999999999998e-05 2024-09-18T04:58:24.124 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.125 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.125 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156941+0000", 2024-09-18T04:58:24.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2856999999999996e-05 2024-09-18T04:58:24.125 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.125 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.125 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.125 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.125 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.125 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60349 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1532928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:24.125 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:58:24.125 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 92.989633526000006, 2024-09-18T04:58:24.125 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044832167999999999, 2024-09-18T04:58:24.126 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.126 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.126 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.126 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.126 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.126 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60349 2024-09-18T04:58:24.126 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.126 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.126 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.126 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:58:24.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:58:24.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.127 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.127 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.127 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:58:24.127 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.127 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112307+0000", 2024-09-18T04:58:24.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.308e-06 2024-09-18T04:58:24.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112310+0000", 2024-09-18T04:58:24.128 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-18T04:58:24.128 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.128 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.128 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.128 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112312+0000", 2024-09-18T04:58:24.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4889999999999998e-06 2024-09-18T04:58:24.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156977+0000", 2024-09-18T04:58:24.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044664598999999999 2024-09-18T04:58:24.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.129 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.129 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157053+0000", 2024-09-18T04:58:24.129 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6014999999999996e-05 2024-09-18T04:58:24.129 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.129 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.130 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.130 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157138+0000", 2024-09-18T04:58:24.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5162999999999998e-05 2024-09-18T04:58:24.130 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.130 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.130 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.130 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.130 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.130 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60350 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1548288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:24.130 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:58:24.130 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 92.989624612, 2024-09-18T04:58:24.130 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045011858000000002, 2024-09-18T04:58:24.130 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.130 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.131 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.131 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.131 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.131 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60350 2024-09-18T04:58:24.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.131 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:58:24.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.131 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.131 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.131 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.131 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:58:24.131 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:58:24.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.132 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112316+0000", 2024-09-18T04:58:24.132 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7579999999999999e-06 2024-09-18T04:58:24.132 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.132 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.132 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112321+0000", 2024-09-18T04:58:24.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1649999999999997e-06 2024-09-18T04:58:24.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112325+0000", 2024-09-18T04:58:24.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.726e-06 2024-09-18T04:58:24.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.133 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.133 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157172+0000", 2024-09-18T04:58:24.133 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.04484718 2024-09-18T04:58:24.133 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.133 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157242+0000", 2024-09-18T04:58:24.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9675e-05 2024-09-18T04:58:24.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.134 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.134 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157327+0000", 2024-09-18T04:58:24.134 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4614e-05 2024-09-18T04:58:24.134 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.134 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.134 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.134 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.134 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.135 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60351 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1563648~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:24.135 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:58:24.135 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 92.989611195999998, 2024-09-18T04:58:24.135 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045132113000000001, 2024-09-18T04:58:24.135 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.135 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.135 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.135 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.135 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.135 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60351 2024-09-18T04:58:24.135 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.135 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.135 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.135 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:58:24.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:58:24.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.136 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.136 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.136 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:58:24.136 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.136 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112330+0000", 2024-09-18T04:58:24.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1660000000000001e-06 2024-09-18T04:58:24.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112336+0000", 2024-09-18T04:58:24.137 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5940000000000001e-06 2024-09-18T04:58:24.137 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.137 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.137 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.137 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112340+0000", 2024-09-18T04:58:24.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5589999999999999e-06 2024-09-18T04:58:24.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157365+0000", 2024-09-18T04:58:24.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045024675 2024-09-18T04:58:24.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.138 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157409+0000", 2024-09-18T04:58:24.138 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4356e-05 2024-09-18T04:58:24.138 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.138 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.138 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.139 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157460+0000", 2024-09-18T04:58:24.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1072000000000002e-05 2024-09-18T04:58:24.139 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.139 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.139 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.139 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.139 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.139 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60502 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1509376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:24.139 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:58:24.139 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 92.983535023000002, 2024-09-18T04:58:24.139 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044092438999999997, 2024-09-18T04:58:24.139 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.139 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.139 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.139 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.140 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.140 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60502 2024-09-18T04:58:24.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.140 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:58:24.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.140 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.140 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.140 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.140 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:58:24.140 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:58:24.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118406+0000", 2024-09-18T04:58:24.141 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:58:24.141 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.141 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.141 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.141 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118412+0000", 2024-09-18T04:58:24.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.524e-06 2024-09-18T04:58:24.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118416+0000", 2024-09-18T04:58:24.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:58:24.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.142 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162418+0000", 2024-09-18T04:58:24.142 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001508000000002 2024-09-18T04:58:24.142 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.142 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.142 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162451+0000", 2024-09-18T04:58:24.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3662999999999999e-05 2024-09-18T04:58:24.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.143 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.143 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162497+0000", 2024-09-18T04:58:24.143 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5550000000000003e-05 2024-09-18T04:58:24.143 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.143 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.143 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.143 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.143 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.143 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60503 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1524736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:24.144 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:58:24.144 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 92.983518864999994, 2024-09-18T04:58:24.144 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044178780000000001, 2024-09-18T04:58:24.144 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.144 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.144 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.144 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.144 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.144 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60503 2024-09-18T04:58:24.144 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.144 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.144 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.144 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.144 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:58:24.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:58:24.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.145 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118420+0000", 2024-09-18T04:58:24.145 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.145 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.145 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.145 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118422+0000", 2024-09-18T04:58:24.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.08e-06 2024-09-18T04:58:24.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118428+0000", 2024-09-18T04:58:24.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.964e-06 2024-09-18T04:58:24.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.146 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.146 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118433+0000", 2024-09-18T04:58:24.146 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.425e-06 2024-09-18T04:58:24.146 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.146 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162519+0000", 2024-09-18T04:58:24.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044086082999999998 2024-09-18T04:58:24.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162552+0000", 2024-09-18T04:58:24.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3719e-05 2024-09-18T04:58:24.147 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.147 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.147 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.147 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162599+0000", 2024-09-18T04:58:24.147 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6873999999999999e-05 2024-09-18T04:58:24.148 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.148 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.148 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.148 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.148 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.148 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60504 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1540096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:24.148 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:58:24.148 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 92.983502759999993, 2024-09-18T04:58:24.148 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044261921000000003, 2024-09-18T04:58:24.148 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.148 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.148 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.149 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.149 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.149 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60504 2024-09-18T04:58:24.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.149 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:58:24.149 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.149 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.149 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.149 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.149 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:58:24.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118436+0000", 2024-09-18T04:58:24.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.150 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.150 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.150 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118438+0000", 2024-09-18T04:58:24.150 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0250000000000001e-06 2024-09-18T04:58:24.150 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118444+0000", 2024-09-18T04:58:24.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4360000000000001e-06 2024-09-18T04:58:24.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118448+0000", 2024-09-18T04:58:24.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.357e-06 2024-09-18T04:58:24.151 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.151 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.151 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.151 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162620+0000", 2024-09-18T04:58:24.151 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044171821 2024-09-18T04:58:24.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162653+0000", 2024-09-18T04:58:24.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3479999999999998e-05 2024-09-18T04:58:24.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.152 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.152 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.152 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162699+0000", 2024-09-18T04:58:24.152 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5113e-05 2024-09-18T04:58:24.152 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.152 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.152 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.152 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.153 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.153 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60585 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1211392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:24.153 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:58:24.153 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 92.981693634999999, 2024-09-18T04:58:24.153 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044389814, 2024-09-18T04:58:24.153 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.153 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.153 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.153 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.153 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.153 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60585 2024-09-18T04:58:24.153 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.153 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:58:24.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:58:24.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.154 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.154 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.154 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.154 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120245+0000", 2024-09-18T04:58:24.154 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120247+0000", 2024-09-18T04:58:24.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0150000000000002e-06 2024-09-18T04:58:24.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120255+0000", 2024-09-18T04:58:24.155 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.858e-06 2024-09-18T04:58:24.155 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.155 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.155 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.155 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120260+0000", 2024-09-18T04:58:24.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6530000000000002e-06 2024-09-18T04:58:24.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164489+0000", 2024-09-18T04:58:24.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044229483999999999 2024-09-18T04:58:24.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.156 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164555+0000", 2024-09-18T04:58:24.156 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.533e-05 2024-09-18T04:58:24.156 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.156 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.156 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.157 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164636+0000", 2024-09-18T04:58:24.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0819e-05 2024-09-18T04:58:24.157 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.157 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.157 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.157 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.157 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.157 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60586 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1226752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:24.157 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:58:24.157 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 92.981675183999997, 2024-09-18T04:58:24.157 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044578214999999997, 2024-09-18T04:58:24.157 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.157 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.158 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.158 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.158 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.158 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60586 2024-09-18T04:58:24.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.158 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:58:24.158 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.158 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.158 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.158 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.158 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:58:24.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:58:24.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.159 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120266+0000", 2024-09-18T04:58:24.159 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0660000000000002e-06 2024-09-18T04:58:24.159 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.159 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.159 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120272+0000", 2024-09-18T04:58:24.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-18T04:58:24.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120276+0000", 2024-09-18T04:58:24.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.476e-06 2024-09-18T04:58:24.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.160 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.160 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164673+0000", 2024-09-18T04:58:24.160 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044396672999999998 2024-09-18T04:58:24.160 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.160 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164749+0000", 2024-09-18T04:58:24.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5811999999999999e-05 2024-09-18T04:58:24.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.161 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.161 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164842+0000", 2024-09-18T04:58:24.161 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3560000000000005e-05 2024-09-18T04:58:24.161 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.161 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.161 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.161 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.161 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.162 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60587 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1242112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:24.162 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:58:24.162 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 92.981626085000002, 2024-09-18T04:58:24.162 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044634794999999998, 2024-09-18T04:58:24.162 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.162 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.162 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.162 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.162 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.162 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60587 2024-09-18T04:58:24.162 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.162 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.162 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:58:24.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:58:24.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.163 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.163 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.163 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.163 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120280+0000", 2024-09-18T04:58:24.163 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999671 2024-09-18T04:58:24.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120316+0000", 2024-09-18T04:58:24.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5728000000000001e-05 2024-09-18T04:58:24.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120329+0000", 2024-09-18T04:58:24.164 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2914e-05 2024-09-18T04:58:24.164 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.164 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.164 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.164 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120335+0000", 2024-09-18T04:58:24.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8289999999999996e-06 2024-09-18T04:58:24.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164858+0000", 2024-09-18T04:58:24.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044523417000000003 2024-09-18T04:58:24.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.165 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.165 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164896+0000", 2024-09-18T04:58:24.165 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7969999999999997e-05 2024-09-18T04:58:24.165 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.165 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.166 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.166 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164948+0000", 2024-09-18T04:58:24.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1996000000000002e-05 2024-09-18T04:58:24.166 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.166 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.166 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.166 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.166 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.166 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60596 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1380352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:24.166 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:58:24.166 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 92.979766548000001, 2024-09-18T04:58:24.166 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044120800000000002, 2024-09-18T04:58:24.166 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.166 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.167 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.167 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.167 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.167 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60596 2024-09-18T04:58:24.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.167 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:58:24.167 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.167 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.167 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.167 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.167 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:58:24.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:58:24.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122175+0000", 2024-09-18T04:58:24.168 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:58:24.168 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.168 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.168 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.168 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122182+0000", 2024-09-18T04:58:24.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3509999999999996e-06 2024-09-18T04:58:24.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122187+0000", 2024-09-18T04:58:24.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7940000000000002e-06 2024-09-18T04:58:24.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166184+0000", 2024-09-18T04:58:24.169 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043997236000000002 2024-09-18T04:58:24.169 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.169 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.169 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.169 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166227+0000", 2024-09-18T04:58:24.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2561000000000001e-05 2024-09-18T04:58:24.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.170 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.170 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166294+0000", 2024-09-18T04:58:24.170 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7068000000000002e-05 2024-09-18T04:58:24.170 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.170 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.170 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.170 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.170 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.170 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60597 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1395712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:24.170 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:58:24.170 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 92.979748520000001, 2024-09-18T04:58:24.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044242166999999999, 2024-09-18T04:58:24.171 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.171 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.171 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.171 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60597 2024-09-18T04:58:24.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.171 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.171 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.171 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.171 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:58:24.171 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.171 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:58:24.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:58:24.172 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.172 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.172 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.172 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.172 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122193+0000", 2024-09-18T04:58:24.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:58:24.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122199+0000", 2024-09-18T04:58:24.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6449999999999999e-06 2024-09-18T04:58:24.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122204+0000", 2024-09-18T04:58:24.173 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4819999999999996e-06 2024-09-18T04:58:24.173 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.173 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.173 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.173 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166329+0000", 2024-09-18T04:58:24.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044125059000000001 2024-09-18T04:58:24.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166370+0000", 2024-09-18T04:58:24.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1012e-05 2024-09-18T04:58:24.174 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.174 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.174 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.174 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166433+0000", 2024-09-18T04:58:24.174 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3178999999999999e-05 2024-09-18T04:58:24.174 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.174 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.174 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.175 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.175 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167074 3.5 3:a712c57a:::rbd_data.11ac138032e7.0000000000000000:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:24.175 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.163703+0000", 2024-09-18T04:58:24.175 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9382363390000004, 2024-09-18T04:58:24.175 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.90992387799999996, 2024-09-18T04:58:24.175 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.175 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:24.175 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.175 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.175 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.175 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167074 2024-09-18T04:58:24.175 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.175 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163703+0000", 2024-09-18T04:58:24.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163703+0000", 2024-09-18T04:58:24.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.176 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.176 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.176 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163702+0000", 2024-09-18T04:58:24.176 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:58:24.176 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163707+0000", 2024-09-18T04:58:24.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5159999999999996e-06 2024-09-18T04:58:24.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163716+0000", 2024-09-18T04:58:24.177 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5270000000000004e-06 2024-09-18T04:58:24.177 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.177 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.177 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.177 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163723+0000", 2024-09-18T04:58:24.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9809999999999997e-06 2024-09-18T04:58:24.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163781+0000", 2024-09-18T04:58:24.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7494999999999998e-05 2024-09-18T04:58:24.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:24.178 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163791+0000", 2024-09-18T04:58:24.178 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0614999999999999e-05 2024-09-18T04:58:24.178 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.178 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.178 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067213+0000", 2024-09-18T04:58:24.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.90342179700000003 2024-09-18T04:58:24.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067327+0000", 2024-09-18T04:58:24.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000114023 2024-09-18T04:58:24.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.179 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.179 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-18T04:58:24.179 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067432+0000", 2024-09-18T04:58:24.179 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104775 2024-09-18T04:58:24.179 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:24.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.071972+0000", 2024-09-18T04:58:24.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0045401219999999997 2024-09-18T04:58:24.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:24.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073530+0000", 2024-09-18T04:58:24.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001558341 2024-09-18T04:58:24.180 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.180 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:24.180 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073582+0000", 2024-09-18T04:58:24.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1924e-05 2024-09-18T04:58:24.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073627+0000", 2024-09-18T04:58:24.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4647000000000001e-05 2024-09-18T04:58:24.181 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.181 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.181 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.181 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167075 3.1 3:864982b2:::rbd_data.11ac138032e7.0000000000000001:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:24.181 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.163740+0000", 2024-09-18T04:58:24.181 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9381990709999997, 2024-09-18T04:58:24.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.90703181499999996, 2024-09-18T04:58:24.181 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.182 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:24.182 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.182 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.182 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.182 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167075 2024-09-18T04:58:24.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.182 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163740+0000", 2024-09-18T04:58:24.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163740+0000", 2024-09-18T04:58:24.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163740+0000", 2024-09-18T04:58:24.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163743+0000", 2024-09-18T04:58:24.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0230000000000001e-06 2024-09-18T04:58:24.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163750+0000", 2024-09-18T04:58:24.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2710000000000003e-06 2024-09-18T04:58:24.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163756+0000", 2024-09-18T04:58:24.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7559999999999996e-06 2024-09-18T04:58:24.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.184 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.184 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.184 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163803+0000", 2024-09-18T04:58:24.184 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6576999999999998e-05 2024-09-18T04:58:24.184 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:24.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163813+0000", 2024-09-18T04:58:24.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0336000000000001e-05 2024-09-18T04:58:24.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.066573+0000", 2024-09-18T04:58:24.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.90276046799999998 2024-09-18T04:58:24.185 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.185 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.185 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.185 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.066694+0000", 2024-09-18T04:58:24.185 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012100299999999999 2024-09-18T04:58:24.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:58:24.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.066791+0000", 2024-09-18T04:58:24.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7141999999999999e-05 2024-09-18T04:58:24.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:24.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.068686+0000", 2024-09-18T04:58:24.186 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018949570000000001 2024-09-18T04:58:24.186 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.186 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.186 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:24.186 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070676+0000", 2024-09-18T04:58:24.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001989158 2024-09-18T04:58:24.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:24.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070733+0000", 2024-09-18T04:58:24.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7642000000000001e-05 2024-09-18T04:58:24.187 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.187 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.187 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.187 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070772+0000", 2024-09-18T04:58:24.187 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8949000000000003e-05 2024-09-18T04:58:24.187 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.188 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.188 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.188 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.188 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.188 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167093 3.1 3:82b38587:::rbd_data.11ac138032e7.0000000000000025:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:24.188 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.173880+0000", 2024-09-18T04:58:24.188 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9280590450000004, 2024-09-18T04:58:24.188 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.89950706999999996, 2024-09-18T04:58:24.188 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.188 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:24.188 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.188 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.188 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.188 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167093 2024-09-18T04:58:24.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.189 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173880+0000", 2024-09-18T04:58:24.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.189 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.189 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173880+0000", 2024-09-18T04:58:24.189 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.189 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.189 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173880+0000", 2024-09-18T04:58:24.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173884+0000", 2024-09-18T04:58:24.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9910000000000004e-06 2024-09-18T04:58:24.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.190 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.190 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.190 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173895+0000", 2024-09-18T04:58:24.190 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1238e-05 2024-09-18T04:58:24.190 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.191 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173903+0000", 2024-09-18T04:58:24.191 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2989999999999997e-06 2024-09-18T04:58:24.191 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.191 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.191 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173954+0000", 2024-09-18T04:58:24.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0862999999999997e-05 2024-09-18T04:58:24.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:24.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173965+0000", 2024-09-18T04:58:24.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0658e-05 2024-09-18T04:58:24.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.192 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.192 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.192 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067554+0000", 2024-09-18T04:58:24.192 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.89358902200000001 2024-09-18T04:58:24.192 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067664+0000", 2024-09-18T04:58:24.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010977200000000001 2024-09-18T04:58:24.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:58:24.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067754+0000", 2024-09-18T04:58:24.193 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0484999999999998e-05 2024-09-18T04:58:24.193 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.193 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.193 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:24.193 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070563+0000", 2024-09-18T04:58:24.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028086650000000001 2024-09-18T04:58:24.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:24.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073294+0000", 2024-09-18T04:58:24.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027316599999999999 2024-09-18T04:58:24.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:24.194 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073351+0000", 2024-09-18T04:58:24.194 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6873999999999998e-05 2024-09-18T04:58:24.194 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.194 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.194 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.195 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073387+0000", 2024-09-18T04:58:24.195 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6254000000000002e-05 2024-09-18T04:58:24.195 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.195 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.195 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.195 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.195 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.195 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167095 3.5 3:a340fea6:::rbd_data.11ac138032e7.0000000000000027:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:24.195 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.173930+0000", 2024-09-18T04:58:24.196 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9280093110000003, 2024-09-18T04:58:24.196 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.89985602499999995, 2024-09-18T04:58:24.196 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.196 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:24.196 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.196 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.196 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.196 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167095 2024-09-18T04:58:24.196 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.196 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173930+0000", 2024-09-18T04:58:24.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173930+0000", 2024-09-18T04:58:24.197 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.197 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.197 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.197 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.197 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173929+0000", 2024-09-18T04:58:24.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173933+0000", 2024-09-18T04:58:24.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4860000000000002e-06 2024-09-18T04:58:24.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.198 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173940+0000", 2024-09-18T04:58:24.198 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4499999999999998e-06 2024-09-18T04:58:24.198 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.198 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.198 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173945+0000", 2024-09-18T04:58:24.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5290000000000002e-06 2024-09-18T04:58:24.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.199 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.199 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173981+0000", 2024-09-18T04:58:24.199 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6548999999999999e-05 2024-09-18T04:58:24.199 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.199 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:24.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173990+0000", 2024-09-18T04:58:24.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1810000000000001e-06 2024-09-18T04:58:24.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.069067+0000", 2024-09-18T04:58:24.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.89507692699999997 2024-09-18T04:58:24.200 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.200 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.200 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.200 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.069196+0000", 2024-09-18T04:58:24.200 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000129572 2024-09-18T04:58:24.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-18T04:58:24.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.069316+0000", 2024-09-18T04:58:24.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120165 2024-09-18T04:58:24.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:24.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.072036+0000", 2024-09-18T04:58:24.201 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027195489999999999 2024-09-18T04:58:24.201 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.201 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.201 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:24.201 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073703+0000", 2024-09-18T04:58:24.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001667555 2024-09-18T04:58:24.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:24.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073749+0000", 2024-09-18T04:58:24.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5636000000000001e-05 2024-09-18T04:58:24.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.202 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.202 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.202 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073786+0000", 2024-09-18T04:58:24.202 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7070000000000003e-05 2024-09-18T04:58:24.202 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.202 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.202 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.202 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.203 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167097 3.c 3:3137739a:::rbd_data.11ac138032e7.0000000000000029:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:24.203 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.178900+0000", 2024-09-18T04:58:24.203 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 4.9230395250000001, 2024-09-18T04:58:24.203 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.89454723999999997, 2024-09-18T04:58:24.203 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.203 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:24.203 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.203 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.203 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.203 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167097 2024-09-18T04:58:24.203 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.203 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.203 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.203 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178900+0000", 2024-09-18T04:58:24.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178900+0000", 2024-09-18T04:58:24.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.204 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.204 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178899+0000", 2024-09-18T04:58:24.204 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:58:24.204 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.204 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178903+0000", 2024-09-18T04:58:24.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7699999999999999e-06 2024-09-18T04:58:24.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178913+0000", 2024-09-18T04:58:24.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0032e-05 2024-09-18T04:58:24.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.205 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.205 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.205 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178918+0000", 2024-09-18T04:58:24.205 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9740000000000001e-06 2024-09-18T04:58:24.205 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178959+0000", 2024-09-18T04:58:24.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1257999999999997e-05 2024-09-18T04:58:24.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:24.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178966+0000", 2024-09-18T04:58:24.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4139999999999999e-06 2024-09-18T04:58:24.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.206 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.206 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.206 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.066273+0000", 2024-09-18T04:58:24.206 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.88730742299999998 2024-09-18T04:58:24.206 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.066394+0000", 2024-09-18T04:58:24.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012086099999999999 2024-09-18T04:58:24.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-18T04:58:24.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.066503+0000", 2024-09-18T04:58:24.207 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109218 2024-09-18T04:58:24.207 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.207 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.207 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:24.207 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070854+0000", 2024-09-18T04:58:24.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0043508219999999998 2024-09-18T04:58:24.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:24.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073373+0000", 2024-09-18T04:58:24.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0025193519999999999 2024-09-18T04:58:24.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:24.208 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073413+0000", 2024-09-18T04:58:24.208 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9592999999999999e-05 2024-09-18T04:58:24.208 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.208 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.208 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.209 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073447+0000", 2024-09-18T04:58:24.209 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4239999999999997e-05 2024-09-18T04:58:24.209 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.209 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.209 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.209 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.209 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.209 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:24.211 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:58:24.211 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:24.208+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:24.311 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:24.309+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:24.360 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:24.360 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:24.361 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:58:24.361 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:24.369 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:58:24.412 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:24.409+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:24.513 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:24.510+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:24.518 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:24.518 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:24.519 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:58:24.519 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:58:24.519 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:24.528 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:58:24.614 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:24.611+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:24.679 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:24.679 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:58:24.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:58:24.679 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:24.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.679 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26200 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 32768~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:24.679 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:58:24.679 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.03595078399999, 2024-09-18T04:58:24.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022501118000000001, 2024-09-18T04:58:24.680 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.680 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.680 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.680 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.680 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.680 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26200 2024-09-18T04:58:24.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.680 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:58:24.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:58:24.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:58:24.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639822+0000", 2024-09-18T04:58:24.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.007e-06 2024-09-18T04:58:24.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639827+0000", 2024-09-18T04:58:24.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5709999999999997e-06 2024-09-18T04:58:24.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.682 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.682 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639831+0000", 2024-09-18T04:58:24.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2819999999999998e-06 2024-09-18T04:58:24.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662181+0000", 2024-09-18T04:58:24.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022349878 2024-09-18T04:58:24.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662252+0000", 2024-09-18T04:58:24.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0542000000000003e-05 2024-09-18T04:58:24.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662321+0000", 2024-09-18T04:58:24.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9167999999999999e-05 2024-09-18T04:58:24.685 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.685 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.685 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.685 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26201 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 61440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:24.685 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:58:24.685 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.035935493, 2024-09-18T04:58:24.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022569783, 2024-09-18T04:58:24.686 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.686 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.686 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.686 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.686 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.686 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26201 2024-09-18T04:58:24.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.686 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:58:24.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:58:24.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.687 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:58:24.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.687 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.687 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639837+0000", 2024-09-18T04:58:24.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.928e-06 2024-09-18T04:58:24.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639843+0000", 2024-09-18T04:58:24.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8130000000000001e-06 2024-09-18T04:58:24.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639847+0000", 2024-09-18T04:58:24.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0999999999999997e-06 2024-09-18T04:58:24.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662335+0000", 2024-09-18T04:58:24.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022488207999999999 2024-09-18T04:58:24.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662370+0000", 2024-09-18T04:58:24.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5292e-05 2024-09-18T04:58:24.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662405+0000", 2024-09-18T04:58:24.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4765999999999999e-05 2024-09-18T04:58:24.690 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.690 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.690 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.690 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26202 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 90112~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:24.690 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:58:24.690 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.035920076, 2024-09-18T04:58:24.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022655937000000001, 2024-09-18T04:58:24.690 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.690 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.690 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.691 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.691 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.691 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26202 2024-09-18T04:58:24.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.691 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:58:24.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.691 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:58:24.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639850+0000", 2024-09-18T04:58:24.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639852+0000", 2024-09-18T04:58:24.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9700000000000002e-06 2024-09-18T04:58:24.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639858+0000", 2024-09-18T04:58:24.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.434e-06 2024-09-18T04:58:24.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639862+0000", 2024-09-18T04:58:24.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3499999999999999e-06 2024-09-18T04:58:24.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662429+0000", 2024-09-18T04:58:24.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022567409 2024-09-18T04:58:24.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662469+0000", 2024-09-18T04:58:24.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9801000000000001e-05 2024-09-18T04:58:24.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662507+0000", 2024-09-18T04:58:24.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7311000000000002e-05 2024-09-18T04:58:24.695 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.695 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.695 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.695 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26203 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 118784~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:24.695 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:58:24.695 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.035904921, 2024-09-18T04:58:24.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022730358999999999, 2024-09-18T04:58:24.695 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.695 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.695 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.695 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.696 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.696 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26203 2024-09-18T04:58:24.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.696 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:58:24.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:58:24.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639865+0000", 2024-09-18T04:58:24.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639867+0000", 2024-09-18T04:58:24.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0889999999999998e-06 2024-09-18T04:58:24.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639873+0000", 2024-09-18T04:58:24.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4979999999999997e-06 2024-09-18T04:58:24.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639877+0000", 2024-09-18T04:58:24.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4510000000000002e-06 2024-09-18T04:58:24.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662528+0000", 2024-09-18T04:58:24.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022650791999999999 2024-09-18T04:58:24.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662563+0000", 2024-09-18T04:58:24.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4530000000000003e-05 2024-09-18T04:58:24.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662596+0000", 2024-09-18T04:58:24.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3463000000000001e-05 2024-09-18T04:58:24.699 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.699 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.699 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.700 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26204 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 147456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:24.700 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:58:24.700 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.035889529, 2024-09-18T04:58:24.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022803233999999999, 2024-09-18T04:58:24.700 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.700 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.700 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.700 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.700 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.701 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26204 2024-09-18T04:58:24.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.701 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:58:24.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:58:24.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:58:24.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639883+0000", 2024-09-18T04:58:24.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6520000000000002e-06 2024-09-18T04:58:24.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639888+0000", 2024-09-18T04:58:24.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2870000000000004e-06 2024-09-18T04:58:24.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639893+0000", 2024-09-18T04:58:24.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3839999999999999e-06 2024-09-18T04:58:24.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662617+0000", 2024-09-18T04:58:24.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022723876 2024-09-18T04:58:24.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662651+0000", 2024-09-18T04:58:24.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3797000000000002e-05 2024-09-18T04:58:24.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662684+0000", 2024-09-18T04:58:24.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3828999999999997e-05 2024-09-18T04:58:24.705 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.705 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.705 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.705 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26205 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 176128~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:24.705 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:58:24.705 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.035866568, 2024-09-18T04:58:24.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022919108000000001, 2024-09-18T04:58:24.705 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.705 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.706 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.706 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.706 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.706 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26205 2024-09-18T04:58:24.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.706 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:58:24.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:58:24.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:58:24.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639906+0000", 2024-09-18T04:58:24.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.531e-06 2024-09-18T04:58:24.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639913+0000", 2024-09-18T04:58:24.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.331e-06 2024-09-18T04:58:24.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639920+0000", 2024-09-18T04:58:24.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5399999999999998e-06 2024-09-18T04:58:24.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.709 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.709 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662706+0000", 2024-09-18T04:58:24.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.02278587 2024-09-18T04:58:24.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662764+0000", 2024-09-18T04:58:24.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7630000000000002e-05 2024-09-18T04:58:24.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662823+0000", 2024-09-18T04:58:24.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9648000000000002e-05 2024-09-18T04:58:24.710 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.711 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.711 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.711 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26206 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 204800~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:24.711 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:58:24.711 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.035846169, 2024-09-18T04:58:24.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023047517, 2024-09-18T04:58:24.711 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.712 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.712 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.712 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.712 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.712 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26206 2024-09-18T04:58:24.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.712 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:58:24.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:58:24.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:58:24.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639926+0000", 2024-09-18T04:58:24.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3120000000000001e-06 2024-09-18T04:58:24.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639932+0000", 2024-09-18T04:58:24.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4890000000000003e-06 2024-09-18T04:58:24.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639936+0000", 2024-09-18T04:58:24.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.07e-06 2024-09-18T04:58:24.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662850+0000", 2024-09-18T04:58:24.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022913579999999999 2024-09-18T04:58:24.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662903+0000", 2024-09-18T04:58:24.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3176e-05 2024-09-18T04:58:24.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662972+0000", 2024-09-18T04:58:24.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9270999999999995e-05 2024-09-18T04:58:24.716 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.716 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.716 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.716 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26207 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 233472~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:24.716 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:58:24.716 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.03583102499999, 2024-09-18T04:58:24.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023139541999999999, 2024-09-18T04:58:24.717 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.717 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.717 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.717 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.717 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.717 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26207 2024-09-18T04:58:24.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.717 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:58:24.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:58:24.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.718 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.718 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639939+0000", 2024-09-18T04:58:24.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639941+0000", 2024-09-18T04:58:24.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0080000000000001e-06 2024-09-18T04:58:24.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639947+0000", 2024-09-18T04:58:24.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.485e-06 2024-09-18T04:58:24.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639951+0000", 2024-09-18T04:58:24.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4549999999999997e-06 2024-09-18T04:58:24.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663005+0000", 2024-09-18T04:58:24.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023053569999999999 2024-09-18T04:58:24.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663042+0000", 2024-09-18T04:58:24.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7245e-05 2024-09-18T04:58:24.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663079+0000", 2024-09-18T04:58:24.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7119000000000002e-05 2024-09-18T04:58:24.721 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.721 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.721 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.721 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26208 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 524288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:24.721 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:58:24.722 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.035814277, 2024-09-18T04:58:24.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023283564999999999, 2024-09-18T04:58:24.722 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.722 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.722 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.722 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.722 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.722 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26208 2024-09-18T04:58:24.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.722 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:58:24.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:58:24.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:58:24.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639958+0000", 2024-09-18T04:58:24.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0899999999999999e-06 2024-09-18T04:58:24.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639963+0000", 2024-09-18T04:58:24.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2209999999999996e-06 2024-09-18T04:58:24.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639967+0000", 2024-09-18T04:58:24.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1169999999999997e-06 2024-09-18T04:58:24.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663114+0000", 2024-09-18T04:58:24.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023146102000000002 2024-09-18T04:58:24.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663183+0000", 2024-09-18T04:58:24.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9686999999999999e-05 2024-09-18T04:58:24.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663240+0000", 2024-09-18T04:58:24.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6728000000000003e-05 2024-09-18T04:58:24.726 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.726 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.726 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.726 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26209 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 552960~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:24.726 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:58:24.726 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.03579650099999, 2024-09-18T04:58:24.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023440966000000001, 2024-09-18T04:58:24.726 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.726 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.727 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26209 2024-09-18T04:58:24.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.727 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.727 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:58:24.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:58:24.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:58:24.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639976+0000", 2024-09-18T04:58:24.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4940000000000002e-06 2024-09-18T04:58:24.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639982+0000", 2024-09-18T04:58:24.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0560000000000001e-06 2024-09-18T04:58:24.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639987+0000", 2024-09-18T04:58:24.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-18T04:58:24.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663278+0000", 2024-09-18T04:58:24.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023290789999999999 2024-09-18T04:58:24.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663348+0000", 2024-09-18T04:58:24.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0837e-05 2024-09-18T04:58:24.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663415+0000", 2024-09-18T04:58:24.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6736000000000006e-05 2024-09-18T04:58:24.730 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.730 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.731 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.731 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26210 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 581632~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:24.731 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:58:24.731 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.03577949300001, 2024-09-18T04:58:24.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023616437000000001, 2024-09-18T04:58:24.731 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.731 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.731 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.731 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.731 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.731 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26210 2024-09-18T04:58:24.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.732 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:58:24.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:58:24.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:58:24.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639993+0000", 2024-09-18T04:58:24.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.419e-06 2024-09-18T04:58:24.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640006+0000", 2024-09-18T04:58:24.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2918000000000001e-05 2024-09-18T04:58:24.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640011+0000", 2024-09-18T04:58:24.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9320000000000004e-06 2024-09-18T04:58:24.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663454+0000", 2024-09-18T04:58:24.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023443035000000001 2024-09-18T04:58:24.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663532+0000", 2024-09-18T04:58:24.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7443999999999999e-05 2024-09-18T04:58:24.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663608+0000", 2024-09-18T04:58:24.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6093000000000005e-05 2024-09-18T04:58:24.735 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.735 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.736 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.736 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26211 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 610304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:24.736 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:58:24.736 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.03575494099999, 2024-09-18T04:58:24.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023774337999999999, 2024-09-18T04:58:24.736 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.736 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.737 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.737 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.737 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.737 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26211 2024-09-18T04:58:24.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.737 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:58:24.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:58:24.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640015+0000", 2024-09-18T04:58:24.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640023+0000", 2024-09-18T04:58:24.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3900000000000004e-06 2024-09-18T04:58:24.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640036+0000", 2024-09-18T04:58:24.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3091999999999999e-05 2024-09-18T04:58:24.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640041+0000", 2024-09-18T04:58:24.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6219999999999997e-06 2024-09-18T04:58:24.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663644+0000", 2024-09-18T04:58:24.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023602643999999999 2024-09-18T04:58:24.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663718+0000", 2024-09-18T04:58:24.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4481999999999996e-05 2024-09-18T04:58:24.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663790+0000", 2024-09-18T04:58:24.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1589999999999997e-05 2024-09-18T04:58:24.741 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.741 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.741 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.742 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26212 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 638976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:24.742 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:58:24.742 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.035721901, 2024-09-18T04:58:24.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023924748999999999, 2024-09-18T04:58:24.742 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.742 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.742 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.742 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.742 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.742 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26212 2024-09-18T04:58:24.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.743 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:58:24.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:58:24.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640048+0000", 2024-09-18T04:58:24.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640051+0000", 2024-09-18T04:58:24.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.095e-06 2024-09-18T04:58:24.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640060+0000", 2024-09-18T04:58:24.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8929999999999994e-06 2024-09-18T04:58:24.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640065+0000", 2024-09-18T04:58:24.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0270000000000003e-06 2024-09-18T04:58:24.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663824+0000", 2024-09-18T04:58:24.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023758873999999999 2024-09-18T04:58:24.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663904+0000", 2024-09-18T04:58:24.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9856000000000003e-05 2024-09-18T04:58:24.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663974+0000", 2024-09-18T04:58:24.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9450999999999996e-05 2024-09-18T04:58:24.746 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.746 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.746 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.747 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26213 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 667648~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:24.747 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:58:24.747 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.03570073, 2024-09-18T04:58:24.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024059812999999999, 2024-09-18T04:58:24.747 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.747 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.747 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.747 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.747 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.747 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26213 2024-09-18T04:58:24.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.748 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:58:24.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:58:24.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:58:24.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640072+0000", 2024-09-18T04:58:24.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4200000000000001e-06 2024-09-18T04:58:24.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640078+0000", 2024-09-18T04:58:24.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0179999999999998e-06 2024-09-18T04:58:24.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640083+0000", 2024-09-18T04:58:24.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:58:24.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664008+0000", 2024-09-18T04:58:24.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023925287 2024-09-18T04:58:24.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664059+0000", 2024-09-18T04:58:24.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0730999999999999e-05 2024-09-18T04:58:24.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664130+0000", 2024-09-18T04:58:24.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0801000000000002e-05 2024-09-18T04:58:24.752 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.752 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.752 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.753 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26214 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 696320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:24.753 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:58:24.753 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.03568376, 2024-09-18T04:58:24.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024223738000000002, 2024-09-18T04:58:24.753 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.753 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.753 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.753 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.753 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.753 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26214 2024-09-18T04:58:24.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.753 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:58:24.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:58:24.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:58:24.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640089+0000", 2024-09-18T04:58:24.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0849999999999999e-06 2024-09-18T04:58:24.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640095+0000", 2024-09-18T04:58:24.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9279999999999998e-06 2024-09-18T04:58:24.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640099+0000", 2024-09-18T04:58:24.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:58:24.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664166+0000", 2024-09-18T04:58:24.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024066513000000001 2024-09-18T04:58:24.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664239+0000", 2024-09-18T04:58:24.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3201999999999995e-05 2024-09-18T04:58:24.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664311+0000", 2024-09-18T04:58:24.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1434999999999996e-05 2024-09-18T04:58:24.757 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.757 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.757 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.757 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26215 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 724992~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:24.757 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:58:24.757 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.035666999, 2024-09-18T04:58:24.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024302284, 2024-09-18T04:58:24.758 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.758 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.758 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26215 2024-09-18T04:58:24.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.758 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:58:24.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:58:24.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640103+0000", 2024-09-18T04:58:24.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640106+0000", 2024-09-18T04:58:24.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2120000000000002e-06 2024-09-18T04:58:24.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640111+0000", 2024-09-18T04:58:24.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7880000000000003e-06 2024-09-18T04:58:24.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640116+0000", 2024-09-18T04:58:24.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6099999999999999e-06 2024-09-18T04:58:24.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664327+0000", 2024-09-18T04:58:24.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024210941999999999 2024-09-18T04:58:24.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664367+0000", 2024-09-18T04:58:24.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9666000000000003e-05 2024-09-18T04:58:24.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664406+0000", 2024-09-18T04:58:24.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9402e-05 2024-09-18T04:58:24.762 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.762 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.762 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.762 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26216 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 753664~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:24.762 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:58:24.762 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.03565050500001, 2024-09-18T04:58:24.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024452854999999999, 2024-09-18T04:58:24.762 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.762 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.762 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.762 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.763 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.763 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26216 2024-09-18T04:58:24.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.763 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:58:24.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:58:24.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.764 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:58:24.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.764 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640122+0000", 2024-09-18T04:58:24.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3599999999999999e-06 2024-09-18T04:58:24.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640128+0000", 2024-09-18T04:58:24.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6339999999999998e-06 2024-09-18T04:58:24.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640132+0000", 2024-09-18T04:58:24.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:58:24.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664436+0000", 2024-09-18T04:58:24.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024303585999999999 2024-09-18T04:58:24.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664507+0000", 2024-09-18T04:58:24.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1046999999999999e-05 2024-09-18T04:58:24.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664573+0000", 2024-09-18T04:58:24.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6217000000000007e-05 2024-09-18T04:58:24.767 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.767 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.767 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.767 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26217 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 782336~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:24.767 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:58:24.767 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 104.03563427100001, 2024-09-18T04:58:24.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024526287000000001, 2024-09-18T04:58:24.767 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.767 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:24.767 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.767 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.768 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.768 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26217 2024-09-18T04:58:24.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.768 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:58:24.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:58:24.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:58:24.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640140+0000", 2024-09-18T04:58:24.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2989999999999998e-06 2024-09-18T04:58:24.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.769 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.769 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640146+0000", 2024-09-18T04:58:24.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9569999999999999e-06 2024-09-18T04:58:24.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640151+0000", 2024-09-18T04:58:24.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:58:24.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664590+0000", 2024-09-18T04:58:24.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0244389 2024-09-18T04:58:24.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664628+0000", 2024-09-18T04:58:24.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7910999999999997e-05 2024-09-18T04:58:24.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664663+0000", 2024-09-18T04:58:24.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5024999999999998e-05 2024-09-18T04:58:24.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.771 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.771 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.771 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167100 3.1e 3:7bd358cc:::rbd_data.11ac138032e7.000000000000004f:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:24.771 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.178944+0000", 2024-09-18T04:58:24.772 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.496826993, 2024-09-18T04:58:24.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.89428526100000005, 2024-09-18T04:58:24.772 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.772 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:24.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.772 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.772 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167100 2024-09-18T04:58:24.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.772 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178944+0000", 2024-09-18T04:58:24.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178944+0000", 2024-09-18T04:58:24.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178942+0000", 2024-09-18T04:58:24.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:58:24.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178948+0000", 2024-09-18T04:58:24.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8100000000000003e-06 2024-09-18T04:58:24.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.774 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.774 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178962+0000", 2024-09-18T04:58:24.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.399e-05 2024-09-18T04:58:24.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178970+0000", 2024-09-18T04:58:24.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.537e-06 2024-09-18T04:58:24.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.179015+0000", 2024-09-18T04:58:24.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5652999999999998e-05 2024-09-18T04:58:24.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:24.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.179021+0000", 2024-09-18T04:58:24.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5280000000000003e-06 2024-09-18T04:58:24.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.065479+0000", 2024-09-18T04:58:24.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.88645855799999995 2024-09-18T04:58:24.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.065788+0000", 2024-09-18T04:58:24.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030816099999999999 2024-09-18T04:58:24.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-18T04:58:24.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.065904+0000", 2024-09-18T04:58:24.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116554 2024-09-18T04:58:24.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:24.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067684+0000", 2024-09-18T04:58:24.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001780195 2024-09-18T04:58:24.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:24.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073137+0000", 2024-09-18T04:58:24.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0054525210000000001 2024-09-18T04:58:24.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:24.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073193+0000", 2024-09-18T04:58:24.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6492000000000001e-05 2024-09-18T04:58:24.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.779 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.779 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073229+0000", 2024-09-18T04:58:24.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5664000000000003e-05 2024-09-18T04:58:24.779 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.779 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.779 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.780 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167111 3.19 3:99c9092f:::rbd_data.11ac138032e7.000000000000005a:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:24.780 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.201980+0000", 2024-09-18T04:58:24.780 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.4737903909999996, 2024-09-18T04:58:24.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.87136403900000003, 2024-09-18T04:58:24.780 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:24.780 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:24.780 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:24.780 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:24.780 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:24.780 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167111 2024-09-18T04:58:24.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.781 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:24.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:24.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.201980+0000", 2024-09-18T04:58:24.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:24.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.201980+0000", 2024-09-18T04:58:24.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:24.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:24.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.201980+0000", 2024-09-18T04:58:24.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:24.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:24.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.201983+0000", 2024-09-18T04:58:24.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8949999999999998e-06 2024-09-18T04:58:24.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:24.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.201990+0000", 2024-09-18T04:58:24.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5129999999999999e-06 2024-09-18T04:58:24.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:24.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.201999+0000", 2024-09-18T04:58:24.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5490000000000006e-06 2024-09-18T04:58:24.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.202033+0000", 2024-09-18T04:58:24.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4076999999999999e-05 2024-09-18T04:58:24.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.784 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:24.784 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.202038+0000", 2024-09-18T04:58:24.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3310000000000003e-06 2024-09-18T04:58:24.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:24.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.068314+0000", 2024-09-18T04:58:24.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.86627544899999998 2024-09-18T04:58:24.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:24.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.068426+0000", 2024-09-18T04:58:24.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011262199999999999 2024-09-18T04:58:24.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 7", 2024-09-18T04:58:24.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.068531+0000", 2024-09-18T04:58:24.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010510600000000001 2024-09-18T04:58:24.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:24.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.071305+0000", 2024-09-18T04:58:24.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027742539999999999 2024-09-18T04:58:24.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:24.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073258+0000", 2024-09-18T04:58:24.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001952264 2024-09-18T04:58:24.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:24.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073303+0000", 2024-09-18T04:58:24.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5286e-05 2024-09-18T04:58:24.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:24.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:24.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:24.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073344+0000", 2024-09-18T04:58:24.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1269000000000001e-05 2024-09-18T04:58:24.788 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.788 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.788 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.788 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:24.788 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:24.788 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:24.789 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:58:24.790 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:24.787+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:24.893 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:24.887+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:24.941 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:24.941 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:24.941 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:58:24.941 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:24.949 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:58:24.991 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:24.988+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:25.092 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:25.090+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:25.102 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:25.102 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:25.102 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:58:25.103 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:58:25.103 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:25.112 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:58:25.133 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T04:58:25.134 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:58:25.193 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:25.191+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:25.265 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:58:25.265 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:58:25.267 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:25.267 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:58:25.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:58:25.267 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:25.267 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.267 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:58:25.267 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:58:25.267 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 133.86968091599999, 2024-09-18T04:58:25.267 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037718511000000003, 2024-09-18T04:58:25.267 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:25.267 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:25.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:25.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:58:25.267 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:58:25.268 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-18T04:58:25.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.268 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:25.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:25.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:58:25.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.268 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.268 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.268 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:25.268 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:58:25.268 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:25.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393125+0000", 2024-09-18T04:58:25.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:58:25.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.269 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:25.269 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393133+0000", 2024-09-18T04:58:25.269 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.225e-06 2024-09-18T04:58:25.269 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.269 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:25.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393144+0000", 2024-09-18T04:58:25.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1279e-05 2024-09-18T04:58:25.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:25.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393153+0000", 2024-09-18T04:58:25.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4540000000000007e-06 2024-09-18T04:58:25.270 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.270 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.270 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.270 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393203+0000", 2024-09-18T04:58:25.270 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9963000000000003e-05 2024-09-18T04:58:25.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:25.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393210+0000", 2024-09-18T04:58:25.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5689999999999999e-06 2024-09-18T04:58:25.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.271 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.271 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427399+0000", 2024-09-18T04:58:25.271 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034188784999999999 2024-09-18T04:58:25.271 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.271 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:25.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427532+0000", 2024-09-18T04:58:25.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013264700000000001 2024-09-18T04:58:25.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:58:25.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427685+0000", 2024-09-18T04:58:25.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015300599999999999 2024-09-18T04:58:25.272 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.272 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.272 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:25.272 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.428520+0000", 2024-09-18T04:58:25.272 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00083502499999999998 2024-09-18T04:58:25.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:25.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430772+0000", 2024-09-18T04:58:25.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022518880000000001 2024-09-18T04:58:25.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:25.273 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430817+0000", 2024-09-18T04:58:25.273 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5077000000000002e-05 2024-09-18T04:58:25.273 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.273 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.273 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:25.274 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430845+0000", 2024-09-18T04:58:25.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8036999999999999e-05 2024-09-18T04:58:25.274 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.274 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:25.274 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.274 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.274 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.274 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:62520 3.13 3.84cdb893 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T04:58:25.274 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:25.274 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 93.965371032999997, 2024-09-18T04:58:25.274 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94452134799999998, 2024-09-18T04:58:25.274 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:25.274 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:58:25.274 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:25.274 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:25.275 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:25.275 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 62520 2024-09-18T04:58:25.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.275 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:25.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:25.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:25.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.275 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.275 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.275 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:25.275 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:25.275 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:25.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:25.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:25.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:25.276 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297441+0000", 2024-09-18T04:58:25.276 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9840000000000001e-06 2024-09-18T04:58:25.276 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.276 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.276 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:25.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297449+0000", 2024-09-18T04:58:25.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3999999999999992e-06 2024-09-18T04:58:25.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:25.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297455+0000", 2024-09-18T04:58:25.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3350000000000003e-06 2024-09-18T04:58:25.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.277 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.277 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.277 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297552+0000", 2024-09-18T04:58:25.277 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6602999999999996e-05 2024-09-18T04:58:25.277 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:25.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297559+0000", 2024-09-18T04:58:25.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.52e-06 2024-09-18T04:58:25.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241943+0000", 2024-09-18T04:58:25.278 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94438357100000003 2024-09-18T04:58:25.278 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.278 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.278 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:25.278 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241958+0000", 2024-09-18T04:58:25.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4598e-05 2024-09-18T04:58:25.279 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.279 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:25.279 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.279 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.279 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.279 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:113179 3.13 3:cf138017:::rbd_data.11ac5f86a3f9.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T04:58:25.279 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:58:25.279 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 66.575580622000004, 2024-09-18T04:58:25.279 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036508304999999998, 2024-09-18T04:58:25.279 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:25.279 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:25.279 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:25.280 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:25.280 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:25.280 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 113179 2024-09-18T04:58:25.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.280 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:25.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:25.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:58:25.280 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.280 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.280 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.280 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:25.280 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:58:25.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:25.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687226+0000", 2024-09-18T04:58:25.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:58:25.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.281 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.281 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:25.281 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688169+0000", 2024-09-18T04:58:25.281 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00094328200000000004 2024-09-18T04:58:25.281 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:25.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688182+0000", 2024-09-18T04:58:25.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2756e-05 2024-09-18T04:58:25.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.282 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:25.282 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688188+0000", 2024-09-18T04:58:25.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0290000000000002e-06 2024-09-18T04:58:25.282 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688225+0000", 2024-09-18T04:58:25.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7301000000000001e-05 2024-09-18T04:58:25.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:25.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688324+0000", 2024-09-18T04:58:25.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9419999999999993e-05 2024-09-18T04:58:25.283 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.283 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.283 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:58:25.283 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.700483+0000", 2024-09-18T04:58:25.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012158771000000001 2024-09-18T04:58:25.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:25.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723586+0000", 2024-09-18T04:58:25.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023102952 2024-09-18T04:58:25.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.284 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:25.284 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723634+0000", 2024-09-18T04:58:25.284 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7468e-05 2024-09-18T04:58:25.284 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.284 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:25.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723693+0000", 2024-09-18T04:58:25.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9539e-05 2024-09-18T04:58:25.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:25.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723735+0000", 2024-09-18T04:58:25.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1829000000000002e-05 2024-09-18T04:58:25.285 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.285 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:25.285 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.286 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146078 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1449984~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:25.286 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:25.286 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.669502778999998, 2024-09-18T04:58:25.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035495325000000001, 2024-09-18T04:58:25.286 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:25.286 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:25.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:25.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:25.286 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:25.286 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146078 2024-09-18T04:58:25.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.286 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:25.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:25.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:25.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:25.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:25.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:25.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:25.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:25.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:25.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593306+0000", 2024-09-18T04:58:25.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.686e-06 2024-09-18T04:58:25.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:25.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593311+0000", 2024-09-18T04:58:25.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8500000000000002e-06 2024-09-18T04:58:25.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:25.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593314+0000", 2024-09-18T04:58:25.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0290000000000001e-06 2024-09-18T04:58:25.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628674+0000", 2024-09-18T04:58:25.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035360346000000001 2024-09-18T04:58:25.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:25.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628726+0000", 2024-09-18T04:58:25.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2219999999999998e-05 2024-09-18T04:58:25.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:25.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628800+0000", 2024-09-18T04:58:25.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3613000000000001e-05 2024-09-18T04:58:25.290 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.291 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:25.291 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.291 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146079 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1499136~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:25.291 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:25.291 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.669491374000003, 2024-09-18T04:58:25.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035604581000000003, 2024-09-18T04:58:25.291 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:25.291 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:25.292 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:25.292 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:25.292 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:25.292 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146079 2024-09-18T04:58:25.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.292 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:25.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:25.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:25.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:25.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:25.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:25.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:25.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:25.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:25.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593317+0000", 2024-09-18T04:58:25.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1599999999999999e-06 2024-09-18T04:58:25.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.294 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:25.294 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593320+0000", 2024-09-18T04:58:25.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0130000000000002e-06 2024-09-18T04:58:25.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:25.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593322+0000", 2024-09-18T04:58:25.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6079999999999998e-06 2024-09-18T04:58:25.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628825+0000", 2024-09-18T04:58:25.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035502747000000001 2024-09-18T04:58:25.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:25.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628868+0000", 2024-09-18T04:58:25.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2719999999999998e-05 2024-09-18T04:58:25.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:25.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628920+0000", 2024-09-18T04:58:25.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2533e-05 2024-09-18T04:58:25.297 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.297 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:25.297 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.297 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146080 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1548288~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:25.297 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:25.297 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.669475876, 2024-09-18T04:58:25.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035755146000000002, 2024-09-18T04:58:25.297 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:25.297 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:25.298 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:25.298 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:25.301 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:25.301 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146080 2024-09-18T04:58:25.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.301 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:25.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:25.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:25.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:25.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:25.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:25.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:25.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:25.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:25.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593333+0000", 2024-09-18T04:58:25.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.858e-06 2024-09-18T04:58:25.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:25.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593336+0000", 2024-09-18T04:58:25.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4910000000000001e-06 2024-09-18T04:58:25.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.303 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:25.303 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593340+0000", 2024-09-18T04:58:25.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2590000000000001e-06 2024-09-18T04:58:25.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628956+0000", 2024-09-18T04:58:25.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035616039000000002 2024-09-18T04:58:25.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:25.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629031+0000", 2024-09-18T04:58:25.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5504000000000001e-05 2024-09-18T04:58:25.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:25.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629086+0000", 2024-09-18T04:58:25.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5429000000000001e-05 2024-09-18T04:58:25.306 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.306 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:25.306 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.306 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146081 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1597440~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:25.306 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:25.306 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.669465303999999, 2024-09-18T04:58:25.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035873223000000003, 2024-09-18T04:58:25.306 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:25.306 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:25.307 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:25.307 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:25.307 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:25.307 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146081 2024-09-18T04:58:25.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.307 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:25.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:25.307 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:25.307 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.307 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:25.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:25.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.308 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:25.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:25.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:25.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:25.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593343+0000", 2024-09-18T04:58:25.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3319999999999999e-06 2024-09-18T04:58:25.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:25.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593346+0000", 2024-09-18T04:58:25.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9059999999999998e-06 2024-09-18T04:58:25.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:25.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593348+0000", 2024-09-18T04:58:25.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4530000000000001e-06 2024-09-18T04:58:25.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629110+0000", 2024-09-18T04:58:25.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035761814000000003 2024-09-18T04:58:25.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:25.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629159+0000", 2024-09-18T04:58:25.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8971000000000003e-05 2024-09-18T04:58:25.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:25.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629215+0000", 2024-09-18T04:58:25.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6107000000000003e-05 2024-09-18T04:58:25.311 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.311 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:25.311 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.311 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146082 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1646592~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:25.311 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:25.311 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.669449254, 2024-09-18T04:58:25.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035987007000000001, 2024-09-18T04:58:25.311 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:25.311 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:25.311 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:25.311 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:25.312 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:25.312 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146082 2024-09-18T04:58:25.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.312 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:25.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:25.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:25.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:25.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:25.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:25.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:25.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:25.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.313 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:25.313 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593360+0000", 2024-09-18T04:58:25.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8560000000000001e-06 2024-09-18T04:58:25.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:25.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593365+0000", 2024-09-18T04:58:25.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3410000000000002e-06 2024-09-18T04:58:25.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:25.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593369+0000", 2024-09-18T04:58:25.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4850000000000003e-06 2024-09-18T04:58:25.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629242+0000", 2024-09-18T04:58:25.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035872464999999999 2024-09-18T04:58:25.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:25.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629288+0000", 2024-09-18T04:58:25.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6592999999999999e-05 2024-09-18T04:58:25.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:25.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629345+0000", 2024-09-18T04:58:25.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6575e-05 2024-09-18T04:58:25.315 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.316 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:25.316 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.316 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146083 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1695744~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:25.316 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:58:25.316 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.669433407, 2024-09-18T04:58:25.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036100587000000003, 2024-09-18T04:58:25.316 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:25.316 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:25.316 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:25.316 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:25.317 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:25.317 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146083 2024-09-18T04:58:25.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.317 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:25.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:25.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:58:25.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:25.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:58:25.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:25.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593373+0000", 2024-09-18T04:58:25.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:25.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.318 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:25.318 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593376+0000", 2024-09-18T04:58:25.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.244e-06 2024-09-18T04:58:25.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:25.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593379+0000", 2024-09-18T04:58:25.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.619e-06 2024-09-18T04:58:25.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:25.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593382+0000", 2024-09-18T04:58:25.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7199999999999998e-06 2024-09-18T04:58:25.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629368+0000", 2024-09-18T04:58:25.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035986281000000002 2024-09-18T04:58:25.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:25.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629416+0000", 2024-09-18T04:58:25.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8055e-05 2024-09-18T04:58:25.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:25.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629474+0000", 2024-09-18T04:58:25.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8176e-05 2024-09-18T04:58:25.320 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.321 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:25.321 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.321 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146084 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1744896~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:25.321 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:25.321 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.669420277, 2024-09-18T04:58:25.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036214349, 2024-09-18T04:58:25.321 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:25.321 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:25.321 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:25.321 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:25.321 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:25.321 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146084 2024-09-18T04:58:25.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.322 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:25.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:25.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:25.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:25.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:25.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:25.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:25.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:25.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:25.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593388+0000", 2024-09-18T04:58:25.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.564e-06 2024-09-18T04:58:25.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:25.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593392+0000", 2024-09-18T04:58:25.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0579999999999999e-06 2024-09-18T04:58:25.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:25.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593395+0000", 2024-09-18T04:58:25.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8890000000000002e-06 2024-09-18T04:58:25.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629500+0000", 2024-09-18T04:58:25.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036105343999999998 2024-09-18T04:58:25.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:25.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629547+0000", 2024-09-18T04:58:25.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7036000000000002e-05 2024-09-18T04:58:25.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:25.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629601+0000", 2024-09-18T04:58:25.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3866999999999998e-05 2024-09-18T04:58:25.325 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.325 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:25.325 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.325 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146085 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1794048~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:25.325 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:25.325 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.669409889999997, 2024-09-18T04:58:25.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036331186000000001, 2024-09-18T04:58:25.325 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:25.326 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:25.326 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:25.326 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:25.326 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:25.326 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146085 2024-09-18T04:58:25.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.326 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:25.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:25.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:25.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:25.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:25.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:25.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:25.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:25.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:25.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593398+0000", 2024-09-18T04:58:25.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2270000000000001e-06 2024-09-18T04:58:25.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:25.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593401+0000", 2024-09-18T04:58:25.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.999e-06 2024-09-18T04:58:25.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:25.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593404+0000", 2024-09-18T04:58:25.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.486e-06 2024-09-18T04:58:25.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629624+0000", 2024-09-18T04:58:25.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036219983999999997 2024-09-18T04:58:25.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:25.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629672+0000", 2024-09-18T04:58:25.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8368000000000002e-05 2024-09-18T04:58:25.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:25.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629729+0000", 2024-09-18T04:58:25.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6359e-05 2024-09-18T04:58:25.330 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.330 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:25.330 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.330 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146086 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1843200~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:25.330 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:25.330 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.669401325999999, 2024-09-18T04:58:25.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036449098999999999, 2024-09-18T04:58:25.330 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:25.330 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:25.330 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:25.330 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:25.330 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:25.331 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146086 2024-09-18T04:58:25.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.331 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:25.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:25.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:25.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:25.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:25.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:25.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:25.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:25.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:25.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593407+0000", 2024-09-18T04:58:25.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1799999999999999e-06 2024-09-18T04:58:25.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:25.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593410+0000", 2024-09-18T04:58:25.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.926e-06 2024-09-18T04:58:25.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:25.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593413+0000", 2024-09-18T04:58:25.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8250000000000001e-06 2024-09-18T04:58:25.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629755+0000", 2024-09-18T04:58:25.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036342212999999998 2024-09-18T04:58:25.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:25.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629803+0000", 2024-09-18T04:58:25.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8619000000000003e-05 2024-09-18T04:58:25.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:25.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629855+0000", 2024-09-18T04:58:25.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1536000000000003e-05 2024-09-18T04:58:25.334 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.334 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:25.334 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.334 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146087 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1892352~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:25.335 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:25.335 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.669392401000003, 2024-09-18T04:58:25.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036564608999999998, 2024-09-18T04:58:25.335 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:25.335 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:25.335 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:25.335 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:25.335 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:25.335 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146087 2024-09-18T04:58:25.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.335 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:25.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:25.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:25.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:25.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:25.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:25.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:25.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:25.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:25.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593416+0000", 2024-09-18T04:58:25.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.218e-06 2024-09-18T04:58:25.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.337 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:25.337 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593419+0000", 2024-09-18T04:58:25.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-18T04:58:25.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:25.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593421+0000", 2024-09-18T04:58:25.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4329999999999998e-06 2024-09-18T04:58:25.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629878+0000", 2024-09-18T04:58:25.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036456578000000003 2024-09-18T04:58:25.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:25.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629925+0000", 2024-09-18T04:58:25.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7595000000000001e-05 2024-09-18T04:58:25.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:25.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629979+0000", 2024-09-18T04:58:25.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4117999999999999e-05 2024-09-18T04:58:25.339 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.339 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:25.339 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.339 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146088 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1941504~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:25.339 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:25.339 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.669370237000003, 2024-09-18T04:58:25.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036676121999999999, 2024-09-18T04:58:25.340 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:25.340 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:25.340 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:25.340 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:25.340 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:25.340 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146088 2024-09-18T04:58:25.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.340 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:25.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:25.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:25.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:25.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:25.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:25.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:25.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:25.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:25.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593438+0000", 2024-09-18T04:58:25.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7570000000000001e-06 2024-09-18T04:58:25.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:25.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593442+0000", 2024-09-18T04:58:25.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2880000000000001e-06 2024-09-18T04:58:25.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:25.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593444+0000", 2024-09-18T04:58:25.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.593e-06 2024-09-18T04:58:25.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630015+0000", 2024-09-18T04:58:25.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036571125000000003 2024-09-18T04:58:25.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:25.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630061+0000", 2024-09-18T04:58:25.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5445000000000003e-05 2024-09-18T04:58:25.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:25.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630113+0000", 2024-09-18T04:58:25.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2358000000000003e-05 2024-09-18T04:58:25.343 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.344 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:25.344 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.344 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146089 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1990656~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:25.344 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:58:25.344 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.669360656000002, 2024-09-18T04:58:25.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036791483999999999, 2024-09-18T04:58:25.344 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:25.344 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:25.344 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:25.344 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:25.344 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:25.345 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146089 2024-09-18T04:58:25.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.345 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:25.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:25.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:58:25.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:25.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:58:25.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:25.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593446+0000", 2024-09-18T04:58:25.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:25.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:25.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593448+0000", 2024-09-18T04:58:25.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1939999999999999e-06 2024-09-18T04:58:25.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:25.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593451+0000", 2024-09-18T04:58:25.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.766e-06 2024-09-18T04:58:25.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:25.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593458+0000", 2024-09-18T04:58:25.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8539999999999999e-06 2024-09-18T04:58:25.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630136+0000", 2024-09-18T04:58:25.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03667757 2024-09-18T04:58:25.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:25.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630183+0000", 2024-09-18T04:58:25.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7289000000000001e-05 2024-09-18T04:58:25.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:25.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630238+0000", 2024-09-18T04:58:25.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5013000000000003e-05 2024-09-18T04:58:25.348 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.348 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:25.348 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.348 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146090 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 2039808~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:25.348 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:25.348 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.669346775999998, 2024-09-18T04:58:25.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036904477999999998, 2024-09-18T04:58:25.349 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:25.349 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:25.349 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:25.349 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:25.349 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:25.349 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146090 2024-09-18T04:58:25.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.349 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:25.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:25.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:25.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:25.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:25.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:25.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:25.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:25.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:25.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593461+0000", 2024-09-18T04:58:25.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3680000000000001e-06 2024-09-18T04:58:25.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:25.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593465+0000", 2024-09-18T04:58:25.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8120000000000001e-06 2024-09-18T04:58:25.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:25.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593468+0000", 2024-09-18T04:58:25.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.582e-06 2024-09-18T04:58:25.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630268+0000", 2024-09-18T04:58:25.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036800449999999998 2024-09-18T04:58:25.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:25.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630313+0000", 2024-09-18T04:58:25.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4836000000000002e-05 2024-09-18T04:58:25.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:25.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630365+0000", 2024-09-18T04:58:25.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1839999999999998e-05 2024-09-18T04:58:25.352 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.352 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:25.352 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.353 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146091 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 2088960~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:25.353 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:25.353 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 51.669336313999999, 2024-09-18T04:58:25.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037011534999999998, 2024-09-18T04:58:25.353 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:25.353 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:25.353 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:25.353 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:25.353 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:25.353 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146091 2024-09-18T04:58:25.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.353 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:25.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:25.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:25.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:25.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:25.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:25.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:25.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:25.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:25.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593472+0000", 2024-09-18T04:58:25.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1620000000000001e-06 2024-09-18T04:58:25.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:25.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593475+0000", 2024-09-18T04:58:25.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0019999999999998e-06 2024-09-18T04:58:25.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:25.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593477+0000", 2024-09-18T04:58:25.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-18T04:58:25.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630381+0000", 2024-09-18T04:58:25.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036904014999999998 2024-09-18T04:58:25.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:25.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630428+0000", 2024-09-18T04:58:25.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6931000000000002e-05 2024-09-18T04:58:25.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:25.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630482+0000", 2024-09-18T04:58:25.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4138999999999998e-05 2024-09-18T04:58:25.358 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.358 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:25.358 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.358 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:162075 3.13 3:cf3f6048:::rbd_header.11ac6967455f:head [watch reconnect cookie 140406508629968 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T04:58:25.358 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:58:25.358 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.772363509000002, 2024-09-18T04:58:25.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.53184266599999996, 2024-09-18T04:58:25.358 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:25.358 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:25.359 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:25.359 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:25.359 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:25.359 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 162075 2024-09-18T04:58:25.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.359 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:25.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:25.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:58:25.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:25.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:58:25.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:25.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490442+0000", 2024-09-18T04:58:25.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:58:25.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:25.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490453+0000", 2024-09-18T04:58:25.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0613e-05 2024-09-18T04:58:25.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:25.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490466+0000", 2024-09-18T04:58:25.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3702e-05 2024-09-18T04:58:25.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:25.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490475+0000", 2024-09-18T04:58:25.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2690000000000005e-06 2024-09-18T04:58:25.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490612+0000", 2024-09-18T04:58:25.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013620900000000001 2024-09-18T04:58:25.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-18T04:58:25.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490620+0000", 2024-09-18T04:58:25.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3089999999999996e-06 2024-09-18T04:58:25.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503041+0000", 2024-09-18T04:58:25.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012420512 2024-09-18T04:58:25.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:25.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503048+0000", 2024-09-18T04:58:25.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0389999999999999e-06 2024-09-18T04:58:25.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.016122+0000", 2024-09-18T04:58:25.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.51307432900000005 2024-09-18T04:58:25.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:25.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.016289+0000", 2024-09-18T04:58:25.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000166875 2024-09-18T04:58:25.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:25.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.022286+0000", 2024-09-18T04:58:25.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059976049999999996 2024-09-18T04:58:25.364 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.364 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:25.364 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.365 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:162076 3.13 3:cf3f6048:::rbd_header.11ac6967455f:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T04:58:25.365 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:25.365 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 42.772240834999998, 2024-09-18T04:58:25.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.53464117600000005, 2024-09-18T04:58:25.365 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:25.365 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:25.365 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:25.365 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:25.365 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:25.365 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 162076 2024-09-18T04:58:25.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.365 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:25.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:25.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:25.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:25.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:25.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:25.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:25.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:58:25.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:25.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490571+0000", 2024-09-18T04:58:25.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8259999999999998e-06 2024-09-18T04:58:25.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:25.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490584+0000", 2024-09-18T04:58:25.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2391999999999999e-05 2024-09-18T04:58:25.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:25.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490591+0000", 2024-09-18T04:58:25.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5460000000000003e-06 2024-09-18T04:58:25.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490714+0000", 2024-09-18T04:58:25.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000122543 2024-09-18T04:58:25.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-18T04:58:25.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490731+0000", 2024-09-18T04:58:25.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7405e-05 2024-09-18T04:58:25.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503069+0000", 2024-09-18T04:58:25.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012337836 2024-09-18T04:58:25.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:25.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503078+0000", 2024-09-18T04:58:25.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8289999999999997e-06 2024-09-18T04:58:25.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019018+0000", 2024-09-18T04:58:25.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.51594035900000002 2024-09-18T04:58:25.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:25.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019091+0000", 2024-09-18T04:58:25.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2608999999999995e-05 2024-09-18T04:58:25.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-18T04:58:25.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019421+0000", 2024-09-18T04:58:25.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00033004300000000002 2024-09-18T04:58:25.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:25.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.021644+0000", 2024-09-18T04:58:25.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022229749999999999 2024-09-18T04:58:25.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:25.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025113+0000", 2024-09-18T04:58:25.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003469228 2024-09-18T04:58:25.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:25.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025167+0000", 2024-09-18T04:58:25.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3569999999999997e-05 2024-09-18T04:58:25.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:25.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025208+0000", 2024-09-18T04:58:25.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0821000000000001e-05 2024-09-18T04:58:25.372 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.372 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:25.372 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.372 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167082 3.15 3:a80f5cd7:::rbd_data.11ac138032e7.0000000000000008:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:25.373 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.163826+0000", 2024-09-18T04:58:25.373 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 6.0989810889999996, 2024-09-18T04:58:25.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.91230620200000001, 2024-09-18T04:58:25.373 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:25.373 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:25.373 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:25.373 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:25.373 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:25.373 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167082 2024-09-18T04:58:25.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.373 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:25.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:25.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163826+0000", 2024-09-18T04:58:25.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:25.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163826+0000", 2024-09-18T04:58:25.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:25.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:25.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163826+0000", 2024-09-18T04:58:25.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:25.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:25.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163829+0000", 2024-09-18T04:58:25.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9069999999999999e-06 2024-09-18T04:58:25.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:25.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163836+0000", 2024-09-18T04:58:25.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4780000000000002e-06 2024-09-18T04:58:25.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:25.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163841+0000", 2024-09-18T04:58:25.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9389999999999996e-06 2024-09-18T04:58:25.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163886+0000", 2024-09-18T04:58:25.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5037999999999998e-05 2024-09-18T04:58:25.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:25.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163895+0000", 2024-09-18T04:58:25.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7819999999999999e-06 2024-09-18T04:58:25.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:25.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070784+0000", 2024-09-18T04:58:25.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.90688966400000004 2024-09-18T04:58:25.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:25.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070917+0000", 2024-09-18T04:58:25.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013228799999999999 2024-09-18T04:58:25.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:58:25.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.071003+0000", 2024-09-18T04:58:25.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5915000000000007e-05 2024-09-18T04:58:25.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:25.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073598+0000", 2024-09-18T04:58:25.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0025949329999999998 2024-09-18T04:58:25.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:25.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.076024+0000", 2024-09-18T04:58:25.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024267310000000001 2024-09-18T04:58:25.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:25.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.076096+0000", 2024-09-18T04:58:25.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1512000000000002e-05 2024-09-18T04:58:25.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:25.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:25.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:25.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.076132+0000", 2024-09-18T04:58:25.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6520999999999998e-05 2024-09-18T04:58:25.379 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.379 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:25.379 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.379 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:25.379 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:25.380 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:25.381 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:58:25.381 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:58:25.382 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:58:25.382 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:58:25.383 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:58:25.383 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:58:25.383 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:58:25.384 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:58:25.384 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:58:25.384 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:58:25.384 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:58:25.385 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:58:25.385 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:58:25.385 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:58:25.386 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:58:25.386 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:25.378+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:25.482 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:25.479+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:25.492 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:25.493 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:25.501 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:58:25.582 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:25.580+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:25.608 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:25.608 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:25.616 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:58:25.683 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:25.681+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:25.723 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:25.723 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:25.732 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:58:25.784 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:25.781+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:25.839 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:25.839 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:25.848 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:58:25.891 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:25.889+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:25.958 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:25.958 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:25.967 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:58:25.985 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:25.983+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:26.087 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:26.084+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:26.093 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-18T04:58:26.120 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:26.121 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:26.130 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:58:26.188 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:26.186+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:26.284 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:26.284 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:26.289 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:26.287+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:26.293 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:58:26.390 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:26.388+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:26.440 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:26.440 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:26.449 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:58:26.491 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:26.488+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:26.555 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:26.555 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:26.564 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:58:26.608 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:26.606+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:26.671 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:26.671 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:26.680 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:58:26.721 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:26.719+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:26.794 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:26.792+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:26.827 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:26.828 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:26.836 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:58:26.896 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:26.893+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:26.942 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:26.942 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:26.951 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:58:26.997 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:26.995+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:27.057 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:27.057 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:27.067 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:58:27.091 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 5, 6, 7, 4] out_osds: [1, 3] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T04:58:27.091 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T04:58:27.091 INFO:tasks.thrashosds.thrasher:Adding osd 3 2024-09-18T04:58:27.091 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 3 2024-09-18T04:58:27.110 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:27.108+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:27.185 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:27.185 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:27.195 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:58:27.199 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:27.196+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:27.300 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:27.298+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:27.345 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:27.345 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:27.358 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:58:27.391 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:58:27.392 INFO:tasks.ceph.mon.a.smithi002.stderr:2024-09-18T04:58:27.390+0000 7f7d3b87f640 -1 mon.a@0(leader).osd e137 definitely_dead 0 2024-09-18T04:58:27.402 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:27.399+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:27.463 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:27.463 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:27.472 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:58:27.514 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:27.511+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:27.515 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:58:27.515 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:58:27.533 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:58:27.533 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:58:27.549 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:58:27.549 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:58:27.565 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:58:27.566 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:58:27.582 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:58:27.582 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:58:27.599 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:58:27.599 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:58:27.602 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:27.600+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:27.617 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:58:27.617 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:58:27.619 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:27.619 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:27.619 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:58:27.619 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:58:27.619 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:27.628 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:58:27.633 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:58:27.633 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:58:27.703 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:27.701+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:27.736 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:27.736 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:58:27.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:58:27.736 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:27.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.736 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60345 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1471488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:27.736 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:58:27.737 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.620410875000005, 2024-09-18T04:58:27.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044180090999999998, 2024-09-18T04:58:27.737 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:27.737 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:27.737 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:27.737 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:27.737 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:27.737 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60345 2024-09-18T04:58:27.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.737 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:27.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:27.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:58:27.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:27.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:58:27.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:27.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112238+0000", 2024-09-18T04:58:27.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:27.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:27.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112240+0000", 2024-09-18T04:58:27.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9910000000000001e-06 2024-09-18T04:58:27.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:27.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112246+0000", 2024-09-18T04:58:27.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0859999999999998e-06 2024-09-18T04:58:27.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:27.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112251+0000", 2024-09-18T04:58:27.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5020000000000003e-06 2024-09-18T04:58:27.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.740 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:27.740 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156274+0000", 2024-09-18T04:58:27.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044023063000000001 2024-09-18T04:58:27.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:27.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156337+0000", 2024-09-18T04:58:27.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3077000000000006e-05 2024-09-18T04:58:27.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:27.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156419+0000", 2024-09-18T04:58:27.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1699000000000005e-05 2024-09-18T04:58:27.741 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.741 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:27.741 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.742 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60346 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1486848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:27.742 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:58:27.742 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.620393093999994, 2024-09-18T04:58:27.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044336331999999999, 2024-09-18T04:58:27.742 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:27.742 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:27.742 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:27.742 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:27.742 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:27.742 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60346 2024-09-18T04:58:27.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.742 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:27.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:27.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:58:27.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:27.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:58:27.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:27.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112256+0000", 2024-09-18T04:58:27.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:27.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:27.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112258+0000", 2024-09-18T04:58:27.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1720000000000001e-06 2024-09-18T04:58:27.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:27.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112265+0000", 2024-09-18T04:58:27.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4280000000000001e-06 2024-09-18T04:58:27.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:27.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112269+0000", 2024-09-18T04:58:27.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.741e-06 2024-09-18T04:58:27.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:27.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156445+0000", 2024-09-18T04:58:27.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044175341 2024-09-18T04:58:27.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:27.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156512+0000", 2024-09-18T04:58:27.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6723000000000005e-05 2024-09-18T04:58:27.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:27.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156593+0000", 2024-09-18T04:58:27.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1284000000000003e-05 2024-09-18T04:58:27.747 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.747 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:27.747 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.748 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60347 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1502208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:27.748 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:58:27.748 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.620364683000005, 2024-09-18T04:58:27.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044482560999999997, 2024-09-18T04:58:27.748 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:27.748 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:27.748 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:27.748 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:27.748 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:27.748 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60347 2024-09-18T04:58:27.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.749 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:27.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:27.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:58:27.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:27.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:58:27.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:27.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112284+0000", 2024-09-18T04:58:27.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:27.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:27.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112286+0000", 2024-09-18T04:58:27.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9709999999999998e-06 2024-09-18T04:58:27.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:27.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112290+0000", 2024-09-18T04:58:27.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9199999999999997e-06 2024-09-18T04:58:27.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:27.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112293+0000", 2024-09-18T04:58:27.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870000000000001e-06 2024-09-18T04:58:27.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:27.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156617+0000", 2024-09-18T04:58:27.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044323960000000003 2024-09-18T04:58:27.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:27.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156687+0000", 2024-09-18T04:58:27.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0141000000000005e-05 2024-09-18T04:58:27.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:27.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156767+0000", 2024-09-18T04:58:27.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0140000000000002e-05 2024-09-18T04:58:27.754 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.754 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:27.754 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.754 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60348 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1517568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:27.754 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:58:27.754 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.620353679000004, 2024-09-18T04:58:27.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044645562, 2024-09-18T04:58:27.755 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:27.755 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:27.755 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:27.755 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:27.755 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:27.755 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60348 2024-09-18T04:58:27.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.755 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:27.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:27.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:58:27.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:27.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:58:27.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:27.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:58:27.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:27.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:27.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112297+0000", 2024-09-18T04:58:27.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.384e-06 2024-09-18T04:58:27.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:27.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112301+0000", 2024-09-18T04:58:27.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0570000000000003e-06 2024-09-18T04:58:27.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:27.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112304+0000", 2024-09-18T04:58:27.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6000000000000001e-06 2024-09-18T04:58:27.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:27.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156793+0000", 2024-09-18T04:58:27.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044489494999999997 2024-09-18T04:58:27.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:27.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156859+0000", 2024-09-18T04:58:27.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5483999999999998e-05 2024-09-18T04:58:27.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:27.758 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156941+0000", 2024-09-18T04:58:27.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2856999999999996e-05 2024-09-18T04:58:27.759 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.759 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:27.759 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.759 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60349 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1532928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:27.759 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:58:27.759 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.620343703000003, 2024-09-18T04:58:27.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044832167999999999, 2024-09-18T04:58:27.760 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:27.760 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:27.760 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:27.760 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:27.760 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:27.760 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60349 2024-09-18T04:58:27.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.760 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:27.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:27.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:58:27.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:27.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:58:27.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:27.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:58:27.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:27.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:27.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112307+0000", 2024-09-18T04:58:27.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.308e-06 2024-09-18T04:58:27.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:27.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112310+0000", 2024-09-18T04:58:27.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-18T04:58:27.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:27.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112312+0000", 2024-09-18T04:58:27.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4889999999999998e-06 2024-09-18T04:58:27.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156977+0000", 2024-09-18T04:58:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044664598999999999 2024-09-18T04:58:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157053+0000", 2024-09-18T04:58:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6014999999999996e-05 2024-09-18T04:58:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157138+0000", 2024-09-18T04:58:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5162999999999998e-05 2024-09-18T04:58:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.763 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:27.764 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.764 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.764 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.764 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60350 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1548288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:27.764 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:58:27.764 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.620334788999997, 2024-09-18T04:58:27.764 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045011858000000002, 2024-09-18T04:58:27.764 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:27.764 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:27.764 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:27.764 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:27.764 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:27.764 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60350 2024-09-18T04:58:27.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.765 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:27.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:27.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:58:27.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:27.765 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:58:27.765 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.765 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.765 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.765 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:27.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:58:27.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:27.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:27.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112316+0000", 2024-09-18T04:58:27.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7579999999999999e-06 2024-09-18T04:58:27.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.766 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.766 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:27.766 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112321+0000", 2024-09-18T04:58:27.766 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1649999999999997e-06 2024-09-18T04:58:27.766 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112325+0000", 2024-09-18T04:58:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.726e-06 2024-09-18T04:58:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157172+0000", 2024-09-18T04:58:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.04484718 2024-09-18T04:58:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157242+0000", 2024-09-18T04:58:27.767 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9675e-05 2024-09-18T04:58:27.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.768 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.768 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:27.768 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157327+0000", 2024-09-18T04:58:27.768 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4614e-05 2024-09-18T04:58:27.768 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.768 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:27.768 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.768 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.769 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.769 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60351 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1563648~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:27.769 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:58:27.769 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.620321372999996, 2024-09-18T04:58:27.769 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045132113000000001, 2024-09-18T04:58:27.769 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:27.769 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:27.769 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:27.769 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:27.769 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:27.769 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60351 2024-09-18T04:58:27.769 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.769 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:27.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:27.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:58:27.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.770 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.770 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.770 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:27.770 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:58:27.770 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:58:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112330+0000", 2024-09-18T04:58:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1660000000000001e-06 2024-09-18T04:58:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.771 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:27.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112336+0000", 2024-09-18T04:58:27.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5940000000000001e-06 2024-09-18T04:58:27.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:27.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112340+0000", 2024-09-18T04:58:27.772 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5589999999999999e-06 2024-09-18T04:58:27.772 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.772 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.772 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:27.772 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157365+0000", 2024-09-18T04:58:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045024675 2024-09-18T04:58:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157409+0000", 2024-09-18T04:58:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4356e-05 2024-09-18T04:58:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157460+0000", 2024-09-18T04:58:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1072000000000002e-05 2024-09-18T04:58:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.773 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:27.774 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.774 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.774 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.774 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60502 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1509376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:27.774 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:58:27.774 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.614245199999999, 2024-09-18T04:58:27.774 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044092438999999997, 2024-09-18T04:58:27.774 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:27.774 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:27.774 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:27.775 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:27.775 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:27.775 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60502 2024-09-18T04:58:27.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.775 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:27.775 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.775 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:27.775 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:58:27.775 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.775 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:27.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:58:27.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:27.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:58:27.776 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:27.776 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.776 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.776 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:27.776 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118406+0000", 2024-09-18T04:58:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:58:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118412+0000", 2024-09-18T04:58:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.524e-06 2024-09-18T04:58:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118416+0000", 2024-09-18T04:58:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:58:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.777 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:27.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162418+0000", 2024-09-18T04:58:27.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001508000000002 2024-09-18T04:58:27.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:27.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162451+0000", 2024-09-18T04:58:27.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3662999999999999e-05 2024-09-18T04:58:27.778 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.778 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.778 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:27.778 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162497+0000", 2024-09-18T04:58:27.778 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5550000000000003e-05 2024-09-18T04:58:27.778 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60503 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1524736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:58:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.614229042000005, 2024-09-18T04:58:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044178780000000001, 2024-09-18T04:58:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:27.779 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60503 2024-09-18T04:58:27.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.780 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:27.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:27.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:58:27.780 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.780 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.780 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.780 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:27.780 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:58:27.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:27.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118420+0000", 2024-09-18T04:58:27.781 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:27.781 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.781 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.781 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:27.781 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118422+0000", 2024-09-18T04:58:27.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.08e-06 2024-09-18T04:58:27.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:27.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118428+0000", 2024-09-18T04:58:27.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.964e-06 2024-09-18T04:58:27.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:27.782 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118433+0000", 2024-09-18T04:58:27.782 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.425e-06 2024-09-18T04:58:27.782 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.782 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.782 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162519+0000", 2024-09-18T04:58:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044086082999999998 2024-09-18T04:58:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162552+0000", 2024-09-18T04:58:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3719e-05 2024-09-18T04:58:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162599+0000", 2024-09-18T04:58:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6873999999999999e-05 2024-09-18T04:58:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.783 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:27.784 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.784 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.784 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.784 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60504 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1540096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:27.784 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:58:27.784 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.614212937000005, 2024-09-18T04:58:27.784 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044261921000000003, 2024-09-18T04:58:27.784 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:27.784 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:27.784 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:27.784 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:27.784 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:27.784 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60504 2024-09-18T04:58:27.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.785 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:27.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:27.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:58:27.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.785 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:27.785 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:58:27.785 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.785 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.785 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118436+0000", 2024-09-18T04:58:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118438+0000", 2024-09-18T04:58:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0250000000000001e-06 2024-09-18T04:58:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118444+0000", 2024-09-18T04:58:27.786 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4360000000000001e-06 2024-09-18T04:58:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118448+0000", 2024-09-18T04:58:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.357e-06 2024-09-18T04:58:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162620+0000", 2024-09-18T04:58:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044171821 2024-09-18T04:58:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:27.787 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162653+0000", 2024-09-18T04:58:27.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3479999999999998e-05 2024-09-18T04:58:27.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.788 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:27.788 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162699+0000", 2024-09-18T04:58:27.788 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5113e-05 2024-09-18T04:58:27.788 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.788 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:27.788 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.788 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.788 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.788 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60585 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1211392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:27.789 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:58:27.789 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.612403811999997, 2024-09-18T04:58:27.789 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044389814, 2024-09-18T04:58:27.789 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:27.789 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:27.789 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:27.789 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:27.789 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:27.790 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60585 2024-09-18T04:58:27.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.790 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:27.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:27.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:58:27.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.790 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:27.790 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:58:27.790 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.790 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.790 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:27.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120245+0000", 2024-09-18T04:58:27.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:27.791 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.791 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.791 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:27.791 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120247+0000", 2024-09-18T04:58:27.791 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0150000000000002e-06 2024-09-18T04:58:27.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:27.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120255+0000", 2024-09-18T04:58:27.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.858e-06 2024-09-18T04:58:27.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:27.792 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120260+0000", 2024-09-18T04:58:27.792 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6530000000000002e-06 2024-09-18T04:58:27.792 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.792 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.792 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:27.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164489+0000", 2024-09-18T04:58:27.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044229483999999999 2024-09-18T04:58:27.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:27.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164555+0000", 2024-09-18T04:58:27.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.533e-05 2024-09-18T04:58:27.793 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.793 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.793 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:27.793 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164636+0000", 2024-09-18T04:58:27.793 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0819e-05 2024-09-18T04:58:27.793 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60586 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1226752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:58:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.612385360999994, 2024-09-18T04:58:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044578214999999997, 2024-09-18T04:58:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:27.794 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60586 2024-09-18T04:58:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:58:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:58:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.795 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:27.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:58:27.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:27.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:27.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120266+0000", 2024-09-18T04:58:27.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0660000000000002e-06 2024-09-18T04:58:27.796 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.796 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.796 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:27.796 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120272+0000", 2024-09-18T04:58:27.796 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-18T04:58:27.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.797 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.797 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:27.797 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120276+0000", 2024-09-18T04:58:27.797 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.476e-06 2024-09-18T04:58:27.797 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:27.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164673+0000", 2024-09-18T04:58:27.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044396672999999998 2024-09-18T04:58:27.798 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.798 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.798 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:27.798 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164749+0000", 2024-09-18T04:58:27.798 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5811999999999999e-05 2024-09-18T04:58:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164842+0000", 2024-09-18T04:58:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3560000000000005e-05 2024-09-18T04:58:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60587 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1242112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:58:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.612336261999999, 2024-09-18T04:58:27.799 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044634794999999998, 2024-09-18T04:58:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60587 2024-09-18T04:58:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:58:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.800 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:58:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120280+0000", 2024-09-18T04:58:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999671 2024-09-18T04:58:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:27.801 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120316+0000", 2024-09-18T04:58:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5728000000000001e-05 2024-09-18T04:58:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120329+0000", 2024-09-18T04:58:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2914e-05 2024-09-18T04:58:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120335+0000", 2024-09-18T04:58:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8289999999999996e-06 2024-09-18T04:58:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.802 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164858+0000", 2024-09-18T04:58:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044523417000000003 2024-09-18T04:58:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164896+0000", 2024-09-18T04:58:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7969999999999997e-05 2024-09-18T04:58:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164948+0000", 2024-09-18T04:58:27.803 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1996000000000002e-05 2024-09-18T04:58:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60596 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1380352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:58:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.610476724999998, 2024-09-18T04:58:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044120800000000002, 2024-09-18T04:58:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:27.804 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60596 2024-09-18T04:58:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:58:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:58:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.805 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:58:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122175+0000", 2024-09-18T04:58:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:58:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:27.806 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122182+0000", 2024-09-18T04:58:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3509999999999996e-06 2024-09-18T04:58:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122187+0000", 2024-09-18T04:58:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7940000000000002e-06 2024-09-18T04:58:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166184+0000", 2024-09-18T04:58:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043997236000000002 2024-09-18T04:58:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.807 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166227+0000", 2024-09-18T04:58:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2561000000000001e-05 2024-09-18T04:58:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166294+0000", 2024-09-18T04:58:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7068000000000002e-05 2024-09-18T04:58:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.808 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60597 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1395712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:58:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 96.610458696999999, 2024-09-18T04:58:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044242166999999999, 2024-09-18T04:58:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60597 2024-09-18T04:58:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.809 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:58:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:58:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:27.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:58:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122193+0000", 2024-09-18T04:58:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:58:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122199+0000", 2024-09-18T04:58:27.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6449999999999999e-06 2024-09-18T04:58:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122204+0000", 2024-09-18T04:58:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4819999999999996e-06 2024-09-18T04:58:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166329+0000", 2024-09-18T04:58:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044125059000000001 2024-09-18T04:58:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166370+0000", 2024-09-18T04:58:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1012e-05 2024-09-18T04:58:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166433+0000", 2024-09-18T04:58:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3178999999999999e-05 2024-09-18T04:58:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167074 3.5 3:a712c57a:::rbd_data.11ac138032e7.0000000000000000:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:27.813 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.163703+0000", 2024-09-18T04:58:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.5689465160000005, 2024-09-18T04:58:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.90992387799999996, 2024-09-18T04:58:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167074 2024-09-18T04:58:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163703+0000", 2024-09-18T04:58:27.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163703+0000", 2024-09-18T04:58:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163702+0000", 2024-09-18T04:58:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:58:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:27.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163707+0000", 2024-09-18T04:58:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5159999999999996e-06 2024-09-18T04:58:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163716+0000", 2024-09-18T04:58:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5270000000000004e-06 2024-09-18T04:58:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163723+0000", 2024-09-18T04:58:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9809999999999997e-06 2024-09-18T04:58:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163781+0000", 2024-09-18T04:58:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7494999999999998e-05 2024-09-18T04:58:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163791+0000", 2024-09-18T04:58:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0614999999999999e-05 2024-09-18T04:58:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067213+0000", 2024-09-18T04:58:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.90342179700000003 2024-09-18T04:58:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067327+0000", 2024-09-18T04:58:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000114023 2024-09-18T04:58:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-18T04:58:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067432+0000", 2024-09-18T04:58:27.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104775 2024-09-18T04:58:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.071972+0000", 2024-09-18T04:58:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0045401219999999997 2024-09-18T04:58:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073530+0000", 2024-09-18T04:58:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001558341 2024-09-18T04:58:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:27.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073582+0000", 2024-09-18T04:58:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1924e-05 2024-09-18T04:58:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073627+0000", 2024-09-18T04:58:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4647000000000001e-05 2024-09-18T04:58:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.820 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167075 3.1 3:864982b2:::rbd_data.11ac138032e7.0000000000000001:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.163740+0000", 2024-09-18T04:58:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.5689092480000006, 2024-09-18T04:58:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.90703181499999996, 2024-09-18T04:58:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167075 2024-09-18T04:58:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163740+0000", 2024-09-18T04:58:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163740+0000", 2024-09-18T04:58:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163740+0000", 2024-09-18T04:58:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163743+0000", 2024-09-18T04:58:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0230000000000001e-06 2024-09-18T04:58:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163750+0000", 2024-09-18T04:58:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2710000000000003e-06 2024-09-18T04:58:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163756+0000", 2024-09-18T04:58:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7559999999999996e-06 2024-09-18T04:58:27.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163803+0000", 2024-09-18T04:58:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6576999999999998e-05 2024-09-18T04:58:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163813+0000", 2024-09-18T04:58:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0336000000000001e-05 2024-09-18T04:58:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.066573+0000", 2024-09-18T04:58:27.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.90276046799999998 2024-09-18T04:58:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.066694+0000", 2024-09-18T04:58:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012100299999999999 2024-09-18T04:58:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:58:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.066791+0000", 2024-09-18T04:58:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7141999999999999e-05 2024-09-18T04:58:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:27.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.068686+0000", 2024-09-18T04:58:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018949570000000001 2024-09-18T04:58:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070676+0000", 2024-09-18T04:58:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001989158 2024-09-18T04:58:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070733+0000", 2024-09-18T04:58:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7642000000000001e-05 2024-09-18T04:58:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070772+0000", 2024-09-18T04:58:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8949000000000003e-05 2024-09-18T04:58:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167093 3.1 3:82b38587:::rbd_data.11ac138032e7.0000000000000025:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.173880+0000", 2024-09-18T04:58:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.5587692220000005, 2024-09-18T04:58:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.89950706999999996, 2024-09-18T04:58:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:27.827 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167093 2024-09-18T04:58:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173880+0000", 2024-09-18T04:58:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:27.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173880+0000", 2024-09-18T04:58:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173880+0000", 2024-09-18T04:58:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173884+0000", 2024-09-18T04:58:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9910000000000004e-06 2024-09-18T04:58:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173895+0000", 2024-09-18T04:58:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1238e-05 2024-09-18T04:58:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173903+0000", 2024-09-18T04:58:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2989999999999997e-06 2024-09-18T04:58:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173954+0000", 2024-09-18T04:58:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0862999999999997e-05 2024-09-18T04:58:27.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:27.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173965+0000", 2024-09-18T04:58:27.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0658e-05 2024-09-18T04:58:27.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:27.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067554+0000", 2024-09-18T04:58:27.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.89358902200000001 2024-09-18T04:58:27.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:27.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067664+0000", 2024-09-18T04:58:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010977200000000001 2024-09-18T04:58:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:58:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067754+0000", 2024-09-18T04:58:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0484999999999998e-05 2024-09-18T04:58:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070563+0000", 2024-09-18T04:58:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028086650000000001 2024-09-18T04:58:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073294+0000", 2024-09-18T04:58:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027316599999999999 2024-09-18T04:58:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073351+0000", 2024-09-18T04:58:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6873999999999998e-05 2024-09-18T04:58:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073387+0000", 2024-09-18T04:58:27.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6254000000000002e-05 2024-09-18T04:58:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167095 3.5 3:a340fea6:::rbd_data.11ac138032e7.0000000000000027:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.173930+0000", 2024-09-18T04:58:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.5587194879999995, 2024-09-18T04:58:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.89985602499999995, 2024-09-18T04:58:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:27.834 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167095 2024-09-18T04:58:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173930+0000", 2024-09-18T04:58:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173930+0000", 2024-09-18T04:58:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173929+0000", 2024-09-18T04:58:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173933+0000", 2024-09-18T04:58:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4860000000000002e-06 2024-09-18T04:58:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173940+0000", 2024-09-18T04:58:27.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4499999999999998e-06 2024-09-18T04:58:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173945+0000", 2024-09-18T04:58:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5290000000000002e-06 2024-09-18T04:58:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173981+0000", 2024-09-18T04:58:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6548999999999999e-05 2024-09-18T04:58:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:27.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173990+0000", 2024-09-18T04:58:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1810000000000001e-06 2024-09-18T04:58:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.069067+0000", 2024-09-18T04:58:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.89507692699999997 2024-09-18T04:58:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.069196+0000", 2024-09-18T04:58:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000129572 2024-09-18T04:58:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-18T04:58:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.069316+0000", 2024-09-18T04:58:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120165 2024-09-18T04:58:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.072036+0000", 2024-09-18T04:58:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027195489999999999 2024-09-18T04:58:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073703+0000", 2024-09-18T04:58:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001667555 2024-09-18T04:58:27.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073749+0000", 2024-09-18T04:58:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5636000000000001e-05 2024-09-18T04:58:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073786+0000", 2024-09-18T04:58:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7070000000000003e-05 2024-09-18T04:58:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167097 3.c 3:3137739a:::rbd_data.11ac138032e7.0000000000000029:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.178900+0000", 2024-09-18T04:58:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.5537497019999993, 2024-09-18T04:58:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.89454723999999997, 2024-09-18T04:58:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167097 2024-09-18T04:58:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.841 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178900+0000", 2024-09-18T04:58:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178900+0000", 2024-09-18T04:58:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178899+0000", 2024-09-18T04:58:27.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:58:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178903+0000", 2024-09-18T04:58:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7699999999999999e-06 2024-09-18T04:58:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178913+0000", 2024-09-18T04:58:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0032e-05 2024-09-18T04:58:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178918+0000", 2024-09-18T04:58:27.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9740000000000001e-06 2024-09-18T04:58:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178959+0000", 2024-09-18T04:58:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1257999999999997e-05 2024-09-18T04:58:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178966+0000", 2024-09-18T04:58:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4139999999999999e-06 2024-09-18T04:58:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:27.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.066273+0000", 2024-09-18T04:58:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.88730742299999998 2024-09-18T04:58:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.066394+0000", 2024-09-18T04:58:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012086099999999999 2024-09-18T04:58:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-18T04:58:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.066503+0000", 2024-09-18T04:58:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109218 2024-09-18T04:58:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070854+0000", 2024-09-18T04:58:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0043508219999999998 2024-09-18T04:58:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073373+0000", 2024-09-18T04:58:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0025193519999999999 2024-09-18T04:58:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073413+0000", 2024-09-18T04:58:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9592999999999999e-05 2024-09-18T04:58:27.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073447+0000", 2024-09-18T04:58:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4239999999999997e-05 2024-09-18T04:58:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:27.847 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:27.847 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:27.849 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:58:27.849 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:27.846+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:27.949 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:27.947+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:27.956 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:27.956 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:27.956 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:58:27.956 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:27.966 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:58:28.020 INFO:teuthology.orchestra.run.smithi002.stderr:marked in osd.3. 2024-09-18T04:58:28.037 INFO:tasks.thrashosds.thrasher:Added osd 3 2024-09-18T04:58:28.050 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:28.048+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:28.109 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T04:58:28.125 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:28.125 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:28.125 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:58:28.125 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:58:28.125 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:28.135 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:58:28.152 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:28.150+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:28.251 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:28.248+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:28.282 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:58:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:58:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:28.282 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26200 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 32768~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:58:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 107.638906268, 2024-09-18T04:58:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022501118000000001, 2024-09-18T04:58:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.283 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.284 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26200 2024-09-18T04:58:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:58:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.285 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:58:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.286 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:58:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639822+0000", 2024-09-18T04:58:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.007e-06 2024-09-18T04:58:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639827+0000", 2024-09-18T04:58:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5709999999999997e-06 2024-09-18T04:58:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.287 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639831+0000", 2024-09-18T04:58:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2819999999999998e-06 2024-09-18T04:58:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662181+0000", 2024-09-18T04:58:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022349878 2024-09-18T04:58:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662252+0000", 2024-09-18T04:58:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0542000000000003e-05 2024-09-18T04:58:28.288 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662321+0000", 2024-09-18T04:58:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9167999999999999e-05 2024-09-18T04:58:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26201 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 61440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:58:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 107.638890977, 2024-09-18T04:58:28.289 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022569783, 2024-09-18T04:58:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26201 2024-09-18T04:58:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:58:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.290 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:58:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:58:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639837+0000", 2024-09-18T04:58:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.928e-06 2024-09-18T04:58:28.291 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639843+0000", 2024-09-18T04:58:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8130000000000001e-06 2024-09-18T04:58:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639847+0000", 2024-09-18T04:58:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0999999999999997e-06 2024-09-18T04:58:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.292 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662335+0000", 2024-09-18T04:58:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022488207999999999 2024-09-18T04:58:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662370+0000", 2024-09-18T04:58:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5292e-05 2024-09-18T04:58:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662405+0000", 2024-09-18T04:58:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4765999999999999e-05 2024-09-18T04:58:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.293 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26202 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 90112~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:58:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 107.63887556, 2024-09-18T04:58:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022655937000000001, 2024-09-18T04:58:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26202 2024-09-18T04:58:28.294 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:58:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:58:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.295 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639850+0000", 2024-09-18T04:58:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639852+0000", 2024-09-18T04:58:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9700000000000002e-06 2024-09-18T04:58:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639858+0000", 2024-09-18T04:58:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.434e-06 2024-09-18T04:58:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.296 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639862+0000", 2024-09-18T04:58:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3499999999999999e-06 2024-09-18T04:58:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662429+0000", 2024-09-18T04:58:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022567409 2024-09-18T04:58:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662469+0000", 2024-09-18T04:58:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9801000000000001e-05 2024-09-18T04:58:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.297 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662507+0000", 2024-09-18T04:58:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7311000000000002e-05 2024-09-18T04:58:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26203 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 118784~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:58:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 107.638860405, 2024-09-18T04:58:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022730358999999999, 2024-09-18T04:58:28.298 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26203 2024-09-18T04:58:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:58:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.299 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:58:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639865+0000", 2024-09-18T04:58:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639867+0000", 2024-09-18T04:58:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0889999999999998e-06 2024-09-18T04:58:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.300 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639873+0000", 2024-09-18T04:58:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4979999999999997e-06 2024-09-18T04:58:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639877+0000", 2024-09-18T04:58:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4510000000000002e-06 2024-09-18T04:58:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662528+0000", 2024-09-18T04:58:28.301 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022650791999999999 2024-09-18T04:58:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662563+0000", 2024-09-18T04:58:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4530000000000003e-05 2024-09-18T04:58:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662596+0000", 2024-09-18T04:58:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3463000000000001e-05 2024-09-18T04:58:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.302 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26204 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 147456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:58:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 107.63884501299999, 2024-09-18T04:58:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022803233999999999, 2024-09-18T04:58:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26204 2024-09-18T04:58:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.303 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:58:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:58:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:58:28.304 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639883+0000", 2024-09-18T04:58:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6520000000000002e-06 2024-09-18T04:58:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639888+0000", 2024-09-18T04:58:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2870000000000004e-06 2024-09-18T04:58:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.305 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639893+0000", 2024-09-18T04:58:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3839999999999999e-06 2024-09-18T04:58:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662617+0000", 2024-09-18T04:58:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022723876 2024-09-18T04:58:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662651+0000", 2024-09-18T04:58:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3797000000000002e-05 2024-09-18T04:58:28.306 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.307 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662684+0000", 2024-09-18T04:58:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3828999999999997e-05 2024-09-18T04:58:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26205 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 176128~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:58:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 107.63882205199999, 2024-09-18T04:58:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022919108000000001, 2024-09-18T04:58:28.308 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26205 2024-09-18T04:58:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:58:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.309 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:58:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:58:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639906+0000", 2024-09-18T04:58:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.531e-06 2024-09-18T04:58:28.310 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639913+0000", 2024-09-18T04:58:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.331e-06 2024-09-18T04:58:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639920+0000", 2024-09-18T04:58:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5399999999999998e-06 2024-09-18T04:58:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662706+0000", 2024-09-18T04:58:28.311 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.02278587 2024-09-18T04:58:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662764+0000", 2024-09-18T04:58:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7630000000000002e-05 2024-09-18T04:58:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662823+0000", 2024-09-18T04:58:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9648000000000002e-05 2024-09-18T04:58:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.312 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26206 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 204800~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:58:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 107.638801653, 2024-09-18T04:58:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023047517, 2024-09-18T04:58:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.313 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26206 2024-09-18T04:58:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:58:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:58:28.314 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:58:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639926+0000", 2024-09-18T04:58:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3120000000000001e-06 2024-09-18T04:58:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.315 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639932+0000", 2024-09-18T04:58:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4890000000000003e-06 2024-09-18T04:58:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639936+0000", 2024-09-18T04:58:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.07e-06 2024-09-18T04:58:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662850+0000", 2024-09-18T04:58:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022913579999999999 2024-09-18T04:58:28.316 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662903+0000", 2024-09-18T04:58:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3176e-05 2024-09-18T04:58:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662972+0000", 2024-09-18T04:58:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9270999999999995e-05 2024-09-18T04:58:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.317 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26207 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 233472~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:58:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 107.638786509, 2024-09-18T04:58:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023139541999999999, 2024-09-18T04:58:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26207 2024-09-18T04:58:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.318 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:58:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:58:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639939+0000", 2024-09-18T04:58:28.319 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639941+0000", 2024-09-18T04:58:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0080000000000001e-06 2024-09-18T04:58:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639947+0000", 2024-09-18T04:58:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.485e-06 2024-09-18T04:58:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.320 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639951+0000", 2024-09-18T04:58:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4549999999999997e-06 2024-09-18T04:58:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663005+0000", 2024-09-18T04:58:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023053569999999999 2024-09-18T04:58:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663042+0000", 2024-09-18T04:58:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7245e-05 2024-09-18T04:58:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.321 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663079+0000", 2024-09-18T04:58:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7119000000000002e-05 2024-09-18T04:58:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26208 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 524288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:58:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 107.63876976100001, 2024-09-18T04:58:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023283564999999999, 2024-09-18T04:58:28.322 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26208 2024-09-18T04:58:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:58:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.323 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:58:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.324 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:58:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639958+0000", 2024-09-18T04:58:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0899999999999999e-06 2024-09-18T04:58:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639963+0000", 2024-09-18T04:58:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2209999999999996e-06 2024-09-18T04:58:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639967+0000", 2024-09-18T04:58:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1169999999999997e-06 2024-09-18T04:58:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663114+0000", 2024-09-18T04:58:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023146102000000002 2024-09-18T04:58:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663183+0000", 2024-09-18T04:58:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9686999999999999e-05 2024-09-18T04:58:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663240+0000", 2024-09-18T04:58:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6728000000000003e-05 2024-09-18T04:58:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26209 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 552960~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:28.327 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:58:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 107.638751985, 2024-09-18T04:58:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023440966000000001, 2024-09-18T04:58:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26209 2024-09-18T04:58:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:58:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:58:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:58:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639976+0000", 2024-09-18T04:58:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4940000000000002e-06 2024-09-18T04:58:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639982+0000", 2024-09-18T04:58:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0560000000000001e-06 2024-09-18T04:58:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639987+0000", 2024-09-18T04:58:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-18T04:58:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663278+0000", 2024-09-18T04:58:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023290789999999999 2024-09-18T04:58:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663348+0000", 2024-09-18T04:58:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0837e-05 2024-09-18T04:58:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.331 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663415+0000", 2024-09-18T04:58:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6736000000000006e-05 2024-09-18T04:58:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26210 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 581632~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:58:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 107.638734977, 2024-09-18T04:58:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023616437000000001, 2024-09-18T04:58:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.332 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.333 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.333 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.333 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.333 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26210 2024-09-18T04:58:28.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.333 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:58:28.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:58:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:58:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639993+0000", 2024-09-18T04:58:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.419e-06 2024-09-18T04:58:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640006+0000", 2024-09-18T04:58:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2918000000000001e-05 2024-09-18T04:58:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640011+0000", 2024-09-18T04:58:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9320000000000004e-06 2024-09-18T04:58:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663454+0000", 2024-09-18T04:58:28.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023443035000000001 2024-09-18T04:58:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663532+0000", 2024-09-18T04:58:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7443999999999999e-05 2024-09-18T04:58:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663608+0000", 2024-09-18T04:58:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6093000000000005e-05 2024-09-18T04:58:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26211 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 610304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:58:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 107.638710425, 2024-09-18T04:58:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023774337999999999, 2024-09-18T04:58:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26211 2024-09-18T04:58:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:58:28.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:58:28.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.338 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.338 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.338 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.338 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640015+0000", 2024-09-18T04:58:28.338 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640023+0000", 2024-09-18T04:58:28.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3900000000000004e-06 2024-09-18T04:58:28.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640036+0000", 2024-09-18T04:58:28.339 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3091999999999999e-05 2024-09-18T04:58:28.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.339 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640041+0000", 2024-09-18T04:58:28.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6219999999999997e-06 2024-09-18T04:58:28.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663644+0000", 2024-09-18T04:58:28.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023602643999999999 2024-09-18T04:58:28.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663718+0000", 2024-09-18T04:58:28.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4481999999999996e-05 2024-09-18T04:58:28.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663790+0000", 2024-09-18T04:58:28.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1589999999999997e-05 2024-09-18T04:58:28.341 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.341 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.341 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.341 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26212 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 638976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:28.341 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:58:28.341 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 107.63867738499999, 2024-09-18T04:58:28.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023924748999999999, 2024-09-18T04:58:28.341 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.341 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.342 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.342 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.342 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.342 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26212 2024-09-18T04:58:28.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.342 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:58:28.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:58:28.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640048+0000", 2024-09-18T04:58:28.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640051+0000", 2024-09-18T04:58:28.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.095e-06 2024-09-18T04:58:28.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640060+0000", 2024-09-18T04:58:28.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8929999999999994e-06 2024-09-18T04:58:28.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640065+0000", 2024-09-18T04:58:28.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0270000000000003e-06 2024-09-18T04:58:28.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663824+0000", 2024-09-18T04:58:28.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023758873999999999 2024-09-18T04:58:28.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663904+0000", 2024-09-18T04:58:28.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9856000000000003e-05 2024-09-18T04:58:28.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.345 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.345 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663974+0000", 2024-09-18T04:58:28.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9450999999999996e-05 2024-09-18T04:58:28.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.345 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.345 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.346 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26213 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 667648~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:28.346 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:58:28.346 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 107.63865621399999, 2024-09-18T04:58:28.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024059812999999999, 2024-09-18T04:58:28.346 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.346 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.346 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.346 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.346 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.346 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26213 2024-09-18T04:58:28.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.346 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:58:28.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:58:28.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:58:28.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640072+0000", 2024-09-18T04:58:28.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4200000000000001e-06 2024-09-18T04:58:28.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640078+0000", 2024-09-18T04:58:28.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0179999999999998e-06 2024-09-18T04:58:28.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640083+0000", 2024-09-18T04:58:28.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:58:28.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664008+0000", 2024-09-18T04:58:28.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023925287 2024-09-18T04:58:28.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664059+0000", 2024-09-18T04:58:28.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0730999999999999e-05 2024-09-18T04:58:28.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664130+0000", 2024-09-18T04:58:28.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0801000000000002e-05 2024-09-18T04:58:28.350 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.350 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.350 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.350 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26214 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 696320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:28.350 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:58:28.350 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 107.638639244, 2024-09-18T04:58:28.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024223738000000002, 2024-09-18T04:58:28.350 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.350 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.351 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.351 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.351 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.351 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26214 2024-09-18T04:58:28.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.351 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:58:28.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:58:28.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:58:28.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.352 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640089+0000", 2024-09-18T04:58:28.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0849999999999999e-06 2024-09-18T04:58:28.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.352 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640095+0000", 2024-09-18T04:58:28.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9279999999999998e-06 2024-09-18T04:58:28.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640099+0000", 2024-09-18T04:58:28.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:58:28.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664166+0000", 2024-09-18T04:58:28.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024066513000000001 2024-09-18T04:58:28.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664239+0000", 2024-09-18T04:58:28.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3201999999999995e-05 2024-09-18T04:58:28.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664311+0000", 2024-09-18T04:58:28.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1434999999999996e-05 2024-09-18T04:58:28.354 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.354 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.354 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.355 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26215 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 724992~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:28.355 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:58:28.355 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 107.63862248300001, 2024-09-18T04:58:28.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024302284, 2024-09-18T04:58:28.355 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.355 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.355 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.355 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.355 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.355 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26215 2024-09-18T04:58:28.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.355 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:58:28.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:58:28.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640103+0000", 2024-09-18T04:58:28.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640106+0000", 2024-09-18T04:58:28.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2120000000000002e-06 2024-09-18T04:58:28.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640111+0000", 2024-09-18T04:58:28.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7880000000000003e-06 2024-09-18T04:58:28.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.357 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640116+0000", 2024-09-18T04:58:28.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6099999999999999e-06 2024-09-18T04:58:28.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664327+0000", 2024-09-18T04:58:28.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024210941999999999 2024-09-18T04:58:28.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664367+0000", 2024-09-18T04:58:28.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9666000000000003e-05 2024-09-18T04:58:28.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664406+0000", 2024-09-18T04:58:28.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9402e-05 2024-09-18T04:58:28.359 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.359 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.359 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.359 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26216 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 753664~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:28.359 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:58:28.359 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 107.638605989, 2024-09-18T04:58:28.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024452854999999999, 2024-09-18T04:58:28.359 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.359 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.359 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.360 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.360 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.360 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26216 2024-09-18T04:58:28.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.360 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:58:28.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:58:28.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:58:28.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.361 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640122+0000", 2024-09-18T04:58:28.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3599999999999999e-06 2024-09-18T04:58:28.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.361 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640128+0000", 2024-09-18T04:58:28.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6339999999999998e-06 2024-09-18T04:58:28.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640132+0000", 2024-09-18T04:58:28.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:58:28.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664436+0000", 2024-09-18T04:58:28.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024303585999999999 2024-09-18T04:58:28.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664507+0000", 2024-09-18T04:58:28.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1046999999999999e-05 2024-09-18T04:58:28.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664573+0000", 2024-09-18T04:58:28.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6217000000000007e-05 2024-09-18T04:58:28.363 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.363 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.363 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.364 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26217 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 782336~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:28.364 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:58:28.364 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 107.638589755, 2024-09-18T04:58:28.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024526287000000001, 2024-09-18T04:58:28.364 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.364 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.364 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.364 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.364 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.364 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26217 2024-09-18T04:58:28.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.364 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:58:28.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:58:28.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:58:28.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640140+0000", 2024-09-18T04:58:28.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2989999999999998e-06 2024-09-18T04:58:28.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.366 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640146+0000", 2024-09-18T04:58:28.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9569999999999999e-06 2024-09-18T04:58:28.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640151+0000", 2024-09-18T04:58:28.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:58:28.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664590+0000", 2024-09-18T04:58:28.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0244389 2024-09-18T04:58:28.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664628+0000", 2024-09-18T04:58:28.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7910999999999997e-05 2024-09-18T04:58:28.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664663+0000", 2024-09-18T04:58:28.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5024999999999998e-05 2024-09-18T04:58:28.368 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.368 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.368 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.368 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167100 3.1e 3:7bd358cc:::rbd_data.11ac138032e7.000000000000004f:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:28.368 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.178944+0000", 2024-09-18T04:58:28.369 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0997824769999998, 2024-09-18T04:58:28.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.89428526100000005, 2024-09-18T04:58:28.369 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.369 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:28.369 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.369 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.369 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.369 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167100 2024-09-18T04:58:28.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.369 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178944+0000", 2024-09-18T04:58:28.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178944+0000", 2024-09-18T04:58:28.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178942+0000", 2024-09-18T04:58:28.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:58:28.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178948+0000", 2024-09-18T04:58:28.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8100000000000003e-06 2024-09-18T04:58:28.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178962+0000", 2024-09-18T04:58:28.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.399e-05 2024-09-18T04:58:28.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.371 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.371 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178970+0000", 2024-09-18T04:58:28.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.537e-06 2024-09-18T04:58:28.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.179015+0000", 2024-09-18T04:58:28.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5652999999999998e-05 2024-09-18T04:58:28.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:28.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.179021+0000", 2024-09-18T04:58:28.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5280000000000003e-06 2024-09-18T04:58:28.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.065479+0000", 2024-09-18T04:58:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.88645855799999995 2024-09-18T04:58:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.065788+0000", 2024-09-18T04:58:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030816099999999999 2024-09-18T04:58:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-18T04:58:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.065904+0000", 2024-09-18T04:58:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116554 2024-09-18T04:58:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:28.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067684+0000", 2024-09-18T04:58:28.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001780195 2024-09-18T04:58:28.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:28.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073137+0000", 2024-09-18T04:58:28.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0054525210000000001 2024-09-18T04:58:28.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:28.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073193+0000", 2024-09-18T04:58:28.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6492000000000001e-05 2024-09-18T04:58:28.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073229+0000", 2024-09-18T04:58:28.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5664000000000003e-05 2024-09-18T04:58:28.375 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.375 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.375 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.375 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167111 3.19 3:99c9092f:::rbd_data.11ac138032e7.000000000000005a:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:28.375 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.201980+0000", 2024-09-18T04:58:28.375 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.0767458750000003, 2024-09-18T04:58:28.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.87136403900000003, 2024-09-18T04:58:28.375 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.376 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:28.376 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.376 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.376 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.376 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167111 2024-09-18T04:58:28.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.376 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.201980+0000", 2024-09-18T04:58:28.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.201980+0000", 2024-09-18T04:58:28.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.201980+0000", 2024-09-18T04:58:28.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.201983+0000", 2024-09-18T04:58:28.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8949999999999998e-06 2024-09-18T04:58:28.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.201990+0000", 2024-09-18T04:58:28.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5129999999999999e-06 2024-09-18T04:58:28.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.201999+0000", 2024-09-18T04:58:28.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5490000000000006e-06 2024-09-18T04:58:28.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.202033+0000", 2024-09-18T04:58:28.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4076999999999999e-05 2024-09-18T04:58:28.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:28.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.202038+0000", 2024-09-18T04:58:28.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3310000000000003e-06 2024-09-18T04:58:28.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.068314+0000", 2024-09-18T04:58:28.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.86627544899999998 2024-09-18T04:58:28.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.068426+0000", 2024-09-18T04:58:28.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011262199999999999 2024-09-18T04:58:28.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 7", 2024-09-18T04:58:28.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.068531+0000", 2024-09-18T04:58:28.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010510600000000001 2024-09-18T04:58:28.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.380 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:28.380 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.071305+0000", 2024-09-18T04:58:28.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027742539999999999 2024-09-18T04:58:28.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:28.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073258+0000", 2024-09-18T04:58:28.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001952264 2024-09-18T04:58:28.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:28.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073303+0000", 2024-09-18T04:58:28.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5286e-05 2024-09-18T04:58:28.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073344+0000", 2024-09-18T04:58:28.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1269000000000001e-05 2024-09-18T04:58:28.382 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.382 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.382 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.382 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.382 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.382 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:28.384 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:58:28.384 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:28.381+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:28.484 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:28.481+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:28.529 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:28.529 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:28.529 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:58:28.529 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:28.538 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:58:28.585 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:28.583+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:28.684 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:28.684 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:28.684 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:58:28.684 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:58:28.684 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:28.685 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:28.683+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:28.693 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:58:28.787 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:28.784+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:28.809 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:28.809 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:58:28.809 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:58:28.809 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:28.809 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.809 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:58:28.809 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:58:28.809 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 137.41244568400001, 2024-09-18T04:58:28.810 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037718511000000003, 2024-09-18T04:58:28.810 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.810 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:28.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:58:28.810 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:58:28.810 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-18T04:58:28.810 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.810 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.810 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.810 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.810 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:58:28.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:58:28.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.811 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393125+0000", 2024-09-18T04:58:28.811 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:58:28.811 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.811 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.811 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393133+0000", 2024-09-18T04:58:28.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.225e-06 2024-09-18T04:58:28.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393144+0000", 2024-09-18T04:58:28.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1279e-05 2024-09-18T04:58:28.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.812 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.812 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.812 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393153+0000", 2024-09-18T04:58:28.812 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4540000000000007e-06 2024-09-18T04:58:28.812 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393203+0000", 2024-09-18T04:58:28.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9963000000000003e-05 2024-09-18T04:58:28.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.813 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.813 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:28.813 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393210+0000", 2024-09-18T04:58:28.813 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5689999999999999e-06 2024-09-18T04:58:28.813 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427399+0000", 2024-09-18T04:58:28.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034188784999999999 2024-09-18T04:58:28.814 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.814 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.814 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.814 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427532+0000", 2024-09-18T04:58:28.814 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013264700000000001 2024-09-18T04:58:28.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:58:28.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427685+0000", 2024-09-18T04:58:28.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015300599999999999 2024-09-18T04:58:28.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:28.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.428520+0000", 2024-09-18T04:58:28.815 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00083502499999999998 2024-09-18T04:58:28.815 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.815 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.815 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:28.815 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430772+0000", 2024-09-18T04:58:28.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022518880000000001 2024-09-18T04:58:28.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:28.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430817+0000", 2024-09-18T04:58:28.816 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5077000000000002e-05 2024-09-18T04:58:28.816 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.816 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.816 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.816 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430845+0000", 2024-09-18T04:58:28.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8036999999999999e-05 2024-09-18T04:58:28.817 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.817 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.817 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.817 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.817 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.817 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:62520 3.13 3.84cdb893 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T04:58:28.817 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:28.817 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 97.508135800999995, 2024-09-18T04:58:28.817 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94452134799999998, 2024-09-18T04:58:28.817 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.818 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:58:28.818 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.818 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.818 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.818 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 62520 2024-09-18T04:58:28.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.818 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.818 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.818 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:28.818 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.818 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.818 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:28.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:28.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.819 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.819 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.819 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.819 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297441+0000", 2024-09-18T04:58:28.819 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9840000000000001e-06 2024-09-18T04:58:28.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297449+0000", 2024-09-18T04:58:28.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3999999999999992e-06 2024-09-18T04:58:28.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297455+0000", 2024-09-18T04:58:28.820 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3350000000000003e-06 2024-09-18T04:58:28.820 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.820 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.820 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.820 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297552+0000", 2024-09-18T04:58:28.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6602999999999996e-05 2024-09-18T04:58:28.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:28.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297559+0000", 2024-09-18T04:58:28.821 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.52e-06 2024-09-18T04:58:28.821 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.821 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.821 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.821 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241943+0000", 2024-09-18T04:58:28.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94438357100000003 2024-09-18T04:58:28.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.822 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.822 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241958+0000", 2024-09-18T04:58:28.822 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4598e-05 2024-09-18T04:58:28.822 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.822 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.822 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.822 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.822 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.822 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:113179 3.13 3:cf138017:::rbd_data.11ac5f86a3f9.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T04:58:28.822 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:58:28.823 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 70.118345390000002, 2024-09-18T04:58:28.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036508304999999998, 2024-09-18T04:58:28.823 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.823 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:28.823 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.823 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.823 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.823 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 113179 2024-09-18T04:58:28.823 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.823 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.823 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.823 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.823 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:58:28.823 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:58:28.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.824 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.824 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687226+0000", 2024-09-18T04:58:28.824 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:58:28.824 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.824 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688169+0000", 2024-09-18T04:58:28.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00094328200000000004 2024-09-18T04:58:28.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.825 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688182+0000", 2024-09-18T04:58:28.825 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2756e-05 2024-09-18T04:58:28.825 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.825 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.825 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688188+0000", 2024-09-18T04:58:28.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0290000000000002e-06 2024-09-18T04:58:28.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.826 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.826 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688225+0000", 2024-09-18T04:58:28.826 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7301000000000001e-05 2024-09-18T04:58:28.826 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.826 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688324+0000", 2024-09-18T04:58:28.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9419999999999993e-05 2024-09-18T04:58:28.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:58:28.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.700483+0000", 2024-09-18T04:58:28.827 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012158771000000001 2024-09-18T04:58:28.827 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.827 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.827 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:28.827 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723586+0000", 2024-09-18T04:58:28.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023102952 2024-09-18T04:58:28.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:28.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723634+0000", 2024-09-18T04:58:28.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7468e-05 2024-09-18T04:58:28.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:28.828 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723693+0000", 2024-09-18T04:58:28.828 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9539e-05 2024-09-18T04:58:28.828 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.828 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.828 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.829 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723735+0000", 2024-09-18T04:58:28.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1829000000000002e-05 2024-09-18T04:58:28.829 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.829 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.829 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.829 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.829 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.829 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146078 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1449984~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:28.829 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:28.829 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.212267547000003, 2024-09-18T04:58:28.829 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035495325000000001, 2024-09-18T04:58:28.829 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.829 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.830 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.830 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.830 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.830 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146078 2024-09-18T04:58:28.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.830 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.830 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.830 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:28.830 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.830 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.830 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:28.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:28.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.831 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.831 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.831 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593306+0000", 2024-09-18T04:58:28.831 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.686e-06 2024-09-18T04:58:28.831 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593311+0000", 2024-09-18T04:58:28.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8500000000000002e-06 2024-09-18T04:58:28.832 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.832 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.832 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.832 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593314+0000", 2024-09-18T04:58:28.832 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0290000000000001e-06 2024-09-18T04:58:28.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628674+0000", 2024-09-18T04:58:28.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035360346000000001 2024-09-18T04:58:28.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.833 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628726+0000", 2024-09-18T04:58:28.833 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2219999999999998e-05 2024-09-18T04:58:28.833 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.833 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.833 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.834 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628800+0000", 2024-09-18T04:58:28.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3613000000000001e-05 2024-09-18T04:58:28.834 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.834 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.834 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.834 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.834 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.834 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146079 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1499136~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:28.834 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:28.834 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.212256142000001, 2024-09-18T04:58:28.834 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035604581000000003, 2024-09-18T04:58:28.834 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.834 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.835 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.835 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.835 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.835 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146079 2024-09-18T04:58:28.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.835 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.835 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:28.835 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.835 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.835 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.835 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:28.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.836 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.836 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.836 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.836 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:28.836 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:28.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593317+0000", 2024-09-18T04:58:28.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1599999999999999e-06 2024-09-18T04:58:28.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.837 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.837 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593320+0000", 2024-09-18T04:58:28.837 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0130000000000002e-06 2024-09-18T04:58:28.837 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.837 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.838 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.838 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593322+0000", 2024-09-18T04:58:28.838 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6079999999999998e-06 2024-09-18T04:58:28.838 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.838 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628825+0000", 2024-09-18T04:58:28.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035502747000000001 2024-09-18T04:58:28.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.839 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628868+0000", 2024-09-18T04:58:28.839 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2719999999999998e-05 2024-09-18T04:58:28.839 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.839 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.839 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.840 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628920+0000", 2024-09-18T04:58:28.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2533e-05 2024-09-18T04:58:28.840 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.840 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.840 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.840 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.840 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.840 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146080 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1548288~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:28.840 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:28.840 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.212240643999998, 2024-09-18T04:58:28.840 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035755146000000002, 2024-09-18T04:58:28.841 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.841 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.841 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.841 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.841 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.841 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146080 2024-09-18T04:58:28.841 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.841 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.841 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.841 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.841 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:28.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.842 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:28.842 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.842 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.842 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.842 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:28.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593333+0000", 2024-09-18T04:58:28.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.858e-06 2024-09-18T04:58:28.843 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.843 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.843 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.843 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593336+0000", 2024-09-18T04:58:28.843 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4910000000000001e-06 2024-09-18T04:58:28.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593340+0000", 2024-09-18T04:58:28.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2590000000000001e-06 2024-09-18T04:58:28.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.844 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.844 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628956+0000", 2024-09-18T04:58:28.844 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035616039000000002 2024-09-18T04:58:28.844 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.844 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629031+0000", 2024-09-18T04:58:28.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5504000000000001e-05 2024-09-18T04:58:28.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.845 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.845 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.845 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629086+0000", 2024-09-18T04:58:28.845 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5429000000000001e-05 2024-09-18T04:58:28.845 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.845 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.845 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.845 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.846 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.846 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146081 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1597440~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:28.846 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:28.846 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.212230071999997, 2024-09-18T04:58:28.846 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035873223000000003, 2024-09-18T04:58:28.846 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.846 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.846 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.846 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.846 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.846 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146081 2024-09-18T04:58:28.846 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.846 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:28.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:28.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.847 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.847 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.847 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.847 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:28.847 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593343+0000", 2024-09-18T04:58:28.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3319999999999999e-06 2024-09-18T04:58:28.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593346+0000", 2024-09-18T04:58:28.848 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9059999999999998e-06 2024-09-18T04:58:28.848 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.848 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.848 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.848 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593348+0000", 2024-09-18T04:58:28.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4530000000000001e-06 2024-09-18T04:58:28.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629110+0000", 2024-09-18T04:58:28.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035761814000000003 2024-09-18T04:58:28.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.849 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.849 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.849 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629159+0000", 2024-09-18T04:58:28.849 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8971000000000003e-05 2024-09-18T04:58:28.849 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.850 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.850 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629215+0000", 2024-09-18T04:58:28.850 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6107000000000003e-05 2024-09-18T04:58:28.850 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.850 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.850 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.850 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.850 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.850 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146082 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1646592~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:28.850 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:28.850 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.212214021999998, 2024-09-18T04:58:28.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035987007000000001, 2024-09-18T04:58:28.851 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.851 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.851 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.851 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.851 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.851 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146082 2024-09-18T04:58:28.851 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.851 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.851 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.851 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.851 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:28.851 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:28.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.852 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.852 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:28.852 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.852 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.852 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593360+0000", 2024-09-18T04:58:28.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8560000000000001e-06 2024-09-18T04:58:28.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593365+0000", 2024-09-18T04:58:28.853 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3410000000000002e-06 2024-09-18T04:58:28.853 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.853 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.853 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.853 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593369+0000", 2024-09-18T04:58:28.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4850000000000003e-06 2024-09-18T04:58:28.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629242+0000", 2024-09-18T04:58:28.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035872464999999999 2024-09-18T04:58:28.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.854 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629288+0000", 2024-09-18T04:58:28.854 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6592999999999999e-05 2024-09-18T04:58:28.854 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.854 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.854 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.855 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629345+0000", 2024-09-18T04:58:28.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6575e-05 2024-09-18T04:58:28.855 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.855 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.855 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.855 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.855 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.855 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146083 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1695744~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:28.855 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:58:28.855 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.212198174999997, 2024-09-18T04:58:28.855 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036100587000000003, 2024-09-18T04:58:28.855 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.855 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.856 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.856 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.856 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.856 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146083 2024-09-18T04:58:28.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.856 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:58:28.856 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.856 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.856 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.856 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.856 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:58:28.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593373+0000", 2024-09-18T04:58:28.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.857 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.857 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.857 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.857 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593376+0000", 2024-09-18T04:58:28.857 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.244e-06 2024-09-18T04:58:28.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593379+0000", 2024-09-18T04:58:28.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.619e-06 2024-09-18T04:58:28.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.859 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593382+0000", 2024-09-18T04:58:28.859 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7199999999999998e-06 2024-09-18T04:58:28.859 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.859 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.859 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629368+0000", 2024-09-18T04:58:28.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035986281000000002 2024-09-18T04:58:28.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629416+0000", 2024-09-18T04:58:28.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8055e-05 2024-09-18T04:58:28.860 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.860 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.860 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.860 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629474+0000", 2024-09-18T04:58:28.860 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8176e-05 2024-09-18T04:58:28.860 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.860 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.861 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.861 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.861 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.861 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146084 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1744896~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:28.861 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:28.861 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.212185044999998, 2024-09-18T04:58:28.861 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036214349, 2024-09-18T04:58:28.861 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.861 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.861 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.861 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.861 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.861 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146084 2024-09-18T04:58:28.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.862 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:28.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.862 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:28.862 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.862 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.862 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.862 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:28.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593388+0000", 2024-09-18T04:58:28.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.564e-06 2024-09-18T04:58:28.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.863 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.863 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593392+0000", 2024-09-18T04:58:28.863 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0579999999999999e-06 2024-09-18T04:58:28.863 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.863 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593395+0000", 2024-09-18T04:58:28.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8890000000000002e-06 2024-09-18T04:58:28.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629500+0000", 2024-09-18T04:58:28.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036105343999999998 2024-09-18T04:58:28.864 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.864 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.864 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.864 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629547+0000", 2024-09-18T04:58:28.864 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7036000000000002e-05 2024-09-18T04:58:28.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.865 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.865 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629601+0000", 2024-09-18T04:58:28.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3866999999999998e-05 2024-09-18T04:58:28.865 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.865 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.865 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.865 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.865 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.865 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146085 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1794048~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:28.865 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:28.865 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.212174658000002, 2024-09-18T04:58:28.865 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036331186000000001, 2024-09-18T04:58:28.866 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.866 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.866 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.866 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.866 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.866 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146085 2024-09-18T04:58:28.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.866 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.866 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.866 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:28.866 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.866 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.866 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:28.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:28.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:28.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.867 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.867 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.867 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593398+0000", 2024-09-18T04:58:28.867 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2270000000000001e-06 2024-09-18T04:58:28.867 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593401+0000", 2024-09-18T04:58:28.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.999e-06 2024-09-18T04:58:28.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593404+0000", 2024-09-18T04:58:28.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.486e-06 2024-09-18T04:58:28.868 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.868 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.868 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629624+0000", 2024-09-18T04:58:28.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036219983999999997 2024-09-18T04:58:28.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629672+0000", 2024-09-18T04:58:28.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8368000000000002e-05 2024-09-18T04:58:28.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.869 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.869 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629729+0000", 2024-09-18T04:58:28.869 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6359e-05 2024-09-18T04:58:28.869 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.869 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.870 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.870 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146086 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1843200~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:28.870 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:28.870 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.212166093999997, 2024-09-18T04:58:28.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036449098999999999, 2024-09-18T04:58:28.870 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.870 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.870 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.870 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.870 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.870 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146086 2024-09-18T04:58:28.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.871 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:28.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:28.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:28.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:28.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593407+0000", 2024-09-18T04:58:28.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1799999999999999e-06 2024-09-18T04:58:28.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593410+0000", 2024-09-18T04:58:28.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.926e-06 2024-09-18T04:58:28.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593413+0000", 2024-09-18T04:58:28.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8250000000000001e-06 2024-09-18T04:58:28.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629755+0000", 2024-09-18T04:58:28.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036342212999999998 2024-09-18T04:58:28.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629803+0000", 2024-09-18T04:58:28.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8619000000000003e-05 2024-09-18T04:58:28.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.874 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.874 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629855+0000", 2024-09-18T04:58:28.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1536000000000003e-05 2024-09-18T04:58:28.875 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.875 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.875 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.875 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146087 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1892352~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:28.875 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:28.875 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.212157169000001, 2024-09-18T04:58:28.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036564608999999998, 2024-09-18T04:58:28.875 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.875 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.875 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.876 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.876 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.876 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146087 2024-09-18T04:58:28.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.876 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:28.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:28.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:28.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:28.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593416+0000", 2024-09-18T04:58:28.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.218e-06 2024-09-18T04:58:28.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593419+0000", 2024-09-18T04:58:28.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-18T04:58:28.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593421+0000", 2024-09-18T04:58:28.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4329999999999998e-06 2024-09-18T04:58:28.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629878+0000", 2024-09-18T04:58:28.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036456578000000003 2024-09-18T04:58:28.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629925+0000", 2024-09-18T04:58:28.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7595000000000001e-05 2024-09-18T04:58:28.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629979+0000", 2024-09-18T04:58:28.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4117999999999999e-05 2024-09-18T04:58:28.879 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.880 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.880 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.880 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.880 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146088 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1941504~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:28.880 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:28.880 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.212135005, 2024-09-18T04:58:28.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036676121999999999, 2024-09-18T04:58:28.880 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.880 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.880 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.880 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.880 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.881 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146088 2024-09-18T04:58:28.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.881 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:28.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:28.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:28.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593438+0000", 2024-09-18T04:58:28.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7570000000000001e-06 2024-09-18T04:58:28.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593442+0000", 2024-09-18T04:58:28.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2880000000000001e-06 2024-09-18T04:58:28.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593444+0000", 2024-09-18T04:58:28.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.593e-06 2024-09-18T04:58:28.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630015+0000", 2024-09-18T04:58:28.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036571125000000003 2024-09-18T04:58:28.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630061+0000", 2024-09-18T04:58:28.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5445000000000003e-05 2024-09-18T04:58:28.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630113+0000", 2024-09-18T04:58:28.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2358000000000003e-05 2024-09-18T04:58:28.884 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.884 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.884 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.884 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146089 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1990656~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:28.884 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:58:28.884 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.212125424, 2024-09-18T04:58:28.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036791483999999999, 2024-09-18T04:58:28.885 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.885 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.885 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.885 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.885 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.885 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146089 2024-09-18T04:58:28.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.885 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.885 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.885 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:58:28.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:58:28.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593446+0000", 2024-09-18T04:58:28.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:28.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593448+0000", 2024-09-18T04:58:28.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1939999999999999e-06 2024-09-18T04:58:28.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593451+0000", 2024-09-18T04:58:28.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.766e-06 2024-09-18T04:58:28.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593458+0000", 2024-09-18T04:58:28.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8539999999999999e-06 2024-09-18T04:58:28.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630136+0000", 2024-09-18T04:58:28.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03667757 2024-09-18T04:58:28.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630183+0000", 2024-09-18T04:58:28.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7289000000000001e-05 2024-09-18T04:58:28.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630238+0000", 2024-09-18T04:58:28.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5013000000000003e-05 2024-09-18T04:58:28.889 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.889 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.889 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.890 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146090 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 2039808~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:28.890 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:28.890 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.212111544000003, 2024-09-18T04:58:28.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036904477999999998, 2024-09-18T04:58:28.890 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.890 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.890 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.890 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.890 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.890 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146090 2024-09-18T04:58:28.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.890 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:28.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:28.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:28.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593461+0000", 2024-09-18T04:58:28.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3680000000000001e-06 2024-09-18T04:58:28.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593465+0000", 2024-09-18T04:58:28.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8120000000000001e-06 2024-09-18T04:58:28.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593468+0000", 2024-09-18T04:58:28.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.582e-06 2024-09-18T04:58:28.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630268+0000", 2024-09-18T04:58:28.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036800449999999998 2024-09-18T04:58:28.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630313+0000", 2024-09-18T04:58:28.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4836000000000002e-05 2024-09-18T04:58:28.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630365+0000", 2024-09-18T04:58:28.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1839999999999998e-05 2024-09-18T04:58:28.894 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.894 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.894 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.894 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146091 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 2088960~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:28.894 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:28.894 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 55.212101081999997, 2024-09-18T04:58:28.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037011534999999998, 2024-09-18T04:58:28.894 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.894 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.895 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.895 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.895 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.895 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146091 2024-09-18T04:58:28.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.895 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:28.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:28.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:28.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:28.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.896 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593472+0000", 2024-09-18T04:58:28.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1620000000000001e-06 2024-09-18T04:58:28.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593475+0000", 2024-09-18T04:58:28.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0019999999999998e-06 2024-09-18T04:58:28.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593477+0000", 2024-09-18T04:58:28.897 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-18T04:58:28.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630381+0000", 2024-09-18T04:58:28.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036904014999999998 2024-09-18T04:58:28.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630428+0000", 2024-09-18T04:58:28.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6931000000000002e-05 2024-09-18T04:58:28.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630482+0000", 2024-09-18T04:58:28.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4138999999999998e-05 2024-09-18T04:58:28.898 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.898 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.899 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.899 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:162075 3.13 3:cf3f6048:::rbd_header.11ac6967455f:head [watch reconnect cookie 140406508629968 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T04:58:28.899 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:58:28.899 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.315128276999999, 2024-09-18T04:58:28.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.53184266599999996, 2024-09-18T04:58:28.899 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.899 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:28.899 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.899 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.899 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.900 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 162075 2024-09-18T04:58:28.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.900 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:58:28.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:58:28.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490442+0000", 2024-09-18T04:58:28.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:58:28.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490453+0000", 2024-09-18T04:58:28.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0613e-05 2024-09-18T04:58:28.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490466+0000", 2024-09-18T04:58:28.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3702e-05 2024-09-18T04:58:28.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490475+0000", 2024-09-18T04:58:28.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2690000000000005e-06 2024-09-18T04:58:28.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490612+0000", 2024-09-18T04:58:28.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013620900000000001 2024-09-18T04:58:28.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-18T04:58:28.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490620+0000", 2024-09-18T04:58:28.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3089999999999996e-06 2024-09-18T04:58:28.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503041+0000", 2024-09-18T04:58:28.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012420512 2024-09-18T04:58:28.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:28.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503048+0000", 2024-09-18T04:58:28.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0389999999999999e-06 2024-09-18T04:58:28.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.016122+0000", 2024-09-18T04:58:28.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.51307432900000005 2024-09-18T04:58:28.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.016289+0000", 2024-09-18T04:58:28.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000166875 2024-09-18T04:58:28.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.022286+0000", 2024-09-18T04:58:28.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059976049999999996 2024-09-18T04:58:28.904 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.905 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.905 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.905 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:162076 3.13 3:cf3f6048:::rbd_header.11ac6967455f:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T04:58:28.905 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:28.905 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 46.315005603000003, 2024-09-18T04:58:28.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.53464117600000005, 2024-09-18T04:58:28.905 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.905 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:28.905 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.905 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.905 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.905 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 162076 2024-09-18T04:58:28.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.906 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:28.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:28.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:28.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:58:28.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490571+0000", 2024-09-18T04:58:28.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8259999999999998e-06 2024-09-18T04:58:28.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490584+0000", 2024-09-18T04:58:28.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2391999999999999e-05 2024-09-18T04:58:28.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490591+0000", 2024-09-18T04:58:28.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5460000000000003e-06 2024-09-18T04:58:28.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490714+0000", 2024-09-18T04:58:28.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000122543 2024-09-18T04:58:28.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-18T04:58:28.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490731+0000", 2024-09-18T04:58:28.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7405e-05 2024-09-18T04:58:28.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503069+0000", 2024-09-18T04:58:28.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012337836 2024-09-18T04:58:28.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:28.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503078+0000", 2024-09-18T04:58:28.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8289999999999997e-06 2024-09-18T04:58:28.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019018+0000", 2024-09-18T04:58:28.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.51594035900000002 2024-09-18T04:58:28.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019091+0000", 2024-09-18T04:58:28.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2608999999999995e-05 2024-09-18T04:58:28.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-18T04:58:28.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019421+0000", 2024-09-18T04:58:28.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00033004300000000002 2024-09-18T04:58:28.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:28.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.021644+0000", 2024-09-18T04:58:28.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022229749999999999 2024-09-18T04:58:28.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:28.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025113+0000", 2024-09-18T04:58:28.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003469228 2024-09-18T04:58:28.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:28.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025167+0000", 2024-09-18T04:58:28.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3569999999999997e-05 2024-09-18T04:58:28.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.912 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.912 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025208+0000", 2024-09-18T04:58:28.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0821000000000001e-05 2024-09-18T04:58:28.912 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.912 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.912 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.912 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.912 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167082 3.15 3:a80f5cd7:::rbd_data.11ac138032e7.0000000000000008:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:28.912 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.163826+0000", 2024-09-18T04:58:28.912 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 9.6417458570000001, 2024-09-18T04:58:28.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.91230620200000001, 2024-09-18T04:58:28.913 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:28.913 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:28.913 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:28.913 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:28.913 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:28.913 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167082 2024-09-18T04:58:28.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.913 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:28.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:28.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163826+0000", 2024-09-18T04:58:28.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:28.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163826+0000", 2024-09-18T04:58:28.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:28.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:28.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163826+0000", 2024-09-18T04:58:28.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:28.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:28.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163829+0000", 2024-09-18T04:58:28.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9069999999999999e-06 2024-09-18T04:58:28.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:28.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163836+0000", 2024-09-18T04:58:28.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4780000000000002e-06 2024-09-18T04:58:28.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:28.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163841+0000", 2024-09-18T04:58:28.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9389999999999996e-06 2024-09-18T04:58:28.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163886+0000", 2024-09-18T04:58:28.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5037999999999998e-05 2024-09-18T04:58:28.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:28.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163895+0000", 2024-09-18T04:58:28.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7819999999999999e-06 2024-09-18T04:58:28.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:28.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070784+0000", 2024-09-18T04:58:28.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.90688966400000004 2024-09-18T04:58:28.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:28.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070917+0000", 2024-09-18T04:58:28.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013228799999999999 2024-09-18T04:58:28.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:58:28.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.071003+0000", 2024-09-18T04:58:28.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5915000000000007e-05 2024-09-18T04:58:28.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:28.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073598+0000", 2024-09-18T04:58:28.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0025949329999999998 2024-09-18T04:58:28.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:28.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.076024+0000", 2024-09-18T04:58:28.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024267310000000001 2024-09-18T04:58:28.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:28.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.076096+0000", 2024-09-18T04:58:28.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1512000000000002e-05 2024-09-18T04:58:28.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:28.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:28.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:28.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.076132+0000", 2024-09-18T04:58:28.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6520999999999998e-05 2024-09-18T04:58:28.919 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.919 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.919 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.919 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:28.919 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:28.919 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:28.921 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:58:28.921 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:28.919+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:29.022 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-18T04:58:29.022 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:29.020+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:29.039 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:29.039 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:29.039 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:58:29.039 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:29.050 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:58:29.135 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:29.133+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:29.209 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:29.210 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:29.210 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:58:29.210 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:58:29.210 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:29.219 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:58:29.223 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:29.221+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:29.324 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:29.322+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:29.369 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:29.369 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:58:29.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:58:29.369 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:58:29.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.369 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9316 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:29.370 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:58:29.370 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 113.59882283899999, 2024-09-18T04:58:29.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044027208999999998, 2024-09-18T04:58:29.370 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:29.370 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:29.370 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:29.370 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:29.370 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:29.370 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9316 2024-09-18T04:58:29.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.370 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:29.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:58:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:58:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:58:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:29.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767144+0000", 2024-09-18T04:58:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3070000000000001e-06 2024-09-18T04:58:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767150+0000", 2024-09-18T04:58:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1700000000000002e-06 2024-09-18T04:58:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:29.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767163+0000", 2024-09-18T04:58:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2670999999999999e-05 2024-09-18T04:58:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810995+0000", 2024-09-18T04:58:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043832310999999999 2024-09-18T04:58:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811087+0000", 2024-09-18T04:58:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1345e-05 2024-09-18T04:58:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811169+0000", 2024-09-18T04:58:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2848000000000004e-05 2024-09-18T04:58:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9317 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1728512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:58:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 113.59879720399999, 2024-09-18T04:58:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044190170000000001, 2024-09-18T04:58:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:29.374 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9317 2024-09-18T04:58:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:58:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:58:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:29.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767167+0000", 2024-09-18T04:58:29.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:29.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:29.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767170+0000", 2024-09-18T04:58:29.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7259999999999998e-06 2024-09-18T04:58:29.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:29.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767176+0000", 2024-09-18T04:58:29.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1460000000000001e-06 2024-09-18T04:58:29.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:29.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767182+0000", 2024-09-18T04:58:29.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1020000000000002e-06 2024-09-18T04:58:29.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:29.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811204+0000", 2024-09-18T04:58:29.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044022039999999998 2024-09-18T04:58:29.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:29.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811286+0000", 2024-09-18T04:58:29.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1623000000000001e-05 2024-09-18T04:58:29.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:29.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811358+0000", 2024-09-18T04:58:29.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2125000000000005e-05 2024-09-18T04:58:29.378 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.378 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9318 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1777664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:58:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 113.598778659, 2024-09-18T04:58:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044380181999999997, 2024-09-18T04:58:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:29.379 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9318 2024-09-18T04:58:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:58:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:58:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:29.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:58:29.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:29.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:29.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767189+0000", 2024-09-18T04:58:29.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5909999999999998e-06 2024-09-18T04:58:29.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:29.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767194+0000", 2024-09-18T04:58:29.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7690000000000002e-06 2024-09-18T04:58:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767199+0000", 2024-09-18T04:58:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189999999999999e-06 2024-09-18T04:58:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811394+0000", 2024-09-18T04:58:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044195350000000001 2024-09-18T04:58:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:29.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811483+0000", 2024-09-18T04:58:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8282999999999995e-05 2024-09-18T04:58:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811567+0000", 2024-09-18T04:58:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3855000000000003e-05 2024-09-18T04:58:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9325 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:58:29.383 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 113.597048121, 2024-09-18T04:58:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044063989999999997, 2024-09-18T04:58:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9325 2024-09-18T04:58:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:58:29.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:29.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:58:29.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:29.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:58:29.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:29.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:29.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768919+0000", 2024-09-18T04:58:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:58:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768925+0000", 2024-09-18T04:58:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2310000000000001e-06 2024-09-18T04:58:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768929+0000", 2024-09-18T04:58:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4469999999999999e-06 2024-09-18T04:58:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812823+0000", 2024-09-18T04:58:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043893975000000002 2024-09-18T04:58:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812908+0000", 2024-09-18T04:58:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4862000000000002e-05 2024-09-18T04:58:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812981+0000", 2024-09-18T04:58:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2892000000000006e-05 2024-09-18T04:58:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.387 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:29.388 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.388 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9334 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:29.388 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:58:29.388 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 113.59502933500001, 2024-09-18T04:58:29.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044042853999999999, 2024-09-18T04:58:29.388 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:29.388 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:29.388 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:29.388 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:29.388 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:29.388 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9334 2024-09-18T04:58:29.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.389 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:29.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:29.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:58:29.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:29.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:58:29.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770935+0000", 2024-09-18T04:58:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770937+0000", 2024-09-18T04:58:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1489999999999999e-06 2024-09-18T04:58:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770940+0000", 2024-09-18T04:58:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9569999999999998e-06 2024-09-18T04:58:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770945+0000", 2024-09-18T04:58:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:58:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814838+0000", 2024-09-18T04:58:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043892635999999999 2024-09-18T04:58:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814920+0000", 2024-09-18T04:58:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2175000000000005e-05 2024-09-18T04:58:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814979+0000", 2024-09-18T04:58:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8372000000000003e-05 2024-09-18T04:58:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9352 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1159168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:58:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 113.592808158, 2024-09-18T04:58:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044011269999999998, 2024-09-18T04:58:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:29.392 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:29.393 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:29.393 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:29.393 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:29.393 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9352 2024-09-18T04:58:29.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.393 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:29.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:29.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:58:29.393 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.393 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:29.393 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:58:29.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:29.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773156+0000", 2024-09-18T04:58:29.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:29.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:29.394 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773159+0000", 2024-09-18T04:58:29.394 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2919999999999998e-06 2024-09-18T04:58:29.394 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.394 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.394 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:29.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773165+0000", 2024-09-18T04:58:29.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0580000000000003e-06 2024-09-18T04:58:29.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:29.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773177+0000", 2024-09-18T04:58:29.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.204e-05 2024-09-18T04:58:29.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:29.395 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817082+0000", 2024-09-18T04:58:29.395 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043905528999999999 2024-09-18T04:58:29.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.395 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817122+0000", 2024-09-18T04:58:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9258999999999998e-05 2024-09-18T04:58:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817168+0000", 2024-09-18T04:58:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6545000000000002e-05 2024-09-18T04:58:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9353 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1208320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:29.396 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:58:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 113.59278386, 2024-09-18T04:58:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044105537, 2024-09-18T04:58:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9353 2024-09-18T04:58:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:29.397 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:58:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:58:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:58:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:29.398 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773183+0000", 2024-09-18T04:58:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2929999999999999e-06 2024-09-18T04:58:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773190+0000", 2024-09-18T04:58:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4330000000000001e-06 2024-09-18T04:58:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773194+0000", 2024-09-18T04:58:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.865e-06 2024-09-18T04:58:29.399 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:29.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817196+0000", 2024-09-18T04:58:29.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044001708 2024-09-18T04:58:29.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.400 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:29.400 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817236+0000", 2024-09-18T04:58:29.400 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0043999999999998e-05 2024-09-18T04:58:29.400 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.400 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817287+0000", 2024-09-18T04:58:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0605000000000001e-05 2024-09-18T04:58:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9354 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:58:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 113.592766349, 2024-09-18T04:58:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044199796, 2024-09-18T04:58:29.401 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9354 2024-09-18T04:58:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:58:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.402 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:29.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:58:29.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:29.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773198+0000", 2024-09-18T04:58:29.403 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:29.403 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.403 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.403 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:29.403 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773200+0000", 2024-09-18T04:58:29.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4159999999999998e-06 2024-09-18T04:58:29.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:29.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773206+0000", 2024-09-18T04:58:29.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.789e-06 2024-09-18T04:58:29.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.404 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:29.404 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773210+0000", 2024-09-18T04:58:29.404 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2679999999999996e-06 2024-09-18T04:58:29.404 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.404 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817314+0000", 2024-09-18T04:58:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044103187000000002 2024-09-18T04:58:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817353+0000", 2024-09-18T04:58:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8973000000000001e-05 2024-09-18T04:58:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:29.405 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817399+0000", 2024-09-18T04:58:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5877999999999997e-05 2024-09-18T04:58:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46591 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2097152~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:58:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.667909932000001, 2024-09-18T04:58:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043987310000000002, 2024-09-18T04:58:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:29.406 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46591 2024-09-18T04:58:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:58:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:58:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.407 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:29.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:58:29.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:29.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:29.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698057+0000", 2024-09-18T04:58:29.408 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.531e-06 2024-09-18T04:58:29.408 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.408 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.408 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:29.408 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698064+0000", 2024-09-18T04:58:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-18T04:58:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698069+0000", 2024-09-18T04:58:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.091e-06 2024-09-18T04:58:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741843+0000", 2024-09-18T04:58:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043773470000000002 2024-09-18T04:58:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.409 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:29.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741928+0000", 2024-09-18T04:58:29.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5113999999999999e-05 2024-09-18T04:58:29.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.410 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:29.410 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742042+0000", 2024-09-18T04:58:29.410 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114527 2024-09-18T04:58:29.410 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.410 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:29.410 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.410 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.410 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46592 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2121728~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:58:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.667891505, 2024-09-18T04:58:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044077948999999998, 2024-09-18T04:58:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46592 2024-09-18T04:58:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.411 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:58:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:58:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698073+0000", 2024-09-18T04:58:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.412 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698076+0000", 2024-09-18T04:58:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.481e-06 2024-09-18T04:58:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698082+0000", 2024-09-18T04:58:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8730000000000004e-06 2024-09-18T04:58:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698086+0000", 2024-09-18T04:58:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:58:29.413 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742055+0000", 2024-09-18T04:58:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043969170000000002 2024-09-18T04:58:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742093+0000", 2024-09-18T04:58:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7707000000000003e-05 2024-09-18T04:58:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742152+0000", 2024-09-18T04:58:29.414 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8323999999999999e-05 2024-09-18T04:58:29.415 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.415 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46593 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2211840~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:58:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.667874601999998, 2024-09-18T04:58:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044259917000000003, 2024-09-18T04:58:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:29.417 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46593 2024-09-18T04:58:29.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.418 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:29.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:29.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:58:29.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.418 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:29.418 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:58:29.418 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:58:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698092+0000", 2024-09-18T04:58:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2249999999999999e-06 2024-09-18T04:58:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:29.419 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698098+0000", 2024-09-18T04:58:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:58:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698103+0000", 2024-09-18T04:58:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:58:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742182+0000", 2024-09-18T04:58:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044079508000000003 2024-09-18T04:58:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.420 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742257+0000", 2024-09-18T04:58:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4678000000000006e-05 2024-09-18T04:58:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742350+0000", 2024-09-18T04:58:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3531999999999998e-05 2024-09-18T04:58:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.421 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46594 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2236416~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:58:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.667858905000003, 2024-09-18T04:58:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044378984000000003, 2024-09-18T04:58:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46594 2024-09-18T04:58:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.422 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:58:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:58:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:58:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.423 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:29.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698108+0000", 2024-09-18T04:58:29.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0880000000000002e-06 2024-09-18T04:58:29.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:29.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698114+0000", 2024-09-18T04:58:29.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6269999999999997e-06 2024-09-18T04:58:29.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.424 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:29.424 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698118+0000", 2024-09-18T04:58:29.424 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4050000000000002e-06 2024-09-18T04:58:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742365+0000", 2024-09-18T04:58:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044246799000000003 2024-09-18T04:58:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742407+0000", 2024-09-18T04:58:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2008000000000002e-05 2024-09-18T04:58:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:29.425 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742485+0000", 2024-09-18T04:58:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8405999999999994e-05 2024-09-18T04:58:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46595 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:58:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.667842832000005, 2024-09-18T04:58:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.04440463, 2024-09-18T04:58:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:29.426 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46595 2024-09-18T04:58:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:58:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:29.427 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:58:29.428 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.428 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.428 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:29.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:58:29.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:29.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:29.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698124+0000", 2024-09-18T04:58:29.429 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1840000000000002e-06 2024-09-18T04:58:29.429 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.429 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.429 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:29.429 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698130+0000", 2024-09-18T04:58:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.947e-06 2024-09-18T04:58:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698136+0000", 2024-09-18T04:58:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6520000000000003e-06 2024-09-18T04:58:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742521+0000", 2024-09-18T04:58:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044385446000000002 2024-09-18T04:58:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742527+0000", 2024-09-18T04:58:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:58:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46596 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:58:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.667824588000002, 2024-09-18T04:58:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044418066999999999, 2024-09-18T04:58:29.431 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:29.432 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:29.432 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:29.432 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:29.432 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:29.432 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46596 2024-09-18T04:58:29.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.432 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:29.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.432 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:29.432 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:58:29.432 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:29.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:58:29.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:29.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:58:29.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:29.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.433 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:29.433 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698146+0000", 2024-09-18T04:58:29.433 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2779999999999999e-06 2024-09-18T04:58:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698153+0000", 2024-09-18T04:58:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3849999999999999e-06 2024-09-18T04:58:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698164+0000", 2024-09-18T04:58:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1375999999999999e-05 2024-09-18T04:58:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.434 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:29.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742554+0000", 2024-09-18T04:58:29.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044389482000000001 2024-09-18T04:58:29.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.435 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:29.435 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742559+0000", 2024-09-18T04:58:29.435 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9540000000000003e-06 2024-09-18T04:58:29.435 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.435 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:29.435 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.435 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46597 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:29.436 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:58:29.436 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.667796222999996, 2024-09-18T04:58:29.436 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044416513999999997, 2024-09-18T04:58:29.436 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:29.436 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:29.436 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:29.436 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:29.436 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:29.436 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46597 2024-09-18T04:58:29.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.436 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:29.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.436 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:29.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:58:29.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:29.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:58:29.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.437 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:29.437 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698168+0000", 2024-09-18T04:58:29.437 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:29.437 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:29.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698175+0000", 2024-09-18T04:58:29.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4520000000000002e-06 2024-09-18T04:58:29.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:29.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698181+0000", 2024-09-18T04:58:29.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4150000000000004e-06 2024-09-18T04:58:29.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.438 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:29.438 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698186+0000", 2024-09-18T04:58:29.438 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7489999999999998e-06 2024-09-18T04:58:29.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:29.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742580+0000", 2024-09-18T04:58:29.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044394325999999998 2024-09-18T04:58:29.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.439 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:29.439 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742585+0000", 2024-09-18T04:58:29.439 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9849999999999997e-06 2024-09-18T04:58:29.439 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.439 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:29.439 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.440 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46598 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:29.440 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:58:29.440 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 99.667688747, 2024-09-18T04:58:29.440 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044335316999999999, 2024-09-18T04:58:29.440 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:29.440 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:29.440 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:29.440 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:29.440 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:29.440 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46598 2024-09-18T04:58:29.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.440 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:29.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:29.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:58:29.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:29.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:58:29.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.441 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:29.441 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698190+0000", 2024-09-18T04:58:29.441 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999137 2024-09-18T04:58:29.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:29.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698280+0000", 2024-09-18T04:58:29.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0409999999999997e-05 2024-09-18T04:58:29.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:29.442 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698289+0000", 2024-09-18T04:58:29.442 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9490000000000002e-06 2024-09-18T04:58:29.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.442 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:29.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698296+0000", 2024-09-18T04:58:29.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2670000000000003e-06 2024-09-18T04:58:29.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:29.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742607+0000", 2024-09-18T04:58:29.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044311072999999999 2024-09-18T04:58:29.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.443 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:29.443 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742612+0000", 2024-09-18T04:58:29.443 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9400000000000001e-06 2024-09-18T04:58:29.443 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.444 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:29.444 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.444 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57986 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3086336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:29.444 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:58:29.444 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.680843023999998, 2024-09-18T04:58:29.444 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044047731999999999, 2024-09-18T04:58:29.444 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:29.444 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:29.444 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:29.444 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:29.444 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:29.444 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57986 2024-09-18T04:58:29.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.445 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:29.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:29.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:58:29.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.445 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:29.445 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:58:29.445 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:29.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:58:29.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:29.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:29.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685124+0000", 2024-09-18T04:58:29.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2859999999999998e-06 2024-09-18T04:58:29.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.446 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:29.446 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685130+0000", 2024-09-18T04:58:29.446 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5400000000000001e-06 2024-09-18T04:58:29.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:29.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685136+0000", 2024-09-18T04:58:29.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.152e-06 2024-09-18T04:58:29.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.447 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:29.447 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.728976+0000", 2024-09-18T04:58:29.447 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043840037999999998 2024-09-18T04:58:29.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:29.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729064+0000", 2024-09-18T04:58:29.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8709999999999996e-05 2024-09-18T04:58:29.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.448 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:29.448 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729170+0000", 2024-09-18T04:58:29.448 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000105386 2024-09-18T04:58:29.448 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.448 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:29.448 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.448 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.448 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57987 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3101696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:29.449 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:58:29.449 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.680794048999999, 2024-09-18T04:58:29.449 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044238123999999997, 2024-09-18T04:58:29.449 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:29.449 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:29.449 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:29.449 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:29.449 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:29.449 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57987 2024-09-18T04:58:29.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.449 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:29.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.449 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:29.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:58:29.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:29.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:58:29.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.450 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:29.450 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685170+0000", 2024-09-18T04:58:29.450 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:58:29.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:29.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685175+0000", 2024-09-18T04:58:29.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9069999999999998e-06 2024-09-18T04:58:29.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:29.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685187+0000", 2024-09-18T04:58:29.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.189e-05 2024-09-18T04:58:29.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.451 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:29.451 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685195+0000", 2024-09-18T04:58:29.451 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9459999999999998e-06 2024-09-18T04:58:29.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:29.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729212+0000", 2024-09-18T04:58:29.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044017239 2024-09-18T04:58:29.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:29.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729301+0000", 2024-09-18T04:58:29.452 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8887999999999993e-05 2024-09-18T04:58:29.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.452 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:29.452 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729409+0000", 2024-09-18T04:58:29.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108119 2024-09-18T04:58:29.453 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.453 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:29.453 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.453 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57988 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3117056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:29.453 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:58:29.453 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.68075193, 2024-09-18T04:58:29.453 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044429835000000001, 2024-09-18T04:58:29.453 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:29.453 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:29.453 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:29.453 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:29.454 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:29.454 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57988 2024-09-18T04:58:29.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.454 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:29.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:29.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:58:29.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.454 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.454 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:29.454 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:58:29.454 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:29.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685212+0000", 2024-09-18T04:58:29.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:29.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:29.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685216+0000", 2024-09-18T04:58:29.455 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7390000000000001e-06 2024-09-18T04:58:29.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.455 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:29.455 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685224+0000", 2024-09-18T04:58:29.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2260000000000005e-06 2024-09-18T04:58:29.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:29.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685230+0000", 2024-09-18T04:58:29.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0299999999999999e-06 2024-09-18T04:58:29.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.456 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:29.456 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729454+0000", 2024-09-18T04:58:29.456 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044223248999999999 2024-09-18T04:58:29.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:29.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729539+0000", 2024-09-18T04:58:29.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5029000000000002e-05 2024-09-18T04:58:29.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.457 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:29.457 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729643+0000", 2024-09-18T04:58:29.457 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104267 2024-09-18T04:58:29.457 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.457 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:29.457 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.458 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57989 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3132416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:29.458 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:58:29.458 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 98.680729178000007, 2024-09-18T04:58:29.458 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044642881000000002, 2024-09-18T04:58:29.458 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:29.458 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:29.458 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:29.458 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:29.458 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:29.458 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57989 2024-09-18T04:58:29.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.458 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:29.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.458 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:29.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:58:29.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:29.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:58:29.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:29.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.459 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.459 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:29.459 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685235+0000", 2024-09-18T04:58:29.459 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:29.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:29.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685238+0000", 2024-09-18T04:58:29.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.667e-06 2024-09-18T04:58:29.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:29.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685245+0000", 2024-09-18T04:58:29.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7909999999999999e-06 2024-09-18T04:58:29.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.460 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:29.460 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685250+0000", 2024-09-18T04:58:29.460 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3730000000000001e-06 2024-09-18T04:58:29.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:29.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729685+0000", 2024-09-18T04:58:29.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044434552000000002 2024-09-18T04:58:29.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:29.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729775+0000", 2024-09-18T04:58:29.461 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0395000000000004e-05 2024-09-18T04:58:29.461 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:29.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:29.461 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:29.461 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729879+0000", 2024-09-18T04:58:29.462 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010361099999999999 2024-09-18T04:58:29.462 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.462 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:29.462 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.462 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:29.462 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:29.462 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:29.463 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:29.461+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:29.464 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:58:29.564 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:29.561+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:29.611 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:29.611 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:29.612 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:58:29.612 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:29.623 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:58:29.645 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:58:29.665 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:29.663+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:29.730 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:29.731 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:29.731 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:58:29.731 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:58:29.731 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:29.739 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:58:29.783 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:29.781+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:29.791 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:58:29.791 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:58:29.810 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:58:29.810 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:58:29.827 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:58:29.827 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:58:29.843 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:58:29.843 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:58:29.850 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:29.851 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:29.860 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:58:29.860 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:58:29.860 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:58:29.867 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:29.865+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:29.876 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:58:29.876 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:58:29.893 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:58:29.893 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:58:29.910 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:58:29.910 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:58:29.969 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:29.966+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:29.984 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:29.984 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:29.993 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:58:30.069 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:30.066+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:30.126 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:30.127 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:30.136 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:58:30.170 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:30.167+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:30.259 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:30.259 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:30.269 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:58:30.270 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:30.268+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:30.371 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:30.369+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:30.380 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:30.380 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:30.389 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:58:30.473 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:30.471+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:30.547 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:30.547 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:30.557 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:58:30.574 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:30.571+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:30.675 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:30.673+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:30.714 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:30.714 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:30.724 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:58:30.776 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:30.773+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:30.827 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:30.828 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:30.837 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:58:30.876 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:30.874+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:30.947 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:30.947 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:30.956 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:58:31.000 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:30.998+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:31.043 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T04:58:31.070 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:31.070 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:31.080 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:58:31.087 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:31.085+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:31.179 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:31.177+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:31.226 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:31.226 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:31.236 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:58:31.321 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:31.319+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:31.381 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:31.378+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:31.381 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:31.381 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:31.390 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:58:31.482 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:31.480+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:31.533 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:31.534 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:31.542 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:58:31.585 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:31.583+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:31.644 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:31.645 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:31.653 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:58:31.684 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:31.682+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:31.785 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:31.783+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:31.795 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:31.795 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:31.803 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:58:31.886 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:31.884+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:31.905 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:31.905 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:31.913 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:58:31.922 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:58:31.987 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:31.984+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:32.016 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:32.016 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:32.024 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:58:32.025 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-18T04:58:32.068 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:58:32.069 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:58:32.087 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:58:32.087 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:58:32.103 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:58:32.103 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:58:32.109 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:32.107+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:32.119 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:58:32.119 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:58:32.135 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:58:32.136 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:58:32.152 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:58:32.152 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:58:32.168 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:58:32.169 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:58:32.170 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:32.170 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:32.179 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:58:32.185 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:58:32.185 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:58:32.221 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:32.218+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:32.290 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:32.287+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:32.324 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:58:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:58:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:58:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:58:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 140.927344592, 2024-09-18T04:58:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037718511000000003, 2024-09-18T04:58:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:32.324 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:58:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:58:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-18T04:58:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:58:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.325 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:58:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393125+0000", 2024-09-18T04:58:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:58:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393133+0000", 2024-09-18T04:58:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.225e-06 2024-09-18T04:58:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.326 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393144+0000", 2024-09-18T04:58:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1279e-05 2024-09-18T04:58:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393153+0000", 2024-09-18T04:58:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4540000000000007e-06 2024-09-18T04:58:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393203+0000", 2024-09-18T04:58:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9963000000000003e-05 2024-09-18T04:58:32.327 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393210+0000", 2024-09-18T04:58:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5689999999999999e-06 2024-09-18T04:58:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427399+0000", 2024-09-18T04:58:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034188784999999999 2024-09-18T04:58:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:32.328 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427532+0000", 2024-09-18T04:58:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013264700000000001 2024-09-18T04:58:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:58:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427685+0000", 2024-09-18T04:58:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015300599999999999 2024-09-18T04:58:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.428520+0000", 2024-09-18T04:58:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00083502499999999998 2024-09-18T04:58:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.329 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430772+0000", 2024-09-18T04:58:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022518880000000001 2024-09-18T04:58:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430817+0000", 2024-09-18T04:58:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5077000000000002e-05 2024-09-18T04:58:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430845+0000", 2024-09-18T04:58:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8036999999999999e-05 2024-09-18T04:58:32.330 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:62520 3.13 3.84cdb893 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T04:58:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 101.023034709, 2024-09-18T04:58:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94452134799999998, 2024-09-18T04:58:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:58:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:32.331 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 62520 2024-09-18T04:58:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.332 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.333 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297441+0000", 2024-09-18T04:58:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9840000000000001e-06 2024-09-18T04:58:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297449+0000", 2024-09-18T04:58:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3999999999999992e-06 2024-09-18T04:58:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297455+0000", 2024-09-18T04:58:32.334 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3350000000000003e-06 2024-09-18T04:58:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297552+0000", 2024-09-18T04:58:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6602999999999996e-05 2024-09-18T04:58:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297559+0000", 2024-09-18T04:58:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.52e-06 2024-09-18T04:58:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.335 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241943+0000", 2024-09-18T04:58:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94438357100000003 2024-09-18T04:58:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241958+0000", 2024-09-18T04:58:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4598e-05 2024-09-18T04:58:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:32.336 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:113179 3.13 3:cf138017:::rbd_data.11ac5f86a3f9.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T04:58:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:58:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.633244297999994, 2024-09-18T04:58:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036508304999999998, 2024-09-18T04:58:32.337 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:32.338 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:32.338 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:32.338 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:32.338 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 113179 2024-09-18T04:58:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.339 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:58:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:32.340 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:58:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687226+0000", 2024-09-18T04:58:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:58:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688169+0000", 2024-09-18T04:58:32.341 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00094328200000000004 2024-09-18T04:58:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688182+0000", 2024-09-18T04:58:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2756e-05 2024-09-18T04:58:32.342 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688188+0000", 2024-09-18T04:58:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0290000000000002e-06 2024-09-18T04:58:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.343 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688225+0000", 2024-09-18T04:58:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7301000000000001e-05 2024-09-18T04:58:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688324+0000", 2024-09-18T04:58:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9419999999999993e-05 2024-09-18T04:58:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.344 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:58:32.345 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.700483+0000", 2024-09-18T04:58:32.345 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012158771000000001 2024-09-18T04:58:32.345 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723586+0000", 2024-09-18T04:58:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023102952 2024-09-18T04:58:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723634+0000", 2024-09-18T04:58:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7468e-05 2024-09-18T04:58:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.346 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723693+0000", 2024-09-18T04:58:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9539e-05 2024-09-18T04:58:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723735+0000", 2024-09-18T04:58:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1829000000000002e-05 2024-09-18T04:58:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.347 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146078 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1449984~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 58.727166455000003, 2024-09-18T04:58:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035495325000000001, 2024-09-18T04:58:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146078 2024-09-18T04:58:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:32.348 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.349 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593306+0000", 2024-09-18T04:58:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.686e-06 2024-09-18T04:58:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593311+0000", 2024-09-18T04:58:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8500000000000002e-06 2024-09-18T04:58:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593314+0000", 2024-09-18T04:58:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0290000000000001e-06 2024-09-18T04:58:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.350 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628674+0000", 2024-09-18T04:58:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035360346000000001 2024-09-18T04:58:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628726+0000", 2024-09-18T04:58:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2219999999999998e-05 2024-09-18T04:58:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628800+0000", 2024-09-18T04:58:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3613000000000001e-05 2024-09-18T04:58:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.351 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146079 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1499136~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 58.72715505, 2024-09-18T04:58:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035604581000000003, 2024-09-18T04:58:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:32.352 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146079 2024-09-18T04:58:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.353 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593317+0000", 2024-09-18T04:58:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1599999999999999e-06 2024-09-18T04:58:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.354 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593320+0000", 2024-09-18T04:58:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0130000000000002e-06 2024-09-18T04:58:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593322+0000", 2024-09-18T04:58:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6079999999999998e-06 2024-09-18T04:58:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628825+0000", 2024-09-18T04:58:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035502747000000001 2024-09-18T04:58:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.355 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628868+0000", 2024-09-18T04:58:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2719999999999998e-05 2024-09-18T04:58:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628920+0000", 2024-09-18T04:58:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2533e-05 2024-09-18T04:58:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.356 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146080 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1548288~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 58.727139551999997, 2024-09-18T04:58:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035755146000000002, 2024-09-18T04:58:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146080 2024-09-18T04:58:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.357 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.358 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593333+0000", 2024-09-18T04:58:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.858e-06 2024-09-18T04:58:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593336+0000", 2024-09-18T04:58:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4910000000000001e-06 2024-09-18T04:58:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593340+0000", 2024-09-18T04:58:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2590000000000001e-06 2024-09-18T04:58:32.359 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628956+0000", 2024-09-18T04:58:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035616039000000002 2024-09-18T04:58:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629031+0000", 2024-09-18T04:58:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5504000000000001e-05 2024-09-18T04:58:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:32.360 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629086+0000", 2024-09-18T04:58:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5429000000000001e-05 2024-09-18T04:58:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146081 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1597440~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 58.727128980000003, 2024-09-18T04:58:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035873223000000003, 2024-09-18T04:58:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:32.361 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146081 2024-09-18T04:58:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.362 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:32.363 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:32.363 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.363 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.363 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.363 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593343+0000", 2024-09-18T04:58:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3319999999999999e-06 2024-09-18T04:58:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593346+0000", 2024-09-18T04:58:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9059999999999998e-06 2024-09-18T04:58:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593348+0000", 2024-09-18T04:58:32.364 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4530000000000001e-06 2024-09-18T04:58:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629110+0000", 2024-09-18T04:58:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035761814000000003 2024-09-18T04:58:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629159+0000", 2024-09-18T04:58:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8971000000000003e-05 2024-09-18T04:58:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.365 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629215+0000", 2024-09-18T04:58:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6107000000000003e-05 2024-09-18T04:58:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146082 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1646592~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 58.727112929999997, 2024-09-18T04:58:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035987007000000001, 2024-09-18T04:58:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:32.366 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146082 2024-09-18T04:58:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:32.367 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:32.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:32.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:32.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593360+0000", 2024-09-18T04:58:32.368 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8560000000000001e-06 2024-09-18T04:58:32.368 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.368 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.368 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:32.368 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593365+0000", 2024-09-18T04:58:32.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3410000000000002e-06 2024-09-18T04:58:32.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.369 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.369 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593369+0000", 2024-09-18T04:58:32.369 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4850000000000003e-06 2024-09-18T04:58:32.369 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.369 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629242+0000", 2024-09-18T04:58:32.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035872464999999999 2024-09-18T04:58:32.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:32.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629288+0000", 2024-09-18T04:58:32.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6592999999999999e-05 2024-09-18T04:58:32.370 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.370 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.370 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:32.370 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629345+0000", 2024-09-18T04:58:32.370 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6575e-05 2024-09-18T04:58:32.370 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.371 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:32.371 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.371 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.371 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.371 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146083 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1695744~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:32.371 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:58:32.371 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 58.727097082999997, 2024-09-18T04:58:32.371 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036100587000000003, 2024-09-18T04:58:32.371 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:32.371 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:32.371 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:32.371 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:32.371 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.371 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146083 2024-09-18T04:58:32.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.372 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:32.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:32.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:58:32.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.372 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:32.372 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:58:32.372 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.372 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.372 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:32.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593373+0000", 2024-09-18T04:58:32.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:32.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593376+0000", 2024-09-18T04:58:32.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.244e-06 2024-09-18T04:58:32.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.373 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.373 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:32.373 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593379+0000", 2024-09-18T04:58:32.373 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.619e-06 2024-09-18T04:58:32.373 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593382+0000", 2024-09-18T04:58:32.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7199999999999998e-06 2024-09-18T04:58:32.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629368+0000", 2024-09-18T04:58:32.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035986281000000002 2024-09-18T04:58:32.374 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.374 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.374 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:32.374 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629416+0000", 2024-09-18T04:58:32.374 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8055e-05 2024-09-18T04:58:32.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.375 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:32.375 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629474+0000", 2024-09-18T04:58:32.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8176e-05 2024-09-18T04:58:32.375 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.375 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:32.375 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.375 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.375 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.375 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146084 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1744896~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:32.375 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:32.375 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 58.727083952999998, 2024-09-18T04:58:32.375 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036214349, 2024-09-18T04:58:32.376 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:32.376 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:32.376 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:32.376 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:32.376 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.376 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146084 2024-09-18T04:58:32.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.376 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:32.376 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.376 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:32.376 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:32.376 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.376 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:32.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:32.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:32.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:32.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.377 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.377 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.377 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:32.377 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593388+0000", 2024-09-18T04:58:32.377 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.564e-06 2024-09-18T04:58:32.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:32.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593392+0000", 2024-09-18T04:58:32.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0579999999999999e-06 2024-09-18T04:58:32.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.378 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593395+0000", 2024-09-18T04:58:32.378 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8890000000000002e-06 2024-09-18T04:58:32.378 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.378 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.378 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629500+0000", 2024-09-18T04:58:32.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036105343999999998 2024-09-18T04:58:32.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:32.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629547+0000", 2024-09-18T04:58:32.379 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7036000000000002e-05 2024-09-18T04:58:32.379 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.379 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.379 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:32.379 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629601+0000", 2024-09-18T04:58:32.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3866999999999998e-05 2024-09-18T04:58:32.380 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.380 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:32.380 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.380 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.380 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.380 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146085 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1794048~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:32.380 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:32.380 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 58.727073566000001, 2024-09-18T04:58:32.380 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036331186000000001, 2024-09-18T04:58:32.380 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:32.380 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:32.381 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:32.381 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:32.381 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.381 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146085 2024-09-18T04:58:32.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.381 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:32.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:32.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:32.381 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.381 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.381 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.381 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:32.381 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:32.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:32.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:32.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:32.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:32.382 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593398+0000", 2024-09-18T04:58:32.382 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2270000000000001e-06 2024-09-18T04:58:32.382 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.382 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.382 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:32.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593401+0000", 2024-09-18T04:58:32.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.999e-06 2024-09-18T04:58:32.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593404+0000", 2024-09-18T04:58:32.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.486e-06 2024-09-18T04:58:32.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.383 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.383 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629624+0000", 2024-09-18T04:58:32.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036219983999999997 2024-09-18T04:58:32.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:32.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629672+0000", 2024-09-18T04:58:32.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8368000000000002e-05 2024-09-18T04:58:32.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:32.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629729+0000", 2024-09-18T04:58:32.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6359e-05 2024-09-18T04:58:32.384 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.384 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:32.384 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.385 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146086 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1843200~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:32.385 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:32.385 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 58.727065002000003, 2024-09-18T04:58:32.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036449098999999999, 2024-09-18T04:58:32.385 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:32.385 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:32.385 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:32.385 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:32.385 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.385 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146086 2024-09-18T04:58:32.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.385 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:32.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:32.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:32.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:32.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:32.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:32.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:32.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:32.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:32.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593407+0000", 2024-09-18T04:58:32.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1799999999999999e-06 2024-09-18T04:58:32.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:32.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593410+0000", 2024-09-18T04:58:32.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.926e-06 2024-09-18T04:58:32.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593413+0000", 2024-09-18T04:58:32.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8250000000000001e-06 2024-09-18T04:58:32.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629755+0000", 2024-09-18T04:58:32.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036342212999999998 2024-09-18T04:58:32.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:32.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629803+0000", 2024-09-18T04:58:32.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8619000000000003e-05 2024-09-18T04:58:32.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:32.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629855+0000", 2024-09-18T04:58:32.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1536000000000003e-05 2024-09-18T04:58:32.389 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.389 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:32.389 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.389 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146087 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1892352~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:32.389 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:32.389 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 58.727056077, 2024-09-18T04:58:32.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036564608999999998, 2024-09-18T04:58:32.389 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:32.389 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:32.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:32.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:32.390 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.390 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146087 2024-09-18T04:58:32.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.390 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:32.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:32.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:32.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:32.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:32.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:32.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:32.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:32.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:32.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593416+0000", 2024-09-18T04:58:32.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.218e-06 2024-09-18T04:58:32.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:32.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593419+0000", 2024-09-18T04:58:32.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-18T04:58:32.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593421+0000", 2024-09-18T04:58:32.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4329999999999998e-06 2024-09-18T04:58:32.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629878+0000", 2024-09-18T04:58:32.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036456578000000003 2024-09-18T04:58:32.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:32.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629925+0000", 2024-09-18T04:58:32.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7595000000000001e-05 2024-09-18T04:58:32.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:32.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629979+0000", 2024-09-18T04:58:32.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4117999999999999e-05 2024-09-18T04:58:32.393 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.393 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:32.393 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.394 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146088 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1941504~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:32.394 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:32.394 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 58.727033913, 2024-09-18T04:58:32.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036676121999999999, 2024-09-18T04:58:32.394 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:32.394 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:32.394 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:32.394 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:32.394 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.394 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146088 2024-09-18T04:58:32.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.394 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:32.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:32.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:32.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:32.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:32.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:32.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:32.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:32.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593438+0000", 2024-09-18T04:58:32.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7570000000000001e-06 2024-09-18T04:58:32.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:32.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593442+0000", 2024-09-18T04:58:32.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2880000000000001e-06 2024-09-18T04:58:32.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593444+0000", 2024-09-18T04:58:32.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.593e-06 2024-09-18T04:58:32.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630015+0000", 2024-09-18T04:58:32.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036571125000000003 2024-09-18T04:58:32.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:32.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630061+0000", 2024-09-18T04:58:32.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5445000000000003e-05 2024-09-18T04:58:32.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:32.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630113+0000", 2024-09-18T04:58:32.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2358000000000003e-05 2024-09-18T04:58:32.398 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.398 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:32.398 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.398 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146089 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1990656~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:32.398 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:58:32.398 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 58.727024331999999, 2024-09-18T04:58:32.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036791483999999999, 2024-09-18T04:58:32.398 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:32.398 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:32.398 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:32.398 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:32.399 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.399 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146089 2024-09-18T04:58:32.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.399 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:32.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:32.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:58:32.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:32.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:58:32.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:32.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593446+0000", 2024-09-18T04:58:32.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:32.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:32.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593448+0000", 2024-09-18T04:58:32.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1939999999999999e-06 2024-09-18T04:58:32.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:32.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593451+0000", 2024-09-18T04:58:32.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.766e-06 2024-09-18T04:58:32.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593458+0000", 2024-09-18T04:58:32.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8539999999999999e-06 2024-09-18T04:58:32.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630136+0000", 2024-09-18T04:58:32.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03667757 2024-09-18T04:58:32.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:32.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630183+0000", 2024-09-18T04:58:32.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7289000000000001e-05 2024-09-18T04:58:32.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.402 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:32.402 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630238+0000", 2024-09-18T04:58:32.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5013000000000003e-05 2024-09-18T04:58:32.402 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.402 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:32.402 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.402 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146090 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 2039808~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:32.402 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:32.403 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 58.727010452000002, 2024-09-18T04:58:32.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036904477999999998, 2024-09-18T04:58:32.403 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:32.403 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:32.403 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:32.403 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:32.403 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.403 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146090 2024-09-18T04:58:32.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.403 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:32.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:32.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:32.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:32.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:32.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:32.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:32.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:32.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593461+0000", 2024-09-18T04:58:32.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3680000000000001e-06 2024-09-18T04:58:32.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:32.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593465+0000", 2024-09-18T04:58:32.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8120000000000001e-06 2024-09-18T04:58:32.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593468+0000", 2024-09-18T04:58:32.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.582e-06 2024-09-18T04:58:32.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630268+0000", 2024-09-18T04:58:32.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036800449999999998 2024-09-18T04:58:32.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:32.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630313+0000", 2024-09-18T04:58:32.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4836000000000002e-05 2024-09-18T04:58:32.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:32.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630365+0000", 2024-09-18T04:58:32.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1839999999999998e-05 2024-09-18T04:58:32.407 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.407 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:32.407 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.407 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146091 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 2088960~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:32.407 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:32.407 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 58.726999990000003, 2024-09-18T04:58:32.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037011534999999998, 2024-09-18T04:58:32.407 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:32.407 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:32.407 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:32.407 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:32.407 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.408 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146091 2024-09-18T04:58:32.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.408 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:32.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:32.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:32.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:32.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:32.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:32.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:32.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:32.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:32.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593472+0000", 2024-09-18T04:58:32.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1620000000000001e-06 2024-09-18T04:58:32.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:32.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593475+0000", 2024-09-18T04:58:32.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0019999999999998e-06 2024-09-18T04:58:32.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593477+0000", 2024-09-18T04:58:32.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-18T04:58:32.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630381+0000", 2024-09-18T04:58:32.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036904014999999998 2024-09-18T04:58:32.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:32.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630428+0000", 2024-09-18T04:58:32.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6931000000000002e-05 2024-09-18T04:58:32.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:32.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630482+0000", 2024-09-18T04:58:32.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4138999999999998e-05 2024-09-18T04:58:32.411 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.411 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:32.411 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.411 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:162075 3.13 3:cf3f6048:::rbd_header.11ac6967455f:head [watch reconnect cookie 140406508629968 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T04:58:32.411 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:58:32.411 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 49.830027184999999, 2024-09-18T04:58:32.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.53184266599999996, 2024-09-18T04:58:32.412 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:32.412 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:32.412 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:32.412 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:32.412 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.412 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 162075 2024-09-18T04:58:32.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.412 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:32.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:32.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:58:32.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:32.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:58:32.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:32.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490442+0000", 2024-09-18T04:58:32.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:58:32.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:32.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490453+0000", 2024-09-18T04:58:32.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0613e-05 2024-09-18T04:58:32.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:32.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490466+0000", 2024-09-18T04:58:32.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3702e-05 2024-09-18T04:58:32.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490475+0000", 2024-09-18T04:58:32.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2690000000000005e-06 2024-09-18T04:58:32.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490612+0000", 2024-09-18T04:58:32.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013620900000000001 2024-09-18T04:58:32.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-18T04:58:32.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490620+0000", 2024-09-18T04:58:32.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3089999999999996e-06 2024-09-18T04:58:32.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503041+0000", 2024-09-18T04:58:32.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012420512 2024-09-18T04:58:32.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:32.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503048+0000", 2024-09-18T04:58:32.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0389999999999999e-06 2024-09-18T04:58:32.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.016122+0000", 2024-09-18T04:58:32.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.51307432900000005 2024-09-18T04:58:32.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:32.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.016289+0000", 2024-09-18T04:58:32.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000166875 2024-09-18T04:58:32.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:32.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.022286+0000", 2024-09-18T04:58:32.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059976049999999996 2024-09-18T04:58:32.417 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.417 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:32.417 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.417 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:162076 3.13 3:cf3f6048:::rbd_header.11ac6967455f:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T04:58:32.417 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:32.417 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 49.829904511000002, 2024-09-18T04:58:32.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.53464117600000005, 2024-09-18T04:58:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 162076 2024-09-18T04:58:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:58:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490571+0000", 2024-09-18T04:58:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8259999999999998e-06 2024-09-18T04:58:32.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490584+0000", 2024-09-18T04:58:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2391999999999999e-05 2024-09-18T04:58:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490591+0000", 2024-09-18T04:58:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5460000000000003e-06 2024-09-18T04:58:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490714+0000", 2024-09-18T04:58:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000122543 2024-09-18T04:58:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-18T04:58:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490731+0000", 2024-09-18T04:58:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7405e-05 2024-09-18T04:58:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503069+0000", 2024-09-18T04:58:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012337836 2024-09-18T04:58:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503078+0000", 2024-09-18T04:58:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8289999999999997e-06 2024-09-18T04:58:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019018+0000", 2024-09-18T04:58:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.51594035900000002 2024-09-18T04:58:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019091+0000", 2024-09-18T04:58:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2608999999999995e-05 2024-09-18T04:58:32.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-18T04:58:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019421+0000", 2024-09-18T04:58:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00033004300000000002 2024-09-18T04:58:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.021644+0000", 2024-09-18T04:58:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022229749999999999 2024-09-18T04:58:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025113+0000", 2024-09-18T04:58:32.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003469228 2024-09-18T04:58:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025167+0000", 2024-09-18T04:58:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3569999999999997e-05 2024-09-18T04:58:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025208+0000", 2024-09-18T04:58:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0821000000000001e-05 2024-09-18T04:58:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167082 3.15 3:a80f5cd7:::rbd_data.11ac138032e7.0000000000000008:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.163826+0000", 2024-09-18T04:58:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 13.156644764999999, 2024-09-18T04:58:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.91230620200000001, 2024-09-18T04:58:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167082 2024-09-18T04:58:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:32.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163826+0000", 2024-09-18T04:58:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163826+0000", 2024-09-18T04:58:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163826+0000", 2024-09-18T04:58:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163829+0000", 2024-09-18T04:58:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9069999999999999e-06 2024-09-18T04:58:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163836+0000", 2024-09-18T04:58:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4780000000000002e-06 2024-09-18T04:58:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163841+0000", 2024-09-18T04:58:32.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9389999999999996e-06 2024-09-18T04:58:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163886+0000", 2024-09-18T04:58:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5037999999999998e-05 2024-09-18T04:58:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163895+0000", 2024-09-18T04:58:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7819999999999999e-06 2024-09-18T04:58:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:32.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070784+0000", 2024-09-18T04:58:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.90688966400000004 2024-09-18T04:58:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070917+0000", 2024-09-18T04:58:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013228799999999999 2024-09-18T04:58:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:58:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.071003+0000", 2024-09-18T04:58:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5915000000000007e-05 2024-09-18T04:58:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073598+0000", 2024-09-18T04:58:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0025949329999999998 2024-09-18T04:58:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.076024+0000", 2024-09-18T04:58:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024267310000000001 2024-09-18T04:58:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.076096+0000", 2024-09-18T04:58:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1512000000000002e-05 2024-09-18T04:58:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:32.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.076132+0000", 2024-09-18T04:58:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6520999999999998e-05 2024-09-18T04:58:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:32.431 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:32.431 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:32.432 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:58:32.433 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:32.431+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:32.533 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:32.531+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:32.541 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:32.541 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:32.541 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:58:32.541 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:32.550 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:58:32.635 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:32.633+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:32.656 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:58:32.656 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:58:32.656 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:32.665 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:58:32.735 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:32.733+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:32.772 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:58:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:58:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:58:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9316 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:58:32.773 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.002491463, 2024-09-18T04:58:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044027208999999998, 2024-09-18T04:58:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9316 2024-09-18T04:58:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:58:32.774 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:58:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:58:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:32.775 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767144+0000", 2024-09-18T04:58:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3070000000000001e-06 2024-09-18T04:58:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767150+0000", 2024-09-18T04:58:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1700000000000002e-06 2024-09-18T04:58:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767163+0000", 2024-09-18T04:58:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2670999999999999e-05 2024-09-18T04:58:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.776 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810995+0000", 2024-09-18T04:58:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043832310999999999 2024-09-18T04:58:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811087+0000", 2024-09-18T04:58:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1345e-05 2024-09-18T04:58:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811169+0000", 2024-09-18T04:58:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2848000000000004e-05 2024-09-18T04:58:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.777 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9317 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1728512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:58:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.002465828, 2024-09-18T04:58:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044190170000000001, 2024-09-18T04:58:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9317 2024-09-18T04:58:32.778 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:58:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:58:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:32.779 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767167+0000", 2024-09-18T04:58:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767170+0000", 2024-09-18T04:58:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7259999999999998e-06 2024-09-18T04:58:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767176+0000", 2024-09-18T04:58:32.780 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1460000000000001e-06 2024-09-18T04:58:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767182+0000", 2024-09-18T04:58:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1020000000000002e-06 2024-09-18T04:58:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811204+0000", 2024-09-18T04:58:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044022039999999998 2024-09-18T04:58:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.781 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811286+0000", 2024-09-18T04:58:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1623000000000001e-05 2024-09-18T04:58:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811358+0000", 2024-09-18T04:58:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2125000000000005e-05 2024-09-18T04:58:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9318 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1777664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:32.782 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:58:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.002447283, 2024-09-18T04:58:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044380181999999997, 2024-09-18T04:58:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9318 2024-09-18T04:58:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:58:32.783 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:58:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:58:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:32.784 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767189+0000", 2024-09-18T04:58:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5909999999999998e-06 2024-09-18T04:58:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767194+0000", 2024-09-18T04:58:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7690000000000002e-06 2024-09-18T04:58:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767199+0000", 2024-09-18T04:58:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189999999999999e-06 2024-09-18T04:58:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:32.785 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811394+0000", 2024-09-18T04:58:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044195350000000001 2024-09-18T04:58:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811483+0000", 2024-09-18T04:58:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8282999999999995e-05 2024-09-18T04:58:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811567+0000", 2024-09-18T04:58:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3855000000000003e-05 2024-09-18T04:58:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:32.786 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9325 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:58:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.00071674500001, 2024-09-18T04:58:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044063989999999997, 2024-09-18T04:58:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9325 2024-09-18T04:58:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.787 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:58:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:58:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:32.788 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:58:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768919+0000", 2024-09-18T04:58:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:58:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768925+0000", 2024-09-18T04:58:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2310000000000001e-06 2024-09-18T04:58:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.789 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768929+0000", 2024-09-18T04:58:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4469999999999999e-06 2024-09-18T04:58:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812823+0000", 2024-09-18T04:58:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043893975000000002 2024-09-18T04:58:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812908+0000", 2024-09-18T04:58:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4862000000000002e-05 2024-09-18T04:58:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.790 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812981+0000", 2024-09-18T04:58:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2892000000000006e-05 2024-09-18T04:58:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9334 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:58:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 116.998697959, 2024-09-18T04:58:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044042853999999999, 2024-09-18T04:58:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:32.791 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9334 2024-09-18T04:58:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:58:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:58:32.792 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770935+0000", 2024-09-18T04:58:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770937+0000", 2024-09-18T04:58:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1489999999999999e-06 2024-09-18T04:58:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:32.793 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770940+0000", 2024-09-18T04:58:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9569999999999998e-06 2024-09-18T04:58:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770945+0000", 2024-09-18T04:58:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:58:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814838+0000", 2024-09-18T04:58:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043892635999999999 2024-09-18T04:58:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814920+0000", 2024-09-18T04:58:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2175000000000005e-05 2024-09-18T04:58:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814979+0000", 2024-09-18T04:58:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8372000000000003e-05 2024-09-18T04:58:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9352 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1159168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:32.795 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:58:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 116.996476782, 2024-09-18T04:58:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044011269999999998, 2024-09-18T04:58:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9352 2024-09-18T04:58:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:32.796 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:58:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:58:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:32.797 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773156+0000", 2024-09-18T04:58:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773159+0000", 2024-09-18T04:58:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2919999999999998e-06 2024-09-18T04:58:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773165+0000", 2024-09-18T04:58:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0580000000000003e-06 2024-09-18T04:58:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.798 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773177+0000", 2024-09-18T04:58:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.204e-05 2024-09-18T04:58:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817082+0000", 2024-09-18T04:58:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043905528999999999 2024-09-18T04:58:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817122+0000", 2024-09-18T04:58:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9258999999999998e-05 2024-09-18T04:58:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.799 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817168+0000", 2024-09-18T04:58:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6545000000000002e-05 2024-09-18T04:58:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9353 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1208320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:58:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 116.996452484, 2024-09-18T04:58:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044105537, 2024-09-18T04:58:32.800 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9353 2024-09-18T04:58:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:58:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.801 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:58:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:58:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:32.802 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773183+0000", 2024-09-18T04:58:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2929999999999999e-06 2024-09-18T04:58:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773190+0000", 2024-09-18T04:58:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4330000000000001e-06 2024-09-18T04:58:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773194+0000", 2024-09-18T04:58:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.865e-06 2024-09-18T04:58:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.803 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817196+0000", 2024-09-18T04:58:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044001708 2024-09-18T04:58:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817236+0000", 2024-09-18T04:58:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0043999999999998e-05 2024-09-18T04:58:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817287+0000", 2024-09-18T04:58:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0605000000000001e-05 2024-09-18T04:58:32.804 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9354 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:58:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 116.99643497300001, 2024-09-18T04:58:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044199796, 2024-09-18T04:58:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:32.805 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9354 2024-09-18T04:58:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:58:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:58:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.806 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773198+0000", 2024-09-18T04:58:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773200+0000", 2024-09-18T04:58:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4159999999999998e-06 2024-09-18T04:58:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773206+0000", 2024-09-18T04:58:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.789e-06 2024-09-18T04:58:32.807 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773210+0000", 2024-09-18T04:58:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2679999999999996e-06 2024-09-18T04:58:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817314+0000", 2024-09-18T04:58:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044103187000000002 2024-09-18T04:58:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817353+0000", 2024-09-18T04:58:32.808 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8973000000000001e-05 2024-09-18T04:58:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817399+0000", 2024-09-18T04:58:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5877999999999997e-05 2024-09-18T04:58:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46591 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2097152~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:58:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 103.07157855600001, 2024-09-18T04:58:32.809 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043987310000000002, 2024-09-18T04:58:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46591 2024-09-18T04:58:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:58:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.810 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:58:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:58:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698057+0000", 2024-09-18T04:58:32.811 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.531e-06 2024-09-18T04:58:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698064+0000", 2024-09-18T04:58:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-18T04:58:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698069+0000", 2024-09-18T04:58:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.091e-06 2024-09-18T04:58:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:32.812 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741843+0000", 2024-09-18T04:58:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043773470000000002 2024-09-18T04:58:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741928+0000", 2024-09-18T04:58:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5113999999999999e-05 2024-09-18T04:58:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742042+0000", 2024-09-18T04:58:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114527 2024-09-18T04:58:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.813 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46592 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2121728~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:58:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 103.07156012900001, 2024-09-18T04:58:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044077948999999998, 2024-09-18T04:58:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.814 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46592 2024-09-18T04:58:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:58:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:58:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.815 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698073+0000", 2024-09-18T04:58:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698076+0000", 2024-09-18T04:58:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.481e-06 2024-09-18T04:58:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698082+0000", 2024-09-18T04:58:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8730000000000004e-06 2024-09-18T04:58:32.816 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698086+0000", 2024-09-18T04:58:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:58:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742055+0000", 2024-09-18T04:58:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043969170000000002 2024-09-18T04:58:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742093+0000", 2024-09-18T04:58:32.817 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7707000000000003e-05 2024-09-18T04:58:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742152+0000", 2024-09-18T04:58:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8323999999999999e-05 2024-09-18T04:58:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46593 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2211840~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:58:32.818 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 103.071543226, 2024-09-18T04:58:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044259917000000003, 2024-09-18T04:58:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46593 2024-09-18T04:58:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:58:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.819 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:58:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:58:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698092+0000", 2024-09-18T04:58:32.820 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2249999999999999e-06 2024-09-18T04:58:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698098+0000", 2024-09-18T04:58:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:58:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698103+0000", 2024-09-18T04:58:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:58:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:32.821 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742182+0000", 2024-09-18T04:58:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044079508000000003 2024-09-18T04:58:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742257+0000", 2024-09-18T04:58:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4678000000000006e-05 2024-09-18T04:58:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742350+0000", 2024-09-18T04:58:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3531999999999998e-05 2024-09-18T04:58:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.822 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46594 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2236416~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:58:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 103.07152752899999, 2024-09-18T04:58:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044378984000000003, 2024-09-18T04:58:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.823 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46594 2024-09-18T04:58:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:58:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:58:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.824 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:58:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698108+0000", 2024-09-18T04:58:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0880000000000002e-06 2024-09-18T04:58:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698114+0000", 2024-09-18T04:58:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6269999999999997e-06 2024-09-18T04:58:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.825 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698118+0000", 2024-09-18T04:58:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4050000000000002e-06 2024-09-18T04:58:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742365+0000", 2024-09-18T04:58:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044246799000000003 2024-09-18T04:58:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:32.826 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742407+0000", 2024-09-18T04:58:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2008000000000002e-05 2024-09-18T04:58:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742485+0000", 2024-09-18T04:58:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8405999999999994e-05 2024-09-18T04:58:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46595 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:58:32.827 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 103.071511456, 2024-09-18T04:58:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.04440463, 2024-09-18T04:58:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46595 2024-09-18T04:58:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:58:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.828 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:58:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:58:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698124+0000", 2024-09-18T04:58:32.829 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1840000000000002e-06 2024-09-18T04:58:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698130+0000", 2024-09-18T04:58:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.947e-06 2024-09-18T04:58:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698136+0000", 2024-09-18T04:58:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6520000000000003e-06 2024-09-18T04:58:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:32.830 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742521+0000", 2024-09-18T04:58:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044385446000000002 2024-09-18T04:58:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742527+0000", 2024-09-18T04:58:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:58:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46596 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:58:32.831 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 103.07149321199999, 2024-09-18T04:58:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044418066999999999, 2024-09-18T04:58:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46596 2024-09-18T04:58:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:58:32.832 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:58:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:58:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:32.833 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698146+0000", 2024-09-18T04:58:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2779999999999999e-06 2024-09-18T04:58:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698153+0000", 2024-09-18T04:58:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3849999999999999e-06 2024-09-18T04:58:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698164+0000", 2024-09-18T04:58:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1375999999999999e-05 2024-09-18T04:58:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.834 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742554+0000", 2024-09-18T04:58:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044389482000000001 2024-09-18T04:58:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742559+0000", 2024-09-18T04:58:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9540000000000003e-06 2024-09-18T04:58:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.835 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46597 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:58:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 103.071464847, 2024-09-18T04:58:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044416513999999997, 2024-09-18T04:58:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46597 2024-09-18T04:58:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:32.836 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:58:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:58:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698168+0000", 2024-09-18T04:58:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.837 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:32.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698175+0000", 2024-09-18T04:58:32.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4520000000000002e-06 2024-09-18T04:58:32.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:32.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698181+0000", 2024-09-18T04:58:32.838 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4150000000000004e-06 2024-09-18T04:58:32.838 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.838 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.838 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.838 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698186+0000", 2024-09-18T04:58:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7489999999999998e-06 2024-09-18T04:58:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742580+0000", 2024-09-18T04:58:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044394325999999998 2024-09-18T04:58:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742585+0000", 2024-09-18T04:58:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9849999999999997e-06 2024-09-18T04:58:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:32.839 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46598 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:58:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 103.071357371, 2024-09-18T04:58:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044335316999999999, 2024-09-18T04:58:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:32.840 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46598 2024-09-18T04:58:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:58:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:58:32.841 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698190+0000", 2024-09-18T04:58:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999137 2024-09-18T04:58:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698280+0000", 2024-09-18T04:58:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0409999999999997e-05 2024-09-18T04:58:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:32.842 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698289+0000", 2024-09-18T04:58:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9490000000000002e-06 2024-09-18T04:58:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698296+0000", 2024-09-18T04:58:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2670000000000003e-06 2024-09-18T04:58:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742607+0000", 2024-09-18T04:58:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044311072999999999 2024-09-18T04:58:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.843 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742612+0000", 2024-09-18T04:58:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9400000000000001e-06 2024-09-18T04:58:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57986 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3086336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:58:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 102.084511648, 2024-09-18T04:58:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044047731999999999, 2024-09-18T04:58:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:32.844 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57986 2024-09-18T04:58:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:58:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.845 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:58:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:58:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685124+0000", 2024-09-18T04:58:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2859999999999998e-06 2024-09-18T04:58:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.846 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685130+0000", 2024-09-18T04:58:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5400000000000001e-06 2024-09-18T04:58:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685136+0000", 2024-09-18T04:58:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.152e-06 2024-09-18T04:58:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.728976+0000", 2024-09-18T04:58:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043840037999999998 2024-09-18T04:58:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.847 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729064+0000", 2024-09-18T04:58:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8709999999999996e-05 2024-09-18T04:58:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729170+0000", 2024-09-18T04:58:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000105386 2024-09-18T04:58:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.848 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57987 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3101696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:58:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 102.084462673, 2024-09-18T04:58:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044238123999999997, 2024-09-18T04:58:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57987 2024-09-18T04:58:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.849 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:58:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:58:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685170+0000", 2024-09-18T04:58:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:58:32.850 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685175+0000", 2024-09-18T04:58:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9069999999999998e-06 2024-09-18T04:58:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685187+0000", 2024-09-18T04:58:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.189e-05 2024-09-18T04:58:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685195+0000", 2024-09-18T04:58:32.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9459999999999998e-06 2024-09-18T04:58:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729212+0000", 2024-09-18T04:58:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044017239 2024-09-18T04:58:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729301+0000", 2024-09-18T04:58:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8887999999999993e-05 2024-09-18T04:58:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:32.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729409+0000", 2024-09-18T04:58:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108119 2024-09-18T04:58:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57988 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3117056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:58:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 102.084420554, 2024-09-18T04:58:32.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044429835000000001, 2024-09-18T04:58:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57988 2024-09-18T04:58:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:32.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:58:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:32.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:58:32.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:32.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685212+0000", 2024-09-18T04:58:32.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:32.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685216+0000", 2024-09-18T04:58:32.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7390000000000001e-06 2024-09-18T04:58:32.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:32.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685224+0000", 2024-09-18T04:58:32.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2260000000000005e-06 2024-09-18T04:58:32.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685230+0000", 2024-09-18T04:58:32.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0299999999999999e-06 2024-09-18T04:58:32.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:32.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729454+0000", 2024-09-18T04:58:32.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044223248999999999 2024-09-18T04:58:32.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:32.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729539+0000", 2024-09-18T04:58:32.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5029000000000002e-05 2024-09-18T04:58:32.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:32.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729643+0000", 2024-09-18T04:58:32.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104267 2024-09-18T04:58:32.860 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.860 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:32.860 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.860 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57989 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3132416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:32.860 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:58:32.860 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 102.084397802, 2024-09-18T04:58:32.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044642881000000002, 2024-09-18T04:58:32.860 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:32.861 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:32.861 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:32.861 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:32.861 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:32.861 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57989 2024-09-18T04:58:32.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.861 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:32.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:32.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:58:32.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:32.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:58:32.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:32.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:32.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685235+0000", 2024-09-18T04:58:32.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:32.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:32.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685238+0000", 2024-09-18T04:58:32.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.667e-06 2024-09-18T04:58:32.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:32.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685245+0000", 2024-09-18T04:58:32.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7909999999999999e-06 2024-09-18T04:58:32.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:32.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685250+0000", 2024-09-18T04:58:32.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3730000000000001e-06 2024-09-18T04:58:32.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:32.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729685+0000", 2024-09-18T04:58:32.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044434552000000002 2024-09-18T04:58:32.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:32.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729775+0000", 2024-09-18T04:58:32.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0395000000000004e-05 2024-09-18T04:58:32.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:32.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:32.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:32.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729879+0000", 2024-09-18T04:58:32.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010361099999999999 2024-09-18T04:58:32.867 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.867 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:32.867 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.867 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:32.867 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:32.867 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:32.869 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:58:32.870 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:32.866+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:32.969 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:32.967+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:33.017 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:33.017 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:33.017 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:58:33.017 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:33.027 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:58:33.038 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 5, 6, 7, 4, 3] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T04:58:33.038 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T04:58:33.038 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-18T04:58:33.038 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-18T04:58:33.070 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:33.067+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:33.143 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:33.143 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:33.143 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:58:33.143 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:58:33.143 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:33.152 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:58:33.196 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:33.194+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:33.262 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:33.262 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:58:33.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:58:33.262 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:58:33.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14125 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:58:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.13236290499999, 2024-09-18T04:58:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052956470999999998, 2024-09-18T04:58:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14125 2024-09-18T04:58:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:58:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:58:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126849+0000", 2024-09-18T04:58:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126852+0000", 2024-09-18T04:58:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3659999999999999e-06 2024-09-18T04:58:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126858+0000", 2024-09-18T04:58:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4540000000000004e-06 2024-09-18T04:58:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126863+0000", 2024-09-18T04:58:33.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1780000000000002e-06 2024-09-18T04:58:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179608+0000", 2024-09-18T04:58:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052745159999999999 2024-09-18T04:58:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179703+0000", 2024-09-18T04:58:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4773999999999998e-05 2024-09-18T04:58:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179806+0000", 2024-09-18T04:58:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010296799999999999 2024-09-18T04:58:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14136 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1732608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:58:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.13020235400001, 2024-09-18T04:58:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052973412999999997, 2024-09-18T04:58:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.267 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14136 2024-09-18T04:58:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:58:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:58:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:58:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129013+0000", 2024-09-18T04:58:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.96e-06 2024-09-18T04:58:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129020+0000", 2024-09-18T04:58:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6809999999999998e-06 2024-09-18T04:58:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129026+0000", 2024-09-18T04:58:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:58:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181873+0000", 2024-09-18T04:58:33.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052847238999999997 2024-09-18T04:58:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181932+0000", 2024-09-18T04:58:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9339000000000002e-05 2024-09-18T04:58:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181984+0000", 2024-09-18T04:58:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1348999999999999e-05 2024-09-18T04:58:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14137 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1781760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:33.272 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:58:33.273 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.130181949, 2024-09-18T04:58:33.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053128914999999999, 2024-09-18T04:58:33.274 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.274 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.274 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.274 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.274 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.275 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14137 2024-09-18T04:58:33.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.275 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:58:33.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.276 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.276 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:58:33.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129030+0000", 2024-09-18T04:58:33.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129033+0000", 2024-09-18T04:58:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7439999999999999e-06 2024-09-18T04:58:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129039+0000", 2024-09-18T04:58:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.314e-06 2024-09-18T04:58:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129044+0000", 2024-09-18T04:58:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9919999999999998e-06 2024-09-18T04:58:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182015+0000", 2024-09-18T04:58:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052970875000000001 2024-09-18T04:58:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.281 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182058+0000", 2024-09-18T04:58:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2997000000000002e-05 2024-09-18T04:58:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182160+0000", 2024-09-18T04:58:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101437 2024-09-18T04:58:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14138 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1830912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:58:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.130163915, 2024-09-18T04:58:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053256435999999997, 2024-09-18T04:58:33.284 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14138 2024-09-18T04:58:33.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:58:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:58:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129048+0000", 2024-09-18T04:58:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129056+0000", 2024-09-18T04:58:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.413e-06 2024-09-18T04:58:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129062+0000", 2024-09-18T04:58:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6810000000000001e-06 2024-09-18T04:58:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129068+0000", 2024-09-18T04:58:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0459999999999996e-06 2024-09-18T04:58:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182191+0000", 2024-09-18T04:58:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053123210999999997 2024-09-18T04:58:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182238+0000", 2024-09-18T04:58:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7404999999999998e-05 2024-09-18T04:58:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182305+0000", 2024-09-18T04:58:33.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7028999999999998e-05 2024-09-18T04:58:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15910 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2236416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:58:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.031076383, 2024-09-18T04:58:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053241054000000003, 2024-09-18T04:58:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.292 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15910 2024-09-18T04:58:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:58:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:58:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:58:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228138+0000", 2024-09-18T04:58:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:58:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228144+0000", 2024-09-18T04:58:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5369999999999997e-06 2024-09-18T04:58:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228149+0000", 2024-09-18T04:58:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1209999999999998e-06 2024-09-18T04:58:33.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280822+0000", 2024-09-18T04:58:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052673194999999999 2024-09-18T04:58:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280903+0000", 2024-09-18T04:58:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1179000000000003e-05 2024-09-18T04:58:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281377+0000", 2024-09-18T04:58:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047417899999999997 2024-09-18T04:58:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15911 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2285568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:58:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.03105965100001, 2024-09-18T04:58:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053415151000000001, 2024-09-18T04:58:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.297 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15911 2024-09-18T04:58:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:58:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:58:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:58:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228155+0000", 2024-09-18T04:58:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:58:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228161+0000", 2024-09-18T04:58:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:58:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228166+0000", 2024-09-18T04:58:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2320000000000001e-06 2024-09-18T04:58:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281405+0000", 2024-09-18T04:58:33.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053239336999999998 2024-09-18T04:58:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281482+0000", 2024-09-18T04:58:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6569999999999994e-05 2024-09-18T04:58:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281568+0000", 2024-09-18T04:58:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6328000000000004e-05 2024-09-18T04:58:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.301 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15918 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2662400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:58:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.02929066999999, 2024-09-18T04:58:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053091065, 2024-09-18T04:58:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.302 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15918 2024-09-18T04:58:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:58:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:58:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:58:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229924+0000", 2024-09-18T04:58:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:58:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229930+0000", 2024-09-18T04:58:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.066e-06 2024-09-18T04:58:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229935+0000", 2024-09-18T04:58:33.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9119999999999997e-06 2024-09-18T04:58:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282840+0000", 2024-09-18T04:58:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052905119 2024-09-18T04:58:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282922+0000", 2024-09-18T04:58:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2081000000000002e-05 2024-09-18T04:58:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283013+0000", 2024-09-18T04:58:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0828000000000005e-05 2024-09-18T04:58:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.308 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15919 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2711552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:58:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.02927339199999, 2024-09-18T04:58:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053293021000000003, 2024-09-18T04:58:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15919 2024-09-18T04:58:33.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:58:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:58:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:58:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229941+0000", 2024-09-18T04:58:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2639999999999998e-06 2024-09-18T04:58:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229947+0000", 2024-09-18T04:58:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9750000000000004e-06 2024-09-18T04:58:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229952+0000", 2024-09-18T04:58:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7929999999999997e-06 2024-09-18T04:58:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283062+0000", 2024-09-18T04:58:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05310952 2024-09-18T04:58:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283162+0000", 2024-09-18T04:58:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100098 2024-09-18T04:58:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283232+0000", 2024-09-18T04:58:33.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0693999999999998e-05 2024-09-18T04:58:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15920 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2760704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:58:33.315 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.029256631, 2024-09-18T04:58:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053502367000000002, 2024-09-18T04:58:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15920 2024-09-18T04:58:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.316 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:58:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:58:33.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:58:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229958+0000", 2024-09-18T04:58:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1770000000000001e-06 2024-09-18T04:58:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229964+0000", 2024-09-18T04:58:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8470000000000001e-06 2024-09-18T04:58:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229968+0000", 2024-09-18T04:58:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5680000000000001e-06 2024-09-18T04:58:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283269+0000", 2024-09-18T04:58:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053300620999999999 2024-09-18T04:58:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283359+0000", 2024-09-18T04:58:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9971999999999998e-05 2024-09-18T04:58:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283458+0000", 2024-09-18T04:58:33.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9483999999999998e-05 2024-09-18T04:58:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15921 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2809856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:58:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.029240081, 2024-09-18T04:58:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053800977, 2024-09-18T04:58:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.321 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15921 2024-09-18T04:58:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:58:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:58:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229972+0000", 2024-09-18T04:58:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229974+0000", 2024-09-18T04:58:33.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2910000000000002e-06 2024-09-18T04:58:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229982+0000", 2024-09-18T04:58:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7249999999999997e-06 2024-09-18T04:58:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229989+0000", 2024-09-18T04:58:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6429999999999998e-06 2024-09-18T04:58:33.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283601+0000", 2024-09-18T04:58:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053612211 2024-09-18T04:58:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283681+0000", 2024-09-18T04:58:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0125999999999998e-05 2024-09-18T04:58:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283774+0000", 2024-09-18T04:58:33.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2362999999999996e-05 2024-09-18T04:58:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.327 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15922 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2859008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:58:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.02919514200001, 2024-09-18T04:58:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053882270000000003, 2024-09-18T04:58:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.328 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15922 2024-09-18T04:58:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:58:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:58:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230017+0000", 2024-09-18T04:58:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230019+0000", 2024-09-18T04:58:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4210000000000002e-06 2024-09-18T04:58:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230027+0000", 2024-09-18T04:58:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000002e-06 2024-09-18T04:58:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230032+0000", 2024-09-18T04:58:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0089999999999998e-06 2024-09-18T04:58:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283800+0000", 2024-09-18T04:58:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053768418999999998 2024-09-18T04:58:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283849+0000", 2024-09-18T04:58:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9339000000000003e-05 2024-09-18T04:58:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283900+0000", 2024-09-18T04:58:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0421999999999999e-05 2024-09-18T04:58:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.333 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15923 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2924544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:58:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.029176273, 2024-09-18T04:58:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054073016000000002, 2024-09-18T04:58:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.334 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15923 2024-09-18T04:58:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:58:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:58:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:58:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230038+0000", 2024-09-18T04:58:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.463e-06 2024-09-18T04:58:33.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230045+0000", 2024-09-18T04:58:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3570000000000003e-06 2024-09-18T04:58:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230050+0000", 2024-09-18T04:58:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7299999999999996e-06 2024-09-18T04:58:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283928+0000", 2024-09-18T04:58:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053878218999999998 2024-09-18T04:58:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284010+0000", 2024-09-18T04:58:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2577000000000006e-05 2024-09-18T04:58:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284109+0000", 2024-09-18T04:58:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9102000000000001e-05 2024-09-18T04:58:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.341 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15924 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:58:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.029157803, 2024-09-18T04:58:33.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054108470999999998, 2024-09-18T04:58:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15924 2024-09-18T04:58:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:58:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:58:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:58:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230061+0000", 2024-09-18T04:58:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8240000000000002e-06 2024-09-18T04:58:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230068+0000", 2024-09-18T04:58:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4320000000000004e-06 2024-09-18T04:58:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230073+0000", 2024-09-18T04:58:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.921e-06 2024-09-18T04:58:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284153+0000", 2024-09-18T04:58:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054080793000000002 2024-09-18T04:58:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284163+0000", 2024-09-18T04:58:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.859e-06 2024-09-18T04:58:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.347 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15925 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:58:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.029133688, 2024-09-18T04:58:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054134632000000002, 2024-09-18T04:58:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.348 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15925 2024-09-18T04:58:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:58:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:58:33.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:58:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230081+0000", 2024-09-18T04:58:33.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4109999999999999e-06 2024-09-18T04:58:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230087+0000", 2024-09-18T04:58:33.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.161e-06 2024-09-18T04:58:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230092+0000", 2024-09-18T04:58:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9259999999999999e-06 2024-09-18T04:58:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284204+0000", 2024-09-18T04:58:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054111546000000003 2024-09-18T04:58:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284214+0000", 2024-09-18T04:58:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9639999999999998e-06 2024-09-18T04:58:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15926 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:58:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.029116364, 2024-09-18T04:58:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054143193999999999, 2024-09-18T04:58:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15926 2024-09-18T04:58:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:58:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:58:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:58:33.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230098+0000", 2024-09-18T04:58:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1399999999999998e-06 2024-09-18T04:58:33.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230104+0000", 2024-09-18T04:58:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-18T04:58:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230109+0000", 2024-09-18T04:58:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9250000000000003e-06 2024-09-18T04:58:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284231+0000", 2024-09-18T04:58:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054122297999999999 2024-09-18T04:58:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.360 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284239+0000", 2024-09-18T04:58:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1689999999999992e-06 2024-09-18T04:58:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15927 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:58:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.029099317, 2024-09-18T04:58:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054168435000000001, 2024-09-18T04:58:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.361 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15927 2024-09-18T04:58:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.362 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:58:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:58:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:58:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230116+0000", 2024-09-18T04:58:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5150000000000001e-06 2024-09-18T04:58:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230122+0000", 2024-09-18T04:58:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1190000000000002e-06 2024-09-18T04:58:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230126+0000", 2024-09-18T04:58:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.792e-06 2024-09-18T04:58:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284273+0000", 2024-09-18T04:58:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054146086000000003 2024-09-18T04:58:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284282+0000", 2024-09-18T04:58:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2590000000000006e-06 2024-09-18T04:58:33.367 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15928 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:33.368 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:58:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.029081934, 2024-09-18T04:58:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054189739000000001, 2024-09-18T04:58:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.369 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15928 2024-09-18T04:58:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:58:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:58:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230130+0000", 2024-09-18T04:58:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230133+0000", 2024-09-18T04:58:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.244e-06 2024-09-18T04:58:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230139+0000", 2024-09-18T04:58:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9830000000000001e-06 2024-09-18T04:58:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230143+0000", 2024-09-18T04:58:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7530000000000001e-06 2024-09-18T04:58:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284313+0000", 2024-09-18T04:58:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054169332000000001 2024-09-18T04:58:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284320+0000", 2024-09-18T04:58:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8699999999999992e-06 2024-09-18T04:58:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15929 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:58:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.02906549399999, 2024-09-18T04:58:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054218771999999998, 2024-09-18T04:58:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15929 2024-09-18T04:58:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:58:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:58:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:58:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230149+0000", 2024-09-18T04:58:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-18T04:58:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230155+0000", 2024-09-18T04:58:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2469999999999997e-06 2024-09-18T04:58:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230160+0000", 2024-09-18T04:58:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7129999999999996e-06 2024-09-18T04:58:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284355+0000", 2024-09-18T04:58:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054194900999999997 2024-09-18T04:58:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284366+0000", 2024-09-18T04:58:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0993e-05 2024-09-18T04:58:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.378 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15930 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:58:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.02904849399999, 2024-09-18T04:58:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054244713999999999, 2024-09-18T04:58:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.379 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.380 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.380 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15930 2024-09-18T04:58:33.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.380 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:58:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:58:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:58:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230166+0000", 2024-09-18T04:58:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1009999999999999e-06 2024-09-18T04:58:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230172+0000", 2024-09-18T04:58:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.981e-06 2024-09-18T04:58:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230184+0000", 2024-09-18T04:58:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2449000000000001e-05 2024-09-18T04:58:33.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284400+0000", 2024-09-18T04:58:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054215452999999997 2024-09-18T04:58:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284409+0000", 2024-09-18T04:58:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0839999999999998e-06 2024-09-18T04:58:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15931 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:58:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.029022705, 2024-09-18T04:58:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054261928000000001, 2024-09-18T04:58:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15931 2024-09-18T04:58:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.384 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:58:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:58:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.385 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230189+0000", 2024-09-18T04:58:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230192+0000", 2024-09-18T04:58:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6599999999999999e-06 2024-09-18T04:58:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230199+0000", 2024-09-18T04:58:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5570000000000001e-06 2024-09-18T04:58:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230204+0000", 2024-09-18T04:58:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2270000000000001e-06 2024-09-18T04:58:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284442+0000", 2024-09-18T04:58:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054238383000000001 2024-09-18T04:58:33.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284452+0000", 2024-09-18T04:58:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6069999999999997e-06 2024-09-18T04:58:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.388 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.388 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:33.391 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:58:33.391 INFO:tasks.ceph.mon.a.smithi002.stderr:2024-09-18T04:58:33.365+0000 7f7d3b87f640 -1 mon.a@0(leader).osd e143 definitely_dead 0 2024-09-18T04:58:33.391 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:33.387+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:33.491 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:33.487+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:33.537 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:33.537 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:33.537 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:58:33.537 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:33.546 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:58:33.592 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:33.590+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:33.693 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:33.693 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:33.693 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:58:33.693 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:58:33.693 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:33.693 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:33.689+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:33.701 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:58:33.794 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:33.790+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:33.850 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:33.850 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:58:33.850 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:58:33.850 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:58:33.850 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160715 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:58:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.192371952999999, 2024-09-18T04:58:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039724306000000001, 2024-09-18T04:58:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160715 2024-09-18T04:58:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.851 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:58:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:58:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.852 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:58:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654696+0000", 2024-09-18T04:58:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1830000000000001e-06 2024-09-18T04:58:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654702+0000", 2024-09-18T04:58:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5110000000000003e-06 2024-09-18T04:58:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.853 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654706+0000", 2024-09-18T04:58:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2470000000000002e-06 2024-09-18T04:58:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694245+0000", 2024-09-18T04:58:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039539259 2024-09-18T04:58:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694327+0000", 2024-09-18T04:58:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1817000000000006e-05 2024-09-18T04:58:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694419+0000", 2024-09-18T04:58:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1639999999999997e-05 2024-09-18T04:58:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160716 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:58:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.192356768000003, 2024-09-18T04:58:33.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039842555000000002, 2024-09-18T04:58:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160716 2024-09-18T04:58:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:58:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:58:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:58:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654711+0000", 2024-09-18T04:58:33.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.99e-06 2024-09-18T04:58:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654717+0000", 2024-09-18T04:58:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6799999999999998e-06 2024-09-18T04:58:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654721+0000", 2024-09-18T04:58:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-18T04:58:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694449+0000", 2024-09-18T04:58:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039727476999999997 2024-09-18T04:58:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694497+0000", 2024-09-18T04:58:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8294000000000002e-05 2024-09-18T04:58:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694552+0000", 2024-09-18T04:58:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5024e-05 2024-09-18T04:58:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.859 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160717 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 727040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:58:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.192316181999999, 2024-09-18T04:58:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040030957999999998, 2024-09-18T04:58:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.860 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160717 2024-09-18T04:58:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:58:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:58:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:58:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654752+0000", 2024-09-18T04:58:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8219999999999998e-06 2024-09-18T04:58:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654769+0000", 2024-09-18T04:58:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6249e-05 2024-09-18T04:58:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654775+0000", 2024-09-18T04:58:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2090000000000002e-06 2024-09-18T04:58:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694589+0000", 2024-09-18T04:58:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039814651999999999 2024-09-18T04:58:33.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694674+0000", 2024-09-18T04:58:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5032999999999997e-05 2024-09-18T04:58:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694781+0000", 2024-09-18T04:58:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106452 2024-09-18T04:58:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160718 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 747520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:58:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.192286291000002, 2024-09-18T04:58:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040152520999999997, 2024-09-18T04:58:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160718 2024-09-18T04:58:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:58:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:58:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654779+0000", 2024-09-18T04:58:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654782+0000", 2024-09-18T04:58:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.633e-06 2024-09-18T04:58:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654789+0000", 2024-09-18T04:58:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8589999999999999e-06 2024-09-18T04:58:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654793+0000", 2024-09-18T04:58:33.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5449999999999997e-06 2024-09-18T04:58:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694817+0000", 2024-09-18T04:58:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040023940000000001 2024-09-18T04:58:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694861+0000", 2024-09-18T04:58:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3961999999999997e-05 2024-09-18T04:58:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694932+0000", 2024-09-18T04:58:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1033999999999998e-05 2024-09-18T04:58:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160719 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 768000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:58:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.192268693000003, 2024-09-18T04:58:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040275321000000003, 2024-09-18T04:58:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.871 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160719 2024-09-18T04:58:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:58:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:58:33.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:58:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654799+0000", 2024-09-18T04:58:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.204e-06 2024-09-18T04:58:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654806+0000", 2024-09-18T04:58:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0500000000000003e-06 2024-09-18T04:58:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654814+0000", 2024-09-18T04:58:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5549999999999997e-06 2024-09-18T04:58:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694967+0000", 2024-09-18T04:58:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040153393000000002 2024-09-18T04:58:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695014+0000", 2024-09-18T04:58:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7194999999999998e-05 2024-09-18T04:58:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695073+0000", 2024-09-18T04:58:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8315e-05 2024-09-18T04:58:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.875 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160720 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 788480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:58:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.192246400000002, 2024-09-18T04:58:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040546450999999997, 2024-09-18T04:58:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.876 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160720 2024-09-18T04:58:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:58:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:58:33.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654819+0000", 2024-09-18T04:58:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654822+0000", 2024-09-18T04:58:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:58:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654828+0000", 2024-09-18T04:58:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2450000000000003e-06 2024-09-18T04:58:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654833+0000", 2024-09-18T04:58:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6419999999999998e-06 2024-09-18T04:58:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695116+0000", 2024-09-18T04:58:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040283126000000002 2024-09-18T04:58:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695203+0000", 2024-09-18T04:58:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7693000000000002e-05 2024-09-18T04:58:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695366+0000", 2024-09-18T04:58:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016283700000000001 2024-09-18T04:58:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160721 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 808960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:33.880 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:58:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.192229064999999, 2024-09-18T04:58:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040737433000000003, 2024-09-18T04:58:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160721 2024-09-18T04:58:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:58:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.882 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:58:33.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.882 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:58:33.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.882 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654839+0000", 2024-09-18T04:58:33.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.43e-06 2024-09-18T04:58:33.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654847+0000", 2024-09-18T04:58:33.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7300000000000005e-06 2024-09-18T04:58:33.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654858+0000", 2024-09-18T04:58:33.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.099e-05 2024-09-18T04:58:33.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695391+0000", 2024-09-18T04:58:33.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040533476999999998 2024-09-18T04:58:33.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695477+0000", 2024-09-18T04:58:33.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5870000000000003e-05 2024-09-18T04:58:33.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695575+0000", 2024-09-18T04:58:33.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7264999999999998e-05 2024-09-18T04:58:33.885 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.885 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.885 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.886 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160722 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 829440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:33.886 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:58:33.886 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.192203534999997, 2024-09-18T04:58:33.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040875739000000001, 2024-09-18T04:58:33.886 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.886 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.886 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.886 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.887 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.887 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160722 2024-09-18T04:58:33.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.887 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:58:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:58:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654862+0000", 2024-09-18T04:58:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654883+0000", 2024-09-18T04:58:33.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0468000000000001e-05 2024-09-18T04:58:33.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654891+0000", 2024-09-18T04:58:33.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8320000000000006e-06 2024-09-18T04:58:33.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654896+0000", 2024-09-18T04:58:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.13e-06 2024-09-18T04:58:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695628+0000", 2024-09-18T04:58:33.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040732011999999998 2024-09-18T04:58:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695678+0000", 2024-09-18T04:58:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0453e-05 2024-09-18T04:58:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695738+0000", 2024-09-18T04:58:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0164999999999997e-05 2024-09-18T04:58:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.892 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160723 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 849920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:58:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.192111728999997, 2024-09-18T04:58:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041073576000000001, 2024-09-18T04:58:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160723 2024-09-18T04:58:33.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:58:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:58:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:58:33.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654958+0000", 2024-09-18T04:58:33.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7230000000000002e-06 2024-09-18T04:58:33.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654968+0000", 2024-09-18T04:58:33.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0665999999999999e-05 2024-09-18T04:58:33.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654974+0000", 2024-09-18T04:58:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9259999999999997e-06 2024-09-18T04:58:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695824+0000", 2024-09-18T04:58:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040850085000000001 2024-09-18T04:58:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.896 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695933+0000", 2024-09-18T04:58:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108738 2024-09-18T04:58:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696028+0000", 2024-09-18T04:58:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4988000000000006e-05 2024-09-18T04:58:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160724 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 870400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:33.897 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:58:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.192087385999997, 2024-09-18T04:58:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041279300999999997, 2024-09-18T04:58:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160724 2024-09-18T04:58:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:58:33.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:58:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654978+0000", 2024-09-18T04:58:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654981+0000", 2024-09-18T04:58:33.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4530000000000001e-06 2024-09-18T04:58:33.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654987+0000", 2024-09-18T04:58:33.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6359999999999997e-06 2024-09-18T04:58:33.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654995+0000", 2024-09-18T04:58:33.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9759999999999995e-06 2024-09-18T04:58:33.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696069+0000", 2024-09-18T04:58:33.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041073447999999999 2024-09-18T04:58:33.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696160+0000", 2024-09-18T04:58:33.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0835999999999996e-05 2024-09-18T04:58:33.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.901 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696258+0000", 2024-09-18T04:58:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8338999999999995e-05 2024-09-18T04:58:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160725 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 890880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:58:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.192066316999998, 2024-09-18T04:58:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041552253999999997, 2024-09-18T04:58:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.902 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.903 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.903 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.903 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.903 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160725 2024-09-18T04:58:33.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.903 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:58:33.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:58:33.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654999+0000", 2024-09-18T04:58:33.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655002+0000", 2024-09-18T04:58:33.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.419e-06 2024-09-18T04:58:33.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655008+0000", 2024-09-18T04:58:33.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2310000000000001e-06 2024-09-18T04:58:33.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655013+0000", 2024-09-18T04:58:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8439999999999997e-06 2024-09-18T04:58:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696353+0000", 2024-09-18T04:58:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041340322999999998 2024-09-18T04:58:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696455+0000", 2024-09-18T04:58:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000102105 2024-09-18T04:58:33.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.907 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.907 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696552+0000", 2024-09-18T04:58:33.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6738999999999997e-05 2024-09-18T04:58:33.907 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.907 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.907 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.907 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160726 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:33.907 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:58:33.907 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.192049085000001, 2024-09-18T04:58:33.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041786440000000001, 2024-09-18T04:58:33.908 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.908 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.908 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.908 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.908 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.908 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160726 2024-09-18T04:58:33.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.908 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:58:33.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:58:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:58:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655019+0000", 2024-09-18T04:58:33.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1100000000000001e-06 2024-09-18T04:58:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655025+0000", 2024-09-18T04:58:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2190000000000001e-06 2024-09-18T04:58:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655030+0000", 2024-09-18T04:58:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5340000000000001e-06 2024-09-18T04:58:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696607+0000", 2024-09-18T04:58:33.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041577309 2024-09-18T04:58:33.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696694+0000", 2024-09-18T04:58:33.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7230000000000003e-05 2024-09-18T04:58:33.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.911 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.911 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696804+0000", 2024-09-18T04:58:33.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0001094 2024-09-18T04:58:33.911 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160727 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 931840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:58:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.192032347999998, 2024-09-18T04:58:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042000527000000003, 2024-09-18T04:58:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.912 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160727 2024-09-18T04:58:33.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.913 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:58:33.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:58:33.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655033+0000", 2024-09-18T04:58:33.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655036+0000", 2024-09-18T04:58:33.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3489999999999999e-06 2024-09-18T04:58:33.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655042+0000", 2024-09-18T04:58:33.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1399999999999997e-06 2024-09-18T04:58:33.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655046+0000", 2024-09-18T04:58:33.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5480000000000003e-06 2024-09-18T04:58:33.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696843+0000", 2024-09-18T04:58:33.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041797103000000002 2024-09-18T04:58:33.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696931+0000", 2024-09-18T04:58:33.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7526000000000002e-05 2024-09-18T04:58:33.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.916 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.916 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697034+0000", 2024-09-18T04:58:33.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010336999999999999 2024-09-18T04:58:33.916 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.916 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.916 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.916 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.916 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160728 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 952320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:33.916 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:58:33.917 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.192015791000003, 2024-09-18T04:58:33.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042191293999999997, 2024-09-18T04:58:33.917 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.917 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.917 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.917 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.917 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.917 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160728 2024-09-18T04:58:33.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.917 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:58:33.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:58:33.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:58:33.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655052+0000", 2024-09-18T04:58:33.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.232e-06 2024-09-18T04:58:33.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655058+0000", 2024-09-18T04:58:33.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9869999999999996e-06 2024-09-18T04:58:33.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655063+0000", 2024-09-18T04:58:33.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8420000000000004e-06 2024-09-18T04:58:33.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697074+0000", 2024-09-18T04:58:33.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042010446 2024-09-18T04:58:33.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697157+0000", 2024-09-18T04:58:33.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3038e-05 2024-09-18T04:58:33.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697242+0000", 2024-09-18T04:58:33.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5115000000000001e-05 2024-09-18T04:58:33.921 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.926 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.927 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.927 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.927 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.927 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160729 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 972800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:33.927 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:58:33.927 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.191999031999998, 2024-09-18T04:58:33.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042367834, 2024-09-18T04:58:33.927 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.927 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.927 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.928 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.928 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.928 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160729 2024-09-18T04:58:33.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.928 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:58:33.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:58:33.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:58:33.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655069+0000", 2024-09-18T04:58:33.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2220000000000001e-06 2024-09-18T04:58:33.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655075+0000", 2024-09-18T04:58:33.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2129999999999996e-06 2024-09-18T04:58:33.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655080+0000", 2024-09-18T04:58:33.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6840000000000004e-06 2024-09-18T04:58:33.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697280+0000", 2024-09-18T04:58:33.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042199819999999999 2024-09-18T04:58:33.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697368+0000", 2024-09-18T04:58:33.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7863999999999998e-05 2024-09-18T04:58:33.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.932 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.932 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697435+0000", 2024-09-18T04:58:33.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7416000000000006e-05 2024-09-18T04:58:33.932 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.932 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.932 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.932 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160730 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 993280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:33.932 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:58:33.932 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.191982267, 2024-09-18T04:58:33.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042559582999999998, 2024-09-18T04:58:33.933 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.933 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.933 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.933 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.933 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.933 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160730 2024-09-18T04:58:33.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.933 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:58:33.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:58:33.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:58:33.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655086+0000", 2024-09-18T04:58:33.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1500000000000002e-06 2024-09-18T04:58:33.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655092+0000", 2024-09-18T04:58:33.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.896e-06 2024-09-18T04:58:33.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655096+0000", 2024-09-18T04:58:33.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7190000000000001e-06 2024-09-18T04:58:33.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697475+0000", 2024-09-18T04:58:33.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042378644 2024-09-18T04:58:33.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697553+0000", 2024-09-18T04:58:33.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7626999999999994e-05 2024-09-18T04:58:33.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697643+0000", 2024-09-18T04:58:33.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0921000000000006e-05 2024-09-18T04:58:33.937 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.937 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.937 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.938 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.938 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160731 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 1013760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:33.938 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:58:33.938 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.191949536999999, 2024-09-18T04:58:33.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042727764000000001, 2024-09-18T04:58:33.938 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.938 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.938 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.938 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.938 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.938 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160731 2024-09-18T04:58:33.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.939 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:58:33.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:58:33.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655116+0000", 2024-09-18T04:58:33.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655119+0000", 2024-09-18T04:58:33.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1240000000000001e-06 2024-09-18T04:58:33.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655131+0000", 2024-09-18T04:58:33.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2145e-05 2024-09-18T04:58:33.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655138+0000", 2024-09-18T04:58:33.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3369999999999996e-06 2024-09-18T04:58:33.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697688+0000", 2024-09-18T04:58:33.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042550604999999998 2024-09-18T04:58:33.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697781+0000", 2024-09-18T04:58:33.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2867000000000004e-05 2024-09-18T04:58:33.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697844+0000", 2024-09-18T04:58:33.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3236000000000002e-05 2024-09-18T04:58:33.943 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.943 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.943 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.943 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160732 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 1034240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:33.943 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:58:33.943 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 57.191923688999999, 2024-09-18T04:58:33.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042857977999999998, 2024-09-18T04:58:33.943 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.943 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:33.944 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.944 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.944 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.944 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160732 2024-09-18T04:58:33.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.944 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:58:33.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:58:33.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:58:33.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655145+0000", 2024-09-18T04:58:33.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7329999999999999e-06 2024-09-18T04:58:33.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655152+0000", 2024-09-18T04:58:33.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8249999999999999e-06 2024-09-18T04:58:33.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655158+0000", 2024-09-18T04:58:33.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5010000000000001e-06 2024-09-18T04:58:33.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697873+0000", 2024-09-18T04:58:33.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042714457999999997 2024-09-18T04:58:33.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697944+0000", 2024-09-18T04:58:33.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1486e-05 2024-09-18T04:58:33.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.948 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.948 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.698000+0000", 2024-09-18T04:58:33.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6332000000000002e-05 2024-09-18T04:58:33.948 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.949 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.949 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.949 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:167084 3.18 3:1f627e86:::rbd_data.11ac138032e7.0000000000000010:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:33.949 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:58:19.165327+0000", 2024-09-18T04:58:33.949 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.681738923999999, 2024-09-18T04:58:33.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.90135632899999996, 2024-09-18T04:58:33.949 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.949 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:33.949 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.950 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.950 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.950 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 167084 2024-09-18T04:58:33.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.950 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.165327+0000", 2024-09-18T04:58:33.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.165327+0000", 2024-09-18T04:58:33.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.165326+0000", 2024-09-18T04:58:33.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:58:33.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.165332+0000", 2024-09-18T04:58:33.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3700000000000003e-06 2024-09-18T04:58:33.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.165343+0000", 2024-09-18T04:58:33.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1263999999999999e-05 2024-09-18T04:58:33.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.165348+0000", 2024-09-18T04:58:33.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9729999999999997e-06 2024-09-18T04:58:33.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.165384+0000", 2024-09-18T04:58:33.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5828999999999999e-05 2024-09-18T04:58:33.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-18T04:58:33.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.165389+0000", 2024-09-18T04:58:33.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2750000000000003e-06 2024-09-18T04:58:33.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.062869+0000", 2024-09-18T04:58:33.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.89747956100000004 2024-09-18T04:58:33.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.063068+0000", 2024-09-18T04:58:33.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000199106 2024-09-18T04:58:33.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:58:33.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.063189+0000", 2024-09-18T04:58:33.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012115100000000001 2024-09-18T04:58:33.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:58:33.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.065943+0000", 2024-09-18T04:58:33.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027540239999999999 2024-09-18T04:58:33.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:33.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.066599+0000", 2024-09-18T04:58:33.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00065621099999999997 2024-09-18T04:58:33.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:58:33.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.066644+0000", 2024-09-18T04:58:33.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.439e-05 2024-09-18T04:58:33.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.066684+0000", 2024-09-18T04:58:33.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9971000000000001e-05 2024-09-18T04:58:33.956 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.956 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.956 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.957 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.957 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.957 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:167107 3.18 3:1fe9e723:::rbd_data.11ac138032e7.0000000000000056:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:33.957 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:58:19.194623+0000", 2024-09-18T04:58:33.957 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 14.652443570999999, 2024-09-18T04:58:33.957 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.87280312599999998, 2024-09-18T04:58:33.957 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:33.957 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:33.957 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:33.957 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:33.957 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:33.958 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 167107 2024-09-18T04:58:33.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.958 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:33.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:33.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.194623+0000", 2024-09-18T04:58:33.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.958 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.958 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.958 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:33.958 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.194623+0000", 2024-09-18T04:58:33.958 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:33.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:33.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.194622+0000", 2024-09-18T04:58:33.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:33.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.959 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:33.959 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.194626+0000", 2024-09-18T04:58:33.959 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8759999999999998e-06 2024-09-18T04:58:33.959 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.959 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:33.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.194634+0000", 2024-09-18T04:58:33.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1419999999999993e-06 2024-09-18T04:58:33.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:33.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.194640+0000", 2024-09-18T04:58:33.960 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3060000000000003e-06 2024-09-18T04:58:33.960 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.960 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.960 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.960 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.194683+0000", 2024-09-18T04:58:33.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2904999999999997e-05 2024-09-18T04:58:33.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-18T04:58:33.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.194689+0000", 2024-09-18T04:58:33.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6250000000000004e-06 2024-09-18T04:58:33.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.961 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.961 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:33.961 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.064039+0000", 2024-09-18T04:58:33.961 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.86935030800000002 2024-09-18T04:58:33.961 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:33.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.064166+0000", 2024-09-18T04:58:33.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012683200000000001 2024-09-18T04:58:33.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:58:33.962 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.064287+0000", 2024-09-18T04:58:33.962 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120658 2024-09-18T04:58:33.962 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.962 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.962 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:58:33.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.066023+0000", 2024-09-18T04:58:33.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017358479999999999 2024-09-18T04:58:33.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:33.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.067339+0000", 2024-09-18T04:58:33.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013168559999999999 2024-09-18T04:58:33.963 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.963 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.963 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:58:33.963 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.067388+0000", 2024-09-18T04:58:33.963 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8764999999999998e-05 2024-09-18T04:58:33.964 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:33.964 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:33.964 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:33.964 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.067426+0000", 2024-09-18T04:58:33.964 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7598999999999997e-05 2024-09-18T04:58:33.964 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.964 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.964 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.964 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:33.964 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:33.964 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:33.966 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:58:33.966 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:33.963+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:34.038 INFO:teuthology.orchestra.run.smithi002.stderr:marked in osd.1. 2024-09-18T04:58:34.051 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T04:58:34.059 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-18T04:58:34.100 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:34.097+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:34.123 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:34.123 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:34.124 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:58:34.124 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:34.133 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:58:34.174 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:34.171+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:34.196 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:58:34.268 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:34.266+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:34.283 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:34.283 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:34.283 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:58:34.283 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:58:34.283 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:34.292 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:58:34.345 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:58:34.345 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:58:34.362 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:58:34.363 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:58:34.369 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:34.366+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:34.379 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:58:34.379 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:58:34.395 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:58:34.395 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:58:34.403 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:34.404 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:34.412 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:58:34.412 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:58:34.413 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:58:34.429 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:58:34.429 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:58:34.446 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:58:34.446 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:58:34.462 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:58:34.463 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:58:34.470 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:34.466+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:34.528 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:34.528 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:34.539 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:58:34.584 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:34.582+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:34.644 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:34.644 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:34.652 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:58:34.672 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:34.669+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:34.772 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:34.770+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:34.795 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:34.795 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:34.803 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:58:34.887 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:34.885+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:34.946 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:34.946 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:34.954 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:58:34.974 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:34.971+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:35.047 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-18T04:58:35.101 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:35.101 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:35.105 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:35.102+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:35.109 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:58:35.177 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:35.174+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:35.210 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:35.211 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:35.219 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:58:35.277 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:35.275+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:35.320 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:35.321 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:35.329 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:58:35.413 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:35.411+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:35.471 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:35.472 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:35.480 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:58:35.480 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:35.477+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:35.580 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:35.578+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:35.623 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:35.623 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:35.632 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:58:35.681 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:35.678+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:35.775 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:35.775 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:35.782 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:35.779+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:35.784 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:58:35.883 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:35.881+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:35.885 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:35.885 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:35.894 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:58:35.985 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:35.982+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:35.995 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:35.995 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:36.004 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:58:36.085 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:36.082+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:36.130 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:36.130 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:36.140 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:58:36.224 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:36.221+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:36.287 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:36.284+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:36.304 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:36.304 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:36.315 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:58:36.395 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:36.391+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:36.474 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:58:36.474 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:36.475 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:36.484 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:58:36.525 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:36.522+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:36.596 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:36.593+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:36.617 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:58:36.617 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:58:36.635 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:58:36.636 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:58:36.658 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:58:36.658 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:58:36.660 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:36.660 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:36.660 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:58:36.660 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:36.670 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:58:36.681 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:58:36.682 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:58:36.705 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:58:36.705 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:58:36.706 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:36.702+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:36.727 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:58:36.727 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:58:36.745 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:58:36.746 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:58:36.762 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:58:36.763 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:58:36.799 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:36.797+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:36.805 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:36.805 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:36.805 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:58:36.806 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:58:36.806 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:36.815 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:58:36.900 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:36.898+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:36.983 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:58:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:58:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:58:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14125 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:58:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 120.85313096199999, 2024-09-18T04:58:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052956470999999998, 2024-09-18T04:58:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:36.984 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14125 2024-09-18T04:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126850+0000", 2024-09-18T04:58:36.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126849+0000", 2024-09-18T04:58:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126852+0000", 2024-09-18T04:58:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3659999999999999e-06 2024-09-18T04:58:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:36.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126858+0000", 2024-09-18T04:58:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4540000000000004e-06 2024-09-18T04:58:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.126863+0000", 2024-09-18T04:58:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1780000000000002e-06 2024-09-18T04:58:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:36.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179608+0000", 2024-09-18T04:58:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052745159999999999 2024-09-18T04:58:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179703+0000", 2024-09-18T04:58:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4773999999999998e-05 2024-09-18T04:58:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.179806+0000", 2024-09-18T04:58:36.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010296799999999999 2024-09-18T04:58:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14136 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1732608~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:58:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 120.85097041100001, 2024-09-18T04:58:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052973412999999997, 2024-09-18T04:58:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:36.989 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:36.990 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:36.990 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14136 2024-09-18T04:58:36.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:36.990 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:36.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:36.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:36.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:58:36.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:36.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:36.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:36.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.143 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:58:37.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129010+0000", 2024-09-18T04:58:37.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.144 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.144 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.144 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129013+0000", 2024-09-18T04:58:37.144 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.96e-06 2024-09-18T04:58:37.144 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129020+0000", 2024-09-18T04:58:37.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6809999999999998e-06 2024-09-18T04:58:37.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129026+0000", 2024-09-18T04:58:37.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:58:37.145 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.145 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.145 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.145 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181873+0000", 2024-09-18T04:58:37.145 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052847238999999997 2024-09-18T04:58:37.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181932+0000", 2024-09-18T04:58:37.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9339000000000002e-05 2024-09-18T04:58:37.146 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.146 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.146 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.146 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.181984+0000", 2024-09-18T04:58:37.146 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1348999999999999e-05 2024-09-18T04:58:37.146 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.146 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.147 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.147 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.147 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.147 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14137 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1781760~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:37.147 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:58:37.147 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 120.85095000600001, 2024-09-18T04:58:37.147 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053128914999999999, 2024-09-18T04:58:37.147 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.148 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.148 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.148 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.148 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.148 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14137 2024-09-18T04:58:37.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.148 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.148 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.148 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:58:37.148 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.148 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.148 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.149 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.149 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129031+0000", 2024-09-18T04:58:37.149 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.149 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.150 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.150 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.150 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129030+0000", 2024-09-18T04:58:37.150 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129033+0000", 2024-09-18T04:58:37.151 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7439999999999999e-06 2024-09-18T04:58:37.151 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.151 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.151 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.151 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129039+0000", 2024-09-18T04:58:37.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.314e-06 2024-09-18T04:58:37.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129044+0000", 2024-09-18T04:58:37.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9919999999999998e-06 2024-09-18T04:58:37.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.152 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.152 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.152 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182015+0000", 2024-09-18T04:58:37.152 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052970875000000001 2024-09-18T04:58:37.152 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182058+0000", 2024-09-18T04:58:37.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2997000000000002e-05 2024-09-18T04:58:37.153 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.153 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.153 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.153 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182160+0000", 2024-09-18T04:58:37.153 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101437 2024-09-18T04:58:37.153 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.154 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.154 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.154 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.154 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.154 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:14138 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 1830912~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:37.154 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:58:37.154 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 120.850931972, 2024-09-18T04:58:37.154 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053256435999999997, 2024-09-18T04:58:37.154 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.154 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.154 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.155 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.155 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.155 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 14138 2024-09-18T04:58:37.155 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.155 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.155 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.155 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.155 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:58:37.155 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.156 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129049+0000", 2024-09-18T04:58:37.156 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.156 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.156 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.156 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129048+0000", 2024-09-18T04:58:37.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129056+0000", 2024-09-18T04:58:37.157 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.413e-06 2024-09-18T04:58:37.157 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.157 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.157 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.157 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129062+0000", 2024-09-18T04:58:37.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6810000000000001e-06 2024-09-18T04:58:37.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.129068+0000", 2024-09-18T04:58:37.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0459999999999996e-06 2024-09-18T04:58:37.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.158 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.158 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.158 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182191+0000", 2024-09-18T04:58:37.158 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053123210999999997 2024-09-18T04:58:37.158 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182238+0000", 2024-09-18T04:58:37.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7404999999999998e-05 2024-09-18T04:58:37.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.159 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.159 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.159 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.182305+0000", 2024-09-18T04:58:37.159 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7028999999999998e-05 2024-09-18T04:58:37.159 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.159 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.159 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.159 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.160 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.160 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15910 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2236416~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:37.160 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:58:37.160 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 120.75184444, 2024-09-18T04:58:37.160 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053241054000000003, 2024-09-18T04:58:37.160 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.160 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.160 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.160 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.160 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.160 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15910 2024-09-18T04:58:37.161 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.161 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.161 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.161 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.162 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:58:37.162 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.163 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.163 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.163 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.163 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:58:37.163 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228136+0000", 2024-09-18T04:58:37.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.164 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228138+0000", 2024-09-18T04:58:37.164 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:58:37.164 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.164 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.164 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228144+0000", 2024-09-18T04:58:37.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5369999999999997e-06 2024-09-18T04:58:37.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228149+0000", 2024-09-18T04:58:37.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1209999999999998e-06 2024-09-18T04:58:37.165 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.165 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.165 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.165 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280822+0000", 2024-09-18T04:58:37.165 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052673194999999999 2024-09-18T04:58:37.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.280903+0000", 2024-09-18T04:58:37.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1179000000000003e-05 2024-09-18T04:58:37.166 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.166 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.166 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.166 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281377+0000", 2024-09-18T04:58:37.166 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00047417899999999997 2024-09-18T04:58:37.166 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.166 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.166 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.167 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.167 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15911 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2285568~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:37.167 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:58:37.167 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 120.75182770799999, 2024-09-18T04:58:37.167 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053415151000000001, 2024-09-18T04:58:37.167 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.167 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.167 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.167 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.167 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.167 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15911 2024-09-18T04:58:37.167 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.167 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:58:37.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:58:37.168 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.168 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.168 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.168 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.168 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228153+0000", 2024-09-18T04:58:37.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228155+0000", 2024-09-18T04:58:37.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:58:37.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.169 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228161+0000", 2024-09-18T04:58:37.169 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:58:37.169 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.169 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.169 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.228166+0000", 2024-09-18T04:58:37.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2320000000000001e-06 2024-09-18T04:58:37.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281405+0000", 2024-09-18T04:58:37.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053239336999999998 2024-09-18T04:58:37.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.170 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.170 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281482+0000", 2024-09-18T04:58:37.170 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.6569999999999994e-05 2024-09-18T04:58:37.170 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.170 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.171 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.171 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.281568+0000", 2024-09-18T04:58:37.171 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6328000000000004e-05 2024-09-18T04:58:37.171 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.171 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.171 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.171 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.171 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.171 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15918 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2662400~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:37.171 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:58:37.171 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 120.750058727, 2024-09-18T04:58:37.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053091065, 2024-09-18T04:58:37.172 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.172 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.172 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.172 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.172 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.172 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15918 2024-09-18T04:58:37.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.172 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.172 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.172 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.172 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:58:37.172 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.172 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:58:37.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229922+0000", 2024-09-18T04:58:37.173 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.173 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.173 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.173 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.173 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229924+0000", 2024-09-18T04:58:37.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3630000000000001e-06 2024-09-18T04:58:37.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229930+0000", 2024-09-18T04:58:37.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.066e-06 2024-09-18T04:58:37.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.174 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.174 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229935+0000", 2024-09-18T04:58:37.174 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9119999999999997e-06 2024-09-18T04:58:37.174 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.174 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282840+0000", 2024-09-18T04:58:37.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.052905119 2024-09-18T04:58:37.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.282922+0000", 2024-09-18T04:58:37.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2081000000000002e-05 2024-09-18T04:58:37.175 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.175 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.175 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.175 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283013+0000", 2024-09-18T04:58:37.175 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0828000000000005e-05 2024-09-18T04:58:37.176 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.176 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.176 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.176 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.176 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.176 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15919 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2711552~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:37.176 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:58:37.176 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 120.75004144899999, 2024-09-18T04:58:37.176 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053293021000000003, 2024-09-18T04:58:37.176 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.176 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.176 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.176 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.177 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.177 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15919 2024-09-18T04:58:37.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.177 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:58:37.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.177 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.177 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.177 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:58:37.177 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.177 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229939+0000", 2024-09-18T04:58:37.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229941+0000", 2024-09-18T04:58:37.178 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2639999999999998e-06 2024-09-18T04:58:37.178 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.178 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.178 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.178 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229947+0000", 2024-09-18T04:58:37.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9750000000000004e-06 2024-09-18T04:58:37.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229952+0000", 2024-09-18T04:58:37.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7929999999999997e-06 2024-09-18T04:58:37.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.179 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283062+0000", 2024-09-18T04:58:37.179 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.05310952 2024-09-18T04:58:37.179 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.179 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.179 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283162+0000", 2024-09-18T04:58:37.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000100098 2024-09-18T04:58:37.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.180 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.180 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283232+0000", 2024-09-18T04:58:37.180 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0693999999999998e-05 2024-09-18T04:58:37.180 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.180 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.180 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.180 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.180 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.180 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15920 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2760704~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:37.181 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:58:37.181 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 120.750024688, 2024-09-18T04:58:37.181 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053502367000000002, 2024-09-18T04:58:37.181 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.181 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.181 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.181 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.181 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.181 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15920 2024-09-18T04:58:37.181 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.181 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:58:37.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.182 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.182 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.182 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:58:37.182 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.182 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.183 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.183 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.183 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229956+0000", 2024-09-18T04:58:37.183 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229958+0000", 2024-09-18T04:58:37.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1770000000000001e-06 2024-09-18T04:58:37.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.184 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229964+0000", 2024-09-18T04:58:37.184 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8470000000000001e-06 2024-09-18T04:58:37.184 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.184 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.184 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229968+0000", 2024-09-18T04:58:37.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5680000000000001e-06 2024-09-18T04:58:37.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283269+0000", 2024-09-18T04:58:37.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053300620999999999 2024-09-18T04:58:37.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.185 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.185 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283359+0000", 2024-09-18T04:58:37.185 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9971999999999998e-05 2024-09-18T04:58:37.185 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.185 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.186 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.186 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283458+0000", 2024-09-18T04:58:37.186 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9483999999999998e-05 2024-09-18T04:58:37.186 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.186 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.186 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.186 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.186 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.186 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15921 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2809856~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:37.186 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:58:37.187 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 120.750008138, 2024-09-18T04:58:37.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053800977, 2024-09-18T04:58:37.187 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.187 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.187 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.187 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.187 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.187 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15921 2024-09-18T04:58:37.187 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.187 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.187 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.187 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.187 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:58:37.187 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229973+0000", 2024-09-18T04:58:37.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.188 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229972+0000", 2024-09-18T04:58:37.188 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.188 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.188 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.188 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229974+0000", 2024-09-18T04:58:37.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2910000000000002e-06 2024-09-18T04:58:37.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229982+0000", 2024-09-18T04:58:37.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7249999999999997e-06 2024-09-18T04:58:37.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.189 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.189 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.229989+0000", 2024-09-18T04:58:37.189 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6429999999999998e-06 2024-09-18T04:58:37.189 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.189 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283601+0000", 2024-09-18T04:58:37.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053612211 2024-09-18T04:58:37.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283681+0000", 2024-09-18T04:58:37.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0125999999999998e-05 2024-09-18T04:58:37.190 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.190 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.190 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.190 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283774+0000", 2024-09-18T04:58:37.190 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2362999999999996e-05 2024-09-18T04:58:37.190 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.191 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.191 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.191 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.191 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.191 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15922 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2859008~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:37.191 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:58:37.191 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 120.74996319900001, 2024-09-18T04:58:37.191 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053882270000000003, 2024-09-18T04:58:37.191 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.191 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.191 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.191 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.192 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.192 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15922 2024-09-18T04:58:37.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.192 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:58:37.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.192 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.192 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.192 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.192 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230018+0000", 2024-09-18T04:58:37.192 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230017+0000", 2024-09-18T04:58:37.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230019+0000", 2024-09-18T04:58:37.193 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4210000000000002e-06 2024-09-18T04:58:37.193 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.193 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.193 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.193 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230027+0000", 2024-09-18T04:58:37.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1500000000000002e-06 2024-09-18T04:58:37.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230032+0000", 2024-09-18T04:58:37.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0089999999999998e-06 2024-09-18T04:58:37.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.194 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283800+0000", 2024-09-18T04:58:37.194 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053768418999999998 2024-09-18T04:58:37.194 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.194 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.194 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283849+0000", 2024-09-18T04:58:37.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9339000000000003e-05 2024-09-18T04:58:37.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.195 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.195 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283900+0000", 2024-09-18T04:58:37.195 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0421999999999999e-05 2024-09-18T04:58:37.195 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.195 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.195 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.195 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.195 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.195 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15923 3.0 3:01e42d3f:::rbd_data.11acf292c51a.0000000000000004:d [read 2924544~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:37.196 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:58:37.196 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 120.74994433000001, 2024-09-18T04:58:37.196 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054073016000000002, 2024-09-18T04:58:37.196 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.196 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.196 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.196 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.196 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.196 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15923 2024-09-18T04:58:37.196 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.196 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.196 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.196 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.196 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:58:37.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:58:37.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.197 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230036+0000", 2024-09-18T04:58:37.197 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.197 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.197 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.197 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230038+0000", 2024-09-18T04:58:37.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.463e-06 2024-09-18T04:58:37.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230045+0000", 2024-09-18T04:58:37.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3570000000000003e-06 2024-09-18T04:58:37.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.198 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.198 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.198 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230050+0000", 2024-09-18T04:58:37.198 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7299999999999996e-06 2024-09-18T04:58:37.198 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.199 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.283928+0000", 2024-09-18T04:58:37.199 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.053878218999999998 2024-09-18T04:58:37.199 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.199 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.199 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284010+0000", 2024-09-18T04:58:37.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2577000000000006e-05 2024-09-18T04:58:37.200 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.200 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.200 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.200 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284109+0000", 2024-09-18T04:58:37.200 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9102000000000001e-05 2024-09-18T04:58:37.200 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.200 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.201 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.201 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.201 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.201 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15924 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:37.201 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:58:37.201 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 120.74992586, 2024-09-18T04:58:37.201 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054108470999999998, 2024-09-18T04:58:37.202 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.202 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:37.202 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.202 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.202 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.202 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15924 2024-09-18T04:58:37.202 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.203 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:58:37.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:58:37.203 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.203 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.203 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.203 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.203 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230055+0000", 2024-09-18T04:58:37.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230061+0000", 2024-09-18T04:58:37.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8240000000000002e-06 2024-09-18T04:58:37.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.204 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230068+0000", 2024-09-18T04:58:37.204 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4320000000000004e-06 2024-09-18T04:58:37.204 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.204 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.204 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230073+0000", 2024-09-18T04:58:37.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.921e-06 2024-09-18T04:58:37.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.205 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.205 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.205 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284153+0000", 2024-09-18T04:58:37.205 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054080793000000002 2024-09-18T04:58:37.205 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.208 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.208 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.208 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284163+0000", 2024-09-18T04:58:37.208 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.859e-06 2024-09-18T04:58:37.208 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.208 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.208 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.209 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.209 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.209 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15925 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:37.209 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:58:37.209 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 120.749901745, 2024-09-18T04:58:37.209 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054134632000000002, 2024-09-18T04:58:37.209 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.209 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:37.209 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.209 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.209 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.210 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15925 2024-09-18T04:58:37.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.210 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:58:37.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.210 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.210 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:58:37.210 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.210 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.210 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230079+0000", 2024-09-18T04:58:37.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230081+0000", 2024-09-18T04:58:37.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4109999999999999e-06 2024-09-18T04:58:37.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.211 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.211 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.211 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230087+0000", 2024-09-18T04:58:37.211 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.161e-06 2024-09-18T04:58:37.211 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230092+0000", 2024-09-18T04:58:37.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9259999999999999e-06 2024-09-18T04:58:37.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284204+0000", 2024-09-18T04:58:37.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054111546000000003 2024-09-18T04:58:37.212 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.212 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.212 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.212 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284214+0000", 2024-09-18T04:58:37.212 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9639999999999998e-06 2024-09-18T04:58:37.213 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.213 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.213 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.213 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.213 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.213 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15926 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:37.213 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:58:37.213 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 120.749884421, 2024-09-18T04:58:37.213 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054143193999999999, 2024-09-18T04:58:37.213 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.213 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:37.213 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.213 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.213 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.214 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15926 2024-09-18T04:58:37.214 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.214 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.214 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.214 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.214 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:58:37.214 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:58:37.215 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.215 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.215 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.215 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.215 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230096+0000", 2024-09-18T04:58:37.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230098+0000", 2024-09-18T04:58:37.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1399999999999998e-06 2024-09-18T04:58:37.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.216 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.216 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.216 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230104+0000", 2024-09-18T04:58:37.216 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0120000000000002e-06 2024-09-18T04:58:37.216 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230109+0000", 2024-09-18T04:58:37.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9250000000000003e-06 2024-09-18T04:58:37.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284231+0000", 2024-09-18T04:58:37.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054122297999999999 2024-09-18T04:58:37.217 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.217 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.217 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.217 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284239+0000", 2024-09-18T04:58:37.217 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1689999999999992e-06 2024-09-18T04:58:37.218 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.218 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.218 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.218 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.218 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.218 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15927 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:37.218 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:58:37.218 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 120.749867374, 2024-09-18T04:58:37.218 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054168435000000001, 2024-09-18T04:58:37.218 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.219 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:37.219 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.219 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.219 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.219 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15927 2024-09-18T04:58:37.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.219 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.219 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.219 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:58:37.219 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.219 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.219 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:58:37.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230113+0000", 2024-09-18T04:58:37.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.220 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.220 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.220 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.220 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230116+0000", 2024-09-18T04:58:37.220 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5150000000000001e-06 2024-09-18T04:58:37.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230122+0000", 2024-09-18T04:58:37.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1190000000000002e-06 2024-09-18T04:58:37.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230126+0000", 2024-09-18T04:58:37.221 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.792e-06 2024-09-18T04:58:37.221 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.221 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.221 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.221 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284273+0000", 2024-09-18T04:58:37.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054146086000000003 2024-09-18T04:58:37.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.222 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.222 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284282+0000", 2024-09-18T04:58:37.222 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2590000000000006e-06 2024-09-18T04:58:37.222 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.222 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.222 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.222 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.222 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.222 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15928 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:37.222 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:58:37.222 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 120.749849991, 2024-09-18T04:58:37.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054189739000000001, 2024-09-18T04:58:37.223 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.223 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:37.223 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.223 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.223 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.223 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15928 2024-09-18T04:58:37.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.223 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.223 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.223 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.223 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:58:37.223 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.223 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230131+0000", 2024-09-18T04:58:37.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230130+0000", 2024-09-18T04:58:37.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.224 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.224 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.224 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.224 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230133+0000", 2024-09-18T04:58:37.224 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.244e-06 2024-09-18T04:58:37.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230139+0000", 2024-09-18T04:58:37.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9830000000000001e-06 2024-09-18T04:58:37.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.225 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230143+0000", 2024-09-18T04:58:37.225 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7530000000000001e-06 2024-09-18T04:58:37.225 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.225 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.225 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284313+0000", 2024-09-18T04:58:37.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054169332000000001 2024-09-18T04:58:37.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.226 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.226 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284320+0000", 2024-09-18T04:58:37.226 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8699999999999992e-06 2024-09-18T04:58:37.226 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.226 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.226 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.226 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.226 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.226 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15929 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:37.226 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:58:37.227 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 120.74983355099999, 2024-09-18T04:58:37.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054218771999999998, 2024-09-18T04:58:37.227 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.227 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:37.227 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.227 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.227 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.227 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15929 2024-09-18T04:58:37.227 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.227 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.227 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.227 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.227 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:58:37.227 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:58:37.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.228 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230147+0000", 2024-09-18T04:58:37.228 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.228 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.228 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.228 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230149+0000", 2024-09-18T04:58:37.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2740000000000002e-06 2024-09-18T04:58:37.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.229 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.229 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.229 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230155+0000", 2024-09-18T04:58:37.229 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2469999999999997e-06 2024-09-18T04:58:37.229 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230160+0000", 2024-09-18T04:58:37.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7129999999999996e-06 2024-09-18T04:58:37.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284355+0000", 2024-09-18T04:58:37.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054194900999999997 2024-09-18T04:58:37.230 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.230 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.230 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.230 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284366+0000", 2024-09-18T04:58:37.230 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0993e-05 2024-09-18T04:58:37.231 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.231 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.231 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.231 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.231 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.231 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15930 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:37.231 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:58:37.231 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 120.749816551, 2024-09-18T04:58:37.231 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054244713999999999, 2024-09-18T04:58:37.231 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.231 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:37.231 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.231 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.231 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.231 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15930 2024-09-18T04:58:37.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.232 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:58:37.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.232 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:58:37.232 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.232 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.232 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.232 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230164+0000", 2024-09-18T04:58:37.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230166+0000", 2024-09-18T04:58:37.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1009999999999999e-06 2024-09-18T04:58:37.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.233 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.233 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230172+0000", 2024-09-18T04:58:37.233 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.981e-06 2024-09-18T04:58:37.233 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.233 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.234 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.234 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230184+0000", 2024-09-18T04:58:37.234 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2449000000000001e-05 2024-09-18T04:58:37.234 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.234 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284400+0000", 2024-09-18T04:58:37.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054215452999999997 2024-09-18T04:58:37.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.235 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.235 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284409+0000", 2024-09-18T04:58:37.235 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0839999999999998e-06 2024-09-18T04:58:37.235 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.235 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.235 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.235 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.235 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.236 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:15931 3.0 3.fcb42780 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:37.236 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:58:37.236 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 120.749790762, 2024-09-18T04:58:37.236 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054261928000000001, 2024-09-18T04:58:37.236 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.236 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:37.236 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.236 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.236 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.236 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 15931 2024-09-18T04:58:37.236 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.236 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.236 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:58:37.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230190+0000", 2024-09-18T04:58:37.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.237 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.237 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230189+0000", 2024-09-18T04:58:37.237 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.237 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.237 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230192+0000", 2024-09-18T04:58:37.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6599999999999999e-06 2024-09-18T04:58:37.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230199+0000", 2024-09-18T04:58:37.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5570000000000001e-06 2024-09-18T04:58:37.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.238 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.238 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.230204+0000", 2024-09-18T04:58:37.238 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2270000000000001e-06 2024-09-18T04:58:37.238 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.238 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284442+0000", 2024-09-18T04:58:37.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.054238383000000001 2024-09-18T04:58:37.239 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.239 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.239 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.239 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:36.284452+0000", 2024-09-18T04:58:37.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6069999999999997e-06 2024-09-18T04:58:37.240 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.240 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.240 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.240 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.240 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.240 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:37.240 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T04:58:37.241 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:37.239+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:37.242 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:58:37.342 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:37.339+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:37.392 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:37.392 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:37.392 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:58:37.392 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:37.403 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:58:37.448 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:37.445+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:37.511 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:37.511 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:37.511 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:58:37.511 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:58:37.511 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:37.520 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:58:37.543 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:37.541+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:37.644 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:37.641+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:37.668 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:37.668 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:58:37.668 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:58:37.668 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:58:37.668 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.668 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160715 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:37.669 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:58:37.669 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.010401442999999, 2024-09-18T04:58:37.669 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039724306000000001, 2024-09-18T04:58:37.669 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.669 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.669 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.669 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.669 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.669 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160715 2024-09-18T04:58:37.669 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.669 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.669 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.669 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:58:37.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:58:37.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.670 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.670 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:58:37.670 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.670 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.670 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654696+0000", 2024-09-18T04:58:37.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1830000000000001e-06 2024-09-18T04:58:37.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654702+0000", 2024-09-18T04:58:37.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5110000000000003e-06 2024-09-18T04:58:37.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.671 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.671 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.671 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654706+0000", 2024-09-18T04:58:37.671 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2470000000000002e-06 2024-09-18T04:58:37.671 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694245+0000", 2024-09-18T04:58:37.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039539259 2024-09-18T04:58:37.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694327+0000", 2024-09-18T04:58:37.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1817000000000006e-05 2024-09-18T04:58:37.672 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.672 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.672 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.672 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694419+0000", 2024-09-18T04:58:37.672 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1639999999999997e-05 2024-09-18T04:58:37.673 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.673 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.673 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.673 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.673 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.673 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160716 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:37.673 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:58:37.673 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.010386257999997, 2024-09-18T04:58:37.673 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039842555000000002, 2024-09-18T04:58:37.673 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.673 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.673 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.674 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160716 2024-09-18T04:58:37.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.674 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.674 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.674 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.674 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:58:37.674 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.674 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:58:37.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:58:37.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.675 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.675 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.675 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.675 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654711+0000", 2024-09-18T04:58:37.675 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.99e-06 2024-09-18T04:58:37.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654717+0000", 2024-09-18T04:58:37.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6799999999999998e-06 2024-09-18T04:58:37.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654721+0000", 2024-09-18T04:58:37.676 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-18T04:58:37.676 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.676 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.676 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.676 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694449+0000", 2024-09-18T04:58:37.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039727476999999997 2024-09-18T04:58:37.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694497+0000", 2024-09-18T04:58:37.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8294000000000002e-05 2024-09-18T04:58:37.677 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.677 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.677 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.677 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694552+0000", 2024-09-18T04:58:37.677 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5024e-05 2024-09-18T04:58:37.677 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.677 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.677 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.678 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160717 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 727040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:37.678 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:58:37.678 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.010345672, 2024-09-18T04:58:37.678 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040030957999999998, 2024-09-18T04:58:37.678 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.678 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.678 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.678 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.678 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.678 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160717 2024-09-18T04:58:37.678 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.678 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.678 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:58:37.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:58:37.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.679 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.679 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.679 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:58:37.679 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.679 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654752+0000", 2024-09-18T04:58:37.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8219999999999998e-06 2024-09-18T04:58:37.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654769+0000", 2024-09-18T04:58:37.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6249e-05 2024-09-18T04:58:37.680 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.680 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.680 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.680 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654775+0000", 2024-09-18T04:58:37.680 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2090000000000002e-06 2024-09-18T04:58:37.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694589+0000", 2024-09-18T04:58:37.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039814651999999999 2024-09-18T04:58:37.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694674+0000", 2024-09-18T04:58:37.681 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5032999999999997e-05 2024-09-18T04:58:37.681 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.681 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.681 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.681 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694781+0000", 2024-09-18T04:58:37.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106452 2024-09-18T04:58:37.682 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.682 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.682 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.682 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.682 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.682 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160718 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 747520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:37.682 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:58:37.682 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.010315781000003, 2024-09-18T04:58:37.682 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040152520999999997, 2024-09-18T04:58:37.682 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.682 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.682 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.683 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.683 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.683 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160718 2024-09-18T04:58:37.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.683 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:58:37.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.683 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.683 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.683 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:58:37.683 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.683 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654779+0000", 2024-09-18T04:58:37.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654782+0000", 2024-09-18T04:58:37.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.633e-06 2024-09-18T04:58:37.684 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.684 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.684 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.684 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654789+0000", 2024-09-18T04:58:37.684 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8589999999999999e-06 2024-09-18T04:58:37.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654793+0000", 2024-09-18T04:58:37.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5449999999999997e-06 2024-09-18T04:58:37.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694817+0000", 2024-09-18T04:58:37.685 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040023940000000001 2024-09-18T04:58:37.685 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.685 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.685 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.685 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694861+0000", 2024-09-18T04:58:37.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3961999999999997e-05 2024-09-18T04:58:37.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.686 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.686 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694932+0000", 2024-09-18T04:58:37.686 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1033999999999998e-05 2024-09-18T04:58:37.686 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.686 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.686 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.686 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.686 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.686 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160719 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 768000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:37.686 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:58:37.686 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.010298183000003, 2024-09-18T04:58:37.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040275321000000003, 2024-09-18T04:58:37.687 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.687 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.687 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.687 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.687 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.687 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160719 2024-09-18T04:58:37.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.687 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.687 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.687 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.687 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:58:37.687 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.687 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:58:37.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:58:37.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.688 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.688 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.688 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.688 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654799+0000", 2024-09-18T04:58:37.688 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.204e-06 2024-09-18T04:58:37.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654806+0000", 2024-09-18T04:58:37.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0500000000000003e-06 2024-09-18T04:58:37.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654814+0000", 2024-09-18T04:58:37.689 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5549999999999997e-06 2024-09-18T04:58:37.689 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.689 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.689 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.689 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694967+0000", 2024-09-18T04:58:37.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040153393000000002 2024-09-18T04:58:37.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695014+0000", 2024-09-18T04:58:37.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7194999999999998e-05 2024-09-18T04:58:37.690 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.690 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.690 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.690 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695073+0000", 2024-09-18T04:58:37.690 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8315e-05 2024-09-18T04:58:37.690 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.690 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.690 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.691 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.691 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160720 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 788480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:37.691 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:58:37.691 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.010275890000003, 2024-09-18T04:58:37.691 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040546450999999997, 2024-09-18T04:58:37.691 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.691 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.691 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.691 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.691 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.691 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160720 2024-09-18T04:58:37.691 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.691 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:58:37.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:58:37.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.692 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.692 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.692 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.692 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654819+0000", 2024-09-18T04:58:37.692 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654822+0000", 2024-09-18T04:58:37.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:58:37.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654828+0000", 2024-09-18T04:58:37.693 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2450000000000003e-06 2024-09-18T04:58:37.693 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.693 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.693 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.693 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654833+0000", 2024-09-18T04:58:37.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6419999999999998e-06 2024-09-18T04:58:37.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695116+0000", 2024-09-18T04:58:37.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040283126000000002 2024-09-18T04:58:37.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695203+0000", 2024-09-18T04:58:37.694 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7693000000000002e-05 2024-09-18T04:58:37.694 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.694 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.694 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.694 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695366+0000", 2024-09-18T04:58:37.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016283700000000001 2024-09-18T04:58:37.695 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.695 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.695 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.695 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.695 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.695 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160721 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 808960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:37.695 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:58:37.695 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.010258555, 2024-09-18T04:58:37.695 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040737433000000003, 2024-09-18T04:58:37.695 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.695 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.696 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.696 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.696 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.696 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160721 2024-09-18T04:58:37.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.696 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.696 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:58:37.696 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.696 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.696 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.696 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:58:37.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:58:37.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.697 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.697 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654839+0000", 2024-09-18T04:58:37.697 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.43e-06 2024-09-18T04:58:37.697 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.697 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654847+0000", 2024-09-18T04:58:37.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7300000000000005e-06 2024-09-18T04:58:37.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654858+0000", 2024-09-18T04:58:37.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.099e-05 2024-09-18T04:58:37.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.698 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.698 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.698 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695391+0000", 2024-09-18T04:58:37.698 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040533476999999998 2024-09-18T04:58:37.698 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695477+0000", 2024-09-18T04:58:37.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5870000000000003e-05 2024-09-18T04:58:37.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.699 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.699 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695575+0000", 2024-09-18T04:58:37.699 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7264999999999998e-05 2024-09-18T04:58:37.699 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.699 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.699 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.699 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.699 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.700 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160722 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 829440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:37.700 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:58:37.700 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.010233024999998, 2024-09-18T04:58:37.700 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040875739000000001, 2024-09-18T04:58:37.700 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.700 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.700 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.700 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160722 2024-09-18T04:58:37.700 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.700 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.700 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:58:37.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.701 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.701 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:58:37.701 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.701 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.701 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654862+0000", 2024-09-18T04:58:37.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654883+0000", 2024-09-18T04:58:37.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0468000000000001e-05 2024-09-18T04:58:37.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.702 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.702 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.702 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654891+0000", 2024-09-18T04:58:37.702 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8320000000000006e-06 2024-09-18T04:58:37.702 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654896+0000", 2024-09-18T04:58:37.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.13e-06 2024-09-18T04:58:37.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695628+0000", 2024-09-18T04:58:37.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040732011999999998 2024-09-18T04:58:37.703 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.703 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.703 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.703 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695678+0000", 2024-09-18T04:58:37.703 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0453e-05 2024-09-18T04:58:37.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.704 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.704 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695738+0000", 2024-09-18T04:58:37.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0164999999999997e-05 2024-09-18T04:58:37.704 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.704 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.704 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.704 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.704 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.704 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160723 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 849920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:37.704 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:58:37.704 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.010141218999998, 2024-09-18T04:58:37.704 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041073576000000001, 2024-09-18T04:58:37.705 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.705 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.705 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.705 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.705 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.705 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160723 2024-09-18T04:58:37.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.705 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.705 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.705 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:58:37.705 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.705 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.705 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:58:37.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:58:37.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.706 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.706 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.706 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.706 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654958+0000", 2024-09-18T04:58:37.706 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7230000000000002e-06 2024-09-18T04:58:37.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654968+0000", 2024-09-18T04:58:37.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0665999999999999e-05 2024-09-18T04:58:37.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654974+0000", 2024-09-18T04:58:37.707 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9259999999999997e-06 2024-09-18T04:58:37.707 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.707 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.707 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.707 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695824+0000", 2024-09-18T04:58:37.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040850085000000001 2024-09-18T04:58:37.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695933+0000", 2024-09-18T04:58:37.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108738 2024-09-18T04:58:37.708 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.708 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.708 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.708 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696028+0000", 2024-09-18T04:58:37.708 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4988000000000006e-05 2024-09-18T04:58:37.708 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.708 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.708 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.709 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160724 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 870400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:37.709 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:58:37.709 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.010116875999998, 2024-09-18T04:58:37.709 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041279300999999997, 2024-09-18T04:58:37.709 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.709 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.709 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.709 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.709 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.709 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160724 2024-09-18T04:58:37.709 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.709 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.709 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:58:37.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.710 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.710 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:58:37.710 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.710 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.710 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654978+0000", 2024-09-18T04:58:37.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654981+0000", 2024-09-18T04:58:37.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4530000000000001e-06 2024-09-18T04:58:37.711 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.711 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.711 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.711 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654987+0000", 2024-09-18T04:58:37.711 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6359999999999997e-06 2024-09-18T04:58:37.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654995+0000", 2024-09-18T04:58:37.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9759999999999995e-06 2024-09-18T04:58:37.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696069+0000", 2024-09-18T04:58:37.712 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041073447999999999 2024-09-18T04:58:37.712 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.712 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.712 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.712 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696160+0000", 2024-09-18T04:58:37.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0835999999999996e-05 2024-09-18T04:58:37.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.713 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.713 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696258+0000", 2024-09-18T04:58:37.713 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8338999999999995e-05 2024-09-18T04:58:37.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.713 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.713 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.713 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.713 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.713 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160725 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 890880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:37.713 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:58:37.713 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.010095806999999, 2024-09-18T04:58:37.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041552253999999997, 2024-09-18T04:58:37.714 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.714 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.714 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.714 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.714 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.714 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160725 2024-09-18T04:58:37.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.714 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.714 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.714 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.714 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:58:37.714 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.714 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:58:37.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654999+0000", 2024-09-18T04:58:37.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.715 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.715 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.715 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.715 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655002+0000", 2024-09-18T04:58:37.715 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.419e-06 2024-09-18T04:58:37.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655008+0000", 2024-09-18T04:58:37.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2310000000000001e-06 2024-09-18T04:58:37.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655013+0000", 2024-09-18T04:58:37.716 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8439999999999997e-06 2024-09-18T04:58:37.716 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.716 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.716 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.716 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696353+0000", 2024-09-18T04:58:37.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041340322999999998 2024-09-18T04:58:37.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696455+0000", 2024-09-18T04:58:37.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000102105 2024-09-18T04:58:37.717 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.717 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.717 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.717 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696552+0000", 2024-09-18T04:58:37.717 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6738999999999997e-05 2024-09-18T04:58:37.717 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.717 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.717 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.718 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160726 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:37.718 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:58:37.718 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.010078575000001, 2024-09-18T04:58:37.718 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041786440000000001, 2024-09-18T04:58:37.718 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.718 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.718 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.718 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.718 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.718 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160726 2024-09-18T04:58:37.718 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.718 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.718 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:58:37.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:58:37.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.719 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.719 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.719 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:58:37.719 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.719 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655019+0000", 2024-09-18T04:58:37.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1100000000000001e-06 2024-09-18T04:58:37.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655025+0000", 2024-09-18T04:58:37.720 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2190000000000001e-06 2024-09-18T04:58:37.720 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.720 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.720 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.720 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655030+0000", 2024-09-18T04:58:37.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5340000000000001e-06 2024-09-18T04:58:37.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696607+0000", 2024-09-18T04:58:37.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041577309 2024-09-18T04:58:37.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.721 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696694+0000", 2024-09-18T04:58:37.721 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7230000000000003e-05 2024-09-18T04:58:37.721 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.721 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.721 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.722 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696804+0000", 2024-09-18T04:58:37.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0001094 2024-09-18T04:58:37.722 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.722 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.722 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.722 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.722 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.722 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160727 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 931840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:37.722 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:58:37.722 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.010061837999999, 2024-09-18T04:58:37.722 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042000527000000003, 2024-09-18T04:58:37.722 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.722 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.722 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.722 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.723 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.723 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160727 2024-09-18T04:58:37.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.723 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.723 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.723 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:58:37.723 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.723 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.723 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:58:37.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655033+0000", 2024-09-18T04:58:37.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.724 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.724 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.724 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.724 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655036+0000", 2024-09-18T04:58:37.724 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3489999999999999e-06 2024-09-18T04:58:37.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655042+0000", 2024-09-18T04:58:37.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1399999999999997e-06 2024-09-18T04:58:37.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.725 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655046+0000", 2024-09-18T04:58:37.725 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5480000000000003e-06 2024-09-18T04:58:37.725 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.725 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.725 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696843+0000", 2024-09-18T04:58:37.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041797103000000002 2024-09-18T04:58:37.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696931+0000", 2024-09-18T04:58:37.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7526000000000002e-05 2024-09-18T04:58:37.726 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.726 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.726 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.726 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697034+0000", 2024-09-18T04:58:37.726 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010336999999999999 2024-09-18T04:58:37.726 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.726 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.727 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.727 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.727 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.727 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160728 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 952320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:37.727 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:58:37.727 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.010045280999996, 2024-09-18T04:58:37.727 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042191293999999997, 2024-09-18T04:58:37.727 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.727 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.727 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.727 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.727 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.727 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160728 2024-09-18T04:58:37.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.728 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:58:37.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.728 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.728 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:58:37.728 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.728 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.728 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:58:37.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.729 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.729 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655052+0000", 2024-09-18T04:58:37.729 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.232e-06 2024-09-18T04:58:37.729 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.729 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655058+0000", 2024-09-18T04:58:37.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9869999999999996e-06 2024-09-18T04:58:37.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655063+0000", 2024-09-18T04:58:37.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8420000000000004e-06 2024-09-18T04:58:37.730 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.730 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.730 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.730 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697074+0000", 2024-09-18T04:58:37.730 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042010446 2024-09-18T04:58:37.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697157+0000", 2024-09-18T04:58:37.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3038e-05 2024-09-18T04:58:37.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.731 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.731 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697242+0000", 2024-09-18T04:58:37.731 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5115000000000001e-05 2024-09-18T04:58:37.731 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.731 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.731 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.731 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.731 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.732 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160729 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 972800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:37.732 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:58:37.732 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.010028521999999, 2024-09-18T04:58:37.732 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042367834, 2024-09-18T04:58:37.732 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.732 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.732 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.732 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.732 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.732 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160729 2024-09-18T04:58:37.732 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.732 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.732 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:58:37.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:58:37.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.733 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.733 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.733 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:58:37.733 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.733 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655069+0000", 2024-09-18T04:58:37.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2220000000000001e-06 2024-09-18T04:58:37.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655075+0000", 2024-09-18T04:58:37.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2129999999999996e-06 2024-09-18T04:58:37.734 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.734 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.734 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.734 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655080+0000", 2024-09-18T04:58:37.734 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6840000000000004e-06 2024-09-18T04:58:37.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697280+0000", 2024-09-18T04:58:37.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042199819999999999 2024-09-18T04:58:37.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697368+0000", 2024-09-18T04:58:37.735 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7863999999999998e-05 2024-09-18T04:58:37.735 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.735 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.735 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.735 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697435+0000", 2024-09-18T04:58:37.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7416000000000006e-05 2024-09-18T04:58:37.736 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.736 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.736 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.736 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.736 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.736 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160730 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 993280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:37.736 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:58:37.736 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.010011757000001, 2024-09-18T04:58:37.736 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042559582999999998, 2024-09-18T04:58:37.736 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.736 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.736 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.736 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.737 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.737 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160730 2024-09-18T04:58:37.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.737 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:58:37.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.737 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.737 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.737 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:58:37.737 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.737 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:58:37.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.738 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655086+0000", 2024-09-18T04:58:37.738 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1500000000000002e-06 2024-09-18T04:58:37.738 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.738 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.738 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655092+0000", 2024-09-18T04:58:37.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.896e-06 2024-09-18T04:58:37.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655096+0000", 2024-09-18T04:58:37.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7190000000000001e-06 2024-09-18T04:58:37.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.739 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697475+0000", 2024-09-18T04:58:37.739 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042378644 2024-09-18T04:58:37.739 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.739 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.739 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697553+0000", 2024-09-18T04:58:37.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7626999999999994e-05 2024-09-18T04:58:37.740 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.740 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.740 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.740 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697643+0000", 2024-09-18T04:58:37.740 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0921000000000006e-05 2024-09-18T04:58:37.740 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.740 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.741 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.741 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.741 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.741 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160731 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 1013760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:37.741 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:58:37.741 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.009979027, 2024-09-18T04:58:37.741 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042727764000000001, 2024-09-18T04:58:37.741 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.741 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.741 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.741 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.741 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.741 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160731 2024-09-18T04:58:37.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.742 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:58:37.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.742 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:58:37.742 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.742 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.742 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.742 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655116+0000", 2024-09-18T04:58:37.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655119+0000", 2024-09-18T04:58:37.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1240000000000001e-06 2024-09-18T04:58:37.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.743 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.743 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655131+0000", 2024-09-18T04:58:37.743 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2145e-05 2024-09-18T04:58:37.743 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.743 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655138+0000", 2024-09-18T04:58:37.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3369999999999996e-06 2024-09-18T04:58:37.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697688+0000", 2024-09-18T04:58:37.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042550604999999998 2024-09-18T04:58:37.744 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.744 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.744 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.744 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697781+0000", 2024-09-18T04:58:37.744 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2867000000000004e-05 2024-09-18T04:58:37.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.745 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.745 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697844+0000", 2024-09-18T04:58:37.745 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3236000000000002e-05 2024-09-18T04:58:37.745 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.745 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.745 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.745 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.745 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.745 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160732 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 1034240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:37.746 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:58:37.746 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 61.009953179, 2024-09-18T04:58:37.746 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042857977999999998, 2024-09-18T04:58:37.746 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.746 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:37.746 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.746 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.746 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.746 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160732 2024-09-18T04:58:37.746 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.746 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.746 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.746 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:58:37.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:58:37.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.747 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:58:37.747 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.747 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.747 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.747 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655145+0000", 2024-09-18T04:58:37.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7329999999999999e-06 2024-09-18T04:58:37.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655152+0000", 2024-09-18T04:58:37.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8249999999999999e-06 2024-09-18T04:58:37.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.748 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.748 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.748 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655158+0000", 2024-09-18T04:58:37.748 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5010000000000001e-06 2024-09-18T04:58:37.748 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697873+0000", 2024-09-18T04:58:37.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042714457999999997 2024-09-18T04:58:37.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697944+0000", 2024-09-18T04:58:37.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1486e-05 2024-09-18T04:58:37.749 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.749 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.749 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.749 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.698000+0000", 2024-09-18T04:58:37.749 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6332000000000002e-05 2024-09-18T04:58:37.750 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.750 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.750 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.750 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.750 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.750 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:167084 3.18 3:1f627e86:::rbd_data.11ac138032e7.0000000000000010:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:37.750 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:58:19.165327+0000", 2024-09-18T04:58:37.750 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.499768413999998, 2024-09-18T04:58:37.750 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.90135632899999996, 2024-09-18T04:58:37.750 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.750 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:37.750 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.750 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.750 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.751 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 167084 2024-09-18T04:58:37.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.751 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.165327+0000", 2024-09-18T04:58:37.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.751 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.165327+0000", 2024-09-18T04:58:37.751 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.751 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.751 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.751 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.165326+0000", 2024-09-18T04:58:37.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:58:37.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.165332+0000", 2024-09-18T04:58:37.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3700000000000003e-06 2024-09-18T04:58:37.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.752 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.752 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.165343+0000", 2024-09-18T04:58:37.752 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1263999999999999e-05 2024-09-18T04:58:37.752 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.752 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.165348+0000", 2024-09-18T04:58:37.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9729999999999997e-06 2024-09-18T04:58:37.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.165384+0000", 2024-09-18T04:58:37.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5828999999999999e-05 2024-09-18T04:58:37.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.753 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.753 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-18T04:58:37.753 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.165389+0000", 2024-09-18T04:58:37.753 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2750000000000003e-06 2024-09-18T04:58:37.753 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.062869+0000", 2024-09-18T04:58:37.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.89747956100000004 2024-09-18T04:58:37.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.063068+0000", 2024-09-18T04:58:37.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000199106 2024-09-18T04:58:37.754 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.754 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.754 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:58:37.754 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.063189+0000", 2024-09-18T04:58:37.754 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012115100000000001 2024-09-18T04:58:37.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:58:37.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.065943+0000", 2024-09-18T04:58:37.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027540239999999999 2024-09-18T04:58:37.755 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.755 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.755 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:37.755 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.066599+0000", 2024-09-18T04:58:37.755 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00065621099999999997 2024-09-18T04:58:37.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:58:37.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.066644+0000", 2024-09-18T04:58:37.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.439e-05 2024-09-18T04:58:37.756 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.756 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.756 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.756 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.066684+0000", 2024-09-18T04:58:37.756 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9971000000000001e-05 2024-09-18T04:58:37.756 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.756 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.756 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.757 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.757 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:167107 3.18 3:1fe9e723:::rbd_data.11ac138032e7.0000000000000056:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:37.757 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:58:19.194623+0000", 2024-09-18T04:58:37.757 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 18.470473061, 2024-09-18T04:58:37.757 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.87280312599999998, 2024-09-18T04:58:37.757 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:37.757 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:37.757 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:37.757 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:37.757 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:37.757 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 167107 2024-09-18T04:58:37.757 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.758 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:37.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:37.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.194623+0000", 2024-09-18T04:58:37.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:37.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.194623+0000", 2024-09-18T04:58:37.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:37.758 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.758 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.758 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:37.758 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.194622+0000", 2024-09-18T04:58:37.758 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:37.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:37.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.194626+0000", 2024-09-18T04:58:37.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8759999999999998e-06 2024-09-18T04:58:37.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:37.759 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.194634+0000", 2024-09-18T04:58:37.759 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1419999999999993e-06 2024-09-18T04:58:37.759 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.759 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.759 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:37.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.194640+0000", 2024-09-18T04:58:37.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3060000000000003e-06 2024-09-18T04:58:37.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.194683+0000", 2024-09-18T04:58:37.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2904999999999997e-05 2024-09-18T04:58:37.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.760 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-18T04:58:37.760 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.194689+0000", 2024-09-18T04:58:37.760 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6250000000000004e-06 2024-09-18T04:58:37.760 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.760 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:37.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.064039+0000", 2024-09-18T04:58:37.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.86935030800000002 2024-09-18T04:58:37.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:37.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.064166+0000", 2024-09-18T04:58:37.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012683200000000001 2024-09-18T04:58:37.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.761 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.761 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:58:37.761 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.064287+0000", 2024-09-18T04:58:37.761 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120658 2024-09-18T04:58:37.761 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:58:37.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.066023+0000", 2024-09-18T04:58:37.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017358479999999999 2024-09-18T04:58:37.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:37.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.067339+0000", 2024-09-18T04:58:37.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013168559999999999 2024-09-18T04:58:37.762 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.762 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.762 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:58:37.762 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.067388+0000", 2024-09-18T04:58:37.762 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8764999999999998e-05 2024-09-18T04:58:37.763 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:37.763 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:37.763 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:37.763 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.067426+0000", 2024-09-18T04:58:37.763 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7598999999999997e-05 2024-09-18T04:58:37.763 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.763 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.763 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.763 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:37.763 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:37.763 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:37.765 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:58:37.765 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:37.761+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:37.865 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:37.862+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:37.913 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:37.913 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:37.913 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:58:37.913 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:37.922 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:58:38.006 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:38.003+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:38.020 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-18T04:58:38.067 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:38.065+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:38.082 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:38.082 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:38.082 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:58:38.082 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:58:38.082 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:38.091 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:58:38.168 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:38.166+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:38.239 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:38.239 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:58:38.239 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:58:38.240 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:58:38.240 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.240 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11154 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:38.240 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:58:38.240 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 122.347007207, 2024-09-18T04:58:38.240 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040591919999999997, 2024-09-18T04:58:38.240 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:38.240 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:38.240 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:38.240 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:38.240 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.240 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11154 2024-09-18T04:58:38.240 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.240 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:38.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:38.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:58:38.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:38.241 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:58:38.241 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.241 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.241 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.241 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:38.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889400+0000", 2024-09-18T04:58:38.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:38.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:38.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889403+0000", 2024-09-18T04:58:38.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3999999999999999e-06 2024-09-18T04:58:38.242 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.242 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.242 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:38.242 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889409+0000", 2024-09-18T04:58:38.242 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3380000000000001e-06 2024-09-18T04:58:38.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889414+0000", 2024-09-18T04:58:38.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5789999999999997e-06 2024-09-18T04:58:38.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.243 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.243 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:38.243 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929811+0000", 2024-09-18T04:58:38.243 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040397486000000003 2024-09-18T04:58:38.243 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:38.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929898+0000", 2024-09-18T04:58:38.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6370000000000001e-05 2024-09-18T04:58:38.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.244 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:38.244 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929993+0000", 2024-09-18T04:58:38.244 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5136000000000005e-05 2024-09-18T04:58:38.244 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.244 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:38.244 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.244 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.244 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.245 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11155 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2015232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:38.245 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:58:38.245 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 122.346990029, 2024-09-18T04:58:38.245 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040769156000000001, 2024-09-18T04:58:38.245 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:38.245 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:38.245 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:38.245 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:38.245 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.245 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11155 2024-09-18T04:58:38.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.246 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:38.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:38.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:58:38.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:38.246 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:58:38.246 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.246 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.246 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.246 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:38.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:58:38.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:38.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:38.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889420+0000", 2024-09-18T04:58:38.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.181e-06 2024-09-18T04:58:38.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:38.247 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889426+0000", 2024-09-18T04:58:38.247 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8599999999999998e-06 2024-09-18T04:58:38.247 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.247 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.247 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889430+0000", 2024-09-18T04:58:38.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4399999999999998e-06 2024-09-18T04:58:38.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:38.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930020+0000", 2024-09-18T04:58:38.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040589698 2024-09-18T04:58:38.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.248 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.248 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:38.248 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930105+0000", 2024-09-18T04:58:38.248 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4812e-05 2024-09-18T04:58:38.248 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.249 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:38.249 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930187+0000", 2024-09-18T04:58:38.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2540999999999995e-05 2024-09-18T04:58:38.249 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.249 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:38.249 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.249 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.249 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.249 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11156 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2064384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:38.249 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:58:38.249 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 122.346949735, 2024-09-18T04:58:38.249 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040934419, 2024-09-18T04:58:38.249 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:38.250 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:38.250 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:38.250 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:38.250 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.250 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11156 2024-09-18T04:58:38.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.250 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:38.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:38.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:58:38.250 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.250 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.250 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.250 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:38.250 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:58:38.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:38.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889457+0000", 2024-09-18T04:58:38.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:58:38.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:38.251 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889462+0000", 2024-09-18T04:58:38.251 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8350000000000003e-06 2024-09-18T04:58:38.251 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.251 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.251 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:38.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889481+0000", 2024-09-18T04:58:38.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8845999999999999e-05 2024-09-18T04:58:38.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889495+0000", 2024-09-18T04:58:38.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4229e-05 2024-09-18T04:58:38.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.252 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:38.252 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930216+0000", 2024-09-18T04:58:38.252 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040721081999999999 2024-09-18T04:58:38.252 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.252 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:38.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930299+0000", 2024-09-18T04:58:38.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2995000000000001e-05 2024-09-18T04:58:38.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.253 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:38.253 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930393+0000", 2024-09-18T04:58:38.253 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3185999999999999e-05 2024-09-18T04:58:38.253 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.253 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:38.253 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.253 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.253 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.253 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11404 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1609728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:38.254 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:58:38.254 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 122.340145573, 2024-09-18T04:58:38.254 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040669012999999997, 2024-09-18T04:58:38.254 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:38.254 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:38.254 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:38.254 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:38.254 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.254 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11404 2024-09-18T04:58:38.254 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.254 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:38.254 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.254 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:38.254 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:58:38.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:38.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:58:38.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:38.255 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:58:38.255 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:58:38.255 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.255 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.255 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:38.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896263+0000", 2024-09-18T04:58:38.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2729999999999999e-06 2024-09-18T04:58:38.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:38.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896266+0000", 2024-09-18T04:58:38.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8200000000000001e-06 2024-09-18T04:58:38.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.256 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.256 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896269+0000", 2024-09-18T04:58:38.256 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3199999999999998e-06 2024-09-18T04:58:38.256 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.256 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:38.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936538+0000", 2024-09-18T04:58:38.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040269394999999999 2024-09-18T04:58:38.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:38.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936657+0000", 2024-09-18T04:58:38.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119498 2024-09-18T04:58:38.257 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.257 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.257 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:38.257 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936931+0000", 2024-09-18T04:58:38.257 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027391900000000002 2024-09-18T04:58:38.258 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.258 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:38.258 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.258 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.258 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.258 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11405 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1658880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:38.258 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:58:38.258 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 122.34013731500001, 2024-09-18T04:58:38.258 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040891034999999999, 2024-09-18T04:58:38.258 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:38.258 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:38.258 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:38.258 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:38.258 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.259 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11405 2024-09-18T04:58:38.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.259 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:38.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:38.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:58:38.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.259 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:38.259 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:58:38.259 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.259 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.259 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:38.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896270+0000", 2024-09-18T04:58:38.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:58:38.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:38.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896272+0000", 2024-09-18T04:58:38.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.124e-06 2024-09-18T04:58:38.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.260 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.260 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:38.260 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896274+0000", 2024-09-18T04:58:38.260 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8660000000000002e-06 2024-09-18T04:58:38.260 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896285+0000", 2024-09-18T04:58:38.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0101e-05 2024-09-18T04:58:38.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:38.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936986+0000", 2024-09-18T04:58:38.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040701213999999999 2024-09-18T04:58:38.261 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.261 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.261 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:38.261 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937070+0000", 2024-09-18T04:58:38.261 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4211999999999999e-05 2024-09-18T04:58:38.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.262 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:38.262 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937162+0000", 2024-09-18T04:58:38.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1728e-05 2024-09-18T04:58:38.262 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.262 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:38.262 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.262 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.262 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.262 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11406 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1708032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:38.262 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:58:38.262 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 122.340121146, 2024-09-18T04:58:38.262 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041085251000000003, 2024-09-18T04:58:38.263 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:38.263 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:38.263 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:38.263 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:38.263 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.263 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11406 2024-09-18T04:58:38.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.263 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:38.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.263 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:38.263 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:58:38.263 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.263 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.263 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:38.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:58:38.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:38.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:58:38.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:58:38.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.264 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.264 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:38.264 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896288+0000", 2024-09-18T04:58:38.264 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2440000000000001e-06 2024-09-18T04:58:38.264 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:38.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896291+0000", 2024-09-18T04:58:38.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8289999999999999e-06 2024-09-18T04:58:38.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896293+0000", 2024-09-18T04:58:38.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5399999999999998e-06 2024-09-18T04:58:38.265 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.265 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.265 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:38.265 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937201+0000", 2024-09-18T04:58:38.265 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040907418000000001 2024-09-18T04:58:38.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:38.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937283+0000", 2024-09-18T04:58:38.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2460999999999995e-05 2024-09-18T04:58:38.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.266 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.266 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:38.266 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937372+0000", 2024-09-18T04:58:38.266 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8987999999999996e-05 2024-09-18T04:58:38.266 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.266 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:38.266 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.266 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.267 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11407 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:38.267 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:58:38.267 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 122.340111326, 2024-09-18T04:58:38.267 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041438580000000003, 2024-09-18T04:58:38.267 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:38.267 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:38.267 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:38.267 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:38.267 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.267 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11407 2024-09-18T04:58:38.267 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.267 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:38.267 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:38.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:58:38.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:38.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:58:38.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.268 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.268 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.268 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:38.268 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896296+0000", 2024-09-18T04:58:38.268 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:38.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:38.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896298+0000", 2024-09-18T04:58:38.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:58:38.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:38.269 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896304+0000", 2024-09-18T04:58:38.269 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6570000000000002e-06 2024-09-18T04:58:38.269 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.269 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.269 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896308+0000", 2024-09-18T04:58:38.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0940000000000001e-06 2024-09-18T04:58:38.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:38.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937540+0000", 2024-09-18T04:58:38.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041231469 2024-09-18T04:58:38.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.270 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:38.270 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937625+0000", 2024-09-18T04:58:38.270 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5767999999999996e-05 2024-09-18T04:58:38.270 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.270 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.271 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:38.271 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937735+0000", 2024-09-18T04:58:38.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010974 2024-09-18T04:58:38.271 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.271 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:38.271 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.271 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.271 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.271 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11408 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:38.271 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:58:38.271 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 122.340095687, 2024-09-18T04:58:38.271 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041653055000000001, 2024-09-18T04:58:38.271 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:38.272 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:38.272 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:38.272 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:38.272 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.272 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11408 2024-09-18T04:58:38.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.272 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:38.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:38.272 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:58:38.272 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.272 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.272 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.272 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:38.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:58:38.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.273 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.273 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:38.273 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:58:38.273 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:38.273 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:38.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896314+0000", 2024-09-18T04:58:38.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0140000000000001e-06 2024-09-18T04:58:38.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:38.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896320+0000", 2024-09-18T04:58:38.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-18T04:58:38.274 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.274 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.274 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.274 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896324+0000", 2024-09-18T04:58:38.274 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4410000000000003e-06 2024-09-18T04:58:38.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:38.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937769+0000", 2024-09-18T04:58:38.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041445243 2024-09-18T04:58:38.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:38.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937861+0000", 2024-09-18T04:58:38.275 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1471000000000006e-05 2024-09-18T04:58:38.275 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.275 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.275 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:38.275 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937965+0000", 2024-09-18T04:58:38.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104516 2024-09-18T04:58:38.276 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.276 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:38.276 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.276 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.276 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.276 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11409 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:38.276 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:58:38.276 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 122.34007963099999, 2024-09-18T04:58:38.276 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041840467999999999, 2024-09-18T04:58:38.276 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:38.276 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:38.276 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:38.277 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:38.277 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.277 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11409 2024-09-18T04:58:38.277 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.277 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:38.277 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.277 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:38.277 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:58:38.277 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:38.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:58:38.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:38.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:58:38.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:38.278 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.278 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.278 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:38.278 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896331+0000", 2024-09-18T04:58:38.278 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5950000000000001e-06 2024-09-18T04:58:38.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:38.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896336+0000", 2024-09-18T04:58:38.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7259999999999999e-06 2024-09-18T04:58:38.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896341+0000", 2024-09-18T04:58:38.279 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5619999999999997e-06 2024-09-18T04:58:38.279 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.279 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.279 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:38.279 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937995+0000", 2024-09-18T04:58:38.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041654060999999999 2024-09-18T04:58:38.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:38.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938078+0000", 2024-09-18T04:58:38.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2869999999999998e-05 2024-09-18T04:58:38.280 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.280 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.280 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:38.280 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938169+0000", 2024-09-18T04:58:38.280 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1040999999999998e-05 2024-09-18T04:58:38.280 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.280 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:38.281 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.281 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.281 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11410 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1921024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:38.281 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:58:38.281 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 122.340063148, 2024-09-18T04:58:38.281 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042060342000000001, 2024-09-18T04:58:38.281 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:38.281 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:38.281 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:38.281 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:38.281 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.281 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11410 2024-09-18T04:58:38.281 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.282 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:38.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:38.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:58:38.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:38.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:58:38.282 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.282 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.282 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.282 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:38.282 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896344+0000", 2024-09-18T04:58:38.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:38.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:38.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896360+0000", 2024-09-18T04:58:38.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5068e-05 2024-09-18T04:58:38.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.283 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:38.283 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896366+0000", 2024-09-18T04:58:38.283 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5119999999999997e-06 2024-09-18T04:58:38.283 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.283 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896371+0000", 2024-09-18T04:58:38.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:58:38.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:38.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938198+0000", 2024-09-18T04:58:38.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041826579000000003 2024-09-18T04:58:38.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.284 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.284 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:38.284 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938297+0000", 2024-09-18T04:58:38.284 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9067000000000005e-05 2024-09-18T04:58:38.284 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.285 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:38.285 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938405+0000", 2024-09-18T04:58:38.285 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108492 2024-09-18T04:58:38.285 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.285 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:38.285 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.285 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.285 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.285 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11411 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1970176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:38.285 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:58:38.285 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 122.340032306, 2024-09-18T04:58:38.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042259191000000002, 2024-09-18T04:58:38.286 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:38.286 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:38.286 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:38.286 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:38.286 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.286 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11411 2024-09-18T04:58:38.286 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.286 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:38.286 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.286 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:38.286 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:58:38.286 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:38.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:58:38.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:38.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896375+0000", 2024-09-18T04:58:38.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:38.287 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.287 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.287 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:38.287 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896377+0000", 2024-09-18T04:58:38.287 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2699999999999999e-06 2024-09-18T04:58:38.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:38.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896383+0000", 2024-09-18T04:58:38.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6740000000000002e-06 2024-09-18T04:58:38.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.288 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896388+0000", 2024-09-18T04:58:38.288 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:58:38.288 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.288 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.288 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:38.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938435+0000", 2024-09-18T04:58:38.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042047169000000002 2024-09-18T04:58:38.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:38.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938534+0000", 2024-09-18T04:58:38.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8939999999999998e-05 2024-09-18T04:58:38.289 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.289 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.289 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:38.289 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938635+0000", 2024-09-18T04:58:38.289 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101208 2024-09-18T04:58:38.289 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.289 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11412 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:58:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 122.340016324, 2024-09-18T04:58:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042828079999999998, 2024-09-18T04:58:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.290 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11412 2024-09-18T04:58:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:58:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:58:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.291 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:38.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896391+0000", 2024-09-18T04:58:38.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:38.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:38.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896393+0000", 2024-09-18T04:58:38.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0810000000000001e-06 2024-09-18T04:58:38.292 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.292 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.292 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:38.292 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896399+0000", 2024-09-18T04:58:38.292 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7230000000000001e-06 2024-09-18T04:58:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896403+0000", 2024-09-18T04:58:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3470000000000001e-06 2024-09-18T04:58:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939043+0000", 2024-09-18T04:58:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042639829999999997 2024-09-18T04:58:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:38.293 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939124+0000", 2024-09-18T04:58:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0809000000000005e-05 2024-09-18T04:58:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939220+0000", 2024-09-18T04:58:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5659e-05 2024-09-18T04:58:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11413 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2068480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:38.294 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:58:38.295 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 122.340000629, 2024-09-18T04:58:38.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043042108000000003, 2024-09-18T04:58:38.295 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:38.295 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:38.295 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:38.295 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:38.295 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.295 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11413 2024-09-18T04:58:38.295 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.295 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:38.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.295 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:38.295 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:58:38.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:58:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:58:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896409+0000", 2024-09-18T04:58:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.165e-06 2024-09-18T04:58:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896415+0000", 2024-09-18T04:58:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6149999999999996e-06 2024-09-18T04:58:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896419+0000", 2024-09-18T04:58:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2649999999999998e-06 2024-09-18T04:58:38.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939262+0000", 2024-09-18T04:58:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042843444000000001 2024-09-18T04:58:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939351+0000", 2024-09-18T04:58:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8944999999999996e-05 2024-09-18T04:58:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939449+0000", 2024-09-18T04:58:38.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8029000000000006e-05 2024-09-18T04:58:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145409 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:58:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.708580980999997, 2024-09-18T04:58:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040684845999999997, 2024-09-18T04:58:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:38.299 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145409 2024-09-18T04:58:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:58:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:58:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:58:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527829+0000", 2024-09-18T04:58:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.441e-06 2024-09-18T04:58:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527835+0000", 2024-09-18T04:58:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1650000000000003e-06 2024-09-18T04:58:38.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527840+0000", 2024-09-18T04:58:38.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8250000000000004e-06 2024-09-18T04:58:38.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:38.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568404+0000", 2024-09-18T04:58:38.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040563872000000001 2024-09-18T04:58:38.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:38.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568457+0000", 2024-09-18T04:58:38.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2713999999999997e-05 2024-09-18T04:58:38.303 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.303 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.303 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:38.303 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568512+0000", 2024-09-18T04:58:38.303 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5148000000000001e-05 2024-09-18T04:58:38.303 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145411 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:58:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.708563119999994, 2024-09-18T04:58:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040792108000000001, 2024-09-18T04:58:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.304 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145411 2024-09-18T04:58:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:58:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:58:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527844+0000", 2024-09-18T04:58:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527847+0000", 2024-09-18T04:58:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0260000000000002e-06 2024-09-18T04:58:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527852+0000", 2024-09-18T04:58:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9889999999999997e-06 2024-09-18T04:58:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527857+0000", 2024-09-18T04:58:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6410000000000001e-06 2024-09-18T04:58:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568537+0000", 2024-09-18T04:58:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040679743999999997 2024-09-18T04:58:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568583+0000", 2024-09-18T04:58:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6286999999999999e-05 2024-09-18T04:58:38.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568637+0000", 2024-09-18T04:58:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.376e-05 2024-09-18T04:58:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145413 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1859584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:58:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.708546740000003, 2024-09-18T04:58:38.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040919516000000003, 2024-09-18T04:58:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145413 2024-09-18T04:58:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:58:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:58:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:58:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527863+0000", 2024-09-18T04:58:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2630000000000002e-06 2024-09-18T04:58:38.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527869+0000", 2024-09-18T04:58:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9009999999999999e-06 2024-09-18T04:58:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527874+0000", 2024-09-18T04:58:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0590000000000002e-06 2024-09-18T04:58:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568663+0000", 2024-09-18T04:58:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040788805999999997 2024-09-18T04:58:38.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:38.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568716+0000", 2024-09-18T04:58:38.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3501999999999997e-05 2024-09-18T04:58:38.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:38.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568781+0000", 2024-09-18T04:58:38.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4338000000000006e-05 2024-09-18T04:58:38.313 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.313 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:38.313 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.313 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145414 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1908736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:38.314 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:58:38.314 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.708529781999999, 2024-09-18T04:58:38.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041030234999999998, 2024-09-18T04:58:38.314 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:38.314 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:38.314 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:38.315 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:38.315 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.315 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145414 2024-09-18T04:58:38.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.315 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:38.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:38.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:58:38.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:38.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:58:38.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:38.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:58:38.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:38.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:38.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527880+0000", 2024-09-18T04:58:38.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4770000000000002e-06 2024-09-18T04:58:38.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:38.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527886+0000", 2024-09-18T04:58:38.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5759999999999996e-06 2024-09-18T04:58:38.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527890+0000", 2024-09-18T04:58:38.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8180000000000003e-06 2024-09-18T04:58:38.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:38.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568807+0000", 2024-09-18T04:58:38.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040916984000000003 2024-09-18T04:58:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568854+0000", 2024-09-18T04:58:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6740999999999999e-05 2024-09-18T04:58:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568908+0000", 2024-09-18T04:58:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4228000000000003e-05 2024-09-18T04:58:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145416 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1957888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:58:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.708513289999999, 2024-09-18T04:58:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041147364999999998, 2024-09-18T04:58:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145416 2024-09-18T04:58:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.320 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:58:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:58:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:38.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527894+0000", 2024-09-18T04:58:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527896+0000", 2024-09-18T04:58:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1789999999999998e-06 2024-09-18T04:58:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527908+0000", 2024-09-18T04:58:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1780000000000001e-05 2024-09-18T04:58:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527914+0000", 2024-09-18T04:58:38.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2029999999999997e-06 2024-09-18T04:58:38.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:38.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568935+0000", 2024-09-18T04:58:38.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041020273000000003 2024-09-18T04:58:38.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:38.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568988+0000", 2024-09-18T04:58:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2911000000000002e-05 2024-09-18T04:58:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569042+0000", 2024-09-18T04:58:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4382000000000002e-05 2024-09-18T04:58:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145418 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 2007040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:38.324 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:58:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.708488892999995, 2024-09-18T04:58:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041247862000000003, 2024-09-18T04:58:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145418 2024-09-18T04:58:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:38.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:58:38.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:38.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:58:38.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:38.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:58:38.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527921+0000", 2024-09-18T04:58:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.667e-06 2024-09-18T04:58:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527929+0000", 2024-09-18T04:58:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.542e-06 2024-09-18T04:58:38.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527934+0000", 2024-09-18T04:58:38.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1440000000000002e-06 2024-09-18T04:58:38.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569068+0000", 2024-09-18T04:58:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041134190000000001 2024-09-18T04:58:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569114+0000", 2024-09-18T04:58:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6261999999999999e-05 2024-09-18T04:58:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:38.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569167+0000", 2024-09-18T04:58:38.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2506999999999997e-05 2024-09-18T04:58:38.330 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.330 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:38.330 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.330 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145420 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 2056192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:38.330 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:58:38.330 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.708469819000001, 2024-09-18T04:58:38.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041351472, 2024-09-18T04:58:38.330 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:38.330 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145420 2024-09-18T04:58:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:58:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:38.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:58:38.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:38.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:38.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:58:38.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:38.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:38.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527940+0000", 2024-09-18T04:58:38.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6570000000000001e-06 2024-09-18T04:58:38.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527948+0000", 2024-09-18T04:58:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8350000000000004e-06 2024-09-18T04:58:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527953+0000", 2024-09-18T04:58:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0690000000000001e-06 2024-09-18T04:58:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:38.333 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569182+0000", 2024-09-18T04:58:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041228734000000003 2024-09-18T04:58:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569234+0000", 2024-09-18T04:58:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1663999999999999e-05 2024-09-18T04:58:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569290+0000", 2024-09-18T04:58:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5937000000000003e-05 2024-09-18T04:58:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:38.334 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.335 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:38.335 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:38.335 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:38.336 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:58:38.337 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:38.333+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:38.437 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:38.434+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:38.444 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:38.444 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:38.444 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:58:38.444 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:38.453 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:58:38.538 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:38.535+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:38.555 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:38.555 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:38.555 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:58:38.555 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:58:38.555 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:38.564 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:58:38.639 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:38.636+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:38.668 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:38.668 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:58:38.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:58:38.668 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:38.668 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.668 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72135 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 394240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:38.668 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:58:38.668 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 101.71068046400001, 2024-09-18T04:58:38.668 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048559739999999997, 2024-09-18T04:58:38.668 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72135 2024-09-18T04:58:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:58:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.669 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:58:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:58:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954005+0000", 2024-09-18T04:58:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5629999999999999e-06 2024-09-18T04:58:38.670 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954008+0000", 2024-09-18T04:58:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-18T04:58:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954012+0000", 2024-09-18T04:58:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4790000000000001e-06 2024-09-18T04:58:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:38.671 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002418+0000", 2024-09-18T04:58:38.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048406049 2024-09-18T04:58:38.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:38.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002460+0000", 2024-09-18T04:58:38.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2188999999999999e-05 2024-09-18T04:58:38.672 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.672 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.672 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:38.672 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002564+0000", 2024-09-18T04:58:38.672 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010377 2024-09-18T04:58:38.672 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.672 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:38.673 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.673 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.673 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.673 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72136 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 409600~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:38.673 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:58:38.673 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 101.710575517, 2024-09-18T04:58:38.673 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048670191000000002, 2024-09-18T04:58:38.673 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:38.673 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:38.673 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:38.673 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:38.673 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72136 2024-09-18T04:58:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:58:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:58:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.674 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954019+0000", 2024-09-18T04:58:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999099 2024-09-18T04:58:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954112+0000", 2024-09-18T04:58:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3273e-05 2024-09-18T04:58:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:38.675 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954121+0000", 2024-09-18T04:58:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0289999999999994e-06 2024-09-18T04:58:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954128+0000", 2024-09-18T04:58:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6189999999999996e-06 2024-09-18T04:58:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002604+0000", 2024-09-18T04:58:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048476048000000001 2024-09-18T04:58:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.676 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002682+0000", 2024-09-18T04:58:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7955999999999997e-05 2024-09-18T04:58:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002779+0000", 2024-09-18T04:58:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7372000000000002e-05 2024-09-18T04:58:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72138 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 440320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:38.677 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:58:38.678 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 101.71014166499999, 2024-09-18T04:58:38.678 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048549430999999997, 2024-09-18T04:58:38.678 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:38.678 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:38.678 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:38.678 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:38.678 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.678 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72138 2024-09-18T04:58:38.678 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.678 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:38.678 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.678 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:38.678 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:58:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:58:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954542+0000", 2024-09-18T04:58:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.679 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954545+0000", 2024-09-18T04:58:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4360000000000001e-06 2024-09-18T04:58:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954551+0000", 2024-09-18T04:58:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4949999999999996e-06 2024-09-18T04:58:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954557+0000", 2024-09-18T04:58:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5099999999999998e-06 2024-09-18T04:58:38.680 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002993+0000", 2024-09-18T04:58:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048436358999999998 2024-09-18T04:58:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003039+0000", 2024-09-18T04:58:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5884000000000003e-05 2024-09-18T04:58:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:38.681 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003092+0000", 2024-09-18T04:58:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3208000000000003e-05 2024-09-18T04:58:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72139 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 455680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:58:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 101.710122857, 2024-09-18T04:58:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048718773, 2024-09-18T04:58:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:38.682 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72139 2024-09-18T04:58:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:58:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:38.683 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:58:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954561+0000", 2024-09-18T04:58:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954564+0000", 2024-09-18T04:58:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4099999999999998e-06 2024-09-18T04:58:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.684 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954570+0000", 2024-09-18T04:58:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.178e-06 2024-09-18T04:58:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954575+0000", 2024-09-18T04:58:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0560000000000004e-06 2024-09-18T04:58:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003114+0000", 2024-09-18T04:58:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048539212999999998 2024-09-18T04:58:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.685 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003176+0000", 2024-09-18T04:58:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2154000000000002e-05 2024-09-18T04:58:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003280+0000", 2024-09-18T04:58:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010421600000000001 2024-09-18T04:58:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.686 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72140 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 471040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:58:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 101.709866916, 2024-09-18T04:58:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048670819999999997, 2024-09-18T04:58:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72140 2024-09-18T04:58:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.687 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:58:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:58:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954579+0000", 2024-09-18T04:58:38.688 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997611 2024-09-18T04:58:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954821+0000", 2024-09-18T04:58:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024241000000000001 2024-09-18T04:58:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954831+0000", 2024-09-18T04:58:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8770000000000005e-06 2024-09-18T04:58:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.689 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954838+0000", 2024-09-18T04:58:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0690000000000004e-06 2024-09-18T04:58:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003317+0000", 2024-09-18T04:58:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048478796999999997 2024-09-18T04:58:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003392+0000", 2024-09-18T04:58:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4411000000000003e-05 2024-09-18T04:58:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.690 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003488+0000", 2024-09-18T04:58:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6927000000000002e-05 2024-09-18T04:58:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72141 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 486400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:58:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 101.709839433, 2024-09-18T04:58:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048808114, 2024-09-18T04:58:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:38.691 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72141 2024-09-18T04:58:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:58:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:58:38.692 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:58:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954847+0000", 2024-09-18T04:58:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3800000000000001e-06 2024-09-18T04:58:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.693 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:38.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954854+0000", 2024-09-18T04:58:38.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6080000000000001e-06 2024-09-18T04:58:38.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954859+0000", 2024-09-18T04:58:38.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:58:38.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.694 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.694 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:38.694 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003526+0000", 2024-09-18T04:58:38.694 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048666433000000002 2024-09-18T04:58:38.694 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003599+0000", 2024-09-18T04:58:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2854000000000004e-05 2024-09-18T04:58:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003653+0000", 2024-09-18T04:58:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4511e-05 2024-09-18T04:58:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.695 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72142 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 501760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:58:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 101.70980063099999, 2024-09-18T04:58:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048885500999999998, 2024-09-18T04:58:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72142 2024-09-18T04:58:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:38.696 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:58:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:58:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954864+0000", 2024-09-18T04:58:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999804 2024-09-18T04:58:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.697 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954886+0000", 2024-09-18T04:58:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2042999999999999e-05 2024-09-18T04:58:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954893+0000", 2024-09-18T04:58:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.255e-06 2024-09-18T04:58:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954899+0000", 2024-09-18T04:58:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8780000000000003e-06 2024-09-18T04:58:38.698 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:38.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003678+0000", 2024-09-18T04:58:38.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048778834 2024-09-18T04:58:38.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:38.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003717+0000", 2024-09-18T04:58:38.699 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8860999999999999e-05 2024-09-18T04:58:38.699 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.699 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.699 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:38.699 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003769+0000", 2024-09-18T04:58:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2389000000000003e-05 2024-09-18T04:58:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72143 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 517120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:58:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 101.70967500499999, 2024-09-18T04:58:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048951738000000002, 2024-09-18T04:58:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:38.700 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:38.701 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.701 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72143 2024-09-18T04:58:38.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.701 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:38.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:38.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:58:38.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.701 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.701 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:38.701 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:58:38.701 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.701 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:38.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954904+0000", 2024-09-18T04:58:38.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998946 2024-09-18T04:58:38.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:38.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955013+0000", 2024-09-18T04:58:38.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108977 2024-09-18T04:58:38.702 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.702 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.702 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:38.702 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955021+0000", 2024-09-18T04:58:38.702 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7570000000000004e-06 2024-09-18T04:58:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955025+0000", 2024-09-18T04:58:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1219999999999997e-06 2024-09-18T04:58:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003792+0000", 2024-09-18T04:58:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048767379 2024-09-18T04:58:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:38.703 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003860+0000", 2024-09-18T04:58:38.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8033000000000004e-05 2024-09-18T04:58:38.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.704 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:38.704 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003961+0000", 2024-09-18T04:58:38.704 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010103300000000001 2024-09-18T04:58:38.704 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.704 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:38.704 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.704 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.704 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.704 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72161 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 13312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:38.704 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:58:38.705 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 101.709411368, 2024-09-18T04:58:38.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048891539999999997, 2024-09-18T04:58:38.705 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:38.705 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:38.705 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:38.705 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:38.705 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.705 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72161 2024-09-18T04:58:38.705 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.705 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:38.705 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.705 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:38.705 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:58:38.705 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:38.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:58:38.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.706 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:38.706 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955028+0000", 2024-09-18T04:58:38.706 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997549 2024-09-18T04:58:38.706 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.706 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:38.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955278+0000", 2024-09-18T04:58:38.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024973299999999998 2024-09-18T04:58:38.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:38.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955289+0000", 2024-09-18T04:58:38.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0812000000000001e-05 2024-09-18T04:58:38.707 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.707 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.707 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.707 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955296+0000", 2024-09-18T04:58:38.707 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2570000000000001e-06 2024-09-18T04:58:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004002+0000", 2024-09-18T04:58:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048705904000000001 2024-09-18T04:58:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004076+0000", 2024-09-18T04:58:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4086999999999997e-05 2024-09-18T04:58:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004165+0000", 2024-09-18T04:58:38.708 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8712e-05 2024-09-18T04:58:38.709 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.709 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:38.709 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.709 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.709 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.709 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72163 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 44032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:38.709 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:58:38.709 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 101.70878593, 2024-09-18T04:58:38.709 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048567011, 2024-09-18T04:58:38.709 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:38.709 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:38.709 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:38.709 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:38.709 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72163 2024-09-18T04:58:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:58:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:58:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.710 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955898+0000", 2024-09-18T04:58:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955901+0000", 2024-09-18T04:58:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.706e-06 2024-09-18T04:58:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955907+0000", 2024-09-18T04:58:38.711 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3559999999999998e-06 2024-09-18T04:58:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955912+0000", 2024-09-18T04:58:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9640000000000002e-06 2024-09-18T04:58:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004377+0000", 2024-09-18T04:58:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048465293999999999 2024-09-18T04:58:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:38.712 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004418+0000", 2024-09-18T04:58:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0163000000000002e-05 2024-09-18T04:58:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004466+0000", 2024-09-18T04:58:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8004000000000003e-05 2024-09-18T04:58:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72164 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:58:38.713 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 101.708767657, 2024-09-18T04:58:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048707253999999998, 2024-09-18T04:58:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72164 2024-09-18T04:58:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:58:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.714 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:58:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:58:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:38.715 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955919+0000", 2024-09-18T04:58:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2520000000000002e-06 2024-09-18T04:58:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955925+0000", 2024-09-18T04:58:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3740000000000003e-06 2024-09-18T04:58:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955930+0000", 2024-09-18T04:58:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:58:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.716 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:38.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004489+0000", 2024-09-18T04:58:38.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048558581000000003 2024-09-18T04:58:38.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:38.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004529+0000", 2024-09-18T04:58:38.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9722999999999999e-05 2024-09-18T04:58:38.717 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.717 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.717 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:38.717 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004624+0000", 2024-09-18T04:58:38.717 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5402999999999994e-05 2024-09-18T04:58:38.717 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72165 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 74752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:58:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 101.70849279799999, 2024-09-18T04:58:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048633223000000003, 2024-09-18T04:58:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.718 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72165 2024-09-18T04:58:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:58:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:58:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:38.719 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955935+0000", 2024-09-18T04:58:38.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999743 2024-09-18T04:58:38.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:38.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956199+0000", 2024-09-18T04:58:38.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000263906 2024-09-18T04:58:38.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.720 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:38.720 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956207+0000", 2024-09-18T04:58:38.720 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2519999999999999e-06 2024-09-18T04:58:38.720 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.720 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956214+0000", 2024-09-18T04:58:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7610000000000002e-06 2024-09-18T04:58:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004661+0000", 2024-09-18T04:58:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048447967000000002 2024-09-18T04:58:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004735+0000", 2024-09-18T04:58:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3789999999999997e-05 2024-09-18T04:58:38.721 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004825+0000", 2024-09-18T04:58:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9724999999999999e-05 2024-09-18T04:58:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72166 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 90112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:58:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 101.708239264, 2024-09-18T04:58:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048595737999999999, 2024-09-18T04:58:38.722 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72166 2024-09-18T04:58:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:58:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.723 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:58:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956221+0000", 2024-09-18T04:58:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997759 2024-09-18T04:58:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956448+0000", 2024-09-18T04:58:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022685500000000001 2024-09-18T04:58:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.724 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:38.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956456+0000", 2024-09-18T04:58:38.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0099999999999995e-06 2024-09-18T04:58:38.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956462+0000", 2024-09-18T04:58:38.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7200000000000003e-06 2024-09-18T04:58:38.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.725 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.725 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:38.725 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004862+0000", 2024-09-18T04:58:38.725 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048399795000000002 2024-09-18T04:58:38.725 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:38.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004941+0000", 2024-09-18T04:58:38.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9414999999999998e-05 2024-09-18T04:58:38.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.726 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:38.726 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005041+0000", 2024-09-18T04:58:38.726 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9942e-05 2024-09-18T04:58:38.726 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.726 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:38.726 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.726 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.726 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.727 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72167 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:38.727 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:58:38.727 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 101.708152226, 2024-09-18T04:58:38.727 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048622894999999999, 2024-09-18T04:58:38.727 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:38.727 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:38.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:38.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:38.727 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.727 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72167 2024-09-18T04:58:38.727 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.727 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:38.727 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.727 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:38.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:58:38.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:38.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:58:38.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.728 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.728 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:38.728 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956468+0000", 2024-09-18T04:58:38.728 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999352 2024-09-18T04:58:38.728 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:38.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956535+0000", 2024-09-18T04:58:38.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7440999999999993e-05 2024-09-18T04:58:38.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:38.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956544+0000", 2024-09-18T04:58:38.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1039999999999996e-06 2024-09-18T04:58:38.729 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.729 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.729 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.729 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956550+0000", 2024-09-18T04:58:38.729 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9850000000000003e-06 2024-09-18T04:58:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005066+0000", 2024-09-18T04:58:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048515614999999998 2024-09-18T04:58:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005107+0000", 2024-09-18T04:58:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0843999999999997e-05 2024-09-18T04:58:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:38.730 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005155+0000", 2024-09-18T04:58:38.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8686000000000001e-05 2024-09-18T04:58:38.731 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.731 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:38.731 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.731 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.731 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.731 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72168 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:38.731 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:58:38.731 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 101.707991163, 2024-09-18T04:58:38.731 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048585179999999999, 2024-09-18T04:58:38.731 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:38.731 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:38.731 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:38.731 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:38.732 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.732 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72168 2024-09-18T04:58:38.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.732 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:38.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:38.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:58:38.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.732 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.732 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.732 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:38.732 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:58:38.732 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956555+0000", 2024-09-18T04:58:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998612 2024-09-18T04:58:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956696+0000", 2024-09-18T04:58:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000140922 2024-09-18T04:58:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956707+0000", 2024-09-18T04:58:38.733 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1555e-05 2024-09-18T04:58:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956714+0000", 2024-09-18T04:58:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2709999999999998e-06 2024-09-18T04:58:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005177+0000", 2024-09-18T04:58:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048463421 2024-09-18T04:58:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.734 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005214+0000", 2024-09-18T04:58:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.676e-05 2024-09-18T04:58:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005279+0000", 2024-09-18T04:58:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4899000000000003e-05 2024-09-18T04:58:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.735 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72199 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:38.736 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:58:38.736 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 101.705459271, 2024-09-18T04:58:38.736 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048626057, 2024-09-18T04:58:38.736 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:38.736 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:38.736 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:38.736 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:38.736 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.736 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72199 2024-09-18T04:58:38.736 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.736 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:38.736 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.736 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:58:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:58:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959195+0000", 2024-09-18T04:58:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999969 2024-09-18T04:58:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.737 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959228+0000", 2024-09-18T04:58:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3609000000000003e-05 2024-09-18T04:58:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959236+0000", 2024-09-18T04:58:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.157e-06 2024-09-18T04:58:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959243+0000", 2024-09-18T04:58:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5910000000000001e-06 2024-09-18T04:58:38.738 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007684+0000", 2024-09-18T04:58:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048440745 2024-09-18T04:58:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007762+0000", 2024-09-18T04:58:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8360000000000002e-05 2024-09-18T04:58:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007851+0000", 2024-09-18T04:58:38.739 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9388000000000005e-05 2024-09-18T04:58:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72240 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 428032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:58:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 101.70136596099999, 2024-09-18T04:58:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048643767999999997, 2024-09-18T04:58:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:38.740 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72240 2024-09-18T04:58:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:58:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:58:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.741 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963240+0000", 2024-09-18T04:58:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999218 2024-09-18T04:58:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963323+0000", 2024-09-18T04:58:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2375999999999999e-05 2024-09-18T04:58:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963332+0000", 2024-09-18T04:58:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1700000000000003e-06 2024-09-18T04:58:38.742 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963338+0000", 2024-09-18T04:58:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2149999999999998e-06 2024-09-18T04:58:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011785+0000", 2024-09-18T04:58:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048446493 2024-09-18T04:58:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011861+0000", 2024-09-18T04:58:38.743 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6472999999999998e-05 2024-09-18T04:58:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011962+0000", 2024-09-18T04:58:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101167 2024-09-18T04:58:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:117375 3.1 3:80b18d38:::rbd_header.11acae5d5365:head [watch reconnect cookie 140406307615440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T04:58:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:58:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.280291423999998, 2024-09-18T04:58:38.744 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.018908156, 2024-09-18T04:58:38.745 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:38.745 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:38.745 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:38.745 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:38.745 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.745 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117375 2024-09-18T04:58:38.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.745 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:38.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.745 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:38.745 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:58:38.745 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.745 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.745 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:38.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:58:38.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:38.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384391+0000", 2024-09-18T04:58:38.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:58:38.746 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.746 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.746 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:38.746 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384401+0000", 2024-09-18T04:58:38.746 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4299999999999995e-06 2024-09-18T04:58:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384418+0000", 2024-09-18T04:58:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6745e-05 2024-09-18T04:58:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384429+0000", 2024-09-18T04:58:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1122e-05 2024-09-18T04:58:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:38.747 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384496+0000", 2024-09-18T04:58:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6841000000000006e-05 2024-09-18T04:58:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-18T04:58:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384505+0000", 2024-09-18T04:58:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4080000000000004e-06 2024-09-18T04:58:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.388971+0000", 2024-09-18T04:58:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0044660000000000004 2024-09-18T04:58:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.748 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:38.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.388981+0000", 2024-09-18T04:58:38.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0118e-05 2024-09-18T04:58:38.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:38.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.399433+0000", 2024-09-18T04:58:38.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.010452055 2024-09-18T04:58:38.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.749 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.749 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:38.749 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.399594+0000", 2024-09-18T04:58:38.749 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000160669 2024-09-18T04:58:38.749 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.403301+0000", 2024-09-18T04:58:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037074370000000001 2024-09-18T04:58:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:117376 3.1 3:80b18d38:::rbd_header.11acae5d5365:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T04:58:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:58:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 75.278239773999999, 2024-09-18T04:58:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0196134539999999, 2024-09-18T04:58:38.750 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117376 2024-09-18T04:58:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:58:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.751 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:58:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386443+0000", 2024-09-18T04:58:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:58:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386457+0000", 2024-09-18T04:58:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4207e-05 2024-09-18T04:58:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.752 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386478+0000", 2024-09-18T04:58:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0349000000000001e-05 2024-09-18T04:58:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386503+0000", 2024-09-18T04:58:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5582e-05 2024-09-18T04:58:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386558+0000", 2024-09-18T04:58:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4892999999999998e-05 2024-09-18T04:58:38.753 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-18T04:58:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386562+0000", 2024-09-18T04:58:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.313e-06 2024-09-18T04:58:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.389010+0000", 2024-09-18T04:58:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024480890000000001 2024-09-18T04:58:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:38.754 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.389018+0000", 2024-09-18T04:58:38.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8469999999999996e-06 2024-09-18T04:58:38.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:38.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400073+0000", 2024-09-18T04:58:38.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.011054286 2024-09-18T04:58:38.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:38.755 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400124+0000", 2024-09-18T04:58:38.755 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1332999999999998e-05 2024-09-18T04:58:38.755 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.755 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.755 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:58:38.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400477+0000", 2024-09-18T04:58:38.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035329300000000002 2024-09-18T04:58:38.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:38.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.401353+0000", 2024-09-18T04:58:38.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00087584200000000005 2024-09-18T04:58:38.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.756 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.756 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:38.756 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406006+0000", 2024-09-18T04:58:38.756 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0046534250000000001 2024-09-18T04:58:38.756 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:38.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406037+0000", 2024-09-18T04:58:38.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0559000000000003e-05 2024-09-18T04:58:38.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.757 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:38.757 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406058+0000", 2024-09-18T04:58:38.757 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1260999999999999e-05 2024-09-18T04:58:38.757 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.757 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:38.757 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.757 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.757 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.758 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167963 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e147)", 2024-09-18T04:58:38.758 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:37.082364+0000", 2024-09-18T04:58:38.758 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 1.5823207610000001, 2024-09-18T04:58:38.758 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94325145700000002, 2024-09-18T04:58:38.758 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:38.758 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:38.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:38.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:38.758 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:38.758 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167963 2024-09-18T04:58:38.758 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.758 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:38.758 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.758 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:38.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:37.082364+0000", 2024-09-18T04:58:38.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:38.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:37.082364+0000", 2024-09-18T04:58:38.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:38.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.759 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:38.759 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:37.082362+0000", 2024-09-18T04:58:38.759 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:58:38.759 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.759 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:38.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:37.082368+0000", 2024-09-18T04:58:38.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8289999999999996e-06 2024-09-18T04:58:38.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:38.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:37.082379+0000", 2024-09-18T04:58:38.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0975000000000001e-05 2024-09-18T04:58:38.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.760 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.760 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:38.760 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:37.082403+0000", 2024-09-18T04:58:38.760 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4029e-05 2024-09-18T04:58:38.760 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:38.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:37.082443+0000", 2024-09-18T04:58:38.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9947999999999998e-05 2024-09-18T04:58:38.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:38.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:37.082450+0000", 2024-09-18T04:58:38.761 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0350000000000004e-06 2024-09-18T04:58:38.761 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.761 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.761 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:38.761 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:38.025368+0000", 2024-09-18T04:58:38.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94291755399999999 2024-09-18T04:58:38.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:38.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:38.025504+0000", 2024-09-18T04:58:38.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013585100000000001 2024-09-18T04:58:38.762 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:38.762 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:38.762 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:38.762 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:38.025615+0000", 2024-09-18T04:58:38.762 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111618 2024-09-18T04:58:38.762 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.762 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:38.762 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.763 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:38.763 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:38.763 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:38.764 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:58:38.765 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:38.761+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:38.778 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:58:38.865 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:38.863+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:38.908 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:38.908 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:38.909 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:58:38.909 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:38.909 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:58:38.909 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:58:38.917 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:58:38.926 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:58:38.926 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:58:38.942 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:58:38.942 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:58:38.957 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:58:38.958 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:58:38.974 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:58:38.975 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:58:38.990 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:58:38.990 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:58:39.000 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:38.998+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:39.007 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:58:39.008 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:58:39.026 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:58:39.026 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:58:39.060 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 5, 6, 7, 4, 3, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T04:58:39.060 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T04:58:39.060 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-18T04:58:39.061 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.000000 2024-09-18T04:58:39.061 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 0 2024-09-18T04:58:39.066 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:39.064+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:39.068 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:39.068 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:39.077 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:58:39.167 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:39.164+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:39.220 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:39.220 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:39.228 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:58:39.269 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:39.266+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:39.332 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:39.332 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:39.341 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:58:39.384 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:39.382+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:39.444 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:39.444 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:39.452 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:58:39.472 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:39.468+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:39.573 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:39.569+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:39.595 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:39.595 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:39.604 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:58:39.674 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:39.671+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:39.706 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:39.706 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:39.714 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:58:39.775 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:39.772+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:39.816 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:39.816 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:39.824 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:58:39.875 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:39.872+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:39.965 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:39.966 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:39.974 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:58:39.976 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:39.973+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:40.044 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T04:58:40.075 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.1 primary-affinity to 0 (802) 2024-09-18T04:58:40.077 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:40.073+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:40.111 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:40.112 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:40.121 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:58:40.177 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:40.174+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:40.269 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:40.269 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:40.277 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:40.274+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:40.279 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:58:40.378 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:40.374+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:40.426 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:40.426 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:40.434 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:58:40.518 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:40.515+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:40.579 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:40.577+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:40.581 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:40.581 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:40.590 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:58:40.680 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:40.677+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:40.737 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:40.737 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:40.746 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:58:40.791 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:40.788+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:40.853 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:40.853 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:40.862 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:58:40.882 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:40.880+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:40.982 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:40.980+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:41.009 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:41.009 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:41.017 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:58:41.039 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:58:41.076 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-18T04:58:41.084 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:41.080+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:41.134 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:41.134 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:41.144 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:58:41.184 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:41.181+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:41.197 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:58:41.197 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:58:41.214 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:58:41.215 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:58:41.231 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:58:41.231 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:58:41.247 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:58:41.247 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:58:41.264 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:58:41.264 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:58:41.280 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:58:41.280 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:58:41.285 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:41.282+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:41.295 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:41.295 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:58:41.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:58:41.295 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:58:41.295 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.295 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160715 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 686080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:41.295 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:58:41.295 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.636345707000004, 2024-09-18T04:58:41.295 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039724306000000001, 2024-09-18T04:58:41.295 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160715 2024-09-18T04:58:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:58:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.296 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:58:41.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654694+0000", 2024-09-18T04:58:41.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.297 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.297 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.297 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654696+0000", 2024-09-18T04:58:41.297 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1830000000000001e-06 2024-09-18T04:58:41.297 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654702+0000", 2024-09-18T04:58:41.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5110000000000003e-06 2024-09-18T04:58:41.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654706+0000", 2024-09-18T04:58:41.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2470000000000002e-06 2024-09-18T04:58:41.298 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.298 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.298 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.298 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694245+0000", 2024-09-18T04:58:41.298 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039539259 2024-09-18T04:58:41.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694327+0000", 2024-09-18T04:58:41.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1817000000000006e-05 2024-09-18T04:58:41.299 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.299 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.299 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.299 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694419+0000", 2024-09-18T04:58:41.299 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1639999999999997e-05 2024-09-18T04:58:41.299 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.299 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.299 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.300 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.300 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160716 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 706560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:41.300 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:58:41.300 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.636330521999994, 2024-09-18T04:58:41.300 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039842555000000002, 2024-09-18T04:58:41.300 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.300 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.300 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.300 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.300 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.300 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160716 2024-09-18T04:58:41.300 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.300 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:58:41.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:58:41.301 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.301 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.301 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.301 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.301 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654709+0000", 2024-09-18T04:58:41.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.302 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.302 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654711+0000", 2024-09-18T04:58:41.302 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.99e-06 2024-09-18T04:58:41.302 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.302 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654717+0000", 2024-09-18T04:58:41.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6799999999999998e-06 2024-09-18T04:58:41.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654721+0000", 2024-09-18T04:58:41.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3989999999999997e-06 2024-09-18T04:58:41.304 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.304 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.304 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.304 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694449+0000", 2024-09-18T04:58:41.304 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039727476999999997 2024-09-18T04:58:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694497+0000", 2024-09-18T04:58:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8294000000000002e-05 2024-09-18T04:58:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694552+0000", 2024-09-18T04:58:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5024e-05 2024-09-18T04:58:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.305 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160717 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 727040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:58:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.636289935999997, 2024-09-18T04:58:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040030957999999998, 2024-09-18T04:58:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160717 2024-09-18T04:58:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.306 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:58:41.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:58:41.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.307 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.307 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.307 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654750+0000", 2024-09-18T04:58:41.307 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.307 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654752+0000", 2024-09-18T04:58:41.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8219999999999998e-06 2024-09-18T04:58:41.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654769+0000", 2024-09-18T04:58:41.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.6249e-05 2024-09-18T04:58:41.308 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.308 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.308 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.308 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654775+0000", 2024-09-18T04:58:41.308 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2090000000000002e-06 2024-09-18T04:58:41.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694589+0000", 2024-09-18T04:58:41.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.039814651999999999 2024-09-18T04:58:41.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694674+0000", 2024-09-18T04:58:41.309 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5032999999999997e-05 2024-09-18T04:58:41.309 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.309 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.309 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.309 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694781+0000", 2024-09-18T04:58:41.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000106452 2024-09-18T04:58:41.310 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.310 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.310 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.310 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.310 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.310 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160718 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 747520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:41.310 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:58:41.310 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.636260045, 2024-09-18T04:58:41.310 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040152520999999997, 2024-09-18T04:58:41.310 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.310 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.310 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.311 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.311 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.311 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160718 2024-09-18T04:58:41.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.311 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:58:41.311 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.311 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.311 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.311 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.311 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654780+0000", 2024-09-18T04:58:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654779+0000", 2024-09-18T04:58:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654782+0000", 2024-09-18T04:58:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.633e-06 2024-09-18T04:58:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.312 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654789+0000", 2024-09-18T04:58:41.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8589999999999999e-06 2024-09-18T04:58:41.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654793+0000", 2024-09-18T04:58:41.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5449999999999997e-06 2024-09-18T04:58:41.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.313 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.313 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.313 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694817+0000", 2024-09-18T04:58:41.313 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040023940000000001 2024-09-18T04:58:41.313 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694861+0000", 2024-09-18T04:58:41.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3961999999999997e-05 2024-09-18T04:58:41.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.314 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.314 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694932+0000", 2024-09-18T04:58:41.314 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1033999999999998e-05 2024-09-18T04:58:41.314 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.314 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.314 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.314 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.314 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.315 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160719 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 768000~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:41.315 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:58:41.315 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.636242447000001, 2024-09-18T04:58:41.315 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040275321000000003, 2024-09-18T04:58:41.315 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.315 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.315 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.315 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.315 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.315 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160719 2024-09-18T04:58:41.315 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.315 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.315 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.315 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:58:41.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:58:41.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.316 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.316 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654797+0000", 2024-09-18T04:58:41.316 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.316 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.316 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654799+0000", 2024-09-18T04:58:41.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.204e-06 2024-09-18T04:58:41.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654806+0000", 2024-09-18T04:58:41.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0500000000000003e-06 2024-09-18T04:58:41.317 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.317 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.317 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.317 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654814+0000", 2024-09-18T04:58:41.317 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.5549999999999997e-06 2024-09-18T04:58:41.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.694967+0000", 2024-09-18T04:58:41.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040153393000000002 2024-09-18T04:58:41.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695014+0000", 2024-09-18T04:58:41.318 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7194999999999998e-05 2024-09-18T04:58:41.318 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.318 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.318 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.318 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695073+0000", 2024-09-18T04:58:41.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8315e-05 2024-09-18T04:58:41.319 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.319 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.319 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.319 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.319 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.319 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160720 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 788480~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:41.319 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:58:41.319 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.636220154, 2024-09-18T04:58:41.319 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040546450999999997, 2024-09-18T04:58:41.319 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.319 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.319 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160720 2024-09-18T04:58:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:58:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654820+0000", 2024-09-18T04:58:41.320 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654819+0000", 2024-09-18T04:58:41.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654822+0000", 2024-09-18T04:58:41.321 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3029999999999998e-06 2024-09-18T04:58:41.321 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.321 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.321 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.321 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654828+0000", 2024-09-18T04:58:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2450000000000003e-06 2024-09-18T04:58:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654833+0000", 2024-09-18T04:58:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6419999999999998e-06 2024-09-18T04:58:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695116+0000", 2024-09-18T04:58:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040283126000000002 2024-09-18T04:58:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.322 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695203+0000", 2024-09-18T04:58:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7693000000000002e-05 2024-09-18T04:58:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695366+0000", 2024-09-18T04:58:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00016283700000000001 2024-09-18T04:58:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.323 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160721 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 808960~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:58:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.636202819000005, 2024-09-18T04:58:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040737433000000003, 2024-09-18T04:58:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160721 2024-09-18T04:58:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.324 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:58:41.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.325 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:58:41.325 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.325 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.325 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.325 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654837+0000", 2024-09-18T04:58:41.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654839+0000", 2024-09-18T04:58:41.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.43e-06 2024-09-18T04:58:41.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.326 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.326 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.326 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654847+0000", 2024-09-18T04:58:41.326 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7300000000000005e-06 2024-09-18T04:58:41.326 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654858+0000", 2024-09-18T04:58:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.099e-05 2024-09-18T04:58:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695391+0000", 2024-09-18T04:58:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040533476999999998 2024-09-18T04:58:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695477+0000", 2024-09-18T04:58:41.327 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5870000000000003e-05 2024-09-18T04:58:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695575+0000", 2024-09-18T04:58:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.7264999999999998e-05 2024-09-18T04:58:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160722 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 829440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:58:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.636177289000003, 2024-09-18T04:58:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040875739000000001, 2024-09-18T04:58:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160722 2024-09-18T04:58:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:58:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.329 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654863+0000", 2024-09-18T04:58:41.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654862+0000", 2024-09-18T04:58:41.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.330 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.330 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.330 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.330 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654883+0000", 2024-09-18T04:58:41.330 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0468000000000001e-05 2024-09-18T04:58:41.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654891+0000", 2024-09-18T04:58:41.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8320000000000006e-06 2024-09-18T04:58:41.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654896+0000", 2024-09-18T04:58:41.331 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.13e-06 2024-09-18T04:58:41.331 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.331 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.331 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.331 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695628+0000", 2024-09-18T04:58:41.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040732011999999998 2024-09-18T04:58:41.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695678+0000", 2024-09-18T04:58:41.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0453e-05 2024-09-18T04:58:41.332 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.332 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.332 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.332 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695738+0000", 2024-09-18T04:58:41.332 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0164999999999997e-05 2024-09-18T04:58:41.333 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.333 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.333 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.333 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.333 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.333 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160723 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 849920~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:41.333 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:58:41.333 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.636085483000002, 2024-09-18T04:58:41.333 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041073576000000001, 2024-09-18T04:58:41.333 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.333 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.333 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.333 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.334 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.334 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160723 2024-09-18T04:58:41.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.334 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:58:41.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.334 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.334 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.334 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:58:41.334 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.334 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654954+0000", 2024-09-18T04:58:41.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654958+0000", 2024-09-18T04:58:41.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7230000000000002e-06 2024-09-18T04:58:41.335 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.335 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.335 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.335 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654968+0000", 2024-09-18T04:58:41.335 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0665999999999999e-05 2024-09-18T04:58:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654974+0000", 2024-09-18T04:58:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9259999999999997e-06 2024-09-18T04:58:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695824+0000", 2024-09-18T04:58:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040850085000000001 2024-09-18T04:58:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.336 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.695933+0000", 2024-09-18T04:58:41.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108738 2024-09-18T04:58:41.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.337 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.337 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696028+0000", 2024-09-18T04:58:41.337 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.4988000000000006e-05 2024-09-18T04:58:41.337 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.337 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.337 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.337 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.337 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.337 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160724 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 870400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:41.337 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:58:41.338 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.636061139999995, 2024-09-18T04:58:41.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041279300999999997, 2024-09-18T04:58:41.338 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.338 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.338 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.338 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.338 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.338 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160724 2024-09-18T04:58:41.338 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.338 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.338 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.338 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.338 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:58:41.338 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654979+0000", 2024-09-18T04:58:41.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.339 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654978+0000", 2024-09-18T04:58:41.339 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.339 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.339 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.339 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654981+0000", 2024-09-18T04:58:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4530000000000001e-06 2024-09-18T04:58:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654987+0000", 2024-09-18T04:58:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.6359999999999997e-06 2024-09-18T04:58:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654995+0000", 2024-09-18T04:58:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9759999999999995e-06 2024-09-18T04:58:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.340 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696069+0000", 2024-09-18T04:58:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041073447999999999 2024-09-18T04:58:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696160+0000", 2024-09-18T04:58:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0835999999999996e-05 2024-09-18T04:58:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696258+0000", 2024-09-18T04:58:41.341 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8338999999999995e-05 2024-09-18T04:58:41.342 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.342 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.342 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.342 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.342 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.342 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160725 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 890880~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:41.342 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:58:41.342 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.636040070999996, 2024-09-18T04:58:41.342 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041552253999999997, 2024-09-18T04:58:41.342 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.342 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.342 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.342 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.342 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160725 2024-09-18T04:58:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:58:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655000+0000", 2024-09-18T04:58:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.343 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.654999+0000", 2024-09-18T04:58:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655002+0000", 2024-09-18T04:58:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.419e-06 2024-09-18T04:58:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655008+0000", 2024-09-18T04:58:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2310000000000001e-06 2024-09-18T04:58:41.344 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655013+0000", 2024-09-18T04:58:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8439999999999997e-06 2024-09-18T04:58:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696353+0000", 2024-09-18T04:58:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041340322999999998 2024-09-18T04:58:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696455+0000", 2024-09-18T04:58:41.345 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000102105 2024-09-18T04:58:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696552+0000", 2024-09-18T04:58:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.6738999999999997e-05 2024-09-18T04:58:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160726 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 911360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:58:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.636022839000006, 2024-09-18T04:58:41.346 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041786440000000001, 2024-09-18T04:58:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160726 2024-09-18T04:58:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:58:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.347 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:58:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655017+0000", 2024-09-18T04:58:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655019+0000", 2024-09-18T04:58:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1100000000000001e-06 2024-09-18T04:58:41.348 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655025+0000", 2024-09-18T04:58:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2190000000000001e-06 2024-09-18T04:58:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655030+0000", 2024-09-18T04:58:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5340000000000001e-06 2024-09-18T04:58:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696607+0000", 2024-09-18T04:58:41.349 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041577309 2024-09-18T04:58:41.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696694+0000", 2024-09-18T04:58:41.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7230000000000003e-05 2024-09-18T04:58:41.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.350 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.350 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.350 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696804+0000", 2024-09-18T04:58:41.350 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0001094 2024-09-18T04:58:41.350 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.350 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.350 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.350 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160727 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 931840~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:58:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.636006101999996, 2024-09-18T04:58:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042000527000000003, 2024-09-18T04:58:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160727 2024-09-18T04:58:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:58:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655034+0000", 2024-09-18T04:58:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655033+0000", 2024-09-18T04:58:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655036+0000", 2024-09-18T04:58:41.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3489999999999999e-06 2024-09-18T04:58:41.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655042+0000", 2024-09-18T04:58:41.353 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1399999999999997e-06 2024-09-18T04:58:41.353 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.353 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.353 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655046+0000", 2024-09-18T04:58:41.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5480000000000003e-06 2024-09-18T04:58:41.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696843+0000", 2024-09-18T04:58:41.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041797103000000002 2024-09-18T04:58:41.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.354 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.354 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.696931+0000", 2024-09-18T04:58:41.354 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7526000000000002e-05 2024-09-18T04:58:41.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697034+0000", 2024-09-18T04:58:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010336999999999999 2024-09-18T04:58:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160728 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 952320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:58:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.635989545000001, 2024-09-18T04:58:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042191293999999997, 2024-09-18T04:58:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.355 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160728 2024-09-18T04:58:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:58:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.356 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:58:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655050+0000", 2024-09-18T04:58:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655052+0000", 2024-09-18T04:58:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.232e-06 2024-09-18T04:58:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.357 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655058+0000", 2024-09-18T04:58:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9869999999999996e-06 2024-09-18T04:58:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655063+0000", 2024-09-18T04:58:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8420000000000004e-06 2024-09-18T04:58:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697074+0000", 2024-09-18T04:58:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042010446 2024-09-18T04:58:41.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697157+0000", 2024-09-18T04:58:41.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3038e-05 2024-09-18T04:58:41.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.359 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.359 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697242+0000", 2024-09-18T04:58:41.359 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5115000000000001e-05 2024-09-18T04:58:41.359 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.359 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.359 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.359 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.359 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.360 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160729 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 972800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:41.360 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:58:41.360 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.635972785999996, 2024-09-18T04:58:41.360 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042367834, 2024-09-18T04:58:41.360 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.360 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.360 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.360 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.360 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.360 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160729 2024-09-18T04:58:41.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.361 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:58:41.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.361 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:58:41.361 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.361 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655067+0000", 2024-09-18T04:58:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655069+0000", 2024-09-18T04:58:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2220000000000001e-06 2024-09-18T04:58:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655075+0000", 2024-09-18T04:58:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2129999999999996e-06 2024-09-18T04:58:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655080+0000", 2024-09-18T04:58:41.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6840000000000004e-06 2024-09-18T04:58:41.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697280+0000", 2024-09-18T04:58:41.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042199819999999999 2024-09-18T04:58:41.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.363 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.363 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.363 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697368+0000", 2024-09-18T04:58:41.363 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.7863999999999998e-05 2024-09-18T04:58:41.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.364 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.364 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697435+0000", 2024-09-18T04:58:41.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7416000000000006e-05 2024-09-18T04:58:41.364 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.364 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.364 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.364 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.364 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160730 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 993280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:41.364 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:58:41.364 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.635956020999998, 2024-09-18T04:58:41.364 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042559582999999998, 2024-09-18T04:58:41.365 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.365 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.365 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.365 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.365 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.365 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160730 2024-09-18T04:58:41.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.365 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.365 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.365 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:58:41.365 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:58:41.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655084+0000", 2024-09-18T04:58:41.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.366 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.366 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.366 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655086+0000", 2024-09-18T04:58:41.366 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1500000000000002e-06 2024-09-18T04:58:41.366 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655092+0000", 2024-09-18T04:58:41.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.896e-06 2024-09-18T04:58:41.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655096+0000", 2024-09-18T04:58:41.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7190000000000001e-06 2024-09-18T04:58:41.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.367 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.367 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.367 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697475+0000", 2024-09-18T04:58:41.367 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042378644 2024-09-18T04:58:41.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697553+0000", 2024-09-18T04:58:41.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.7626999999999994e-05 2024-09-18T04:58:41.368 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.368 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.368 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697643+0000", 2024-09-18T04:58:41.368 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0921000000000006e-05 2024-09-18T04:58:41.368 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.368 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.368 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160731 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 1013760~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:58:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.635923290999997, 2024-09-18T04:58:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042727764000000001, 2024-09-18T04:58:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160731 2024-09-18T04:58:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.369 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:58:41.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655117+0000", 2024-09-18T04:58:41.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.370 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.370 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.370 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655116+0000", 2024-09-18T04:58:41.370 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655119+0000", 2024-09-18T04:58:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.1240000000000001e-06 2024-09-18T04:58:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655131+0000", 2024-09-18T04:58:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2145e-05 2024-09-18T04:58:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.371 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655138+0000", 2024-09-18T04:58:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3369999999999996e-06 2024-09-18T04:58:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697688+0000", 2024-09-18T04:58:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042550604999999998 2024-09-18T04:58:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697781+0000", 2024-09-18T04:58:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.2867000000000004e-05 2024-09-18T04:58:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.372 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697844+0000", 2024-09-18T04:58:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3236000000000002e-05 2024-09-18T04:58:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:160732 3.0 3:041ece2f:::rbd_data.11ac33b7ee50.0000000000000002:30 [read 1034240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e86)", 2024-09-18T04:58:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:58:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 64.635897443000005, 2024-09-18T04:58:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042857977999999998, 2024-09-18T04:58:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.373 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 160732 2024-09-18T04:58:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:58:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.374 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:58:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655142+0000", 2024-09-18T04:58:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655145+0000", 2024-09-18T04:58:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7329999999999999e-06 2024-09-18T04:58:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.375 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655152+0000", 2024-09-18T04:58:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.8249999999999999e-06 2024-09-18T04:58:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.655158+0000", 2024-09-18T04:58:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5010000000000001e-06 2024-09-18T04:58:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697873+0000", 2024-09-18T04:58:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042714457999999997 2024-09-18T04:58:41.376 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.697944+0000", 2024-09-18T04:58:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.1486e-05 2024-09-18T04:58:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:36.698000+0000", 2024-09-18T04:58:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6332000000000002e-05 2024-09-18T04:58:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.377 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.378 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:167084 3.18 3:1f627e86:::rbd_data.11ac138032e7.0000000000000010:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:41.378 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:58:19.165327+0000", 2024-09-18T04:58:41.378 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.125712677999999, 2024-09-18T04:58:41.378 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.90135632899999996, 2024-09-18T04:58:41.378 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.378 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:41.378 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.378 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.378 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.378 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 167084 2024-09-18T04:58:41.378 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.379 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.165327+0000", 2024-09-18T04:58:41.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.379 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.165327+0000", 2024-09-18T04:58:41.379 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.379 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.165326+0000", 2024-09-18T04:58:41.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:58:41.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.165332+0000", 2024-09-18T04:58:41.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3700000000000003e-06 2024-09-18T04:58:41.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.380 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.380 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.165343+0000", 2024-09-18T04:58:41.380 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1263999999999999e-05 2024-09-18T04:58:41.380 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.380 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.165348+0000", 2024-09-18T04:58:41.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9729999999999997e-06 2024-09-18T04:58:41.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.165384+0000", 2024-09-18T04:58:41.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.5828999999999999e-05 2024-09-18T04:58:41.381 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.381 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.381 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-18T04:58:41.381 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.165389+0000", 2024-09-18T04:58:41.381 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2750000000000003e-06 2024-09-18T04:58:41.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.062869+0000", 2024-09-18T04:58:41.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.89747956100000004 2024-09-18T04:58:41.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.063068+0000", 2024-09-18T04:58:41.382 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000199106 2024-09-18T04:58:41.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.382 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:58:41.382 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.063189+0000", 2024-09-18T04:58:41.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012115100000000001 2024-09-18T04:58:41.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:58:41.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.065943+0000", 2024-09-18T04:58:41.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0027540239999999999 2024-09-18T04:58:41.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.383 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:41.383 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.066599+0000", 2024-09-18T04:58:41.383 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00065621099999999997 2024-09-18T04:58:41.383 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.383 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:58:41.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.066644+0000", 2024-09-18T04:58:41.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.439e-05 2024-09-18T04:58:41.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.384 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.384 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.066684+0000", 2024-09-18T04:58:41.384 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9971000000000001e-05 2024-09-18T04:58:41.384 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.384 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.384 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.384 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.385 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:167107 3.18 3:1fe9e723:::rbd_data.11ac138032e7.0000000000000056:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:41.385 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:58:19.194623+0000", 2024-09-18T04:58:41.385 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 22.096417325000001, 2024-09-18T04:58:41.385 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.87280312599999998, 2024-09-18T04:58:41.385 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.385 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:41.385 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.385 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.385 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.385 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 167107 2024-09-18T04:58:41.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.386 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.194623+0000", 2024-09-18T04:58:41.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.386 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.386 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.194623+0000", 2024-09-18T04:58:41.386 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.194622+0000", 2024-09-18T04:58:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.194626+0000", 2024-09-18T04:58:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8759999999999998e-06 2024-09-18T04:58:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.194634+0000", 2024-09-18T04:58:41.387 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1419999999999993e-06 2024-09-18T04:58:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.194640+0000", 2024-09-18T04:58:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3060000000000003e-06 2024-09-18T04:58:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.194683+0000", 2024-09-18T04:58:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2904999999999997e-05 2024-09-18T04:58:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for peered", 2024-09-18T04:58:41.388 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:19.194689+0000", 2024-09-18T04:58:41.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6250000000000004e-06 2024-09-18T04:58:41.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.064039+0000", 2024-09-18T04:58:41.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.86935030800000002 2024-09-18T04:58:41.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.389 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.064166+0000", 2024-09-18T04:58:41.389 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00012683200000000001 2024-09-18T04:58:41.389 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.389 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "waiting for subops from 0", 2024-09-18T04:58:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.064287+0000", 2024-09-18T04:58:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000120658 2024-09-18T04:58:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "op_commit", 2024-09-18T04:58:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.066023+0000", 2024-09-18T04:58:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0017358479999999999 2024-09-18T04:58:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.067339+0000", 2024-09-18T04:58:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.0013168559999999999 2024-09-18T04:58:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.390 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "commit_sent", 2024-09-18T04:58:41.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.067388+0000", 2024-09-18T04:58:41.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8764999999999998e-05 2024-09-18T04:58:41.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.391 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.391 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.391 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:58:20.067426+0000", 2024-09-18T04:58:41.391 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7598999999999997e-05 2024-09-18T04:58:41.391 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.391 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.391 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.391 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.391 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.391 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:41.392 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:58:41.393 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:58:41.394 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:58:41.394 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:58:41.395 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:58:41.395 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:41.389+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:41.494 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:41.491+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:41.541 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:41.541 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:58:41.542 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:41.550 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:58:41.634 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:41.631+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:41.696 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:41.693+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:41.697 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:41.698 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:41.698 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:58:41.698 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:58:41.698 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:41.706 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:58:41.797 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:41.794+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:41.855 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:41.855 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:58:41.855 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:58:41.855 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:58:41.855 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.855 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11154 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1966080~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:41.855 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:58:41.855 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 125.9622639, 2024-09-18T04:58:41.856 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040591919999999997, 2024-09-18T04:58:41.856 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.856 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.856 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.856 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.856 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.856 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11154 2024-09-18T04:58:41.856 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.856 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.856 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.856 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:58:41.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.857 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.857 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889401+0000", 2024-09-18T04:58:41.857 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.857 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.857 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889400+0000", 2024-09-18T04:58:41.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889403+0000", 2024-09-18T04:58:41.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3999999999999999e-06 2024-09-18T04:58:41.858 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.858 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.858 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.858 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889409+0000", 2024-09-18T04:58:41.858 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3380000000000001e-06 2024-09-18T04:58:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889414+0000", 2024-09-18T04:58:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5789999999999997e-06 2024-09-18T04:58:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929811+0000", 2024-09-18T04:58:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040397486000000003 2024-09-18T04:58:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.859 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929898+0000", 2024-09-18T04:58:41.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.6370000000000001e-05 2024-09-18T04:58:41.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.860 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.860 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.929993+0000", 2024-09-18T04:58:41.860 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5136000000000005e-05 2024-09-18T04:58:41.860 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.860 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.860 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.860 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.860 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.860 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11155 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2015232~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:41.860 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:58:41.861 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 125.962246722, 2024-09-18T04:58:41.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040769156000000001, 2024-09-18T04:58:41.861 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.861 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.861 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.861 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.861 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.861 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11155 2024-09-18T04:58:41.861 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.861 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.861 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.861 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.861 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:58:41.861 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:58:41.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.862 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.862 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.862 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.862 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889418+0000", 2024-09-18T04:58:41.862 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889420+0000", 2024-09-18T04:58:41.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.181e-06 2024-09-18T04:58:41.863 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.863 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.863 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.863 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889426+0000", 2024-09-18T04:58:41.863 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8599999999999998e-06 2024-09-18T04:58:41.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889430+0000", 2024-09-18T04:58:41.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4399999999999998e-06 2024-09-18T04:58:41.864 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.864 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.864 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.864 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930020+0000", 2024-09-18T04:58:41.864 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040589698 2024-09-18T04:58:41.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.865 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930105+0000", 2024-09-18T04:58:41.865 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4812e-05 2024-09-18T04:58:41.865 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.865 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.865 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.866 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930187+0000", 2024-09-18T04:58:41.866 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2540999999999995e-05 2024-09-18T04:58:41.866 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.866 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.866 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.866 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.866 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11156 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2064384~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:58:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 125.962206428, 2024-09-18T04:58:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040934419, 2024-09-18T04:58:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11156 2024-09-18T04:58:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.867 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:58:41.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889458+0000", 2024-09-18T04:58:41.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.868 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.868 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889457+0000", 2024-09-18T04:58:41.868 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:58:41.868 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.868 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889462+0000", 2024-09-18T04:58:41.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8350000000000003e-06 2024-09-18T04:58:41.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889481+0000", 2024-09-18T04:58:41.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.8845999999999999e-05 2024-09-18T04:58:41.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.869 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.869 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.869 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.889495+0000", 2024-09-18T04:58:41.869 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.4229e-05 2024-09-18T04:58:41.869 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930216+0000", 2024-09-18T04:58:41.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040721081999999999 2024-09-18T04:58:41.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930299+0000", 2024-09-18T04:58:41.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2995000000000001e-05 2024-09-18T04:58:41.870 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.870 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.870 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.870 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.930393+0000", 2024-09-18T04:58:41.870 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3185999999999999e-05 2024-09-18T04:58:41.871 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.871 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.871 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.871 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.871 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.871 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11404 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1609728~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:41.871 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:58:41.871 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 125.95540226599999, 2024-09-18T04:58:41.871 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040669012999999997, 2024-09-18T04:58:41.871 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.871 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.871 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.872 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.872 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.872 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11404 2024-09-18T04:58:41.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.872 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:58:41.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.872 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.872 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.872 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.872 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:58:41.872 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896262+0000", 2024-09-18T04:58:41.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:58:41.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.873 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896263+0000", 2024-09-18T04:58:41.873 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2729999999999999e-06 2024-09-18T04:58:41.873 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.873 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.873 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896266+0000", 2024-09-18T04:58:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8200000000000001e-06 2024-09-18T04:58:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896269+0000", 2024-09-18T04:58:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3199999999999998e-06 2024-09-18T04:58:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936538+0000", 2024-09-18T04:58:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040269394999999999 2024-09-18T04:58:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.874 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936657+0000", 2024-09-18T04:58:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000119498 2024-09-18T04:58:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936931+0000", 2024-09-18T04:58:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00027391900000000002 2024-09-18T04:58:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11405 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1658880~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:41.875 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:58:41.876 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 125.955394008, 2024-09-18T04:58:41.876 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040891034999999999, 2024-09-18T04:58:41.876 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.876 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.876 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.876 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.876 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.876 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11405 2024-09-18T04:58:41.876 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.876 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.876 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.876 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:58:41.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896271+0000", 2024-09-18T04:58:41.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.877 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.877 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.877 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896270+0000", 2024-09-18T04:58:41.877 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:58:41.877 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896272+0000", 2024-09-18T04:58:41.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.124e-06 2024-09-18T04:58:41.878 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.878 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.878 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.878 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896274+0000", 2024-09-18T04:58:41.878 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8660000000000002e-06 2024-09-18T04:58:41.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896285+0000", 2024-09-18T04:58:41.879 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.0101e-05 2024-09-18T04:58:41.879 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.879 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.879 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.879 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.936986+0000", 2024-09-18T04:58:41.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040701213999999999 2024-09-18T04:58:41.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937070+0000", 2024-09-18T04:58:41.880 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4211999999999999e-05 2024-09-18T04:58:41.880 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.880 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.880 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.880 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937162+0000", 2024-09-18T04:58:41.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1728e-05 2024-09-18T04:58:41.881 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.881 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.881 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.881 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.881 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.881 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11406 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1708032~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:41.881 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:58:41.881 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 125.95537783899999, 2024-09-18T04:58:41.881 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041085251000000003, 2024-09-18T04:58:41.881 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.881 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.882 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.882 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.882 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.882 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11406 2024-09-18T04:58:41.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.882 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.882 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:58:41.882 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.882 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.882 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.882 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:58:41.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896287+0000", 2024-09-18T04:58:41.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967296 2024-09-18T04:58:41.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.883 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.883 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896288+0000", 2024-09-18T04:58:41.883 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2440000000000001e-06 2024-09-18T04:58:41.883 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.883 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896291+0000", 2024-09-18T04:58:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.8289999999999999e-06 2024-09-18T04:58:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896293+0000", 2024-09-18T04:58:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5399999999999998e-06 2024-09-18T04:58:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937201+0000", 2024-09-18T04:58:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040907418000000001 2024-09-18T04:58:41.884 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937283+0000", 2024-09-18T04:58:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2460999999999995e-05 2024-09-18T04:58:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937372+0000", 2024-09-18T04:58:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8987999999999996e-05 2024-09-18T04:58:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.885 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.886 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11407 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1757184~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:41.886 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:58:41.886 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 125.95536801900001, 2024-09-18T04:58:41.886 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041438580000000003, 2024-09-18T04:58:41.886 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.886 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.886 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.886 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.886 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.886 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11407 2024-09-18T04:58:41.886 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.886 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.886 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.886 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:58:41.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896297+0000", 2024-09-18T04:58:41.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.887 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.887 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896296+0000", 2024-09-18T04:58:41.887 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.887 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.887 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896298+0000", 2024-09-18T04:58:41.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:58:41.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896304+0000", 2024-09-18T04:58:41.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6570000000000002e-06 2024-09-18T04:58:41.888 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.888 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.888 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.888 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896308+0000", 2024-09-18T04:58:41.888 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0940000000000001e-06 2024-09-18T04:58:41.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937540+0000", 2024-09-18T04:58:41.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041231469 2024-09-18T04:58:41.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.889 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.889 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937625+0000", 2024-09-18T04:58:41.889 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5767999999999996e-05 2024-09-18T04:58:41.889 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.889 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.890 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.890 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937735+0000", 2024-09-18T04:58:41.890 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010974 2024-09-18T04:58:41.890 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.890 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.890 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.890 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.890 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.890 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11408 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1806336~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:41.890 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:58:41.890 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 125.95535237999999, 2024-09-18T04:58:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041653055000000001, 2024-09-18T04:58:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11408 2024-09-18T04:58:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:58:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.891 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:58:41.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.892 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.892 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.892 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.892 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896312+0000", 2024-09-18T04:58:41.892 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896314+0000", 2024-09-18T04:58:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0140000000000001e-06 2024-09-18T04:58:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896320+0000", 2024-09-18T04:58:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6980000000000003e-06 2024-09-18T04:58:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.893 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896324+0000", 2024-09-18T04:58:41.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4410000000000003e-06 2024-09-18T04:58:41.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.894 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937769+0000", 2024-09-18T04:58:41.894 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041445243 2024-09-18T04:58:41.894 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.894 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.894 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937861+0000", 2024-09-18T04:58:41.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1471000000000006e-05 2024-09-18T04:58:41.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.895 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.895 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937965+0000", 2024-09-18T04:58:41.895 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104516 2024-09-18T04:58:41.895 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.895 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.895 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.895 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.895 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.896 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11409 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1871872~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:41.896 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:58:41.896 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 125.955336324, 2024-09-18T04:58:41.896 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041840467999999999, 2024-09-18T04:58:41.896 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.896 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.896 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.896 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.896 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.896 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11409 2024-09-18T04:58:41.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.897 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:58:41.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.897 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.897 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.897 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:58:41.897 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.897 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896328+0000", 2024-09-18T04:58:41.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.898 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.898 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.898 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896331+0000", 2024-09-18T04:58:41.898 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5950000000000001e-06 2024-09-18T04:58:41.898 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896336+0000", 2024-09-18T04:58:41.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7259999999999999e-06 2024-09-18T04:58:41.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.899 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.899 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.899 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896341+0000", 2024-09-18T04:58:41.899 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5619999999999997e-06 2024-09-18T04:58:41.899 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.937995+0000", 2024-09-18T04:58:41.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041654060999999999 2024-09-18T04:58:41.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.900 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938078+0000", 2024-09-18T04:58:41.900 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2869999999999998e-05 2024-09-18T04:58:41.900 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.900 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.900 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.901 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938169+0000", 2024-09-18T04:58:41.901 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1040999999999998e-05 2024-09-18T04:58:41.901 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.901 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.901 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.901 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.901 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.901 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11410 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1921024~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:41.901 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:58:41.901 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 125.955319841, 2024-09-18T04:58:41.902 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042060342000000001, 2024-09-18T04:58:41.902 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.902 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.902 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.902 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.902 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.902 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11410 2024-09-18T04:58:41.902 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.902 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.902 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.902 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.902 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:58:41.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896345+0000", 2024-09-18T04:58:41.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.903 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.903 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896344+0000", 2024-09-18T04:58:41.903 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.903 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.903 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896360+0000", 2024-09-18T04:58:41.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.5068e-05 2024-09-18T04:58:41.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896366+0000", 2024-09-18T04:58:41.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5119999999999997e-06 2024-09-18T04:58:41.904 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.904 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.904 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.904 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896371+0000", 2024-09-18T04:58:41.904 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:58:41.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938198+0000", 2024-09-18T04:58:41.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041826579000000003 2024-09-18T04:58:41.905 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.905 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.905 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.905 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938297+0000", 2024-09-18T04:58:41.905 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.9067000000000005e-05 2024-09-18T04:58:41.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.906 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.906 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938405+0000", 2024-09-18T04:58:41.906 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108492 2024-09-18T04:58:41.906 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.906 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.906 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.906 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.906 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.907 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11411 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 1970176~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:41.907 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:58:41.907 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 125.955288999, 2024-09-18T04:58:41.907 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042259191000000002, 2024-09-18T04:58:41.907 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.907 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.907 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.907 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.907 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.907 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11411 2024-09-18T04:58:41.907 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.907 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.907 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:58:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896376+0000", 2024-09-18T04:58:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896375+0000", 2024-09-18T04:58:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.908 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896377+0000", 2024-09-18T04:58:41.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2699999999999999e-06 2024-09-18T04:58:41.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896383+0000", 2024-09-18T04:58:41.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6740000000000002e-06 2024-09-18T04:58:41.909 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.909 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.909 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.909 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896388+0000", 2024-09-18T04:58:41.909 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:58:41.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938435+0000", 2024-09-18T04:58:41.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042047169000000002 2024-09-18T04:58:41.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938534+0000", 2024-09-18T04:58:41.910 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8939999999999998e-05 2024-09-18T04:58:41.910 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.910 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.910 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.910 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.938635+0000", 2024-09-18T04:58:41.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000101208 2024-09-18T04:58:41.911 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.911 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.911 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.911 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.911 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.911 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11412 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2019328~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:41.911 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:58:41.911 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 125.955273017, 2024-09-18T04:58:41.911 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042828079999999998, 2024-09-18T04:58:41.911 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.911 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.911 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.911 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11412 2024-09-18T04:58:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:58:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896392+0000", 2024-09-18T04:58:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.912 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896391+0000", 2024-09-18T04:58:41.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896393+0000", 2024-09-18T04:58:41.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0810000000000001e-06 2024-09-18T04:58:41.913 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.913 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.913 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.913 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896399+0000", 2024-09-18T04:58:41.913 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7230000000000001e-06 2024-09-18T04:58:41.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896403+0000", 2024-09-18T04:58:41.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.3470000000000001e-06 2024-09-18T04:58:41.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939043+0000", 2024-09-18T04:58:41.914 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042639829999999997 2024-09-18T04:58:41.914 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.914 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.914 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.914 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939124+0000", 2024-09-18T04:58:41.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.0809000000000005e-05 2024-09-18T04:58:41.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.915 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.915 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939220+0000", 2024-09-18T04:58:41.915 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.5659e-05 2024-09-18T04:58:41.915 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.915 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.915 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.915 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.915 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.916 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:11413 3.b 3:d3ef3a8d:::rbd_data.11acf292c51a.0000000000000009:d [read 2068480~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:41.916 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:58:41.916 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 125.95525732199999, 2024-09-18T04:58:41.916 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043042108000000003, 2024-09-18T04:58:41.916 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.916 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.916 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.916 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.916 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.916 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 11413 2024-09-18T04:58:41.916 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.917 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:58:41.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.917 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:58:41.917 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.917 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.917 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.917 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896407+0000", 2024-09-18T04:58:41.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896409+0000", 2024-09-18T04:58:41.918 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.165e-06 2024-09-18T04:58:41.918 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.918 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.918 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.918 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896415+0000", 2024-09-18T04:58:41.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6149999999999996e-06 2024-09-18T04:58:41.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.896419+0000", 2024-09-18T04:58:41.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2649999999999998e-06 2024-09-18T04:58:41.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.919 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.919 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.919 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939262+0000", 2024-09-18T04:58:41.919 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.042843444000000001 2024-09-18T04:58:41.919 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939351+0000", 2024-09-18T04:58:41.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8944999999999996e-05 2024-09-18T04:58:41.920 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.920 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.920 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.920 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.939449+0000", 2024-09-18T04:58:41.920 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.8029000000000006e-05 2024-09-18T04:58:41.920 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.920 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.921 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.921 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.921 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.921 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145409 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1761280~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:41.921 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:58:41.921 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 68.323837674000004, 2024-09-18T04:58:41.921 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040684845999999997, 2024-09-18T04:58:41.921 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.921 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.921 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.921 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.921 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.922 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145409 2024-09-18T04:58:41.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.922 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:58:41.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.922 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.922 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.922 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.922 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:58:41.922 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527827+0000", 2024-09-18T04:58:41.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.923 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.923 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.923 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527829+0000", 2024-09-18T04:58:41.923 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.441e-06 2024-09-18T04:58:41.923 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527835+0000", 2024-09-18T04:58:41.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1650000000000003e-06 2024-09-18T04:58:41.924 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.924 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.924 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.924 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527840+0000", 2024-09-18T04:58:41.924 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8250000000000004e-06 2024-09-18T04:58:41.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568404+0000", 2024-09-18T04:58:41.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040563872000000001 2024-09-18T04:58:41.925 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.925 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.925 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.925 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568457+0000", 2024-09-18T04:58:41.925 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2713999999999997e-05 2024-09-18T04:58:41.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.926 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.926 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568512+0000", 2024-09-18T04:58:41.926 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5148000000000001e-05 2024-09-18T04:58:41.926 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.926 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.926 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.926 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.926 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.927 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145411 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1810432~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:41.927 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:58:41.927 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 68.323819813, 2024-09-18T04:58:41.927 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040792108000000001, 2024-09-18T04:58:41.927 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.927 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.927 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.927 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.927 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.928 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145411 2024-09-18T04:58:41.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.928 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:58:41.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.928 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.928 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.928 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527845+0000", 2024-09-18T04:58:41.928 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.928 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527844+0000", 2024-09-18T04:58:41.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.929 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527847+0000", 2024-09-18T04:58:41.929 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0260000000000002e-06 2024-09-18T04:58:41.929 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.929 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.929 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527852+0000", 2024-09-18T04:58:41.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9889999999999997e-06 2024-09-18T04:58:41.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.930 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527857+0000", 2024-09-18T04:58:41.930 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6410000000000001e-06 2024-09-18T04:58:41.930 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.930 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.930 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568537+0000", 2024-09-18T04:58:41.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040679743999999997 2024-09-18T04:58:41.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568583+0000", 2024-09-18T04:58:41.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6286999999999999e-05 2024-09-18T04:58:41.931 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.931 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.931 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.931 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568637+0000", 2024-09-18T04:58:41.931 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.376e-05 2024-09-18T04:58:41.931 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.931 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.932 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.932 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.932 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.932 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145413 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1859584~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:41.932 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:58:41.932 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 68.323803432999995, 2024-09-18T04:58:41.932 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040919516000000003, 2024-09-18T04:58:41.932 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.932 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.933 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.933 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.933 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.933 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145413 2024-09-18T04:58:41.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.933 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.933 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:58:41.933 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.933 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.933 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.933 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:58:41.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.934 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527861+0000", 2024-09-18T04:58:41.934 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.934 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.934 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.934 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527863+0000", 2024-09-18T04:58:41.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2630000000000002e-06 2024-09-18T04:58:41.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527869+0000", 2024-09-18T04:58:41.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.9009999999999999e-06 2024-09-18T04:58:41.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.935 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.935 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.935 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527874+0000", 2024-09-18T04:58:41.935 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0590000000000002e-06 2024-09-18T04:58:41.935 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568663+0000", 2024-09-18T04:58:41.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040788805999999997 2024-09-18T04:58:41.936 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.936 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.936 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.936 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568716+0000", 2024-09-18T04:58:41.936 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3501999999999997e-05 2024-09-18T04:58:41.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.937 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.937 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568781+0000", 2024-09-18T04:58:41.937 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4338000000000006e-05 2024-09-18T04:58:41.937 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.937 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.937 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.937 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.937 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.937 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145414 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1908736~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:41.937 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:58:41.938 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 68.323786475000006, 2024-09-18T04:58:41.938 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041030234999999998, 2024-09-18T04:58:41.938 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.938 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.938 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.938 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.938 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.938 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145414 2024-09-18T04:58:41.938 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.938 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:58:41.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.939 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.939 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.939 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:58:41.939 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.939 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527878+0000", 2024-09-18T04:58:41.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.940 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.940 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527880+0000", 2024-09-18T04:58:41.940 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4770000000000002e-06 2024-09-18T04:58:41.940 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.940 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527886+0000", 2024-09-18T04:58:41.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5759999999999996e-06 2024-09-18T04:58:41.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.941 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.941 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527890+0000", 2024-09-18T04:58:41.941 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8180000000000003e-06 2024-09-18T04:58:41.941 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.941 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568807+0000", 2024-09-18T04:58:41.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.040916984000000003 2024-09-18T04:58:41.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.942 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.942 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568854+0000", 2024-09-18T04:58:41.942 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6740999999999999e-05 2024-09-18T04:58:41.942 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.942 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.943 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.943 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568908+0000", 2024-09-18T04:58:41.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4228000000000003e-05 2024-09-18T04:58:41.943 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.943 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.943 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.943 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.943 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.943 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145416 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 1957888~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:41.943 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:58:41.943 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 68.323769983000005, 2024-09-18T04:58:41.943 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041147364999999998, 2024-09-18T04:58:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145416 2024-09-18T04:58:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:58:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.944 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527895+0000", 2024-09-18T04:58:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527894+0000", 2024-09-18T04:58:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527896+0000", 2024-09-18T04:58:41.945 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1789999999999998e-06 2024-09-18T04:58:41.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527908+0000", 2024-09-18T04:58:41.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1780000000000001e-05 2024-09-18T04:58:41.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.946 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527914+0000", 2024-09-18T04:58:41.946 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2029999999999997e-06 2024-09-18T04:58:41.946 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.946 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.946 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568935+0000", 2024-09-18T04:58:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041020273000000003 2024-09-18T04:58:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.568988+0000", 2024-09-18T04:58:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2911000000000002e-05 2024-09-18T04:58:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569042+0000", 2024-09-18T04:58:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.4382000000000002e-05 2024-09-18T04:58:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.947 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.948 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.948 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.948 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145418 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 2007040~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:41.948 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:58:41.948 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 68.323745586000001, 2024-09-18T04:58:41.948 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041247862000000003, 2024-09-18T04:58:41.948 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.948 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.948 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.948 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.948 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.948 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145418 2024-09-18T04:58:41.948 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.949 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:58:41.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.949 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:58:41.949 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.949 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.949 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.949 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527919+0000", 2024-09-18T04:58:41.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527921+0000", 2024-09-18T04:58:41.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.667e-06 2024-09-18T04:58:41.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.950 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.950 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527929+0000", 2024-09-18T04:58:41.950 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.542e-06 2024-09-18T04:58:41.950 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.950 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527934+0000", 2024-09-18T04:58:41.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1440000000000002e-06 2024-09-18T04:58:41.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569068+0000", 2024-09-18T04:58:41.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041134190000000001 2024-09-18T04:58:41.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.951 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.951 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.951 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569114+0000", 2024-09-18T04:58:41.951 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6261999999999999e-05 2024-09-18T04:58:41.951 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.952 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.952 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569167+0000", 2024-09-18T04:58:41.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.2506999999999997e-05 2024-09-18T04:58:41.952 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.952 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.952 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.952 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.952 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.952 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:145420 3.1e 3:7cc8e267:::rbd_data.11ac6967455f.0000000000000003:2d [read 2056192~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:41.952 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:58:41.952 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 68.323726511999993, 2024-09-18T04:58:41.952 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041351472, 2024-09-18T04:58:41.952 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:41.953 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:41.953 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:41.953 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:41.953 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:41.953 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 145420 2024-09-18T04:58:41.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.953 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:41.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.953 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:41.953 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:58:41.953 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.953 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.953 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:41.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:58:41.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:41.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:41.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527938+0000", 2024-09-18T04:58:41.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:41.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.954 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.954 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:41.954 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527940+0000", 2024-09-18T04:58:41.954 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.6570000000000001e-06 2024-09-18T04:58:41.954 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:41.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527948+0000", 2024-09-18T04:58:41.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8350000000000004e-06 2024-09-18T04:58:41.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:41.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.527953+0000", 2024-09-18T04:58:41.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0690000000000001e-06 2024-09-18T04:58:41.955 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.955 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.955 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:41.955 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569182+0000", 2024-09-18T04:58:41.955 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.041228734000000003 2024-09-18T04:58:41.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:41.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569234+0000", 2024-09-18T04:58:41.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.1663999999999999e-05 2024-09-18T04:58:41.956 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:41.956 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:41.956 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:41.956 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:57:33.569290+0000", 2024-09-18T04:58:41.956 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.5937000000000003e-05 2024-09-18T04:58:41.956 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.956 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.956 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.956 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:41.957 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:41.957 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:41.958 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:58:41.959 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:41.955+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:42.059 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:42.056+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:42.069 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:42.070 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:42.070 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:58:42.070 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:42.080 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:58:42.163 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:42.161+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:42.228 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:42.229 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:42.229 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:58:42.229 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:58:42.229 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:42.238 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:58:42.261 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:42.258+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:42.362 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:42.359+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:42.382 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:42.382 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:58:42.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:58:42.383 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:42.383 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.383 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72135 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 394240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:42.383 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:58:42.383 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 105.42518437299999, 2024-09-18T04:58:42.383 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048559739999999997, 2024-09-18T04:58:42.383 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.383 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.383 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.383 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.383 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.383 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72135 2024-09-18T04:58:42.383 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.384 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:58:42.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.384 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:58:42.384 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.384 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.384 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.384 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:58:42.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:42.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954005+0000", 2024-09-18T04:58:42.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5629999999999999e-06 2024-09-18T04:58:42.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.385 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.385 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.385 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954008+0000", 2024-09-18T04:58:42.385 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-18T04:58:42.385 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954012+0000", 2024-09-18T04:58:42.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4790000000000001e-06 2024-09-18T04:58:42.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002418+0000", 2024-09-18T04:58:42.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048406049 2024-09-18T04:58:42.386 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.386 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.386 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.386 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002460+0000", 2024-09-18T04:58:42.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2188999999999999e-05 2024-09-18T04:58:42.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.387 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.387 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002564+0000", 2024-09-18T04:58:42.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010377 2024-09-18T04:58:42.387 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.387 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.387 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.387 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72136 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 409600~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:42.387 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:58:42.387 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 105.425079426, 2024-09-18T04:58:42.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048670191000000002, 2024-09-18T04:58:42.388 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.388 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.388 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.388 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.388 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.388 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72136 2024-09-18T04:58:42.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.388 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:58:42.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:58:42.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954019+0000", 2024-09-18T04:58:42.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999099 2024-09-18T04:58:42.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954112+0000", 2024-09-18T04:58:42.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3273e-05 2024-09-18T04:58:42.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954121+0000", 2024-09-18T04:58:42.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0289999999999994e-06 2024-09-18T04:58:42.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954128+0000", 2024-09-18T04:58:42.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6189999999999996e-06 2024-09-18T04:58:42.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002604+0000", 2024-09-18T04:58:42.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048476048000000001 2024-09-18T04:58:42.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002682+0000", 2024-09-18T04:58:42.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7955999999999997e-05 2024-09-18T04:58:42.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002779+0000", 2024-09-18T04:58:42.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7372000000000002e-05 2024-09-18T04:58:42.392 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.392 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.392 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.392 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72138 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 440320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:42.392 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:58:42.392 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 105.424645574, 2024-09-18T04:58:42.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048549430999999997, 2024-09-18T04:58:42.393 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.393 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.393 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.393 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.393 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.393 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72138 2024-09-18T04:58:42.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.393 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:58:42.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:58:42.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954542+0000", 2024-09-18T04:58:42.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:42.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954545+0000", 2024-09-18T04:58:42.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4360000000000001e-06 2024-09-18T04:58:42.397 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.397 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.397 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.397 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954551+0000", 2024-09-18T04:58:42.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4949999999999996e-06 2024-09-18T04:58:42.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954557+0000", 2024-09-18T04:58:42.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5099999999999998e-06 2024-09-18T04:58:42.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002993+0000", 2024-09-18T04:58:42.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048436358999999998 2024-09-18T04:58:42.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003039+0000", 2024-09-18T04:58:42.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5884000000000003e-05 2024-09-18T04:58:42.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003092+0000", 2024-09-18T04:58:42.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3208000000000003e-05 2024-09-18T04:58:42.401 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.401 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.401 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.402 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.402 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72139 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 455680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:42.402 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:58:42.402 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 105.424626766, 2024-09-18T04:58:42.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048718773, 2024-09-18T04:58:42.402 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.402 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.402 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.402 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.402 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.402 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72139 2024-09-18T04:58:42.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.402 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:58:42.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:58:42.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954561+0000", 2024-09-18T04:58:42.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:42.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954564+0000", 2024-09-18T04:58:42.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4099999999999998e-06 2024-09-18T04:58:42.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954570+0000", 2024-09-18T04:58:42.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.178e-06 2024-09-18T04:58:42.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954575+0000", 2024-09-18T04:58:42.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0560000000000004e-06 2024-09-18T04:58:42.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003114+0000", 2024-09-18T04:58:42.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048539212999999998 2024-09-18T04:58:42.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003176+0000", 2024-09-18T04:58:42.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2154000000000002e-05 2024-09-18T04:58:42.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003280+0000", 2024-09-18T04:58:42.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010421600000000001 2024-09-18T04:58:42.406 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.406 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.406 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.406 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72140 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 471040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:42.407 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:58:42.407 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 105.424370825, 2024-09-18T04:58:42.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048670819999999997, 2024-09-18T04:58:42.407 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.407 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.407 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.407 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.407 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.407 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72140 2024-09-18T04:58:42.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.407 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:58:42.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:58:42.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954579+0000", 2024-09-18T04:58:42.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997611 2024-09-18T04:58:42.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954821+0000", 2024-09-18T04:58:42.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024241000000000001 2024-09-18T04:58:42.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954831+0000", 2024-09-18T04:58:42.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8770000000000005e-06 2024-09-18T04:58:42.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954838+0000", 2024-09-18T04:58:42.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0690000000000004e-06 2024-09-18T04:58:42.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003317+0000", 2024-09-18T04:58:42.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048478796999999997 2024-09-18T04:58:42.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003392+0000", 2024-09-18T04:58:42.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4411000000000003e-05 2024-09-18T04:58:42.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003488+0000", 2024-09-18T04:58:42.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6927000000000002e-05 2024-09-18T04:58:42.411 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.411 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.411 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.411 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72141 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 486400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:42.411 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:58:42.411 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 105.424343342, 2024-09-18T04:58:42.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048808114, 2024-09-18T04:58:42.412 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.412 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.412 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.412 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.412 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.412 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72141 2024-09-18T04:58:42.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.412 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:58:42.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:58:42.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:58:42.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:42.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954847+0000", 2024-09-18T04:58:42.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3800000000000001e-06 2024-09-18T04:58:42.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954854+0000", 2024-09-18T04:58:42.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6080000000000001e-06 2024-09-18T04:58:42.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954859+0000", 2024-09-18T04:58:42.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:58:42.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003526+0000", 2024-09-18T04:58:42.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048666433000000002 2024-09-18T04:58:42.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003599+0000", 2024-09-18T04:58:42.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2854000000000004e-05 2024-09-18T04:58:42.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.415 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003653+0000", 2024-09-18T04:58:42.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4511e-05 2024-09-18T04:58:42.416 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.416 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.416 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.416 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72142 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 501760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:42.416 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:58:42.416 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 105.42430453999999, 2024-09-18T04:58:42.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048885500999999998, 2024-09-18T04:58:42.416 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.416 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.417 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.417 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.417 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.417 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72142 2024-09-18T04:58:42.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.417 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:58:42.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:58:42.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954864+0000", 2024-09-18T04:58:42.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999804 2024-09-18T04:58:42.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954886+0000", 2024-09-18T04:58:42.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2042999999999999e-05 2024-09-18T04:58:42.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954893+0000", 2024-09-18T04:58:42.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.255e-06 2024-09-18T04:58:42.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954899+0000", 2024-09-18T04:58:42.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8780000000000003e-06 2024-09-18T04:58:42.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003678+0000", 2024-09-18T04:58:42.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048778834 2024-09-18T04:58:42.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003717+0000", 2024-09-18T04:58:42.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8860999999999999e-05 2024-09-18T04:58:42.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003769+0000", 2024-09-18T04:58:42.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2389000000000003e-05 2024-09-18T04:58:42.420 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.420 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.420 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.421 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72143 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 517120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:42.421 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:58:42.421 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 105.424178914, 2024-09-18T04:58:42.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048951738000000002, 2024-09-18T04:58:42.421 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.421 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.421 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.421 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.421 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.421 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72143 2024-09-18T04:58:42.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.421 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:58:42.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:58:42.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.422 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.422 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954904+0000", 2024-09-18T04:58:42.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998946 2024-09-18T04:58:42.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955013+0000", 2024-09-18T04:58:42.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108977 2024-09-18T04:58:42.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955021+0000", 2024-09-18T04:58:42.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7570000000000004e-06 2024-09-18T04:58:42.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955025+0000", 2024-09-18T04:58:42.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1219999999999997e-06 2024-09-18T04:58:42.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003792+0000", 2024-09-18T04:58:42.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048767379 2024-09-18T04:58:42.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003860+0000", 2024-09-18T04:58:42.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8033000000000004e-05 2024-09-18T04:58:42.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003961+0000", 2024-09-18T04:58:42.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010103300000000001 2024-09-18T04:58:42.425 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.425 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.425 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.425 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72161 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 13312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:42.425 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:58:42.425 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 105.42391527700001, 2024-09-18T04:58:42.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048891539999999997, 2024-09-18T04:58:42.425 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.425 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.425 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.426 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.426 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.426 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72161 2024-09-18T04:58:42.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.426 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:58:42.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:58:42.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955028+0000", 2024-09-18T04:58:42.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997549 2024-09-18T04:58:42.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955278+0000", 2024-09-18T04:58:42.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024973299999999998 2024-09-18T04:58:42.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955289+0000", 2024-09-18T04:58:42.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0812000000000001e-05 2024-09-18T04:58:42.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955296+0000", 2024-09-18T04:58:42.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2570000000000001e-06 2024-09-18T04:58:42.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004002+0000", 2024-09-18T04:58:42.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048705904000000001 2024-09-18T04:58:42.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004076+0000", 2024-09-18T04:58:42.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4086999999999997e-05 2024-09-18T04:58:42.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004165+0000", 2024-09-18T04:58:42.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8712e-05 2024-09-18T04:58:42.429 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.429 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.429 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.430 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72163 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 44032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:42.430 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:58:42.430 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 105.42328983900001, 2024-09-18T04:58:42.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048567011, 2024-09-18T04:58:42.430 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.430 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.430 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.430 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.430 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.430 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72163 2024-09-18T04:58:42.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.430 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:58:42.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:58:42.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955898+0000", 2024-09-18T04:58:42.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:42.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955901+0000", 2024-09-18T04:58:42.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.706e-06 2024-09-18T04:58:42.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.432 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955907+0000", 2024-09-18T04:58:42.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3559999999999998e-06 2024-09-18T04:58:42.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.432 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955912+0000", 2024-09-18T04:58:42.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9640000000000002e-06 2024-09-18T04:58:42.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004377+0000", 2024-09-18T04:58:42.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048465293999999999 2024-09-18T04:58:42.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004418+0000", 2024-09-18T04:58:42.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0163000000000002e-05 2024-09-18T04:58:42.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004466+0000", 2024-09-18T04:58:42.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8004000000000003e-05 2024-09-18T04:58:42.434 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.434 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.434 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.434 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72164 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:42.434 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:58:42.434 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 105.423271566, 2024-09-18T04:58:42.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048707253999999998, 2024-09-18T04:58:42.434 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.434 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.435 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.435 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.435 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.435 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72164 2024-09-18T04:58:42.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.435 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:58:42.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:58:42.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:58:42.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:42.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955919+0000", 2024-09-18T04:58:42.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2520000000000002e-06 2024-09-18T04:58:42.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955925+0000", 2024-09-18T04:58:42.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3740000000000003e-06 2024-09-18T04:58:42.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955930+0000", 2024-09-18T04:58:42.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:58:42.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004489+0000", 2024-09-18T04:58:42.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048558581000000003 2024-09-18T04:58:42.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004529+0000", 2024-09-18T04:58:42.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9722999999999999e-05 2024-09-18T04:58:42.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004624+0000", 2024-09-18T04:58:42.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5402999999999994e-05 2024-09-18T04:58:42.438 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.438 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.438 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.439 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72165 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 74752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:42.439 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:58:42.439 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 105.422996707, 2024-09-18T04:58:42.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048633223000000003, 2024-09-18T04:58:42.439 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.439 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.439 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.439 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.439 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.439 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72165 2024-09-18T04:58:42.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.439 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:58:42.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:58:42.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955935+0000", 2024-09-18T04:58:42.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999743 2024-09-18T04:58:42.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956199+0000", 2024-09-18T04:58:42.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000263906 2024-09-18T04:58:42.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956207+0000", 2024-09-18T04:58:42.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2519999999999999e-06 2024-09-18T04:58:42.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956214+0000", 2024-09-18T04:58:42.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7610000000000002e-06 2024-09-18T04:58:42.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004661+0000", 2024-09-18T04:58:42.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048447967000000002 2024-09-18T04:58:42.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004735+0000", 2024-09-18T04:58:42.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3789999999999997e-05 2024-09-18T04:58:42.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004825+0000", 2024-09-18T04:58:42.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9724999999999999e-05 2024-09-18T04:58:42.443 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.443 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.443 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.443 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72166 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 90112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:42.443 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:58:42.444 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 105.422743173, 2024-09-18T04:58:42.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048595737999999999, 2024-09-18T04:58:42.444 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.444 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.444 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.444 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72166 2024-09-18T04:58:42.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.444 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:58:42.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:58:42.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956221+0000", 2024-09-18T04:58:42.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997759 2024-09-18T04:58:42.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956448+0000", 2024-09-18T04:58:42.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022685500000000001 2024-09-18T04:58:42.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956456+0000", 2024-09-18T04:58:42.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0099999999999995e-06 2024-09-18T04:58:42.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956462+0000", 2024-09-18T04:58:42.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7200000000000003e-06 2024-09-18T04:58:42.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004862+0000", 2024-09-18T04:58:42.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048399795000000002 2024-09-18T04:58:42.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004941+0000", 2024-09-18T04:58:42.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9414999999999998e-05 2024-09-18T04:58:42.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005041+0000", 2024-09-18T04:58:42.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9942e-05 2024-09-18T04:58:42.448 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.448 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.448 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.448 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72167 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:42.448 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:58:42.448 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 105.422656135, 2024-09-18T04:58:42.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048622894999999999, 2024-09-18T04:58:42.448 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.449 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.449 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.449 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.449 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.449 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72167 2024-09-18T04:58:42.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.449 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:58:42.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:58:42.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956468+0000", 2024-09-18T04:58:42.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999352 2024-09-18T04:58:42.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956535+0000", 2024-09-18T04:58:42.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7440999999999993e-05 2024-09-18T04:58:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956544+0000", 2024-09-18T04:58:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1039999999999996e-06 2024-09-18T04:58:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956550+0000", 2024-09-18T04:58:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9850000000000003e-06 2024-09-18T04:58:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005066+0000", 2024-09-18T04:58:42.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048515614999999998 2024-09-18T04:58:42.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005107+0000", 2024-09-18T04:58:42.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0843999999999997e-05 2024-09-18T04:58:42.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005155+0000", 2024-09-18T04:58:42.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8686000000000001e-05 2024-09-18T04:58:42.452 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.452 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.453 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.453 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72168 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:42.453 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:58:42.453 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 105.422495072, 2024-09-18T04:58:42.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048585179999999999, 2024-09-18T04:58:42.453 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.453 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.453 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.453 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.453 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.453 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72168 2024-09-18T04:58:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:58:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:58:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956555+0000", 2024-09-18T04:58:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998612 2024-09-18T04:58:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956696+0000", 2024-09-18T04:58:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000140922 2024-09-18T04:58:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956707+0000", 2024-09-18T04:58:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1555e-05 2024-09-18T04:58:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956714+0000", 2024-09-18T04:58:42.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2709999999999998e-06 2024-09-18T04:58:42.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005177+0000", 2024-09-18T04:58:42.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048463421 2024-09-18T04:58:42.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005214+0000", 2024-09-18T04:58:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.676e-05 2024-09-18T04:58:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005279+0000", 2024-09-18T04:58:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4899000000000003e-05 2024-09-18T04:58:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72199 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:58:42.457 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 105.41996318, 2024-09-18T04:58:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048626057, 2024-09-18T04:58:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72199 2024-09-18T04:58:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:58:42.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:58:42.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959195+0000", 2024-09-18T04:58:42.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999969 2024-09-18T04:58:42.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959228+0000", 2024-09-18T04:58:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3609000000000003e-05 2024-09-18T04:58:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959236+0000", 2024-09-18T04:58:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.157e-06 2024-09-18T04:58:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959243+0000", 2024-09-18T04:58:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5910000000000001e-06 2024-09-18T04:58:42.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007684+0000", 2024-09-18T04:58:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048440745 2024-09-18T04:58:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007762+0000", 2024-09-18T04:58:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8360000000000002e-05 2024-09-18T04:58:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007851+0000", 2024-09-18T04:58:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9388000000000005e-05 2024-09-18T04:58:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72240 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 428032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:58:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 105.41586986999999, 2024-09-18T04:58:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048643767999999997, 2024-09-18T04:58:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.462 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72240 2024-09-18T04:58:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:58:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:58:42.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963240+0000", 2024-09-18T04:58:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999218 2024-09-18T04:58:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963323+0000", 2024-09-18T04:58:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2375999999999999e-05 2024-09-18T04:58:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963332+0000", 2024-09-18T04:58:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1700000000000003e-06 2024-09-18T04:58:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963338+0000", 2024-09-18T04:58:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2149999999999998e-06 2024-09-18T04:58:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011785+0000", 2024-09-18T04:58:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048446493 2024-09-18T04:58:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011861+0000", 2024-09-18T04:58:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6472999999999998e-05 2024-09-18T04:58:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011962+0000", 2024-09-18T04:58:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101167 2024-09-18T04:58:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.466 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:117375 3.1 3:80b18d38:::rbd_header.11acae5d5365:head [watch reconnect cookie 140406307615440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T04:58:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:58:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.994795332999999, 2024-09-18T04:58:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.018908156, 2024-09-18T04:58:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117375 2024-09-18T04:58:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:58:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:58:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384391+0000", 2024-09-18T04:58:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:58:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384401+0000", 2024-09-18T04:58:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4299999999999995e-06 2024-09-18T04:58:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384418+0000", 2024-09-18T04:58:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6745e-05 2024-09-18T04:58:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384429+0000", 2024-09-18T04:58:42.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1122e-05 2024-09-18T04:58:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384496+0000", 2024-09-18T04:58:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6841000000000006e-05 2024-09-18T04:58:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-18T04:58:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384505+0000", 2024-09-18T04:58:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4080000000000004e-06 2024-09-18T04:58:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.388971+0000", 2024-09-18T04:58:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0044660000000000004 2024-09-18T04:58:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.388981+0000", 2024-09-18T04:58:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0118e-05 2024-09-18T04:58:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.399433+0000", 2024-09-18T04:58:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.010452055 2024-09-18T04:58:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.399594+0000", 2024-09-18T04:58:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000160669 2024-09-18T04:58:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.403301+0000", 2024-09-18T04:58:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037074370000000001 2024-09-18T04:58:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:117376 3.1 3:80b18d38:::rbd_header.11acae5d5365:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T04:58:42.472 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:58:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 78.992743683, 2024-09-18T04:58:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0196134539999999, 2024-09-18T04:58:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117376 2024-09-18T04:58:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:58:42.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:58:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386443+0000", 2024-09-18T04:58:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:58:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386457+0000", 2024-09-18T04:58:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4207e-05 2024-09-18T04:58:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386478+0000", 2024-09-18T04:58:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0349000000000001e-05 2024-09-18T04:58:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386503+0000", 2024-09-18T04:58:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5582e-05 2024-09-18T04:58:42.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386558+0000", 2024-09-18T04:58:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4892999999999998e-05 2024-09-18T04:58:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-18T04:58:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386562+0000", 2024-09-18T04:58:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.313e-06 2024-09-18T04:58:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.389010+0000", 2024-09-18T04:58:42.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024480890000000001 2024-09-18T04:58:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.389018+0000", 2024-09-18T04:58:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8469999999999996e-06 2024-09-18T04:58:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400073+0000", 2024-09-18T04:58:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.011054286 2024-09-18T04:58:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400124+0000", 2024-09-18T04:58:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1332999999999998e-05 2024-09-18T04:58:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:58:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400477+0000", 2024-09-18T04:58:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035329300000000002 2024-09-18T04:58:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.401353+0000", 2024-09-18T04:58:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00087584200000000005 2024-09-18T04:58:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406006+0000", 2024-09-18T04:58:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0046534250000000001 2024-09-18T04:58:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406037+0000", 2024-09-18T04:58:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0559000000000003e-05 2024-09-18T04:58:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406058+0000", 2024-09-18T04:58:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1260999999999999e-05 2024-09-18T04:58:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.480 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167963 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e147)", 2024-09-18T04:58:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:37.082364+0000", 2024-09-18T04:58:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 5.2968246700000003, 2024-09-18T04:58:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94325145700000002, 2024-09-18T04:58:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167963 2024-09-18T04:58:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:37.082364+0000", 2024-09-18T04:58:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:37.082364+0000", 2024-09-18T04:58:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:37.082362+0000", 2024-09-18T04:58:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:58:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:37.082368+0000", 2024-09-18T04:58:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8289999999999996e-06 2024-09-18T04:58:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:37.082379+0000", 2024-09-18T04:58:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0975000000000001e-05 2024-09-18T04:58:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:37.082403+0000", 2024-09-18T04:58:42.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4029e-05 2024-09-18T04:58:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:37.082443+0000", 2024-09-18T04:58:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9947999999999998e-05 2024-09-18T04:58:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:37.082450+0000", 2024-09-18T04:58:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0350000000000004e-06 2024-09-18T04:58:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:38.025368+0000", 2024-09-18T04:58:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94291755399999999 2024-09-18T04:58:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:38.025504+0000", 2024-09-18T04:58:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013585100000000001 2024-09-18T04:58:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:38.025615+0000", 2024-09-18T04:58:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111618 2024-09-18T04:58:42.485 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.486 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.486 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:42.487 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:58:42.488 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:42.484+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:42.588 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:42.586+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:42.630 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:42.630 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:42.631 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:58:42.631 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:42.639 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:58:42.724 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:42.722+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:42.782 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:42.782 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:42.782 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:58:42.782 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:58:42.782 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:42.790 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:42.787+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:42.791 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:58:42.890 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:42.888+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:42.935 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:42.935 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:58:42.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:58:42.935 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:42.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.935 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60345 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1471488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:42.935 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:58:42.935 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 111.819326207, 2024-09-18T04:58:42.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044180090999999998, 2024-09-18T04:58:42.935 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.935 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.936 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.936 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.936 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.936 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60345 2024-09-18T04:58:42.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.936 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:58:42.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:58:42.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112238+0000", 2024-09-18T04:58:42.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:42.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112240+0000", 2024-09-18T04:58:42.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9910000000000001e-06 2024-09-18T04:58:42.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112246+0000", 2024-09-18T04:58:42.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0859999999999998e-06 2024-09-18T04:58:42.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112251+0000", 2024-09-18T04:58:42.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5020000000000003e-06 2024-09-18T04:58:42.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156274+0000", 2024-09-18T04:58:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044023063000000001 2024-09-18T04:58:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156337+0000", 2024-09-18T04:58:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3077000000000006e-05 2024-09-18T04:58:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156419+0000", 2024-09-18T04:58:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1699000000000005e-05 2024-09-18T04:58:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.939 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.940 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.940 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60346 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1486848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:42.940 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:58:42.940 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 111.81930842600001, 2024-09-18T04:58:42.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044336331999999999, 2024-09-18T04:58:42.940 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.940 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.940 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.940 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.940 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.940 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60346 2024-09-18T04:58:42.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.941 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:58:42.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:58:42.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112256+0000", 2024-09-18T04:58:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112258+0000", 2024-09-18T04:58:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1720000000000001e-06 2024-09-18T04:58:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112265+0000", 2024-09-18T04:58:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4280000000000001e-06 2024-09-18T04:58:42.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112269+0000", 2024-09-18T04:58:42.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.741e-06 2024-09-18T04:58:42.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156445+0000", 2024-09-18T04:58:42.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044175341 2024-09-18T04:58:42.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156512+0000", 2024-09-18T04:58:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6723000000000005e-05 2024-09-18T04:58:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156593+0000", 2024-09-18T04:58:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1284000000000003e-05 2024-09-18T04:58:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60347 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1502208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:42.944 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:58:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 111.819280015, 2024-09-18T04:58:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044482560999999997, 2024-09-18T04:58:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60347 2024-09-18T04:58:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:58:42.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:58:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112284+0000", 2024-09-18T04:58:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112286+0000", 2024-09-18T04:58:42.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9709999999999998e-06 2024-09-18T04:58:42.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112290+0000", 2024-09-18T04:58:42.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9199999999999997e-06 2024-09-18T04:58:42.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112293+0000", 2024-09-18T04:58:42.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870000000000001e-06 2024-09-18T04:58:42.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156617+0000", 2024-09-18T04:58:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044323960000000003 2024-09-18T04:58:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156687+0000", 2024-09-18T04:58:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0141000000000005e-05 2024-09-18T04:58:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156767+0000", 2024-09-18T04:58:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0140000000000002e-05 2024-09-18T04:58:42.948 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60348 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1517568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:58:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 111.819269011, 2024-09-18T04:58:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044645562, 2024-09-18T04:58:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.949 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60348 2024-09-18T04:58:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:58:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:58:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:58:42.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:42.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112297+0000", 2024-09-18T04:58:42.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.384e-06 2024-09-18T04:58:42.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112301+0000", 2024-09-18T04:58:42.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0570000000000003e-06 2024-09-18T04:58:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112304+0000", 2024-09-18T04:58:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6000000000000001e-06 2024-09-18T04:58:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156793+0000", 2024-09-18T04:58:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044489494999999997 2024-09-18T04:58:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156859+0000", 2024-09-18T04:58:42.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5483999999999998e-05 2024-09-18T04:58:42.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156941+0000", 2024-09-18T04:58:42.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2856999999999996e-05 2024-09-18T04:58:42.953 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.953 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.953 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.953 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60349 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1532928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:42.953 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:58:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 111.819259035, 2024-09-18T04:58:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044832167999999999, 2024-09-18T04:58:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60349 2024-09-18T04:58:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:58:42.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:58:42.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:58:42.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112307+0000", 2024-09-18T04:58:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.308e-06 2024-09-18T04:58:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112310+0000", 2024-09-18T04:58:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-18T04:58:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112312+0000", 2024-09-18T04:58:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4889999999999998e-06 2024-09-18T04:58:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156977+0000", 2024-09-18T04:58:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044664598999999999 2024-09-18T04:58:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157053+0000", 2024-09-18T04:58:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6014999999999996e-05 2024-09-18T04:58:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157138+0000", 2024-09-18T04:58:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5162999999999998e-05 2024-09-18T04:58:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60350 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1548288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:58:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 111.819250121, 2024-09-18T04:58:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045011858000000002, 2024-09-18T04:58:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.958 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.959 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.959 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.959 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.959 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60350 2024-09-18T04:58:42.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.959 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:58:42.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:58:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:58:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112316+0000", 2024-09-18T04:58:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7579999999999999e-06 2024-09-18T04:58:42.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112321+0000", 2024-09-18T04:58:42.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1649999999999997e-06 2024-09-18T04:58:42.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112325+0000", 2024-09-18T04:58:42.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.726e-06 2024-09-18T04:58:42.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157172+0000", 2024-09-18T04:58:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.04484718 2024-09-18T04:58:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157242+0000", 2024-09-18T04:58:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9675e-05 2024-09-18T04:58:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157327+0000", 2024-09-18T04:58:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4614e-05 2024-09-18T04:58:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.962 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.963 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60351 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1563648~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:42.963 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:58:42.963 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 111.81923670499999, 2024-09-18T04:58:42.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045132113000000001, 2024-09-18T04:58:42.963 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.963 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.963 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.963 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.963 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.963 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60351 2024-09-18T04:58:42.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:58:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:58:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:58:42.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:42.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112330+0000", 2024-09-18T04:58:42.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1660000000000001e-06 2024-09-18T04:58:42.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112336+0000", 2024-09-18T04:58:42.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5940000000000001e-06 2024-09-18T04:58:42.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112340+0000", 2024-09-18T04:58:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5589999999999999e-06 2024-09-18T04:58:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157365+0000", 2024-09-18T04:58:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045024675 2024-09-18T04:58:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157409+0000", 2024-09-18T04:58:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4356e-05 2024-09-18T04:58:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157460+0000", 2024-09-18T04:58:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1072000000000002e-05 2024-09-18T04:58:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60502 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1509376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:42.967 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:58:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 111.813160532, 2024-09-18T04:58:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044092438999999997, 2024-09-18T04:58:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60502 2024-09-18T04:58:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:58:42.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:58:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:58:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118406+0000", 2024-09-18T04:58:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:58:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118412+0000", 2024-09-18T04:58:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.524e-06 2024-09-18T04:58:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118416+0000", 2024-09-18T04:58:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:58:42.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162418+0000", 2024-09-18T04:58:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001508000000002 2024-09-18T04:58:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162451+0000", 2024-09-18T04:58:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3662999999999999e-05 2024-09-18T04:58:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162497+0000", 2024-09-18T04:58:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5550000000000003e-05 2024-09-18T04:58:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60503 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1524736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:58:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 111.813144374, 2024-09-18T04:58:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044178780000000001, 2024-09-18T04:58:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.972 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60503 2024-09-18T04:58:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:58:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:58:42.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118420+0000", 2024-09-18T04:58:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118422+0000", 2024-09-18T04:58:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.08e-06 2024-09-18T04:58:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118428+0000", 2024-09-18T04:58:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.964e-06 2024-09-18T04:58:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118433+0000", 2024-09-18T04:58:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.425e-06 2024-09-18T04:58:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162519+0000", 2024-09-18T04:58:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044086082999999998 2024-09-18T04:58:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162552+0000", 2024-09-18T04:58:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3719e-05 2024-09-18T04:58:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162599+0000", 2024-09-18T04:58:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6873999999999999e-05 2024-09-18T04:58:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60504 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1540096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:58:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 111.813128269, 2024-09-18T04:58:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044261921000000003, 2024-09-18T04:58:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60504 2024-09-18T04:58:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:58:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:58:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118436+0000", 2024-09-18T04:58:42.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118438+0000", 2024-09-18T04:58:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0250000000000001e-06 2024-09-18T04:58:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118444+0000", 2024-09-18T04:58:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4360000000000001e-06 2024-09-18T04:58:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118448+0000", 2024-09-18T04:58:42.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.357e-06 2024-09-18T04:58:42.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162620+0000", 2024-09-18T04:58:42.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044171821 2024-09-18T04:58:42.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162653+0000", 2024-09-18T04:58:42.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3479999999999998e-05 2024-09-18T04:58:42.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162699+0000", 2024-09-18T04:58:42.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5113e-05 2024-09-18T04:58:42.981 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.981 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.981 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.981 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60585 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1211392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:42.981 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:58:42.981 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 111.811319144, 2024-09-18T04:58:42.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044389814, 2024-09-18T04:58:42.982 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.982 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.982 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.982 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.982 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.982 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60585 2024-09-18T04:58:42.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.982 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:58:42.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:58:42.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120245+0000", 2024-09-18T04:58:42.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:42.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120247+0000", 2024-09-18T04:58:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0150000000000002e-06 2024-09-18T04:58:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120255+0000", 2024-09-18T04:58:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.858e-06 2024-09-18T04:58:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120260+0000", 2024-09-18T04:58:42.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6530000000000002e-06 2024-09-18T04:58:42.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164489+0000", 2024-09-18T04:58:42.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044229483999999999 2024-09-18T04:58:42.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.985 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164555+0000", 2024-09-18T04:58:42.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.533e-05 2024-09-18T04:58:42.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164636+0000", 2024-09-18T04:58:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0819e-05 2024-09-18T04:58:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60586 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1226752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:58:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 111.81130069300001, 2024-09-18T04:58:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044578214999999997, 2024-09-18T04:58:42.986 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.987 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.987 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.987 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.987 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.987 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60586 2024-09-18T04:58:42.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.987 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:58:42.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:58:42.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:58:42.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:42.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120266+0000", 2024-09-18T04:58:42.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0660000000000002e-06 2024-09-18T04:58:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120272+0000", 2024-09-18T04:58:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-18T04:58:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120276+0000", 2024-09-18T04:58:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.476e-06 2024-09-18T04:58:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164673+0000", 2024-09-18T04:58:42.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044396672999999998 2024-09-18T04:58:42.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164749+0000", 2024-09-18T04:58:42.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5811999999999999e-05 2024-09-18T04:58:42.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.990 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.990 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164842+0000", 2024-09-18T04:58:42.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3560000000000005e-05 2024-09-18T04:58:42.990 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.990 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60587 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1242112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:58:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 111.811251594, 2024-09-18T04:58:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044634794999999998, 2024-09-18T04:58:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60587 2024-09-18T04:58:42.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:58:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:58:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120280+0000", 2024-09-18T04:58:42.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999671 2024-09-18T04:58:42.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120316+0000", 2024-09-18T04:58:42.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5728000000000001e-05 2024-09-18T04:58:42.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120329+0000", 2024-09-18T04:58:42.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2914e-05 2024-09-18T04:58:42.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120335+0000", 2024-09-18T04:58:42.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8289999999999996e-06 2024-09-18T04:58:42.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164858+0000", 2024-09-18T04:58:42.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044523417000000003 2024-09-18T04:58:42.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164896+0000", 2024-09-18T04:58:42.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7969999999999997e-05 2024-09-18T04:58:42.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:42.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164948+0000", 2024-09-18T04:58:42.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1996000000000002e-05 2024-09-18T04:58:42.995 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.995 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:42.995 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:42.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.996 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60596 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1380352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:42.996 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:58:42.996 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 111.809392057, 2024-09-18T04:58:42.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044120800000000002, 2024-09-18T04:58:42.996 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:42.996 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:42.996 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:42.996 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:42.996 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:42.996 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60596 2024-09-18T04:58:42.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.996 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:42.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:42.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:58:42.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:42.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:58:42.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:42.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.997 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.997 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:42.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:58:42.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:42.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:42.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122175+0000", 2024-09-18T04:58:42.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:58:42.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:42.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122182+0000", 2024-09-18T04:58:42.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3509999999999996e-06 2024-09-18T04:58:42.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:42.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122187+0000", 2024-09-18T04:58:42.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7940000000000002e-06 2024-09-18T04:58:42.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:42.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166184+0000", 2024-09-18T04:58:42.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043997236000000002 2024-09-18T04:58:42.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:42.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:42.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166227+0000", 2024-09-18T04:58:42.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2561000000000001e-05 2024-09-18T04:58:42.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:42.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166294+0000", 2024-09-18T04:58:43.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7068000000000002e-05 2024-09-18T04:58:43.000 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.000 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.000 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.000 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60597 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1395712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:43.000 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:58:43.000 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 111.809374029, 2024-09-18T04:58:43.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044242166999999999, 2024-09-18T04:58:43.000 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:43.001 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:43.001 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:43.001 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:43.001 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:43.001 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60597 2024-09-18T04:58:43.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.001 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:43.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:43.001 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:58:43.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.001 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:43.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:58:43.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:43.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:58:43.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:43.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.002 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:43.002 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122193+0000", 2024-09-18T04:58:43.002 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:58:43.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:43.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122199+0000", 2024-09-18T04:58:43.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6449999999999999e-06 2024-09-18T04:58:43.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:43.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122204+0000", 2024-09-18T04:58:43.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4819999999999996e-06 2024-09-18T04:58:43.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166329+0000", 2024-09-18T04:58:43.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044125059000000001 2024-09-18T04:58:43.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:43.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166370+0000", 2024-09-18T04:58:43.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1012e-05 2024-09-18T04:58:43.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166433+0000", 2024-09-18T04:58:43.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3178999999999999e-05 2024-09-18T04:58:43.004 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.005 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.005 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.005 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167074 3.5 3:a712c57a:::rbd_data.11ac138032e7.0000000000000000:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:43.005 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.163703+0000", 2024-09-18T04:58:43.005 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.767861847999999, 2024-09-18T04:58:43.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.90992387799999996, 2024-09-18T04:58:43.005 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:43.005 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:43.005 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:43.005 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:43.005 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:43.006 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167074 2024-09-18T04:58:43.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.006 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:43.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:43.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163703+0000", 2024-09-18T04:58:43.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:43.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163703+0000", 2024-09-18T04:58:43.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:43.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163702+0000", 2024-09-18T04:58:43.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:58:43.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:43.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163707+0000", 2024-09-18T04:58:43.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5159999999999996e-06 2024-09-18T04:58:43.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:43.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163716+0000", 2024-09-18T04:58:43.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5270000000000004e-06 2024-09-18T04:58:43.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:43.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163723+0000", 2024-09-18T04:58:43.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9809999999999997e-06 2024-09-18T04:58:43.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163781+0000", 2024-09-18T04:58:43.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7494999999999998e-05 2024-09-18T04:58:43.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:43.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163791+0000", 2024-09-18T04:58:43.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0614999999999999e-05 2024-09-18T04:58:43.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067213+0000", 2024-09-18T04:58:43.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.90342179700000003 2024-09-18T04:58:43.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:43.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067327+0000", 2024-09-18T04:58:43.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000114023 2024-09-18T04:58:43.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-18T04:58:43.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067432+0000", 2024-09-18T04:58:43.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104775 2024-09-18T04:58:43.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:43.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.071972+0000", 2024-09-18T04:58:43.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0045401219999999997 2024-09-18T04:58:43.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.010 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:43.010 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073530+0000", 2024-09-18T04:58:43.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001558341 2024-09-18T04:58:43.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:43.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073582+0000", 2024-09-18T04:58:43.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1924e-05 2024-09-18T04:58:43.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.011 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.011 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.011 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073627+0000", 2024-09-18T04:58:43.011 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4647000000000001e-05 2024-09-18T04:58:43.011 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.011 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.011 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.012 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167075 3.1 3:864982b2:::rbd_data.11ac138032e7.0000000000000001:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:43.012 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.163740+0000", 2024-09-18T04:58:43.012 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.767824579999999, 2024-09-18T04:58:43.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.90703181499999996, 2024-09-18T04:58:43.012 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:43.012 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:43.012 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:43.012 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:43.012 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:43.012 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167075 2024-09-18T04:58:43.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.012 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:43.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:43.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163740+0000", 2024-09-18T04:58:43.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:43.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163740+0000", 2024-09-18T04:58:43.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:43.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163740+0000", 2024-09-18T04:58:43.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:43.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:43.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163743+0000", 2024-09-18T04:58:43.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0230000000000001e-06 2024-09-18T04:58:43.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:43.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163750+0000", 2024-09-18T04:58:43.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2710000000000003e-06 2024-09-18T04:58:43.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:43.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163756+0000", 2024-09-18T04:58:43.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7559999999999996e-06 2024-09-18T04:58:43.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163803+0000", 2024-09-18T04:58:43.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6576999999999998e-05 2024-09-18T04:58:43.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:43.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163813+0000", 2024-09-18T04:58:43.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0336000000000001e-05 2024-09-18T04:58:43.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.066573+0000", 2024-09-18T04:58:43.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.90276046799999998 2024-09-18T04:58:43.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:43.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.066694+0000", 2024-09-18T04:58:43.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012100299999999999 2024-09-18T04:58:43.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:58:43.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.066791+0000", 2024-09-18T04:58:43.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7141999999999999e-05 2024-09-18T04:58:43.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:43.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.068686+0000", 2024-09-18T04:58:43.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018949570000000001 2024-09-18T04:58:43.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:43.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070676+0000", 2024-09-18T04:58:43.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001989158 2024-09-18T04:58:43.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:43.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070733+0000", 2024-09-18T04:58:43.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7642000000000001e-05 2024-09-18T04:58:43.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070772+0000", 2024-09-18T04:58:43.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8949000000000003e-05 2024-09-18T04:58:43.018 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.018 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.018 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.018 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167093 3.1 3:82b38587:::rbd_data.11ac138032e7.0000000000000025:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:43.018 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.173880+0000", 2024-09-18T04:58:43.018 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.757684554000001, 2024-09-18T04:58:43.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.89950706999999996, 2024-09-18T04:58:43.019 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:43.019 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:43.019 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:43.019 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:43.019 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:43.019 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167093 2024-09-18T04:58:43.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.019 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:43.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:43.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173880+0000", 2024-09-18T04:58:43.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:43.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173880+0000", 2024-09-18T04:58:43.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:43.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173880+0000", 2024-09-18T04:58:43.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:43.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.020 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:43.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173884+0000", 2024-09-18T04:58:43.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9910000000000004e-06 2024-09-18T04:58:43.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:43.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173895+0000", 2024-09-18T04:58:43.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1238e-05 2024-09-18T04:58:43.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:43.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173903+0000", 2024-09-18T04:58:43.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2989999999999997e-06 2024-09-18T04:58:43.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173954+0000", 2024-09-18T04:58:43.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0862999999999997e-05 2024-09-18T04:58:43.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:43.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173965+0000", 2024-09-18T04:58:43.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0658e-05 2024-09-18T04:58:43.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067554+0000", 2024-09-18T04:58:43.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.89358902200000001 2024-09-18T04:58:43.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:43.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067664+0000", 2024-09-18T04:58:43.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010977200000000001 2024-09-18T04:58:43.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:58:43.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067754+0000", 2024-09-18T04:58:43.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0484999999999998e-05 2024-09-18T04:58:43.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:43.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070563+0000", 2024-09-18T04:58:43.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028086650000000001 2024-09-18T04:58:43.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:43.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073294+0000", 2024-09-18T04:58:43.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027316599999999999 2024-09-18T04:58:43.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:43.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073351+0000", 2024-09-18T04:58:43.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6873999999999998e-05 2024-09-18T04:58:43.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.025 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073387+0000", 2024-09-18T04:58:43.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6254000000000002e-05 2024-09-18T04:58:43.025 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.025 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.025 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.025 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167095 3.5 3:a340fea6:::rbd_data.11ac138032e7.0000000000000027:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:43.025 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.173930+0000", 2024-09-18T04:58:43.025 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.75763482, 2024-09-18T04:58:43.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.89985602499999995, 2024-09-18T04:58:43.025 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:43.025 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:43.025 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:43.026 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:43.026 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:43.026 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167095 2024-09-18T04:58:43.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.026 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:43.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:43.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173930+0000", 2024-09-18T04:58:43.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:43.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173930+0000", 2024-09-18T04:58:43.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:43.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173929+0000", 2024-09-18T04:58:43.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:43.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:43.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173933+0000", 2024-09-18T04:58:43.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4860000000000002e-06 2024-09-18T04:58:43.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:43.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173940+0000", 2024-09-18T04:58:43.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4499999999999998e-06 2024-09-18T04:58:43.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:43.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173945+0000", 2024-09-18T04:58:43.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5290000000000002e-06 2024-09-18T04:58:43.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173981+0000", 2024-09-18T04:58:43.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6548999999999999e-05 2024-09-18T04:58:43.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:43.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173990+0000", 2024-09-18T04:58:43.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1810000000000001e-06 2024-09-18T04:58:43.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.069067+0000", 2024-09-18T04:58:43.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.89507692699999997 2024-09-18T04:58:43.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.029 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:43.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.069196+0000", 2024-09-18T04:58:43.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000129572 2024-09-18T04:58:43.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-18T04:58:43.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.069316+0000", 2024-09-18T04:58:43.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120165 2024-09-18T04:58:43.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:43.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.072036+0000", 2024-09-18T04:58:43.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027195489999999999 2024-09-18T04:58:43.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:43.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073703+0000", 2024-09-18T04:58:43.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001667555 2024-09-18T04:58:43.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:43.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073749+0000", 2024-09-18T04:58:43.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5636000000000001e-05 2024-09-18T04:58:43.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073786+0000", 2024-09-18T04:58:43.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7070000000000003e-05 2024-09-18T04:58:43.031 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.031 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.032 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.032 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167097 3.c 3:3137739a:::rbd_data.11ac138032e7.0000000000000029:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:43.032 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.178900+0000", 2024-09-18T04:58:43.032 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 23.752665034, 2024-09-18T04:58:43.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.89454723999999997, 2024-09-18T04:58:43.032 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:43.032 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:43.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:43.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:43.033 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:43.033 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167097 2024-09-18T04:58:43.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.033 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:43.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:43.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178900+0000", 2024-09-18T04:58:43.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:43.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178900+0000", 2024-09-18T04:58:43.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:43.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178899+0000", 2024-09-18T04:58:43.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:58:43.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:43.034 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178903+0000", 2024-09-18T04:58:43.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7699999999999999e-06 2024-09-18T04:58:43.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:43.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178913+0000", 2024-09-18T04:58:43.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0032e-05 2024-09-18T04:58:43.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:43.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178918+0000", 2024-09-18T04:58:43.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9740000000000001e-06 2024-09-18T04:58:43.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178959+0000", 2024-09-18T04:58:43.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1257999999999997e-05 2024-09-18T04:58:43.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:43.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178966+0000", 2024-09-18T04:58:43.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4139999999999999e-06 2024-09-18T04:58:43.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.066273+0000", 2024-09-18T04:58:43.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.88730742299999998 2024-09-18T04:58:43.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:43.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.066394+0000", 2024-09-18T04:58:43.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012086099999999999 2024-09-18T04:58:43.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-18T04:58:43.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.066503+0000", 2024-09-18T04:58:43.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109218 2024-09-18T04:58:43.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:43.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070854+0000", 2024-09-18T04:58:43.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0043508219999999998 2024-09-18T04:58:43.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:43.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073373+0000", 2024-09-18T04:58:43.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0025193519999999999 2024-09-18T04:58:43.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:43.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073413+0000", 2024-09-18T04:58:43.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9592999999999999e-05 2024-09-18T04:58:43.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.038 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073447+0000", 2024-09-18T04:58:43.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4239999999999997e-05 2024-09-18T04:58:43.038 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.038 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.038 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.039 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.039 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.039 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:43.040 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:58:43.041 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:43.037+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:43.096 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-09-18T04:58:43.143 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:43.139+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:43.188 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:43.188 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:43.189 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:58:43.189 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:43.197 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:58:43.242 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:43.239+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:43.300 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:43.300 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:58:43.301 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:58:43.301 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:43.309 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:58:43.350 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:43.348+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:43.395 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:58:43.444 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:43.440+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:43.455 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:43.455 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:58:43.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:58:43.455 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:43.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.455 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26200 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 32768~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:58:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 122.81233050100001, 2024-09-18T04:58:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022501118000000001, 2024-09-18T04:58:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26200 2024-09-18T04:58:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:58:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:58:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:58:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639822+0000", 2024-09-18T04:58:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.007e-06 2024-09-18T04:58:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639827+0000", 2024-09-18T04:58:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5709999999999997e-06 2024-09-18T04:58:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639831+0000", 2024-09-18T04:58:43.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2819999999999998e-06 2024-09-18T04:58:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662181+0000", 2024-09-18T04:58:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022349878 2024-09-18T04:58:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662252+0000", 2024-09-18T04:58:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0542000000000003e-05 2024-09-18T04:58:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662321+0000", 2024-09-18T04:58:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9167999999999999e-05 2024-09-18T04:58:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26201 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 61440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:58:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 122.81231520999999, 2024-09-18T04:58:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022569783, 2024-09-18T04:58:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:43.460 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26201 2024-09-18T04:58:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:58:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:58:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:58:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639837+0000", 2024-09-18T04:58:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.928e-06 2024-09-18T04:58:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639843+0000", 2024-09-18T04:58:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8130000000000001e-06 2024-09-18T04:58:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639847+0000", 2024-09-18T04:58:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0999999999999997e-06 2024-09-18T04:58:43.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662335+0000", 2024-09-18T04:58:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022488207999999999 2024-09-18T04:58:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662370+0000", 2024-09-18T04:58:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5292e-05 2024-09-18T04:58:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662405+0000", 2024-09-18T04:58:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4765999999999999e-05 2024-09-18T04:58:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26202 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 90112~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:58:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 122.81229979299999, 2024-09-18T04:58:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022655937000000001, 2024-09-18T04:58:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:43.465 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26202 2024-09-18T04:58:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:58:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:58:43.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639850+0000", 2024-09-18T04:58:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639852+0000", 2024-09-18T04:58:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9700000000000002e-06 2024-09-18T04:58:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:43.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639858+0000", 2024-09-18T04:58:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.434e-06 2024-09-18T04:58:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639862+0000", 2024-09-18T04:58:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3499999999999999e-06 2024-09-18T04:58:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662429+0000", 2024-09-18T04:58:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022567409 2024-09-18T04:58:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:43.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662469+0000", 2024-09-18T04:58:43.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9801000000000001e-05 2024-09-18T04:58:43.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662507+0000", 2024-09-18T04:58:43.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7311000000000002e-05 2024-09-18T04:58:43.469 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.469 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.469 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26203 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 118784~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:58:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 122.81228463799999, 2024-09-18T04:58:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022730358999999999, 2024-09-18T04:58:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26203 2024-09-18T04:58:43.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:58:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:58:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:43.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639865+0000", 2024-09-18T04:58:43.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:43.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:43.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639867+0000", 2024-09-18T04:58:43.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0889999999999998e-06 2024-09-18T04:58:43.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639873+0000", 2024-09-18T04:58:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4979999999999997e-06 2024-09-18T04:58:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639877+0000", 2024-09-18T04:58:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4510000000000002e-06 2024-09-18T04:58:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662528+0000", 2024-09-18T04:58:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022650791999999999 2024-09-18T04:58:43.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662563+0000", 2024-09-18T04:58:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4530000000000003e-05 2024-09-18T04:58:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662596+0000", 2024-09-18T04:58:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3463000000000001e-05 2024-09-18T04:58:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26204 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 147456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:58:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 122.812269246, 2024-09-18T04:58:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022803233999999999, 2024-09-18T04:58:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26204 2024-09-18T04:58:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:58:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:58:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:58:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:43.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639883+0000", 2024-09-18T04:58:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6520000000000002e-06 2024-09-18T04:58:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639888+0000", 2024-09-18T04:58:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2870000000000004e-06 2024-09-18T04:58:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639893+0000", 2024-09-18T04:58:43.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3839999999999999e-06 2024-09-18T04:58:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662617+0000", 2024-09-18T04:58:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022723876 2024-09-18T04:58:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662651+0000", 2024-09-18T04:58:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3797000000000002e-05 2024-09-18T04:58:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662684+0000", 2024-09-18T04:58:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3828999999999997e-05 2024-09-18T04:58:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26205 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 176128~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:58:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 122.812246285, 2024-09-18T04:58:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022919108000000001, 2024-09-18T04:58:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:43.479 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26205 2024-09-18T04:58:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:58:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:58:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:43.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:58:43.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:43.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:43.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639906+0000", 2024-09-18T04:58:43.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.531e-06 2024-09-18T04:58:43.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639913+0000", 2024-09-18T04:58:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.331e-06 2024-09-18T04:58:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639920+0000", 2024-09-18T04:58:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5399999999999998e-06 2024-09-18T04:58:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662706+0000", 2024-09-18T04:58:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.02278587 2024-09-18T04:58:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662764+0000", 2024-09-18T04:58:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7630000000000002e-05 2024-09-18T04:58:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662823+0000", 2024-09-18T04:58:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9648000000000002e-05 2024-09-18T04:58:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.483 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26206 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 204800~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:58:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 122.81222588599999, 2024-09-18T04:58:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023047517, 2024-09-18T04:58:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26206 2024-09-18T04:58:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.484 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:58:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:58:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:43.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:58:43.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:43.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:43.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639926+0000", 2024-09-18T04:58:43.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3120000000000001e-06 2024-09-18T04:58:43.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639932+0000", 2024-09-18T04:58:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4890000000000003e-06 2024-09-18T04:58:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639936+0000", 2024-09-18T04:58:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.07e-06 2024-09-18T04:58:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662850+0000", 2024-09-18T04:58:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022913579999999999 2024-09-18T04:58:43.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662903+0000", 2024-09-18T04:58:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3176e-05 2024-09-18T04:58:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662972+0000", 2024-09-18T04:58:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9270999999999995e-05 2024-09-18T04:58:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26207 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 233472~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:58:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 122.812210742, 2024-09-18T04:58:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023139541999999999, 2024-09-18T04:58:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26207 2024-09-18T04:58:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:43.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:58:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:58:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639939+0000", 2024-09-18T04:58:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:43.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639941+0000", 2024-09-18T04:58:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0080000000000001e-06 2024-09-18T04:58:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639947+0000", 2024-09-18T04:58:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.485e-06 2024-09-18T04:58:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639951+0000", 2024-09-18T04:58:43.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4549999999999997e-06 2024-09-18T04:58:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663005+0000", 2024-09-18T04:58:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023053569999999999 2024-09-18T04:58:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663042+0000", 2024-09-18T04:58:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7245e-05 2024-09-18T04:58:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663079+0000", 2024-09-18T04:58:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7119000000000002e-05 2024-09-18T04:58:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26208 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 524288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:58:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 122.812193994, 2024-09-18T04:58:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023283564999999999, 2024-09-18T04:58:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:43.493 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:43.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:43.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:43.494 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26208 2024-09-18T04:58:43.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.494 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:43.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:43.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:58:43.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:43.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:58:43.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:43.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:58:43.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:43.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:43.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639958+0000", 2024-09-18T04:58:43.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0899999999999999e-06 2024-09-18T04:58:43.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:43.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639963+0000", 2024-09-18T04:58:43.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2209999999999996e-06 2024-09-18T04:58:43.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639967+0000", 2024-09-18T04:58:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1169999999999997e-06 2024-09-18T04:58:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663114+0000", 2024-09-18T04:58:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023146102000000002 2024-09-18T04:58:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663183+0000", 2024-09-18T04:58:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9686999999999999e-05 2024-09-18T04:58:43.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663240+0000", 2024-09-18T04:58:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6728000000000003e-05 2024-09-18T04:58:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26209 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 552960~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:58:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 122.812176218, 2024-09-18T04:58:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023440966000000001, 2024-09-18T04:58:43.498 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26209 2024-09-18T04:58:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:58:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:58:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:58:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639976+0000", 2024-09-18T04:58:43.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4940000000000002e-06 2024-09-18T04:58:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639982+0000", 2024-09-18T04:58:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0560000000000001e-06 2024-09-18T04:58:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639987+0000", 2024-09-18T04:58:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-18T04:58:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.501 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663278+0000", 2024-09-18T04:58:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023290789999999999 2024-09-18T04:58:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663348+0000", 2024-09-18T04:58:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0837e-05 2024-09-18T04:58:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663415+0000", 2024-09-18T04:58:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6736000000000006e-05 2024-09-18T04:58:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.502 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26210 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 581632~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:58:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 122.81215921, 2024-09-18T04:58:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023616437000000001, 2024-09-18T04:58:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:43.503 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26210 2024-09-18T04:58:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:58:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:58:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:43.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:58:43.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:43.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:43.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639993+0000", 2024-09-18T04:58:43.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.419e-06 2024-09-18T04:58:43.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640006+0000", 2024-09-18T04:58:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2918000000000001e-05 2024-09-18T04:58:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640011+0000", 2024-09-18T04:58:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9320000000000004e-06 2024-09-18T04:58:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663454+0000", 2024-09-18T04:58:43.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023443035000000001 2024-09-18T04:58:43.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:43.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663532+0000", 2024-09-18T04:58:43.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7443999999999999e-05 2024-09-18T04:58:43.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.507 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663608+0000", 2024-09-18T04:58:43.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6093000000000005e-05 2024-09-18T04:58:43.507 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.507 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.507 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26211 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 610304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:58:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 122.81213465800001, 2024-09-18T04:58:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023774337999999999, 2024-09-18T04:58:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26211 2024-09-18T04:58:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.508 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:58:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:58:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:43.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640015+0000", 2024-09-18T04:58:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640023+0000", 2024-09-18T04:58:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3900000000000004e-06 2024-09-18T04:58:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640036+0000", 2024-09-18T04:58:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3091999999999999e-05 2024-09-18T04:58:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640041+0000", 2024-09-18T04:58:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6219999999999997e-06 2024-09-18T04:58:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663644+0000", 2024-09-18T04:58:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023602643999999999 2024-09-18T04:58:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663718+0000", 2024-09-18T04:58:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4481999999999996e-05 2024-09-18T04:58:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663790+0000", 2024-09-18T04:58:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1589999999999997e-05 2024-09-18T04:58:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26212 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 638976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:58:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 122.812101618, 2024-09-18T04:58:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023924748999999999, 2024-09-18T04:58:43.512 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26212 2024-09-18T04:58:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:58:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:43.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:58:43.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.514 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:43.514 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640048+0000", 2024-09-18T04:58:43.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:43.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640051+0000", 2024-09-18T04:58:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.095e-06 2024-09-18T04:58:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640060+0000", 2024-09-18T04:58:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8929999999999994e-06 2024-09-18T04:58:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640065+0000", 2024-09-18T04:58:43.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0270000000000003e-06 2024-09-18T04:58:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663824+0000", 2024-09-18T04:58:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023758873999999999 2024-09-18T04:58:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663904+0000", 2024-09-18T04:58:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9856000000000003e-05 2024-09-18T04:58:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663974+0000", 2024-09-18T04:58:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9450999999999996e-05 2024-09-18T04:58:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26213 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 667648~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:58:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 122.812080447, 2024-09-18T04:58:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024059812999999999, 2024-09-18T04:58:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:43.517 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26213 2024-09-18T04:58:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:58:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:58:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:58:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640072+0000", 2024-09-18T04:58:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4200000000000001e-06 2024-09-18T04:58:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:43.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640078+0000", 2024-09-18T04:58:43.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0179999999999998e-06 2024-09-18T04:58:43.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:43.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640083+0000", 2024-09-18T04:58:43.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:58:43.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664008+0000", 2024-09-18T04:58:43.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023925287 2024-09-18T04:58:43.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:43.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664059+0000", 2024-09-18T04:58:43.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0730999999999999e-05 2024-09-18T04:58:43.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664130+0000", 2024-09-18T04:58:43.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0801000000000002e-05 2024-09-18T04:58:43.521 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.521 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.521 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26214 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 696320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:58:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 122.812063477, 2024-09-18T04:58:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024223738000000002, 2024-09-18T04:58:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26214 2024-09-18T04:58:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.522 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:58:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:58:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:43.523 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:58:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640089+0000", 2024-09-18T04:58:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0849999999999999e-06 2024-09-18T04:58:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640095+0000", 2024-09-18T04:58:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9279999999999998e-06 2024-09-18T04:58:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640099+0000", 2024-09-18T04:58:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:58:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664166+0000", 2024-09-18T04:58:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024066513000000001 2024-09-18T04:58:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664239+0000", 2024-09-18T04:58:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3201999999999995e-05 2024-09-18T04:58:43.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664311+0000", 2024-09-18T04:58:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1434999999999996e-05 2024-09-18T04:58:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26215 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 724992~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:58:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 122.812046716, 2024-09-18T04:58:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024302284, 2024-09-18T04:58:43.526 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26215 2024-09-18T04:58:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:58:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:58:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640103+0000", 2024-09-18T04:58:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640106+0000", 2024-09-18T04:58:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2120000000000002e-06 2024-09-18T04:58:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640111+0000", 2024-09-18T04:58:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7880000000000003e-06 2024-09-18T04:58:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640116+0000", 2024-09-18T04:58:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6099999999999999e-06 2024-09-18T04:58:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664327+0000", 2024-09-18T04:58:43.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024210941999999999 2024-09-18T04:58:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664367+0000", 2024-09-18T04:58:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9666000000000003e-05 2024-09-18T04:58:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664406+0000", 2024-09-18T04:58:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9402e-05 2024-09-18T04:58:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26216 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 753664~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:58:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 122.812030222, 2024-09-18T04:58:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024452854999999999, 2024-09-18T04:58:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26216 2024-09-18T04:58:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.531 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:58:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:58:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:58:43.532 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640122+0000", 2024-09-18T04:58:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3599999999999999e-06 2024-09-18T04:58:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640128+0000", 2024-09-18T04:58:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6339999999999998e-06 2024-09-18T04:58:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.533 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640132+0000", 2024-09-18T04:58:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:58:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664436+0000", 2024-09-18T04:58:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024303585999999999 2024-09-18T04:58:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664507+0000", 2024-09-18T04:58:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1046999999999999e-05 2024-09-18T04:58:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.534 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664573+0000", 2024-09-18T04:58:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6217000000000007e-05 2024-09-18T04:58:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26217 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 782336~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:58:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 122.812013988, 2024-09-18T04:58:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024526287000000001, 2024-09-18T04:58:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:43.535 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26217 2024-09-18T04:58:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:58:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.536 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:58:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:58:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640140+0000", 2024-09-18T04:58:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2989999999999998e-06 2024-09-18T04:58:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.537 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640146+0000", 2024-09-18T04:58:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9569999999999999e-06 2024-09-18T04:58:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640151+0000", 2024-09-18T04:58:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:58:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664590+0000", 2024-09-18T04:58:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0244389 2024-09-18T04:58:43.538 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664628+0000", 2024-09-18T04:58:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7910999999999997e-05 2024-09-18T04:58:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664663+0000", 2024-09-18T04:58:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5024999999999998e-05 2024-09-18T04:58:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.539 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167100 3.1e 3:7bd358cc:::rbd_data.11ac138032e7.000000000000004f:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.178944+0000", 2024-09-18T04:58:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.27320671, 2024-09-18T04:58:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.89428526100000005, 2024-09-18T04:58:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167100 2024-09-18T04:58:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:43.540 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178944+0000", 2024-09-18T04:58:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178944+0000", 2024-09-18T04:58:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178942+0000", 2024-09-18T04:58:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:58:43.541 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178948+0000", 2024-09-18T04:58:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8100000000000003e-06 2024-09-18T04:58:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178962+0000", 2024-09-18T04:58:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.399e-05 2024-09-18T04:58:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178970+0000", 2024-09-18T04:58:43.542 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.537e-06 2024-09-18T04:58:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.179015+0000", 2024-09-18T04:58:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5652999999999998e-05 2024-09-18T04:58:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.179021+0000", 2024-09-18T04:58:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5280000000000003e-06 2024-09-18T04:58:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.543 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.065479+0000", 2024-09-18T04:58:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.88645855799999995 2024-09-18T04:58:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.065788+0000", 2024-09-18T04:58:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030816099999999999 2024-09-18T04:58:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-18T04:58:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.065904+0000", 2024-09-18T04:58:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116554 2024-09-18T04:58:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.544 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067684+0000", 2024-09-18T04:58:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001780195 2024-09-18T04:58:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073137+0000", 2024-09-18T04:58:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0054525210000000001 2024-09-18T04:58:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073193+0000", 2024-09-18T04:58:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6492000000000001e-05 2024-09-18T04:58:43.545 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073229+0000", 2024-09-18T04:58:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5664000000000003e-05 2024-09-18T04:58:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167111 3.19 3:99c9092f:::rbd_data.11ac138032e7.000000000000005a:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.201980+0000", 2024-09-18T04:58:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 24.250170107999999, 2024-09-18T04:58:43.546 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.87136403900000003, 2024-09-18T04:58:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167111 2024-09-18T04:58:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.201980+0000", 2024-09-18T04:58:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.547 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.201980+0000", 2024-09-18T04:58:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.201980+0000", 2024-09-18T04:58:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.201983+0000", 2024-09-18T04:58:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8949999999999998e-06 2024-09-18T04:58:43.548 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.201990+0000", 2024-09-18T04:58:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5129999999999999e-06 2024-09-18T04:58:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.201999+0000", 2024-09-18T04:58:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5490000000000006e-06 2024-09-18T04:58:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.202033+0000", 2024-09-18T04:58:43.549 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4076999999999999e-05 2024-09-18T04:58:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.202038+0000", 2024-09-18T04:58:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3310000000000003e-06 2024-09-18T04:58:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.068314+0000", 2024-09-18T04:58:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.86627544899999998 2024-09-18T04:58:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:43.550 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.068426+0000", 2024-09-18T04:58:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011262199999999999 2024-09-18T04:58:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 7", 2024-09-18T04:58:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.068531+0000", 2024-09-18T04:58:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010510600000000001 2024-09-18T04:58:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.071305+0000", 2024-09-18T04:58:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027742539999999999 2024-09-18T04:58:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.551 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073258+0000", 2024-09-18T04:58:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001952264 2024-09-18T04:58:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073303+0000", 2024-09-18T04:58:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5286e-05 2024-09-18T04:58:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073344+0000", 2024-09-18T04:58:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1269000000000001e-05 2024-09-18T04:58:43.552 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.553 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.553 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.553 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:43.553 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:43.553 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:43.554 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:58:43.554 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:58:43.555 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:58:43.555 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:58:43.555 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:58:43.556 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:58:43.556 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:58:43.556 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:43.552+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:43.566 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:58:43.566 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:58:43.582 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:58:43.583 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:58:43.598 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:58:43.598 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:58:43.614 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:58:43.614 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:58:43.632 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:58:43.632 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:58:43.655 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:43.652+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:43.661 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:43.661 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:43.670 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:58:43.756 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:43.754+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:43.812 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:43.812 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:43.820 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:58:43.857 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:43.854+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:43.922 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:43.922 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:43.931 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:58:43.958 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:43.955+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:44.044 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:44.045 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:44.053 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:58:44.058 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:44.057+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:44.087 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is set 2024-09-18T04:58:44.159 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:44.157+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:44.179 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:44.179 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:44.189 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:58:44.260 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:44.258+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:44.296 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:44.297 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:44.305 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:58:44.361 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:44.359+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:44.412 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:44.413 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:44.421 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:58:44.462 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:44.460+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:44.528 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:44.528 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:44.537 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:58:44.563 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:44.561+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:44.643 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:44.644 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:44.652 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:58:44.664 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:44.662+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:44.759 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:44.759 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:44.764 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:44.763+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:44.768 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:58:44.866 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:44.864+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:44.874 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:44.874 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:44.883 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:58:44.966 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:44.965+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:44.989 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:44.989 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:44.998 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:58:45.068 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:45.066+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:45.091 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 5, 6, 7, 4, 3, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-18T04:58:45.092 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-18T04:58:45.092 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-18T04:58:45.092 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-09-18T04:58:45.111 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:45.111 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:45.121 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:58:45.205 INFO:tasks.ceph.osd.1.smithi002.stderr:2024-09-18T04:58:45.203+0000 7f3a18232640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 33022) UID: 0 2024-09-18T04:58:45.270 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:45.267+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:45.285 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:45.285 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:45.294 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:58:45.370 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:45.369+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:45.403 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:58:45.403 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:45.403 INFO:teuthology.orchestra.run.smithi002.stdout: "epoch": 155, 2024-09-18T04:58:45.403 INFO:teuthology.orchestra.run.smithi002.stdout: "fsid": "c0761d0c-69d5-4b64-b8b9-c7d098b2d06d", 2024-09-18T04:58:45.403 INFO:teuthology.orchestra.run.smithi002.stdout: "created": "2024-09-18T04:55:58.216538+0000", 2024-09-18T04:58:45.403 INFO:teuthology.orchestra.run.smithi002.stdout: "modified": "2024-09-18T04:58:45.087642+0000", 2024-09-18T04:58:45.403 INFO:teuthology.orchestra.run.smithi002.stdout: "last_up_change": "2024-09-18T04:56:08.369924+0000", 2024-09-18T04:58:45.403 INFO:teuthology.orchestra.run.smithi002.stdout: "last_in_change": "2024-09-18T04:58:34.032750+0000", 2024-09-18T04:58:45.404 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-18T04:58:45.404 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_num": 5801984, 2024-09-18T04:58:45.404 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_set": [ 2024-09-18T04:58:45.404 INFO:teuthology.orchestra.run.smithi002.stdout: "noscrub", 2024-09-18T04:58:45.404 INFO:teuthology.orchestra.run.smithi002.stdout: "pglog_hardlimit", 2024-09-18T04:58:45.404 INFO:teuthology.orchestra.run.smithi002.stdout: "purged_snapdirs", 2024-09-18T04:58:45.404 INFO:teuthology.orchestra.run.smithi002.stdout: "recovery_deletes", 2024-09-18T04:58:45.404 INFO:teuthology.orchestra.run.smithi002.stdout: "sortbitwise" 2024-09-18T04:58:45.404 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:58:45.404 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_version": 4, 2024-09-18T04:58:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: "full_ratio": 0.94999998807907104, 2024-09-18T04:58:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-18T04:58:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-18T04:58:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_snapshot": "", 2024-09-18T04:58:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_max": 3, 2024-09-18T04:58:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: "max_osd": 8, 2024-09-18T04:58:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: "require_min_compat_client": "luminous", 2024-09-18T04:58:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: "min_compat_client": "luminous", 2024-09-18T04:58:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: "require_osd_release": "reef", 2024-09-18T04:58:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: "allow_crimson": false, 2024-09-18T04:58:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: "pools": [ 2024-09-18T04:58:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.405 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 1, 2024-09-18T04:58:45.406 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "rbd", 2024-09-18T04:58:45.406 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-18T04:56:08.983344+0000", 2024-09-18T04:58:45.406 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-18T04:58:45.406 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T04:58:45.406 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-18T04:58:45.406 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-18T04:58:45.406 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-18T04:58:45.406 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-18T04:58:45.406 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-18T04:58:45.407 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-18T04:58:45.407 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T04:58:45.407 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T04:58:45.407 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-18T04:58:45.407 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-18T04:58:45.407 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 8, 2024-09-18T04:58:45.408 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 8, 2024-09-18T04:58:45.408 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 8, 2024-09-18T04:58:45.408 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 8, 2024-09-18T04:58:45.408 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 8, 2024-09-18T04:58:45.408 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-18T04:58:45.408 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-18T04:58:45.409 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-18T04:58:45.409 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-18T04:58:45.409 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-18T04:58:45.409 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-18T04:58:45.409 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-18T04:58:45.409 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.409 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "17", 2024-09-18T04:58:45.409 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-18T04:58:45.409 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T04:58:45.409 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T04:58:45.409 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-18T04:58:45.409 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-18T04:58:45.410 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 3, 2024-09-18T04:58:45.410 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 17, 2024-09-18T04:58:45.410 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-18T04:58:45.410 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-18T04:58:45.411 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-18T04:58:45.411 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-18T04:58:45.411 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-18T04:58:45.411 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-18T04:58:45.411 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-18T04:58:45.412 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-18T04:58:45.412 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-18T04:58:45.412 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-18T04:58:45.412 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-18T04:58:45.412 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T04:58:45.412 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T04:58:45.413 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T04:58:45.413 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-18T04:58:45.413 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-18T04:58:45.413 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-18T04:58:45.413 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-18T04:58:45.414 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-18T04:58:45.414 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.414 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-18T04:58:45.414 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-18T04:58:45.414 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-18T04:58:45.414 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-18T04:58:45.415 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-18T04:58:45.415 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T04:58:45.416 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-18T04:58:45.416 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-18T04:58:45.416 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-18T04:58:45.416 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-18T04:58:45.416 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-18T04:58:45.416 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-18T04:58:45.416 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-18T04:58:45.417 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-18T04:58:45.417 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.417 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-18T04:58:45.417 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 3.0099999904632568, 2024-09-18T04:58:45.417 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 3.0099999904632568, 2024-09-18T04:58:45.417 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 1.3300000429153442, 2024-09-18T04:58:45.417 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 4, 2024-09-18T04:58:45.417 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 4, 2024-09-18T04:58:45.417 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 0.75, 2024-09-18T04:58:45.417 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-18T04:58:45.417 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 0.75 2024-09-18T04:58:45.417 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.418 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.418 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.418 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 2, 2024-09-18T04:58:45.418 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": ".mgr", 2024-09-18T04:58:45.418 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-18T04:56:10.313988+0000", 2024-09-18T04:58:45.418 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 1, 2024-09-18T04:58:45.418 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool", 2024-09-18T04:58:45.418 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-18T04:58:45.418 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-18T04:58:45.418 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-18T04:58:45.418 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-18T04:58:45.418 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-18T04:58:45.418 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-18T04:58:45.419 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T04:58:45.419 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T04:58:45.419 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-18T04:58:45.419 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-18T04:58:45.419 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 1, 2024-09-18T04:58:45.419 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 1, 2024-09-18T04:58:45.419 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 1, 2024-09-18T04:58:45.419 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 1, 2024-09-18T04:58:45.419 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 1, 2024-09-18T04:58:45.419 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-18T04:58:45.419 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-18T04:58:45.419 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-18T04:58:45.420 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-18T04:58:45.420 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-18T04:58:45.420 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-18T04:58:45.420 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-18T04:58:45.420 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.420 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "17", 2024-09-18T04:58:45.420 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-18T04:58:45.420 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T04:58:45.420 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T04:58:45.420 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-18T04:58:45.420 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-18T04:58:45.420 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 0, 2024-09-18T04:58:45.420 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 0, 2024-09-18T04:58:45.421 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-18T04:58:45.421 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-18T04:58:45.421 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-18T04:58:45.421 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-18T04:58:45.421 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-18T04:58:45.421 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-18T04:58:45.421 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-18T04:58:45.421 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-18T04:58:45.421 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-18T04:58:45.421 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-18T04:58:45.421 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-18T04:58:45.421 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T04:58:45.421 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T04:58:45.421 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T04:58:45.422 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-18T04:58:45.422 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-18T04:58:45.422 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-18T04:58:45.422 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-18T04:58:45.422 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-18T04:58:45.422 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.422 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-18T04:58:45.422 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-18T04:58:45.422 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-18T04:58:45.422 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-18T04:58:45.422 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-18T04:58:45.422 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T04:58:45.422 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-18T04:58:45.423 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-18T04:58:45.423 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-18T04:58:45.423 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-18T04:58:45.423 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-18T04:58:45.423 INFO:teuthology.orchestra.run.smithi002.stdout: "options": { 2024-09-18T04:58:45.423 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_max": 32, 2024-09-18T04:58:45.423 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_min": 1 2024-09-18T04:58:45.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.423 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-18T04:58:45.423 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": {} 2024-09-18T04:58:45.423 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.423 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-18T04:58:45.423 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 6.059999942779541, 2024-09-18T04:58:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 6.059999942779541, 2024-09-18T04:58:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: "optimal_score": 0.33000001311302185, 2024-09-18T04:58:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_acting": 2, 2024-09-18T04:58:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: "raw_score_stable": 2, 2024-09-18T04:58:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity_weighted": 0.75, 2024-09-18T04:58:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity": 1, 2024-09-18T04:58:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: "average_primary_affinity_weighted": 0.75 2024-09-18T04:58:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-18T04:58:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_name": "pool_client.0", 2024-09-18T04:58:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: "create_time": "2024-09-18T04:56:19.562579+0000", 2024-09-18T04:58:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: "flags": 8193, 2024-09-18T04:58:45.425 INFO:teuthology.orchestra.run.smithi002.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-18T04:58:45.425 INFO:teuthology.orchestra.run.smithi002.stdout: "type": 1, 2024-09-18T04:58:45.425 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 2, 2024-09-18T04:58:45.425 INFO:teuthology.orchestra.run.smithi002.stdout: "min_size": 1, 2024-09-18T04:58:45.425 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_rule": 0, 2024-09-18T04:58:45.425 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_count": 0, 2024-09-18T04:58:45.425 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_target": 0, 2024-09-18T04:58:45.425 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_barrier": 0, 2024-09-18T04:58:45.425 INFO:teuthology.orchestra.run.smithi002.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-18T04:58:45.425 INFO:teuthology.orchestra.run.smithi002.stdout: "object_hash": 2, 2024-09-18T04:58:45.425 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_autoscale_mode": "off", 2024-09-18T04:58:45.425 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num": 32, 2024-09-18T04:58:45.425 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num": 32, 2024-09-18T04:58:45.426 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_placement_num_target": 32, 2024-09-18T04:58:45.426 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_target": 32, 2024-09-18T04:58:45.426 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_num_pending": 32, 2024-09-18T04:58:45.426 INFO:teuthology.orchestra.run.smithi002.stdout: "last_pg_merge_meta": { 2024-09-18T04:58:45.426 INFO:teuthology.orchestra.run.smithi002.stdout: "source_pgid": "0.0", 2024-09-18T04:58:45.426 INFO:teuthology.orchestra.run.smithi002.stdout: "ready_epoch": 0, 2024-09-18T04:58:45.426 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_started": 0, 2024-09-18T04:58:45.426 INFO:teuthology.orchestra.run.smithi002.stdout: "last_epoch_clean": 0, 2024-09-18T04:58:45.426 INFO:teuthology.orchestra.run.smithi002.stdout: "source_version": "0'0", 2024-09-18T04:58:45.426 INFO:teuthology.orchestra.run.smithi002.stdout: "target_version": "0'0" 2024-09-18T04:58:45.426 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.426 INFO:teuthology.orchestra.run.smithi002.stdout: "last_change": "155", 2024-09-18T04:58:45.426 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend": "0", 2024-09-18T04:58:45.427 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-18T04:58:45.427 INFO:teuthology.orchestra.run.smithi002.stdout: "last_force_op_resend_preluminous": "0", 2024-09-18T04:58:45.427 INFO:teuthology.orchestra.run.smithi002.stdout: "auid": 0, 2024-09-18T04:58:45.427 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_mode": "selfmanaged", 2024-09-18T04:58:45.427 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_seq": 90, 2024-09-18T04:58:45.427 INFO:teuthology.orchestra.run.smithi002.stdout: "snap_epoch": 155, 2024-09-18T04:58:45.427 INFO:teuthology.orchestra.run.smithi002.stdout: "pool_snaps": [], 2024-09-18T04:58:45.427 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps": "[]", 2024-09-18T04:58:45.427 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_bytes": 0, 2024-09-18T04:58:45.427 INFO:teuthology.orchestra.run.smithi002.stdout: "quota_max_objects": 0, 2024-09-18T04:58:45.427 INFO:teuthology.orchestra.run.smithi002.stdout: "tiers": [], 2024-09-18T04:58:45.427 INFO:teuthology.orchestra.run.smithi002.stdout: "tier_of": -1, 2024-09-18T04:58:45.427 INFO:teuthology.orchestra.run.smithi002.stdout: "read_tier": -1, 2024-09-18T04:58:45.427 INFO:teuthology.orchestra.run.smithi002.stdout: "write_tier": -1, 2024-09-18T04:58:45.428 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_mode": "none", 2024-09-18T04:58:45.428 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_bytes": 0, 2024-09-18T04:58:45.428 INFO:teuthology.orchestra.run.smithi002.stdout: "target_max_objects": 0, 2024-09-18T04:58:45.428 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-18T04:58:45.428 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-18T04:58:45.428 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-18T04:58:45.428 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_flush_age": 0, 2024-09-18T04:58:45.428 INFO:teuthology.orchestra.run.smithi002.stdout: "cache_min_evict_age": 0, 2024-09-18T04:58:45.428 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profile": "", 2024-09-18T04:58:45.428 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_params": { 2024-09-18T04:58:45.428 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "none" 2024-09-18T04:58:45.428 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.428 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_period": 0, 2024-09-18T04:58:45.429 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_count": 0, 2024-09-18T04:58:45.429 INFO:teuthology.orchestra.run.smithi002.stdout: "use_gmt_hitset": true, 2024-09-18T04:58:45.429 INFO:teuthology.orchestra.run.smithi002.stdout: "min_read_recency_for_promote": 0, 2024-09-18T04:58:45.429 INFO:teuthology.orchestra.run.smithi002.stdout: "min_write_recency_for_promote": 0, 2024-09-18T04:58:45.429 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_grade_decay_rate": 0, 2024-09-18T04:58:45.429 INFO:teuthology.orchestra.run.smithi002.stdout: "hit_set_search_last_n": 0, 2024-09-18T04:58:45.429 INFO:teuthology.orchestra.run.smithi002.stdout: "grade_table": [], 2024-09-18T04:58:45.429 INFO:teuthology.orchestra.run.smithi002.stdout: "stripe_width": 0, 2024-09-18T04:58:45.429 INFO:teuthology.orchestra.run.smithi002.stdout: "expected_num_objects": 0, 2024-09-18T04:58:45.429 INFO:teuthology.orchestra.run.smithi002.stdout: "fast_read": false, 2024-09-18T04:58:45.429 INFO:teuthology.orchestra.run.smithi002.stdout: "options": {}, 2024-09-18T04:58:45.429 INFO:teuthology.orchestra.run.smithi002.stdout: "application_metadata": { 2024-09-18T04:58:45.429 INFO:teuthology.orchestra.run.smithi002.stdout: "rbd": {} 2024-09-18T04:58:45.430 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.430 INFO:teuthology.orchestra.run.smithi002.stdout: "read_balance": { 2024-09-18T04:58:45.430 INFO:teuthology.orchestra.run.smithi002.stdout: "error_message": "pool 3 has some PGs where all OSDs are with primary_affinity 0 (at least pg 3.6), read balance score may not be reliable", 2024-09-18T04:58:45.430 INFO:teuthology.orchestra.run.smithi002.stdout: "score_acting": 0, 2024-09-18T04:58:45.430 INFO:teuthology.orchestra.run.smithi002.stdout: "score_stable": 0 2024-09-18T04:58:45.430 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.430 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.430 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:58:45.430 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-18T04:58:45.430 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.430 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-18T04:58:45.430 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "c43f8024-3a3e-424e-aa54-398310ceb70d", 2024-09-18T04:58:45.430 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:58:45.430 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:58:45.431 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 0.95001220703125, 2024-09-18T04:58:45.431 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 0, 2024-09-18T04:58:45.431 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:58:45.431 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:58:45.431 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:58:45.431 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 150, 2024-09-18T04:58:45.431 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:58:45.431 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:58:45.431 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:58:45.431 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.431 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.431 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.431 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6824", 2024-09-18T04:58:45.431 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:58:45.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.432 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.432 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6825", 2024-09-18T04:58:45.432 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:58:45.432 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.432 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.432 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.432 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:58:45.432 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.432 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.432 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.432 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6826", 2024-09-18T04:58:45.432 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:58:45.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.433 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.433 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6827", 2024-09-18T04:58:45.433 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:58:45.433 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.433 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.433 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.433 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:58:45.433 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.433 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.434 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.434 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6830", 2024-09-18T04:58:45.434 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:58:45.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.434 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.434 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.434 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6831", 2024-09-18T04:58:45.434 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:58:45.434 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.434 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.434 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.434 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:58:45.435 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.435 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.435 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6828", 2024-09-18T04:58:45.435 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:58:45.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.435 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.435 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.435 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6829", 2024-09-18T04:58:45.435 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 4106704527 2024-09-18T04:58:45.435 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.435 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.435 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.435 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6825/4106704527", 2024-09-18T04:58:45.436 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6827/4106704527", 2024-09-18T04:58:45.436 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6831/4106704527", 2024-09-18T04:58:45.436 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6829/4106704527", 2024-09-18T04:58:45.436 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:58:45.436 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:58:45.436 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:58:45.436 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.436 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.436 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.436 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-18T04:58:45.436 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "c664b970-145c-4b6f-8852-2f6dcf1fa30d", 2024-09-18T04:58:45.436 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:58:45.436 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:58:45.437 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:58:45.437 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 0, 2024-09-18T04:58:45.437 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:58:45.437 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:58:45.437 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:58:45.437 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 148, 2024-09-18T04:58:45.437 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:58:45.437 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:58:45.437 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:58:45.437 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.437 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.437 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.437 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6808", 2024-09-18T04:58:45.437 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:58:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6809", 2024-09-18T04:58:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:58:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:58:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6810", 2024-09-18T04:58:45.438 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:58:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6811", 2024-09-18T04:58:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:58:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:58:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6814", 2024-09-18T04:58:45.439 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:58:45.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.440 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.440 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6815", 2024-09-18T04:58:45.440 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:58:45.440 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.440 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.440 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:58:45.440 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.440 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.440 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.440 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6812", 2024-09-18T04:58:45.440 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:58:45.440 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.441 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.441 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.441 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6813", 2024-09-18T04:58:45.441 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1781959259 2024-09-18T04:58:45.441 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.441 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.441 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.441 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6809/1781959259", 2024-09-18T04:58:45.441 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6811/1781959259", 2024-09-18T04:58:45.441 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6815/1781959259", 2024-09-18T04:58:45.441 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6813/1781959259", 2024-09-18T04:58:45.441 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:58:45.441 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:58:45.441 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:58:45.442 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.442 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.442 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.442 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-18T04:58:45.442 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "0609ea8c-d975-490e-8844-04a3e4512efd", 2024-09-18T04:58:45.442 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:58:45.442 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:58:45.442 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:58:45.442 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:58:45.442 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:58:45.442 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:58:45.442 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:58:45.442 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 150, 2024-09-18T04:58:45.442 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:58:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:58:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:58:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6800", 2024-09-18T04:58:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:58:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6801", 2024-09-18T04:58:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:58:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.443 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.444 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:58:45.444 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.444 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.444 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6802", 2024-09-18T04:58:45.444 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:58:45.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.444 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.444 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.444 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6803", 2024-09-18T04:58:45.444 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:58:45.444 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.444 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.444 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.445 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:58:45.445 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.445 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.445 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6806", 2024-09-18T04:58:45.445 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:58:45.445 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.445 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.445 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.445 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6807", 2024-09-18T04:58:45.445 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:58:45.445 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.446 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.446 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:58:45.446 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.446 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.446 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6804", 2024-09-18T04:58:45.446 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:58:45.446 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.446 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.446 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.446 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6805", 2024-09-18T04:58:45.446 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 1153570802 2024-09-18T04:58:45.447 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.447 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.447 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6801/1153570802", 2024-09-18T04:58:45.447 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6803/1153570802", 2024-09-18T04:58:45.447 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6807/1153570802", 2024-09-18T04:58:45.447 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6805/1153570802", 2024-09-18T04:58:45.447 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:58:45.447 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:58:45.447 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:58:45.447 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.447 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.447 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.447 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-18T04:58:45.448 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "c6bd5ac8-ab89-4fa2-b843-3b166d1b238b", 2024-09-18T04:58:45.448 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:58:45.448 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:58:45.448 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 0.95001220703125, 2024-09-18T04:58:45.448 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:58:45.448 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:58:45.448 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:58:45.448 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:58:45.448 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 150, 2024-09-18T04:58:45.448 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:58:45.448 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:58:45.448 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:58:45.448 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.448 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.449 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.449 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6816", 2024-09-18T04:58:45.449 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:58:45.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.449 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.449 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.449 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6817", 2024-09-18T04:58:45.449 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:58:45.449 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.449 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.449 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.449 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:58:45.449 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6818", 2024-09-18T04:58:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:58:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6819", 2024-09-18T04:58:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:58:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:58:45.450 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.451 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.451 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6822", 2024-09-18T04:58:45.451 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:58:45.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.451 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.451 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.451 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6823", 2024-09-18T04:58:45.451 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:58:45.451 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.451 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.451 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.451 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:58:45.452 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.452 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.452 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6820", 2024-09-18T04:58:45.452 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:58:45.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.452 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.452 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.452 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.2:6821", 2024-09-18T04:58:45.452 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2008321375 2024-09-18T04:58:45.452 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.452 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.452 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.453 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.2:6817/2008321375", 2024-09-18T04:58:45.453 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.2:6819/2008321375", 2024-09-18T04:58:45.453 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.2:6823/2008321375", 2024-09-18T04:58:45.453 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.2:6821/2008321375", 2024-09-18T04:58:45.453 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:58:45.453 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:58:45.453 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:58:45.453 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.453 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.453 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.453 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-18T04:58:45.453 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "89d6a371-4f4e-465e-a023-d5083cce6e6c", 2024-09-18T04:58:45.453 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:58:45.453 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:58:45.454 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:58:45.454 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:58:45.454 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:58:45.454 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:58:45.454 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:58:45.454 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 147, 2024-09-18T04:58:45.454 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:58:45.454 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:58:45.454 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:58:45.454 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.454 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.454 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.454 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6824", 2024-09-18T04:58:45.455 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:58:45.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.455 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.455 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.455 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6825", 2024-09-18T04:58:45.455 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:58:45.455 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.455 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.455 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.455 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:58:45.456 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.456 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.456 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6826", 2024-09-18T04:58:45.456 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:58:45.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.456 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.456 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.456 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6827", 2024-09-18T04:58:45.456 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:58:45.456 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.456 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.456 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.457 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:58:45.457 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.457 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.457 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6830", 2024-09-18T04:58:45.457 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:58:45.457 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.457 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.457 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.457 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6831", 2024-09-18T04:58:45.458 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:58:45.458 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.458 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.458 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:58:45.458 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.458 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.458 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6828", 2024-09-18T04:58:45.458 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:58:45.458 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.458 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.458 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.459 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6829", 2024-09-18T04:58:45.459 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2939896598 2024-09-18T04:58:45.459 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.459 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.459 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.459 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6825/2939896598", 2024-09-18T04:58:45.459 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6827/2939896598", 2024-09-18T04:58:45.459 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6831/2939896598", 2024-09-18T04:58:45.459 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6829/2939896598", 2024-09-18T04:58:45.459 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:58:45.460 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:58:45.460 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:58:45.460 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.460 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.460 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.460 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-18T04:58:45.460 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "da99686b-d979-4753-b4aa-a3b26e9a0497", 2024-09-18T04:58:45.460 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:58:45.460 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:58:45.460 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 0.95001220703125, 2024-09-18T04:58:45.460 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:58:45.460 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:58:45.461 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:58:45.461 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:58:45.461 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 142, 2024-09-18T04:58:45.461 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:58:45.461 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:58:45.461 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:58:45.461 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.461 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.461 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.461 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6816", 2024-09-18T04:58:45.462 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:58:45.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.462 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.462 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6817", 2024-09-18T04:58:45.462 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:58:45.462 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.462 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.462 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.462 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:58:45.462 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.462 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.462 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.463 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6818", 2024-09-18T04:58:45.463 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:58:45.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.463 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.463 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.463 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6819", 2024-09-18T04:58:45.463 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:58:45.463 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.463 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.463 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.463 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:58:45.464 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.464 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.464 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6822", 2024-09-18T04:58:45.464 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:58:45.464 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.464 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.464 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.464 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6823", 2024-09-18T04:58:45.464 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:58:45.465 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.465 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.465 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:58:45.465 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.465 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.465 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.465 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6820", 2024-09-18T04:58:45.465 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:58:45.465 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.466 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.466 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.466 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6821", 2024-09-18T04:58:45.466 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 3440879618 2024-09-18T04:58:45.466 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.466 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.466 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.466 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6817/3440879618", 2024-09-18T04:58:45.466 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6819/3440879618", 2024-09-18T04:58:45.467 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6823/3440879618", 2024-09-18T04:58:45.467 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6821/3440879618", 2024-09-18T04:58:45.467 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:58:45.467 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:58:45.467 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:58:45.467 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.467 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.467 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.467 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-18T04:58:45.467 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "ceec6aa6-603d-4706-bfea-3c07dbdaed3a", 2024-09-18T04:58:45.468 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:58:45.468 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:58:45.468 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:58:45.468 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:58:45.468 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:58:45.468 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:58:45.468 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:58:45.468 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 150, 2024-09-18T04:58:45.468 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:58:45.468 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:58:45.468 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:58:45.468 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.468 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.469 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.469 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6800", 2024-09-18T04:58:45.469 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:58:45.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.469 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.469 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.469 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6801", 2024-09-18T04:58:45.469 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:58:45.469 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.469 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.469 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.469 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:58:45.470 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.470 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.470 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6802", 2024-09-18T04:58:45.470 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:58:45.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.470 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.470 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.470 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6803", 2024-09-18T04:58:45.470 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:58:45.470 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.470 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.470 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.470 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:58:45.471 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.471 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.471 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6806", 2024-09-18T04:58:45.471 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:58:45.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.471 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.471 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.471 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6807", 2024-09-18T04:58:45.471 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:58:45.471 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.471 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.471 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.471 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:58:45.472 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.472 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.472 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6804", 2024-09-18T04:58:45.472 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:58:45.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.472 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.472 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.472 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6805", 2024-09-18T04:58:45.472 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 2493830504 2024-09-18T04:58:45.472 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.472 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.472 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.472 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6801/2493830504", 2024-09-18T04:58:45.473 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6803/2493830504", 2024-09-18T04:58:45.473 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6807/2493830504", 2024-09-18T04:58:45.473 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6805/2493830504", 2024-09-18T04:58:45.473 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:58:45.473 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:58:45.473 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:58:45.473 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.473 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.473 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.473 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-18T04:58:45.473 INFO:teuthology.orchestra.run.smithi002.stdout: "uuid": "6779dfad-f60b-41f4-b666-a99db6dc1814", 2024-09-18T04:58:45.473 INFO:teuthology.orchestra.run.smithi002.stdout: "up": 1, 2024-09-18T04:58:45.473 INFO:teuthology.orchestra.run.smithi002.stdout: "in": 1, 2024-09-18T04:58:45.474 INFO:teuthology.orchestra.run.smithi002.stdout: "weight": 1, 2024-09-18T04:58:45.474 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_affinity": 1, 2024-09-18T04:58:45.474 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_begin": 0, 2024-09-18T04:58:45.474 INFO:teuthology.orchestra.run.smithi002.stdout: "last_clean_end": 0, 2024-09-18T04:58:45.474 INFO:teuthology.orchestra.run.smithi002.stdout: "up_from": 13, 2024-09-18T04:58:45.474 INFO:teuthology.orchestra.run.smithi002.stdout: "up_thru": 147, 2024-09-18T04:58:45.474 INFO:teuthology.orchestra.run.smithi002.stdout: "down_at": 0, 2024-09-18T04:58:45.474 INFO:teuthology.orchestra.run.smithi002.stdout: "lost_at": 0, 2024-09-18T04:58:45.474 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addrs": { 2024-09-18T04:58:45.474 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.474 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.474 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.474 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6808", 2024-09-18T04:58:45.474 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:58:45.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.475 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.475 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6809", 2024-09-18T04:58:45.475 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:58:45.475 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.475 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.475 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.475 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addrs": { 2024-09-18T04:58:45.475 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.475 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.475 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.475 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6810", 2024-09-18T04:58:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:58:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6811", 2024-09-18T04:58:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:58:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addrs": { 2024-09-18T04:58:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6814", 2024-09-18T04:58:45.476 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:58:45.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.477 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.477 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6815", 2024-09-18T04:58:45.477 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:58:45.477 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.477 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.477 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.477 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addrs": { 2024-09-18T04:58:45.477 INFO:teuthology.orchestra.run.smithi002.stdout: "addrvec": [ 2024-09-18T04:58:45.477 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.477 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v2", 2024-09-18T04:58:45.477 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6812", 2024-09-18T04:58:45.477 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:58:45.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.478 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.478 INFO:teuthology.orchestra.run.smithi002.stdout: "type": "v1", 2024-09-18T04:58:45.478 INFO:teuthology.orchestra.run.smithi002.stdout: "addr": "172.21.15.99:6813", 2024-09-18T04:58:45.478 INFO:teuthology.orchestra.run.smithi002.stdout: "nonce": 91038858 2024-09-18T04:58:45.478 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.478 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.478 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.478 INFO:teuthology.orchestra.run.smithi002.stdout: "public_addr": "172.21.15.99:6809/91038858", 2024-09-18T04:58:45.478 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_addr": "172.21.15.99:6811/91038858", 2024-09-18T04:58:45.478 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_back_addr": "172.21.15.99:6815/91038858", 2024-09-18T04:58:45.478 INFO:teuthology.orchestra.run.smithi002.stdout: "heartbeat_front_addr": "172.21.15.99:6813/91038858", 2024-09-18T04:58:45.478 INFO:teuthology.orchestra.run.smithi002.stdout: "state": [ 2024-09-18T04:58:45.478 INFO:teuthology.orchestra.run.smithi002.stdout: "exists", 2024-09-18T04:58:45.479 INFO:teuthology.orchestra.run.smithi002.stdout: "up" 2024-09-18T04:58:45.479 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.479 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.479 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:58:45.479 INFO:teuthology.orchestra.run.smithi002.stdout: "osd_xinfo": [ 2024-09-18T04:58:45.479 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.479 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 0, 2024-09-18T04:58:45.479 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:58:45.479 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:58:45.479 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:58:45.479 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:58:45.479 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:58:45.479 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.468213+0000", 2024-09-18T04:58:45.479 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:58:45.479 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.480 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 1, 2024-09-18T04:58:45.480 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:58:45.480 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:58:45.480 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:58:45.480 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:58:45.480 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:58:45.480 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.538610+0000", 2024-09-18T04:58:45.480 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:58:45.480 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.480 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.480 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 2, 2024-09-18T04:58:45.480 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:58:45.481 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:58:45.481 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:58:45.481 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:58:45.481 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:58:45.481 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.500881+0000", 2024-09-18T04:58:45.481 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:58:45.481 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.481 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.481 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 3, 2024-09-18T04:58:45.481 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:58:45.481 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:58:45.481 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:58:45.481 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:58:45.481 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:58:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.474179+0000", 2024-09-18T04:58:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:58:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 4, 2024-09-18T04:58:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:58:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:58:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:58:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:58:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:58:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.483737+0000", 2024-09-18T04:58:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:58:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.482 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 5, 2024-09-18T04:58:45.483 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:58:45.483 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:58:45.483 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:58:45.483 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:58:45.483 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:58:45.483 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.460693+0000", 2024-09-18T04:58:45.483 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:58:45.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.483 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 6, 2024-09-18T04:58:45.483 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:58:45.483 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:58:45.483 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:58:45.483 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:58:45.484 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:58:45.484 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.475970+0000", 2024-09-18T04:58:45.484 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:58:45.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.484 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.484 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": 7, 2024-09-18T04:58:45.484 INFO:teuthology.orchestra.run.smithi002.stdout: "down_stamp": "0.000000", 2024-09-18T04:58:45.484 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_probability": 0, 2024-09-18T04:58:45.484 INFO:teuthology.orchestra.run.smithi002.stdout: "laggy_interval": 0, 2024-09-18T04:58:45.484 INFO:teuthology.orchestra.run.smithi002.stdout: "features": 4540138322906710015, 2024-09-18T04:58:45.484 INFO:teuthology.orchestra.run.smithi002.stdout: "old_weight": 0, 2024-09-18T04:58:45.484 INFO:teuthology.orchestra.run.smithi002.stdout: "last_purged_snaps_scrub": "2024-09-18T04:56:05.519287+0000", 2024-09-18T04:58:45.484 INFO:teuthology.orchestra.run.smithi002.stdout: "dead_epoch": 0 2024-09-18T04:58:45.485 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.485 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:58:45.485 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap": [ 2024-09-18T04:58:45.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.485 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "1.4", 2024-09-18T04:58:45.485 INFO:teuthology.orchestra.run.smithi002.stdout: "osds": [ 2024-09-18T04:58:45.485 INFO:teuthology.orchestra.run.smithi002.stdout: 6, 2024-09-18T04:58:45.485 INFO:teuthology.orchestra.run.smithi002.stdout: 7 2024-09-18T04:58:45.485 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.485 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.485 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:58:45.485 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_items": [ 2024-09-18T04:58:45.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.486 INFO:teuthology.orchestra.run.smithi002.stdout: "pgid": "3.d", 2024-09-18T04:58:45.486 INFO:teuthology.orchestra.run.smithi002.stdout: "mappings": [ 2024-09-18T04:58:45.486 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.486 INFO:teuthology.orchestra.run.smithi002.stdout: "from": 7, 2024-09-18T04:58:45.486 INFO:teuthology.orchestra.run.smithi002.stdout: "to": 2 2024-09-18T04:58:45.486 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.486 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.486 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.486 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:58:45.486 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_upmap_primaries": [], 2024-09-18T04:58:45.486 INFO:teuthology.orchestra.run.smithi002.stdout: "pg_temp": [], 2024-09-18T04:58:45.486 INFO:teuthology.orchestra.run.smithi002.stdout: "primary_temp": [], 2024-09-18T04:58:45.486 INFO:teuthology.orchestra.run.smithi002.stdout: "blocklist": {}, 2024-09-18T04:58:45.486 INFO:teuthology.orchestra.run.smithi002.stdout: "range_blocklist": {}, 2024-09-18T04:58:45.487 INFO:teuthology.orchestra.run.smithi002.stdout: "erasure_code_profiles": { 2024-09-18T04:58:45.487 INFO:teuthology.orchestra.run.smithi002.stdout: "default": { 2024-09-18T04:58:45.487 INFO:teuthology.orchestra.run.smithi002.stdout: "crush-failure-domain": "osd", 2024-09-18T04:58:45.487 INFO:teuthology.orchestra.run.smithi002.stdout: "k": "2", 2024-09-18T04:58:45.487 INFO:teuthology.orchestra.run.smithi002.stdout: "m": "1", 2024-09-18T04:58:45.487 INFO:teuthology.orchestra.run.smithi002.stdout: "plugin": "jerasure", 2024-09-18T04:58:45.487 INFO:teuthology.orchestra.run.smithi002.stdout: "technique": "reed_sol_van" 2024-09-18T04:58:45.487 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.487 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:45.487 INFO:teuthology.orchestra.run.smithi002.stdout: "removed_snaps_queue": [ 2024-09-18T04:58:45.487 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.487 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-18T04:58:45.487 INFO:teuthology.orchestra.run.smithi002.stdout: "snaps": [ 2024-09-18T04:58:45.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.488 INFO:teuthology.orchestra.run.smithi002.stdout: "begin": 7, 2024-09-18T04:58:45.488 INFO:teuthology.orchestra.run.smithi002.stdout: "length": 5 2024-09-18T04:58:45.488 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.488 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.488 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.488 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:58:45.488 INFO:teuthology.orchestra.run.smithi002.stdout: "new_removed_snaps": [ 2024-09-18T04:58:45.488 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.488 INFO:teuthology.orchestra.run.smithi002.stdout: "pool": 3, 2024-09-18T04:58:45.489 INFO:teuthology.orchestra.run.smithi002.stdout: "snaps": [ 2024-09-18T04:58:45.489 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:45.489 INFO:teuthology.orchestra.run.smithi002.stdout: "begin": 7, 2024-09-18T04:58:45.489 INFO:teuthology.orchestra.run.smithi002.stdout: "length": 1 2024-09-18T04:58:45.489 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.489 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:45.489 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.489 INFO:teuthology.orchestra.run.smithi002.stdout: ], 2024-09-18T04:58:45.489 INFO:teuthology.orchestra.run.smithi002.stdout: "new_purged_snaps": [], 2024-09-18T04:58:45.489 INFO:teuthology.orchestra.run.smithi002.stdout: "crush_node_flags": {}, 2024-09-18T04:58:45.489 INFO:teuthology.orchestra.run.smithi002.stdout: "device_class_flags": {}, 2024-09-18T04:58:45.489 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode": { 2024-09-18T04:58:45.489 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_enabled": false, 2024-09-18T04:58:45.489 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_bucket_count": 0, 2024-09-18T04:58:45.489 INFO:teuthology.orchestra.run.smithi002.stdout: "degraded_stretch_mode": 0, 2024-09-18T04:58:45.490 INFO:teuthology.orchestra.run.smithi002.stdout: "recovering_stretch_mode": 0, 2024-09-18T04:58:45.490 INFO:teuthology.orchestra.run.smithi002.stdout: "stretch_mode_bucket": 0 2024-09-18T04:58:45.490 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:45.490 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:45.493 INFO:tasks.thrashosds.thrasher:j is {'epoch': 155, 'fsid': 'c0761d0c-69d5-4b64-b8b9-c7d098b2d06d', 'created': '2024-09-18T04:55:58.216538+0000', 'modified': '2024-09-18T04:58:45.087642+0000', 'last_up_change': '2024-09-18T04:56:08.369924+0000', 'last_in_change': '2024-09-18T04:58:34.032750+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'reef', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-18T04:56:08.983344+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 17, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 3.009999990463257, 'score_stable': 3.009999990463257, 'optimal_score': 1.3300000429153442, 'raw_score_acting': 4, 'raw_score_stable': 4, 'primary_affinity_weighted': 0.75, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.75}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-09-18T04:56:10.313988+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.75, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.75}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-09-18T04:56:19.562579+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 90, 'snap_epoch': 155, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'error_message': 'pool 3 has some PGs where all OSDs are with primary_affinity 0 (at least pg 3.6), read balance score may not be reliable', 'score_acting': 0, 'score_stable': 0}}], 'osds': [{'osd': 0, 'uuid': 'c43f8024-3a3e-424e-aa54-398310ceb70d', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 150, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6824', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6825', 'nonce': 4106704527}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6826', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6827', 'nonce': 4106704527}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6830', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6831', 'nonce': 4106704527}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6828', 'nonce': 4106704527}, {'type': 'v1', 'addr': '172.21.15.2:6829', 'nonce': 4106704527}]}, 'public_addr': '172.21.15.2:6825/4106704527', 'cluster_addr': '172.21.15.2:6827/4106704527', 'heartbeat_back_addr': '172.21.15.2:6831/4106704527', 'heartbeat_front_addr': '172.21.15.2:6829/4106704527', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c664b970-145c-4b6f-8852-2f6dcf1fa30d', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 148, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6808', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6809', 'nonce': 1781959259}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6810', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6811', 'nonce': 1781959259}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6814', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6815', 'nonce': 1781959259}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6812', 'nonce': 1781959259}, {'type': 'v1', 'addr': '172.21.15.2:6813', 'nonce': 1781959259}]}, 'public_addr': '172.21.15.2:6809/1781959259', 'cluster_addr': '172.21.15.2:6811/1781959259', 'heartbeat_back_addr': '172.21.15.2:6815/1781959259', 'heartbeat_front_addr': '172.21.15.2:6813/1781959259', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '0609ea8c-d975-490e-8844-04a3e4512efd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 150, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6800', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6801', 'nonce': 1153570802}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6802', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6803', 'nonce': 1153570802}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6806', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6807', 'nonce': 1153570802}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6804', 'nonce': 1153570802}, {'type': 'v1', 'addr': '172.21.15.2:6805', 'nonce': 1153570802}]}, 'public_addr': '172.21.15.2:6801/1153570802', 'cluster_addr': '172.21.15.2:6803/1153570802', 'heartbeat_back_addr': '172.21.15.2:6807/1153570802', 'heartbeat_front_addr': '172.21.15.2:6805/1153570802', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'c6bd5ac8-ab89-4fa2-b843-3b166d1b238b', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 150, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6816', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6817', 'nonce': 2008321375}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6818', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6819', 'nonce': 2008321375}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6822', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6823', 'nonce': 2008321375}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6820', 'nonce': 2008321375}, {'type': 'v1', 'addr': '172.21.15.2:6821', 'nonce': 2008321375}]}, 'public_addr': '172.21.15.2:6817/2008321375', 'cluster_addr': '172.21.15.2:6819/2008321375', 'heartbeat_back_addr': '172.21.15.2:6823/2008321375', 'heartbeat_front_addr': '172.21.15.2:6821/2008321375', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '89d6a371-4f4e-465e-a023-d5083cce6e6c', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 147, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6824', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6825', 'nonce': 2939896598}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6826', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6827', 'nonce': 2939896598}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6830', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6831', 'nonce': 2939896598}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6828', 'nonce': 2939896598}, {'type': 'v1', 'addr': '172.21.15.99:6829', 'nonce': 2939896598}]}, 'public_addr': '172.21.15.99:6825/2939896598', 'cluster_addr': '172.21.15.99:6827/2939896598', 'heartbeat_back_addr': '172.21.15.99:6831/2939896598', 'heartbeat_front_addr': '172.21.15.99:6829/2939896598', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': 'da99686b-d979-4753-b4aa-a3b26e9a0497', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 142, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6816', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6817', 'nonce': 3440879618}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6818', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6819', 'nonce': 3440879618}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6822', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6823', 'nonce': 3440879618}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6820', 'nonce': 3440879618}, {'type': 'v1', 'addr': '172.21.15.99:6821', 'nonce': 3440879618}]}, 'public_addr': '172.21.15.99:6817/3440879618', 'cluster_addr': '172.21.15.99:6819/3440879618', 'heartbeat_back_addr': '172.21.15.99:6823/3440879618', 'heartbeat_front_addr': '172.21.15.99:6821/3440879618', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'ceec6aa6-603d-4706-bfea-3c07dbdaed3a', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 150, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6800', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6801', 'nonce': 2493830504}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6802', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6803', 'nonce': 2493830504}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6806', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6807', 'nonce': 2493830504}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6804', 'nonce': 2493830504}, {'type': 'v1', 'addr': '172.21.15.99:6805', 'nonce': 2493830504}]}, 'public_addr': '172.21.15.99:6801/2493830504', 'cluster_addr': '172.21.15.99:6803/2493830504', 'heartbeat_back_addr': '172.21.15.99:6807/2493830504', 'heartbeat_front_addr': '172.21.15.99:6805/2493830504', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '6779dfad-f60b-41f4-b666-a99db6dc1814', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 147, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6808', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6809', 'nonce': 91038858}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6810', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6811', 'nonce': 91038858}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6814', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6815', 'nonce': 91038858}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.99:6812', 'nonce': 91038858}, {'type': 'v1', 'addr': '172.21.15.99:6813', 'nonce': 91038858}]}, 'public_addr': '172.21.15.99:6809/91038858', 'cluster_addr': '172.21.15.99:6811/91038858', 'heartbeat_back_addr': '172.21.15.99:6815/91038858', 'heartbeat_front_addr': '172.21.15.99:6813/91038858', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.468213+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.538610+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.500881+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.474179+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.483737+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.460693+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.475970+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138322906710015, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-18T04:56:05.519287+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '1.4', 'osds': [6, 7]}], 'pg_upmap_items': [{'pgid': '3.d', 'mappings': [{'from': 7, 'to': 2}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 7, 'length': 5}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 7, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-18T04:58:45.493 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T04:58:45.494 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:45.494 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:45.494 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:58:45.495 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:45.489+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:45.592 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:45.590+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:45.600 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:45.600 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:45.609 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:58:45.643 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-18T04:58:45.693 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:45.690+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:45.712 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:45.712 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:45.721 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:58:45.766 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:58:45.767 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:58:45.784 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:58:45.785 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:58:45.800 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:58:45.800 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:58:45.805 INFO:tasks.ceph.osd.5.smithi099.stderr:2024-09-18T04:58:45.803+0000 7f3c25f62640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 33000) UID: 0 2024-09-18T04:58:45.816 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:58:45.816 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:58:45.825 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:58:45.826 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-18T04:58:45.833 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:58:45.833 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:58:45.838 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":226,"stamp":"2024-09-18T04:58:45.093151+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":130583193,"num_objects":155,"num_object_clones":24,"num_object_copies":310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":155,"num_whiteouts":59,"num_read":220064,"num_read_kb":682581,"num_write":29870,"num_write_kb":380157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8539,"num_bytes_recovered":185732385,"num_keys_recovered":562,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23826,"ondisk_log_size":23826,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":432948,"kb_used_data":17344,"kb_used_omap":12,"kb_used_meta":415347,"kb_avail":754541772,"statfs":{"total":773094113280,"available":772650774528,"internally_reserved":0,"allocated":17760256,"data_stored":31947699,"data_compressed":115975,"data_compressed_allocated":827392,"data_compressed_original":22568876,"omap_allocated":12742,"internal_metadata":425315898},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-114134262,"num_objects":-74,"num_object_clones":-24,"num_object_copies":-148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-74,"num_whiteouts":-12,"num_read":1085,"num_read_kb":927,"num_write":179,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.021511"},"pg_stats":[{"pgid":"3.1f","version":"146'764","reported_seq":5110,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.089801+0000","last_change":"2024-09-18T04:58:44.090115+0000","last_active":"2024-09-18T04:58:44.089801+0000","last_peered":"2024-09-18T04:58:44.089801+0000","last_clean":"2024-09-18T04:58:44.089801+0000","last_became_active":"2024-09-18T04:57:24.397820+0000","last_became_peered":"2024-09-18T04:57:24.397820+0000","last_unstale":"2024-09-18T04:58:44.089801+0000","last_undegraded":"2024-09-18T04:58:44.089801+0000","last_fullsized":"2024-09-18T04:58:44.089801+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":764,"log_dups_size":0,"ondisk_log_size":764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:39:52.529276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.005402993,"stat_sum":{"num_bytes":262160,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":6345,"num_read_kb":20851,"num_write":868,"num_write_kb":9417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"3.1e","version":"154'806","reported_seq":2728,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.091660+0000","last_change":"2024-09-18T04:58:44.091660+0000","last_active":"2024-09-18T04:58:44.091660+0000","last_peered":"2024-09-18T04:58:44.091660+0000","last_clean":"2024-09-18T04:58:44.091660+0000","last_became_active":"2024-09-18T04:58:32.029310+0000","last_became_peered":"2024-09-18T04:58:32.029310+0000","last_unstale":"2024-09-18T04:58:44.091660+0000","last_undegraded":"2024-09-18T04:58:44.091660+0000","last_fullsized":"2024-09-18T04:58:44.091660+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":806,"log_dups_size":0,"ondisk_log_size":806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:13:02.688292+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0068936420000000002,"stat_sum":{"num_bytes":65566,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":3332,"num_read_kb":18009,"num_write":1110,"num_write_kb":8161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":3252478,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"3.1d","version":"154'497","reported_seq":10715,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.093559+0000","last_change":"2024-09-18T04:58:44.093559+0000","last_active":"2024-09-18T04:58:44.093559+0000","last_peered":"2024-09-18T04:58:44.093559+0000","last_clean":"2024-09-18T04:58:44.093559+0000","last_became_active":"2024-09-18T04:58:27.115104+0000","last_became_peered":"2024-09-18T04:58:27.115104+0000","last_unstale":"2024-09-18T04:58:44.093559+0000","last_undegraded":"2024-09-18T04:58:44.093559+0000","last_fullsized":"2024-09-18T04:58:44.093559+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:33:08.572507+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0088643060000000006,"stat_sum":{"num_bytes":10485760,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":9761,"num_read_kb":28168,"num_write":547,"num_write_kb":23097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7293282,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"3.1c","version":"154'834","reported_seq":7921,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.120086+0000","last_change":"2024-09-18T04:58:44.085088+0000","last_active":"2024-09-18T04:58:44.120086+0000","last_peered":"2024-09-18T04:58:44.120086+0000","last_clean":"2024-09-18T04:58:44.120086+0000","last_became_active":"2024-09-18T04:58:28.028905+0000","last_became_peered":"2024-09-18T04:58:28.028905+0000","last_unstale":"2024-09-18T04:58:44.120086+0000","last_undegraded":"2024-09-18T04:58:44.120086+0000","last_fullsized":"2024-09-18T04:58:44.120086+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":834,"log_dups_size":0,"ondisk_log_size":834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:15:17.569793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042698600000000002,"stat_sum":{"num_bytes":196682,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":8568,"num_read_kb":9060,"num_write":1159,"num_write_kb":5309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":1412856,"num_keys_recovered":106,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"3.1b","version":"154'471","reported_seq":3894,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.190656+0000","last_change":"2024-09-18T04:58:44.096481+0000","last_active":"2024-09-18T04:58:44.190656+0000","last_peered":"2024-09-18T04:58:44.190656+0000","last_clean":"2024-09-18T04:58:44.190656+0000","last_became_active":"2024-09-18T04:58:31.031695+0000","last_became_peered":"2024-09-18T04:58:31.031695+0000","last_unstale":"2024-09-18T04:58:44.190656+0000","last_undegraded":"2024-09-18T04:58:44.190656+0000","last_fullsized":"2024-09-18T04:58:44.190656+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":471,"log_dups_size":0,"ondisk_log_size":471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:07:39.967715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024008190000000002,"stat_sum":{"num_bytes":185099,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":3,"num_read":2677,"num_read_kb":15514,"num_write":501,"num_write_kb":9112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":608,"num_bytes_recovered":7005168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"3.1a","version":"154'453","reported_seq":3892,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.174764+0000","last_change":"2024-09-18T04:58:44.096238+0000","last_active":"2024-09-18T04:58:44.174764+0000","last_peered":"2024-09-18T04:58:44.174764+0000","last_clean":"2024-09-18T04:58:44.174764+0000","last_became_active":"2024-09-18T04:58:38.023634+0000","last_became_peered":"2024-09-18T04:58:38.023634+0000","last_unstale":"2024-09-18T04:58:44.174764+0000","last_undegraded":"2024-09-18T04:58:44.174764+0000","last_fullsized":"2024-09-18T04:58:44.174764+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:42:06.464495+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.002278254,"stat_sum":{"num_bytes":167252,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":3,"num_read":2967,"num_read_kb":11726,"num_write":512,"num_write_kb":18285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":403,"num_bytes_recovered":11494081,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"3.19","version":"148'496","reported_seq":5351,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.085460+0000","last_change":"2024-09-18T04:58:44.085460+0000","last_active":"2024-09-18T04:58:44.085460+0000","last_peered":"2024-09-18T04:58:44.085460+0000","last_clean":"2024-09-18T04:58:44.085460+0000","last_became_active":"2024-09-18T04:58:41.077052+0000","last_became_peered":"2024-09-18T04:58:41.077052+0000","last_unstale":"2024-09-18T04:58:44.085460+0000","last_undegraded":"2024-09-18T04:58:44.085460+0000","last_fullsized":"2024-09-18T04:58:44.085460+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:42:28.597809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075927199999999996,"stat_sum":{"num_bytes":283237,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":2,"num_read":4375,"num_read_kb":26418,"num_write":519,"num_write_kb":13063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":23811007,"num_keys_recovered":60,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"3.18","version":"152'564","reported_seq":3370,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.085726+0000","last_change":"2024-09-18T04:58:44.086316+0000","last_active":"2024-09-18T04:58:44.085726+0000","last_peered":"2024-09-18T04:58:44.085726+0000","last_clean":"2024-09-18T04:58:44.085726+0000","last_became_active":"2024-09-18T04:58:32.029074+0000","last_became_peered":"2024-09-18T04:58:32.029074+0000","last_unstale":"2024-09-18T04:58:44.085726+0000","last_undegraded":"2024-09-18T04:58:44.085726+0000","last_fullsized":"2024-09-18T04:58:44.085726+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":564,"log_dups_size":0,"ondisk_log_size":564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:15:38.950060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014771879999999999,"stat_sum":{"num_bytes":37,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":6,"num_read":3372,"num_read_kb":16934,"num_write":686,"num_write_kb":12982,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":4650417,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"3.17","version":"153'506","reported_seq":4224,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.090959+0000","last_change":"2024-09-18T04:58:44.091396+0000","last_active":"2024-09-18T04:58:44.090959+0000","last_peered":"2024-09-18T04:58:44.090959+0000","last_clean":"2024-09-18T04:58:44.090959+0000","last_became_active":"2024-09-18T04:57:24.397890+0000","last_became_peered":"2024-09-18T04:57:24.397890+0000","last_unstale":"2024-09-18T04:58:44.090959+0000","last_undegraded":"2024-09-18T04:58:44.090959+0000","last_fullsized":"2024-09-18T04:58:44.090959+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:00:33.151325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.006528196,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4899,"num_read_kb":13699,"num_write":541,"num_write_kb":6077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"3.16","version":"154'885","reported_seq":5001,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.089627+0000","last_change":"2024-09-18T04:58:44.090230+0000","last_active":"2024-09-18T04:58:44.089627+0000","last_peered":"2024-09-18T04:58:44.089627+0000","last_clean":"2024-09-18T04:58:44.089627+0000","last_became_active":"2024-09-18T04:56:21.131133+0000","last_became_peered":"2024-09-18T04:56:21.131133+0000","last_unstale":"2024-09-18T04:58:44.089627+0000","last_undegraded":"2024-09-18T04:58:44.089627+0000","last_fullsized":"2024-09-18T04:58:44.089627+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:01:57.036904+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0055370829999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6340,"num_read_kb":26579,"num_write":1141,"num_write_kb":12621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"3.15","version":"153'628","reported_seq":5242,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.090014+0000","last_change":"2024-09-18T04:58:44.090335+0000","last_active":"2024-09-18T04:58:44.090014+0000","last_peered":"2024-09-18T04:58:44.090014+0000","last_clean":"2024-09-18T04:58:44.090014+0000","last_became_active":"2024-09-18T04:58:32.032762+0000","last_became_peered":"2024-09-18T04:58:32.032762+0000","last_unstale":"2024-09-18T04:58:44.090014+0000","last_undegraded":"2024-09-18T04:58:44.090014+0000","last_fullsized":"2024-09-18T04:58:44.090014+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:33:25.010425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016431200000000001,"stat_sum":{"num_bytes":21084894,"num_objects":10,"num_object_clones":3,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":4,"num_read":5290,"num_read_kb":26012,"num_write":593,"num_write_kb":7300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":447,"num_bytes_recovered":1594801,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"3.14","version":"153'500","reported_seq":2777,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.096529+0000","last_change":"2024-09-18T04:58:44.096755+0000","last_active":"2024-09-18T04:58:44.096529+0000","last_peered":"2024-09-18T04:58:44.096529+0000","last_clean":"2024-09-18T04:58:44.096529+0000","last_became_active":"2024-09-18T04:58:30.025558+0000","last_became_peered":"2024-09-18T04:58:30.025558+0000","last_unstale":"2024-09-18T04:58:44.096529+0000","last_undegraded":"2024-09-18T04:58:44.096529+0000","last_fullsized":"2024-09-18T04:58:44.096529+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:57:31.113016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00245794,"stat_sum":{"num_bytes":429135,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3131,"num_read_kb":27325,"num_write":607,"num_write_kb":11672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":12427258,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"1.7","version":"0'0","reported_seq":170,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-18T04:58:43.022255+0000","last_change":"2024-09-18T04:58:29.025271+0000","last_active":"2024-09-18T04:58:43.022255+0000","last_peered":"2024-09-18T04:58:43.022255+0000","last_clean":"2024-09-18T04:58:43.022255+0000","last_became_active":"2024-09-18T04:58:29.024911+0000","last_became_peered":"2024-09-18T04:58:29.024911+0000","last_unstale":"2024-09-18T04:58:43.022255+0000","last_undegraded":"2024-09-18T04:58:43.022255+0000","last_fullsized":"2024-09-18T04:58:43.022255+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:53:21.863837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063436799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"152'2479","reported_seq":9858,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.171367+0000","last_change":"2024-09-18T04:58:44.090895+0000","last_active":"2024-09-18T04:58:44.171367+0000","last_peered":"2024-09-18T04:58:44.171367+0000","last_clean":"2024-09-18T04:58:44.171367+0000","last_became_active":"2024-09-18T04:58:33.027340+0000","last_became_peered":"2024-09-18T04:58:33.027340+0000","last_unstale":"2024-09-18T04:58:44.171367+0000","last_undegraded":"2024-09-18T04:58:44.171367+0000","last_fullsized":"2024-09-18T04:58:44.171367+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":2479,"log_dups_size":0,"ondisk_log_size":2479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:47:47.826132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061509499999999996,"stat_sum":{"num_bytes":16,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":18529,"num_read_kb":32990,"num_write":4378,"num_write_kb":20206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":5439803,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"1.6","version":"0'0","reported_seq":154,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-18T04:58:43.022393+0000","last_change":"2024-09-18T04:56:12.389971+0000","last_active":"2024-09-18T04:58:43.022393+0000","last_peered":"2024-09-18T04:58:43.022393+0000","last_clean":"2024-09-18T04:58:43.022393+0000","last_became_active":"2024-09-18T04:56:10.412595+0000","last_became_peered":"2024-09-18T04:56:10.412595+0000","last_unstale":"2024-09-18T04:58:43.022393+0000","last_undegraded":"2024-09-18T04:58:43.022393+0000","last_fullsized":"2024-09-18T04:58:43.022393+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:45.638420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038576799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"153'804","reported_seq":7827,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.088636+0000","last_change":"2024-09-18T04:58:44.088835+0000","last_active":"2024-09-18T04:58:44.088636+0000","last_peered":"2024-09-18T04:58:44.088636+0000","last_clean":"2024-09-18T04:58:44.088636+0000","last_became_active":"2024-09-18T04:58:41.079093+0000","last_became_peered":"2024-09-18T04:58:41.079093+0000","last_unstale":"2024-09-18T04:58:44.088636+0000","last_undegraded":"2024-09-18T04:58:44.088636+0000","last_fullsized":"2024-09-18T04:58:44.088636+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:13:10.471671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029195799999999999,"stat_sum":{"num_bytes":183797,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":8940,"num_read_kb":28417,"num_write":1050,"num_write_kb":11406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":5706670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"1.5","version":"0'0","reported_seq":31,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-18T04:58:43.026948+0000","last_change":"2024-09-18T04:58:23.021504+0000","last_active":"2024-09-18T04:58:43.026948+0000","last_peered":"2024-09-18T04:58:43.026948+0000","last_clean":"2024-09-18T04:58:43.026948+0000","last_became_active":"2024-09-18T04:58:23.021173+0000","last_became_peered":"2024-09-18T04:58:23.021173+0000","last_unstale":"2024-09-18T04:58:43.026948+0000","last_undegraded":"2024-09-18T04:58:43.026948+0000","last_fullsized":"2024-09-18T04:58:43.026948+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:05:51.723821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"154'741","reported_seq":6122,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.188017+0000","last_change":"2024-09-18T04:58:44.092005+0000","last_active":"2024-09-18T04:58:44.188017+0000","last_peered":"2024-09-18T04:58:44.188017+0000","last_clean":"2024-09-18T04:58:44.188017+0000","last_became_active":"2024-09-18T04:58:33.026806+0000","last_became_peered":"2024-09-18T04:58:33.026806+0000","last_unstale":"2024-09-18T04:58:44.188017+0000","last_undegraded":"2024-09-18T04:58:44.188017+0000","last_fullsized":"2024-09-18T04:58:44.188017+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:59:41.673907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0070657200000000002,"stat_sum":{"num_bytes":1787848,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":5,"num_read":6871,"num_read_kb":21241,"num_write":927,"num_write_kb":5204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":571544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"1.4","version":"0'0","reported_seq":122,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-18T04:58:43.022166+0000","last_change":"2024-09-18T04:56:46.254241+0000","last_active":"2024-09-18T04:58:43.022166+0000","last_peered":"2024-09-18T04:58:43.022166+0000","last_clean":"2024-09-18T04:58:43.022166+0000","last_became_active":"2024-09-18T04:56:46.253894+0000","last_became_peered":"2024-09-18T04:56:46.253894+0000","last_unstale":"2024-09-18T04:58:43.022166+0000","last_undegraded":"2024-09-18T04:58:43.022166+0000","last_fullsized":"2024-09-18T04:58:43.022166+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:58:42.098099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045144100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"154'497","reported_seq":4978,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.167432+0000","last_change":"2024-09-18T04:58:44.091114+0000","last_active":"2024-09-18T04:58:44.167432+0000","last_peered":"2024-09-18T04:58:44.167432+0000","last_clean":"2024-09-18T04:58:44.167432+0000","last_became_active":"2024-09-18T04:58:41.078435+0000","last_became_peered":"2024-09-18T04:58:41.078435+0000","last_unstale":"2024-09-18T04:58:44.167432+0000","last_undegraded":"2024-09-18T04:58:44.167432+0000","last_fullsized":"2024-09-18T04:58:44.167432+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:34:50.214360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043771499999999998,"stat_sum":{"num_bytes":10485797,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4217,"num_read_kb":33042,"num_write":470,"num_write_kb":37350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":25349011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"3.1","version":"154'484","reported_seq":9255,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.096148+0000","last_change":"2024-09-18T04:58:44.096148+0000","last_active":"2024-09-18T04:58:44.096148+0000","last_peered":"2024-09-18T04:58:44.096148+0000","last_clean":"2024-09-18T04:58:44.096148+0000","last_became_active":"2024-09-18T04:58:29.030467+0000","last_became_peered":"2024-09-18T04:58:29.030467+0000","last_unstale":"2024-09-18T04:58:44.096148+0000","last_undegraded":"2024-09-18T04:58:44.096148+0000","last_fullsized":"2024-09-18T04:58:44.096148+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:04:49.574686+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0022181530000000001,"stat_sum":{"num_bytes":196608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":9360,"num_read_kb":28051,"num_write":498,"num_write_kb":12488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":354,"num_bytes_recovered":11362815,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"2.0","version":"17'117","reported_seq":340,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-18T04:58:43.026712+0000","last_change":"2024-09-18T04:58:38.025236+0000","last_active":"2024-09-18T04:58:43.026712+0000","last_peered":"2024-09-18T04:58:43.026712+0000","last_clean":"2024-09-18T04:58:43.026712+0000","last_became_active":"2024-09-18T04:58:38.024740+0000","last_became_peered":"2024-09-18T04:58:38.024740+0000","last_unstale":"2024-09-18T04:58:43.026712+0000","last_undegraded":"2024-09-18T04:58:43.026712+0000","last_fullsized":"2024-09-18T04:58:43.026712+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_clean_scrub_stamp":"2024-09-18T04:56:10.373785+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:10:45.251654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":139,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-18T04:58:43.022164+0000","last_change":"2024-09-18T04:58:41.078319+0000","last_active":"2024-09-18T04:58:43.022164+0000","last_peered":"2024-09-18T04:58:43.022164+0000","last_clean":"2024-09-18T04:58:43.022164+0000","last_became_active":"2024-09-18T04:58:41.077968+0000","last_became_peered":"2024-09-18T04:58:41.077968+0000","last_unstale":"2024-09-18T04:58:43.022164+0000","last_undegraded":"2024-09-18T04:58:43.022164+0000","last_fullsized":"2024-09-18T04:58:43.022164+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:22:10.969871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"154'800","reported_seq":13385,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.163266+0000","last_change":"2024-09-18T04:58:44.091461+0000","last_active":"2024-09-18T04:58:44.163266+0000","last_peered":"2024-09-18T04:58:44.163266+0000","last_clean":"2024-09-18T04:58:44.163266+0000","last_became_active":"2024-09-18T04:58:41.079029+0000","last_became_peered":"2024-09-18T04:58:41.079029+0000","last_unstale":"2024-09-18T04:58:44.163266+0000","last_undegraded":"2024-09-18T04:58:44.163266+0000","last_fullsized":"2024-09-18T04:58:44.163266+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:42:14.548551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022670920000000001,"stat_sum":{"num_bytes":19874166,"num_objects":11,"num_object_clones":3,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":3,"num_read":14834,"num_read_kb":27168,"num_write":1042,"num_write_kb":7785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":1280827,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"1.2","version":"17'2","reported_seq":156,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-18T04:58:43.026780+0000","last_change":"2024-09-18T04:56:12.398631+0000","last_active":"2024-09-18T04:58:43.026780+0000","last_peered":"2024-09-18T04:58:43.026780+0000","last_clean":"2024-09-18T04:58:43.026780+0000","last_became_active":"2024-09-18T04:56:10.412614+0000","last_became_peered":"2024-09-18T04:56:10.412614+0000","last_unstale":"2024-09-18T04:58:43.026780+0000","last_undegraded":"2024-09-18T04:58:43.026780+0000","last_fullsized":"2024-09-18T04:58:43.026780+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:44:13.401320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075126299999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"154'584","reported_seq":2906,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.168686+0000","last_change":"2024-09-18T04:58:44.096697+0000","last_active":"2024-09-18T04:58:44.168686+0000","last_peered":"2024-09-18T04:58:44.168686+0000","last_clean":"2024-09-18T04:58:44.168686+0000","last_became_active":"2024-09-18T04:58:30.027757+0000","last_became_peered":"2024-09-18T04:58:30.027757+0000","last_unstale":"2024-09-18T04:58:44.168686+0000","last_undegraded":"2024-09-18T04:58:44.168686+0000","last_fullsized":"2024-09-18T04:58:44.168686+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:29:00.106938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002559525,"stat_sum":{"num_bytes":98304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":2192,"num_read_kb":23943,"num_write":576,"num_write_kb":16085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":10562840,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"1.1","version":"0'0","reported_seq":154,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-18T04:58:43.023849+0000","last_change":"2024-09-18T04:56:12.390044+0000","last_active":"2024-09-18T04:58:43.023849+0000","last_peered":"2024-09-18T04:58:43.023849+0000","last_clean":"2024-09-18T04:58:43.023849+0000","last_became_active":"2024-09-18T04:56:10.413042+0000","last_became_peered":"2024-09-18T04:56:10.413042+0000","last_unstale":"2024-09-18T04:58:43.023849+0000","last_undegraded":"2024-09-18T04:58:43.023849+0000","last_fullsized":"2024-09-18T04:58:43.023849+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:46:11.074243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"150'522","reported_seq":4773,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.091781+0000","last_change":"2024-09-18T04:58:44.092245+0000","last_active":"2024-09-18T04:58:44.091781+0000","last_peered":"2024-09-18T04:58:44.091781+0000","last_clean":"2024-09-18T04:58:44.091781+0000","last_became_active":"2024-09-18T04:58:32.027104+0000","last_became_peered":"2024-09-18T04:58:32.027104+0000","last_unstale":"2024-09-18T04:58:44.091781+0000","last_undegraded":"2024-09-18T04:58:44.091781+0000","last_fullsized":"2024-09-18T04:58:44.091781+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":522,"log_dups_size":0,"ondisk_log_size":522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:41:37.134684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0072807189999999997,"stat_sum":{"num_bytes":262160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":5901,"num_read_kb":13839,"num_write":500,"num_write_kb":5042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":2187784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-09-18T04:58:43.026685+0000","last_change":"2024-09-18T04:57:40.472681+0000","last_active":"2024-09-18T04:58:43.026685+0000","last_peered":"2024-09-18T04:58:43.026685+0000","last_clean":"2024-09-18T04:58:43.026685+0000","last_became_active":"2024-09-18T04:57:40.472217+0000","last_became_peered":"2024-09-18T04:57:40.472217+0000","last_unstale":"2024-09-18T04:58:43.026685+0000","last_undegraded":"2024-09-18T04:58:43.026685+0000","last_fullsized":"2024-09-18T04:58:43.026685+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:27:11.849311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"154'478","reported_seq":3571,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.091655+0000","last_change":"2024-09-18T04:58:44.091655+0000","last_active":"2024-09-18T04:58:44.091655+0000","last_peered":"2024-09-18T04:58:44.091655+0000","last_clean":"2024-09-18T04:58:44.091655+0000","last_became_active":"2024-09-18T04:58:38.024916+0000","last_became_peered":"2024-09-18T04:58:38.024916+0000","last_unstale":"2024-09-18T04:58:44.091655+0000","last_undegraded":"2024-09-18T04:58:44.091655+0000","last_fullsized":"2024-09-18T04:58:44.091655+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:32:42.202542+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0069547680000000001,"stat_sum":{"num_bytes":864981,"num_objects":6,"num_object_clones":1,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":3,"num_read":2758,"num_read_kb":7938,"num_write":471,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":458,"num_bytes_recovered":2219067,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"3.9","version":"153'711","reported_seq":11968,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.096215+0000","last_change":"2024-09-18T04:58:44.096429+0000","last_active":"2024-09-18T04:58:44.096215+0000","last_peered":"2024-09-18T04:58:44.096215+0000","last_clean":"2024-09-18T04:58:44.096215+0000","last_became_active":"2024-09-18T04:58:26.101490+0000","last_became_peered":"2024-09-18T04:58:26.101490+0000","last_unstale":"2024-09-18T04:58:44.096215+0000","last_undegraded":"2024-09-18T04:58:44.096215+0000","last_fullsized":"2024-09-18T04:58:44.096215+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":711,"log_dups_size":0,"ondisk_log_size":711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:30:15.538062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024584049999999999,"stat_sum":{"num_bytes":425984,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":12956,"num_read_kb":26590,"num_write":845,"num_write_kb":15777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":5822953,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"3.a","version":"154'462","reported_seq":6026,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.085963+0000","last_change":"2024-09-18T04:58:44.085963+0000","last_active":"2024-09-18T04:58:44.085963+0000","last_peered":"2024-09-18T04:58:44.085963+0000","last_clean":"2024-09-18T04:58:44.085963+0000","last_became_active":"2024-09-18T04:58:27.115137+0000","last_became_peered":"2024-09-18T04:58:27.115137+0000","last_unstale":"2024-09-18T04:58:44.085963+0000","last_undegraded":"2024-09-18T04:58:44.085963+0000","last_fullsized":"2024-09-18T04:58:44.085963+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:57:42.359624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049114899999999997,"stat_sum":{"num_bytes":826965,"num_objects":10,"num_object_clones":2,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":5,"num_read":4723,"num_read_kb":14771,"num_write":474,"num_write_kb":8139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":6055124,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"3.b","version":"151'1398","reported_seq":5730,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.084893+0000","last_change":"2024-09-18T04:58:44.085328+0000","last_active":"2024-09-18T04:58:44.084893+0000","last_peered":"2024-09-18T04:58:44.084893+0000","last_clean":"2024-09-18T04:58:44.084893+0000","last_became_active":"2024-09-18T04:58:31.026041+0000","last_became_peered":"2024-09-18T04:58:31.026041+0000","last_unstale":"2024-09-18T04:58:44.084893+0000","last_undegraded":"2024-09-18T04:58:44.084893+0000","last_fullsized":"2024-09-18T04:58:44.084893+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":1398,"log_dups_size":0,"ondisk_log_size":1398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:55:29.526466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055981300000000004,"stat_sum":{"num_bytes":393263,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":8851,"num_read_kb":15183,"num_write":2047,"num_write_kb":8862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":1966128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"3.c","version":"154'907","reported_seq":7162,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.089814+0000","last_change":"2024-09-18T04:58:44.089814+0000","last_active":"2024-09-18T04:58:44.089814+0000","last_peered":"2024-09-18T04:58:44.089814+0000","last_clean":"2024-09-18T04:58:44.089814+0000","last_became_active":"2024-09-18T04:58:33.027525+0000","last_became_peered":"2024-09-18T04:58:33.027525+0000","last_unstale":"2024-09-18T04:58:44.089814+0000","last_undegraded":"2024-09-18T04:58:44.089814+0000","last_fullsized":"2024-09-18T04:58:44.089814+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":907,"log_dups_size":0,"ondisk_log_size":907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:27:34.074204+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0050376689999999998,"stat_sum":{"num_bytes":196608,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9983,"num_read_kb":24174,"num_write":927,"num_write_kb":8107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":86198,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"3.d","version":"154'598","reported_seq":6945,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.189799+0000","last_change":"2024-09-18T04:58:44.088488+0000","last_active":"2024-09-18T04:58:44.189799+0000","last_peered":"2024-09-18T04:58:44.189799+0000","last_clean":"2024-09-18T04:58:44.189799+0000","last_became_active":"2024-09-18T04:58:07.017827+0000","last_became_peered":"2024-09-18T04:58:07.017827+0000","last_unstale":"2024-09-18T04:58:44.189799+0000","last_undegraded":"2024-09-18T04:58:44.189799+0000","last_fullsized":"2024-09-18T04:58:44.189799+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":598,"log_dups_size":0,"ondisk_log_size":598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:42:31.897588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035750300000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":8463,"num_read_kb":18677,"num_write":584,"num_write_kb":12466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":2872264,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"3.e","version":"143'1048","reported_seq":5219,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.089417+0000","last_change":"2024-09-18T04:58:44.090104+0000","last_active":"2024-09-18T04:58:44.089417+0000","last_peered":"2024-09-18T04:58:44.089417+0000","last_clean":"2024-09-18T04:58:44.089417+0000","last_became_active":"2024-09-18T04:58:31.028189+0000","last_became_peered":"2024-09-18T04:58:31.028189+0000","last_unstale":"2024-09-18T04:58:44.089417+0000","last_undegraded":"2024-09-18T04:58:44.089417+0000","last_fullsized":"2024-09-18T04:58:44.089417+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":1048,"log_dups_size":0,"ondisk_log_size":1048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:26:10.430573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001408023,"stat_sum":{"num_bytes":364403,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":7660,"num_read_kb":17955,"num_write":1661,"num_write_kb":5543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":2749742,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"3.f","version":"148'527","reported_seq":4944,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.090792+0000","last_change":"2024-09-18T04:58:44.091033+0000","last_active":"2024-09-18T04:58:44.090792+0000","last_peered":"2024-09-18T04:58:44.090792+0000","last_clean":"2024-09-18T04:58:44.090792+0000","last_became_active":"2024-09-18T04:58:29.029625+0000","last_became_peered":"2024-09-18T04:58:29.029625+0000","last_unstale":"2024-09-18T04:58:44.090792+0000","last_undegraded":"2024-09-18T04:58:44.090792+0000","last_fullsized":"2024-09-18T04:58:44.090792+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":527,"log_dups_size":0,"ondisk_log_size":527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:51:09.257320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021221780000000002,"stat_sum":{"num_bytes":11583496,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":4488,"num_read_kb":26295,"num_write":617,"num_write_kb":8102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":5504104,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"3.10","version":"154'799","reported_seq":4730,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.166127+0000","last_change":"2024-09-18T04:58:44.089641+0000","last_active":"2024-09-18T04:58:44.166127+0000","last_peered":"2024-09-18T04:58:44.166127+0000","last_clean":"2024-09-18T04:58:44.166127+0000","last_became_active":"2024-09-18T04:56:21.131202+0000","last_became_peered":"2024-09-18T04:56:21.131202+0000","last_unstale":"2024-09-18T04:58:44.166127+0000","last_undegraded":"2024-09-18T04:58:44.166127+0000","last_fullsized":"2024-09-18T04:58:44.166127+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":799,"log_dups_size":0,"ondisk_log_size":799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:17:35.438723+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0042270390000000001,"stat_sum":{"num_bytes":22256551,"num_objects":8,"num_object_clones":3,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":5877,"num_read_kb":14669,"num_write":989,"num_write_kb":10354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"3.11","version":"152'911","reported_seq":8133,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.088945+0000","last_change":"2024-09-18T04:58:44.088945+0000","last_active":"2024-09-18T04:58:44.088945+0000","last_peered":"2024-09-18T04:58:44.088945+0000","last_clean":"2024-09-18T04:58:44.088945+0000","last_became_active":"2024-09-18T04:58:31.027738+0000","last_became_peered":"2024-09-18T04:58:31.027738+0000","last_unstale":"2024-09-18T04:58:44.088945+0000","last_undegraded":"2024-09-18T04:58:44.088945+0000","last_fullsized":"2024-09-18T04:58:44.088945+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":911,"log_dups_size":0,"ondisk_log_size":911,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:35:02.050975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045370799999999999,"stat_sum":{"num_bytes":18327001,"num_objects":7,"num_object_clones":3,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":1,"num_read":10766,"num_read_kb":19560,"num_write":905,"num_write_kb":18676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":9230703,"num_keys_recovered":59,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"3.12","version":"144'809","reported_seq":5837,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.089048+0000","last_change":"2024-09-18T04:58:44.089456+0000","last_active":"2024-09-18T04:58:44.089048+0000","last_peered":"2024-09-18T04:58:44.089048+0000","last_clean":"2024-09-18T04:58:44.089048+0000","last_became_active":"2024-09-18T04:57:24.398252+0000","last_became_peered":"2024-09-18T04:57:24.398252+0000","last_unstale":"2024-09-18T04:58:44.089048+0000","last_undegraded":"2024-09-18T04:58:44.089048+0000","last_fullsized":"2024-09-18T04:58:44.089048+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:02:29.228991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00094802700000000005,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7388,"num_read_kb":33089,"num_write":1110,"num_write_kb":12855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"8","length":"4"}]},{"pgid":"3.13","version":"143'743","reported_seq":2774,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-09-18T04:58:44.089105+0000","last_change":"2024-09-18T04:58:44.089351+0000","last_active":"2024-09-18T04:58:44.089105+0000","last_peered":"2024-09-18T04:58:44.089105+0000","last_clean":"2024-09-18T04:58:44.089105+0000","last_became_active":"2024-09-18T04:58:31.028633+0000","last_became_peered":"2024-09-18T04:58:31.028633+0000","last_unstale":"2024-09-18T04:58:44.089105+0000","last_undegraded":"2024-09-18T04:58:44.089105+0000","last_fullsized":"2024-09-18T04:58:44.089105+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":743,"log_dups_size":0,"ondisk_log_size":743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:04:25.747259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082179799999999999,"stat_sum":{"num_bytes":8705036,"num_objects":8,"num_object_clones":2,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":2,"num_read":4134,"num_read_kb":14594,"num_write":854,"num_write_kb":9988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":12642724,"num_keys_recovered":71,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"8","length":"4"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":129992806,"num_objects":151,"num_object_clones":24,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":59,"num_read":219948,"num_read_kb":682481,"num_write":29709,"num_write_kb":377739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8532,"num_bytes_recovered":184551649,"num_keys_recovered":562,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2269184,"data_stored":22425992,"data_compressed":94968,"data_compressed_allocated":729088,"data_compressed_original":21028864,"omap_allocated":0,"internal_metadata":0},"log_size":23706,"ondisk_log_size":23706,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28167,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574883,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51756,"kb_used_data":2480,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320084,"statfs":{"total":96636764160,"available":96583766016,"internally_reserved":0,"allocated":2539520,"data_stored":8350134,"data_compressed":37596,"data_compressed_allocated":245760,"data_compressed_original":7028736,"omap_allocated":1589,"internal_metadata":50330059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.45600000000000002,"15min":0.46400000000000002},"min":{"1min":0.23799999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53100000000000003,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.437,"5min":0.41499999999999998,"15min":0.41099999999999998},"min":{"1min":0.312,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.49299999999999999}]},{"osd":1,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.46899999999999997,"15min":0.47799999999999998},"min":{"1min":0.249,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.55100000000000005,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.504},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.436,"15min":0.434},"min":{"1min":0.28399999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.57999999999999996}]},{"osd":2,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.44700000000000001,"15min":0.44500000000000001},"min":{"1min":0.35399999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45200000000000001,"15min":0.45000000000000001},"min":{"1min":0.34200000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.56200000000000006,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.53000000000000003}]},{"osd":3,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52600000000000002,"15min":0.51900000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.4399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.34100000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.4809999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":0.56399999999999995}]},{"osd":4,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.42199999999999999,"15min":0.43099999999999999},"min":{"1min":0.26500000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.63700000000000001,"5min":1.954,"15min":1.954},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.41699999999999998,"15min":0.42399999999999999},"min":{"1min":0.246,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.629,"5min":1.478,"15min":1.478},"last":0.33400000000000002}]},{"osd":5,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.50800000000000001,"15min":0.52200000000000002},"min":{"1min":0.30299999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.71799999999999997,"5min":3.145,"15min":3.145},"last":0.314},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.443,"15min":0.441},"min":{"1min":0.27700000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.65800000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.379}]},{"osd":6,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.47899999999999998,"15min":0.48299999999999998},"min":{"1min":0.33300000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.69299999999999995,"5min":0.746,"15min":0.746},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.48899999999999999,"15min":0.49299999999999999},"min":{"1min":0.35199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.623,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.38800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574882,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55392,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316448,"statfs":{"total":96636764160,"available":96580042752,"internally_reserved":0,"allocated":2183168,"data_stored":5055174,"data_compressed":17970,"data_compressed_allocated":131072,"data_compressed_original":3923926,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.71899999999999997,"15min":0.76900000000000002},"min":{"1min":0.23899999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.56799999999999995,"5min":6.1319999999999997,"15min":6.1319999999999997},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.75700000000000001,"15min":0.80500000000000005},"min":{"1min":0.35799999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.81100000000000005,"5min":6.1429999999999998,"15min":6.1429999999999998},"last":0.433}]},{"osd":1,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.70999999999999996,"15min":0.76200000000000001},"min":{"1min":0.29299999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.44900000000000001,"5min":6.0410000000000004,"15min":6.0410000000000004},"last":0.372},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.70899999999999996,"15min":0.753},"min":{"1min":0.33000000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.83599999999999997,"5min":5.5380000000000003,"15min":5.5380000000000003},"last":0.38500000000000001}]},{"osd":2,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.41999999999999998,"15min":0.42599999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.54400000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.46100000000000002,"15min":0.46800000000000003},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.55900000000000005,"5min":1.04,"15min":1.04},"last":0.33700000000000002}]},{"osd":3,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46000000000000002,"15min":0.46100000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.47799999999999998,"15min":0.48099999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.625,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.40400000000000003}]},{"osd":4,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.36799999999999999,"15min":0.374},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.47599999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.43099999999999999,"15min":0.437},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.27500000000000002}]},{"osd":5,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.372,"15min":0.378},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.312},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.42499999999999999,"15min":0.435},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.53000000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.29499999999999998}]},{"osd":7,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.46800000000000003,"15min":0.46600000000000003},"min":{"1min":0.28100000000000003,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.48599999999999999,"15min":0.47699999999999998},"min":{"1min":0.34399999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.36199999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65248,"kb_used_data":2124,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306592,"statfs":{"total":96636764160,"available":96569950208,"internally_reserved":0,"allocated":2174976,"data_stored":1310785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.54600000000000004,"15min":0.57099999999999995},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.59199999999999997,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.504,"15min":0.51000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71599999999999997,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.36399999999999999}]},{"osd":1,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.53300000000000003,"15min":0.55400000000000005},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.55600000000000005,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.32500000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63800000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.34399999999999997}]},{"osd":2,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47699999999999998,"15min":0.47899999999999998},"min":{"1min":0.28699999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.629,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.496},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.57399999999999995,"15min":0.59999999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.626,"5min":3.1070000000000002,"15min":3.1070000000000002},"last":0.40600000000000003}]},{"osd":3,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.496,"15min":0.502},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.77000000000000002,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.311},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.504,"15min":0.50800000000000001},"min":{"1min":0.35099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.63,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.47499999999999998}]},{"osd":4,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.46800000000000003,"15min":0.47399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.59499999999999997,"5min":1.599,"15min":1.599},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.56299999999999994,"15min":0.59299999999999997},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.53500000000000003,"5min":3.379,"15min":3.379},"last":0.376}]},{"osd":6,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.45500000000000002,"15min":0.46600000000000003},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54200000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.56200000000000006,"15min":0.58499999999999996},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.085,"5min":3.1669999999999998,"15min":3.1669999999999998},"last":0.32900000000000001}]},{"osd":7,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.48399999999999999,"15min":0.495},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80200000000000005,"5min":1.224,"15min":1.224},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.439,"5min":0.54200000000000004,"15min":0.55900000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.58599999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.39500000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59816,"kb_used_data":2456,"kb_used_omap":1,"kb_used_meta":57342,"kb_avail":94312024,"statfs":{"total":96636764160,"available":96575512576,"internally_reserved":0,"allocated":2514944,"data_stored":8188644,"data_compressed":30357,"data_compressed_allocated":229376,"data_compressed_original":6766592,"omap_allocated":1594,"internal_metadata":58718662},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.46000000000000002,"15min":0.44400000000000001},"min":{"1min":0.23499999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":3.7069999999999999,"5min":3.7069999999999999,"15min":3.7069999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.46100000000000002,"15min":0.441},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":3.3700000000000001,"5min":3.3700000000000001,"15min":3.3700000000000001},"last":0.26000000000000001}]},{"osd":1,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.47099999999999997,"15min":0.45500000000000002},"min":{"1min":0.28199999999999997,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":3.2879999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.46600000000000003,"15min":0.44600000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.661,"5min":3.661,"15min":3.661},"last":0.32400000000000001}]},{"osd":2,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.52200000000000002,"15min":0.50600000000000001},"min":{"1min":0.29999999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.1890000000000001,"5min":3.1890000000000001,"15min":3.1890000000000001},"last":0.442},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.51600000000000001,"15min":0.502},"min":{"1min":0.32300000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.3860000000000001,"5min":3.3860000000000001,"15min":3.3860000000000001},"last":0.39700000000000002}]},{"osd":3,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.504,"15min":0.48899999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":3.4500000000000002,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.48299999999999998,"15min":0.46100000000000002},"min":{"1min":0.372,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.3210000000000002,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.41099999999999998}]},{"osd":5,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.42599999999999999,"15min":0.40200000000000002},"min":{"1min":0.25,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":3.9319999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.43099999999999999,"15min":0.40699999999999997},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":3.9449999999999998,"5min":3.9449999999999998,"15min":3.9449999999999998},"last":0.48899999999999999}]},{"osd":6,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.45100000000000001,"15min":0.42899999999999999},"min":{"1min":0.29299999999999998,"5min":0.251,"15min":0.251},"max":{"1min":3.3530000000000002,"5min":3.3530000000000002,"15min":3.3530000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":1.29,"5min":0.626,"15min":0.51600000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":17.422999999999998,"5min":17.422999999999998,"15min":17.422999999999998},"last":0.433}]},{"osd":7,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.48399999999999999,"15min":0.46400000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":3.7330000000000001,"5min":3.7330000000000001,"15min":3.7330000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.52100000000000002,"15min":0.499},"min":{"1min":0.33300000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":3.6930000000000001,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":0.5}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87348,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284492,"statfs":{"total":96636764160,"available":96547319808,"internally_reserved":0,"allocated":2134016,"data_stored":1237831,"data_compressed":982,"data_compressed_allocated":16384,"data_compressed_original":126976,"omap_allocated":1592,"internal_metadata":87292360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.39500000000000002,"15min":0.40500000000000003},"min":{"1min":0.23899999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.45200000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.51800000000000002,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.33800000000000002}]},{"osd":1,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.376,"15min":0.375},"min":{"1min":0.249,"5min":0.245,"15min":0.245},"max":{"1min":0.51800000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.38},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.40500000000000003,"15min":0.41499999999999998},"min":{"1min":0.249,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.54400000000000004,"5min":0.755,"15min":0.755},"last":0.36599999999999999}]},{"osd":2,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.41599999999999998,"15min":0.41999999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.56999999999999995,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.45200000000000001,"15min":0.45300000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.59399999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.438}]},{"osd":4,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.52000000000000002,"15min":0.52900000000000003},"min":{"1min":0.31,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71999999999999997,"5min":0.755,"15min":0.755},"last":0.316},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.76100000000000001,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.51000000000000001}]},{"osd":5,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.55900000000000005,"15min":0.56899999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68600000000000005,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.503,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.66900000000000004,"5min":0.874,"15min":0.874},"last":0.39200000000000002}]},{"osd":6,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58699999999999997,"15min":0.59599999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80400000000000005,"5min":0.874,"15min":0.874},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72399999999999998,"5min":1.095,"15min":1.095},"last":0.71399999999999997}]},{"osd":7,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.65600000000000003,"15min":0.67200000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84499999999999997,"5min":1.335,"15min":1.335},"last":0.628},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.61599999999999999,"15min":0.624},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.753,"5min":1.087,"15min":1.087},"last":0.64600000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1941504,"data_stored":1048749,"data_compressed":555,"data_compressed_allocated":12288,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.33700000000000002,"15min":0.33600000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.191},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.39600000000000002,"15min":0.39800000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.63800000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.32800000000000001}]},{"osd":1,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.56999999999999995,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.253},{"interface":"front","average":{"1min":0.375,"5min":0.36299999999999999,"15min":0.36099999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.28299999999999997}]},{"osd":3,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.40799999999999997,"15min":0.40400000000000003},"min":{"1min":0.26200000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.40999999999999998,"15min":0.40400000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.23400000000000001}]},{"osd":4,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.58899999999999997,"15min":0.57899999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.4220000000000002,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":0.36599999999999999}]},{"osd":5,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.61199999999999999,"15min":0.56399999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":7.4480000000000004,"5min":7.4480000000000004,"15min":7.4480000000000004},"last":0.377},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54800000000000004,"15min":0.54700000000000004},"min":{"1min":0.377,"5min":0.375,"15min":0.375},"max":{"1min":0.73699999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.33800000000000002}]},{"osd":6,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.84599999999999997,"5min":1.171,"15min":1.171},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.438}]},{"osd":7,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80800000000000005,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":1.1719999999999999,"5min":0.69999999999999996,"15min":0.622},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":11.138,"5min":11.138,"15min":11.138},"last":0.45900000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29044,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342796,"statfs":{"total":96636764160,"available":96607023104,"internally_reserved":0,"allocated":2068480,"data_stored":1767854,"data_compressed":10509,"data_compressed_allocated":53248,"data_compressed_original":737280,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.34999999999999998,"15min":0.35799999999999998},"min":{"1min":0.20899999999999999,"5min":0.185,"15min":0.185},"max":{"1min":0.622,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.20200000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.63100000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":0.318}]},{"osd":2,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.371,"15min":0.373},"min":{"1min":0.222,"5min":0.221,"15min":0.221},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41599999999999998,"15min":0.41399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.35899999999999999}]},{"osd":3,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.38800000000000001,"15min":0.38400000000000001},"min":{"1min":0.23899999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.40600000000000003,"15min":0.40100000000000002},"min":{"1min":0.29599999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.41699999999999998}]},{"osd":4,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.50800000000000001,"15min":0.51700000000000002},"min":{"1min":0.32400000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.77000000000000002,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.50700000000000001,"15min":0.50900000000000001},"min":{"1min":0.315,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.86099999999999999,"5min":0.999,"15min":0.999},"last":0.53200000000000003}]},{"osd":5,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50900000000000001,"15min":0.51300000000000001},"min":{"1min":0.33600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80400000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.5,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.373,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.83899999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.55400000000000005}]},{"osd":6,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82799999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.74299999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.65500000000000003}]},{"osd":7,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.56999999999999995,"15min":0.57799999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78200000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.40999999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0309999999999999,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.624}]}]},{"osd":0,"up_from":13,"seq":55834574883,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51324,"kb_used_data":2152,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320516,"statfs":{"total":96636764160,"available":96584208384,"internally_reserved":0,"allocated":2203648,"data_stored":4988528,"data_compressed":18006,"data_compressed_allocated":139264,"data_compressed_original":3903446,"omap_allocated":1593,"internal_metadata":50330055},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35999999999999999,"15min":0.36299999999999999},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.56799999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.216},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.34799999999999998,"15min":0.34599999999999997},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.53700000000000003,"5min":0.621,"15min":0.621},"last":0.316}]},{"osd":2,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.39900000000000002,"15min":0.40100000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.249},{"interface":"front","average":{"1min":0.435,"5min":0.42599999999999999,"15min":0.42399999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.35799999999999998}]},{"osd":3,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.434,"15min":0.42399999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.40400000000000003,"15min":0.39700000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.33100000000000002}]},{"osd":4,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.63400000000000001,"15min":0.65600000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.89400000000000002,"5min":3.4710000000000001,"15min":3.4710000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.504,"15min":0.502},"min":{"1min":0.35899999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.52800000000000002}]},{"osd":5,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.81699999999999995,"15min":0.84099999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":4.2640000000000002,"5min":4.2640000000000002,"15min":4.2640000000000002},"last":0.435},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.54500000000000004}]},{"osd":6,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.66000000000000003,"15min":0.68000000000000005},"min":{"1min":0.39500000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81699999999999995,"5min":3.536,"15min":3.536},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57099999999999995,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.47399999999999998}]},{"osd":7,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55200000000000005,"15min":0.54700000000000004},"min":{"1min":0.41999999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.68400000000000005,"15min":0.70599999999999996},"min":{"1min":0.39100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.85699999999999998,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.51200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9734,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9734,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3965700,"data_compressed":17148,"data_compressed_allocated":126976,"data_compressed_original":3788800,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":277031,"data_compressed":775,"data_compressed_allocated":16384,"data_compressed_original":147456,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":148294,"data_compressed":555,"data_compressed_allocated":12288,"data_compressed_original":81920,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":241664,"data_stored":306788,"data_compressed":982,"data_compressed_allocated":16384,"data_compressed_original":126976,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":524288,"data_stored":6851140,"data_compressed":29435,"data_compressed_allocated":221184,"data_compressed_original":6569984,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":83402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":4019989,"data_compressed":17176,"data_compressed_allocated":126976,"data_compressed_original":3858432,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":6773648,"data_compressed":28897,"data_compressed_allocated":208896,"data_compressed_original":6455296,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T04:58:45.842 INFO:tasks.thrashosds.thrasher:Setting 1.5 to [1, 4] 2024-09-18T04:58:45.842 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '1.5', '1', '4'] 2024-09-18T04:58:45.842 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 1.5 1 4 2024-09-18T04:58:45.850 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:58:45.850 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:58:45.867 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:45.867 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:58:45.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:58:45.868 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:45.868 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.868 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72135 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 394240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:45.868 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:58:45.868 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.909796919, 2024-09-18T04:58:45.868 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048559739999999997, 2024-09-18T04:58:45.868 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:45.868 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:45.869 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:45.869 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:45.869 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:45.869 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72135 2024-09-18T04:58:45.869 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.869 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:45.869 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:45.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:58:45.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.870 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.870 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:45.870 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:58:45.870 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.870 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:45.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954004+0000", 2024-09-18T04:58:45.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:45.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.871 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:45.871 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954005+0000", 2024-09-18T04:58:45.871 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.5629999999999999e-06 2024-09-18T04:58:45.871 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.871 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:45.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954008+0000", 2024-09-18T04:58:45.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9790000000000002e-06 2024-09-18T04:58:45.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.872 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:45.872 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954012+0000", 2024-09-18T04:58:45.872 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4790000000000001e-06 2024-09-18T04:58:45.872 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.872 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:45.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002418+0000", 2024-09-18T04:58:45.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048406049 2024-09-18T04:58:45.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:45.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002460+0000", 2024-09-18T04:58:45.873 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2188999999999999e-05 2024-09-18T04:58:45.873 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.873 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.873 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:45.873 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002564+0000", 2024-09-18T04:58:45.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010377 2024-09-18T04:58:45.874 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.874 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:45.874 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.874 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.874 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.874 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72136 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 409600~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:45.874 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:58:45.874 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.909691972, 2024-09-18T04:58:45.874 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048670191000000002, 2024-09-18T04:58:45.874 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:45.875 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:45.875 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:45.875 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:45.875 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:45.875 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72136 2024-09-18T04:58:45.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.875 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:45.875 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.875 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:45.875 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:58:45.875 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.875 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:45.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954109+0000", 2024-09-18T04:58:45.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.876 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.876 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:45.876 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954019+0000", 2024-09-18T04:58:45.876 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999099 2024-09-18T04:58:45.876 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:45.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954112+0000", 2024-09-18T04:58:45.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3273e-05 2024-09-18T04:58:45.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.877 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.877 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:45.877 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954121+0000", 2024-09-18T04:58:45.877 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0289999999999994e-06 2024-09-18T04:58:45.877 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:45.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954128+0000", 2024-09-18T04:58:45.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6189999999999996e-06 2024-09-18T04:58:45.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:45.878 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002604+0000", 2024-09-18T04:58:45.878 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048476048000000001 2024-09-18T04:58:45.878 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.878 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.878 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002682+0000", 2024-09-18T04:58:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7955999999999997e-05 2024-09-18T04:58:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002779+0000", 2024-09-18T04:58:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7372000000000002e-05 2024-09-18T04:58:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.879 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.880 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72138 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 440320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:45.880 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:58:45.880 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.90925812, 2024-09-18T04:58:45.880 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048549430999999997, 2024-09-18T04:58:45.880 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:45.880 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:45.880 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:45.880 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:45.880 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:45.880 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72138 2024-09-18T04:58:45.880 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.881 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:45.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:45.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:58:45.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.881 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.881 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:45.881 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954543+0000", 2024-09-18T04:58:45.881 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.881 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954542+0000", 2024-09-18T04:58:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954545+0000", 2024-09-18T04:58:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4360000000000001e-06 2024-09-18T04:58:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:45.882 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954551+0000", 2024-09-18T04:58:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4949999999999996e-06 2024-09-18T04:58:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954557+0000", 2024-09-18T04:58:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5099999999999998e-06 2024-09-18T04:58:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.002993+0000", 2024-09-18T04:58:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048436358999999998 2024-09-18T04:58:45.883 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:45.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003039+0000", 2024-09-18T04:58:45.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5884000000000003e-05 2024-09-18T04:58:45.884 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.884 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.884 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:45.884 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003092+0000", 2024-09-18T04:58:45.884 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3208000000000003e-05 2024-09-18T04:58:45.884 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.884 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:45.885 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.885 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.885 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.885 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72139 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 455680~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:45.885 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:58:45.885 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.909239312, 2024-09-18T04:58:45.885 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048718773, 2024-09-18T04:58:45.885 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:45.885 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:45.885 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:45.885 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:45.886 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:45.886 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72139 2024-09-18T04:58:45.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.886 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:45.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:45.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:58:45.886 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.886 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.886 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.886 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:45.886 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954562+0000", 2024-09-18T04:58:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954561+0000", 2024-09-18T04:58:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954564+0000", 2024-09-18T04:58:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4099999999999998e-06 2024-09-18T04:58:45.887 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:45.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954570+0000", 2024-09-18T04:58:45.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.178e-06 2024-09-18T04:58:45.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.888 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:45.888 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954575+0000", 2024-09-18T04:58:45.888 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0560000000000004e-06 2024-09-18T04:58:45.888 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.888 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003114+0000", 2024-09-18T04:58:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048539212999999998 2024-09-18T04:58:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003176+0000", 2024-09-18T04:58:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2154000000000002e-05 2024-09-18T04:58:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.889 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:45.890 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003280+0000", 2024-09-18T04:58:45.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010421600000000001 2024-09-18T04:58:45.890 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.890 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:45.890 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.890 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.890 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.890 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72140 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 471040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:45.890 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:58:45.890 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.90898337100001, 2024-09-18T04:58:45.890 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048670819999999997, 2024-09-18T04:58:45.891 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:45.891 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:45.891 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:45.891 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:45.891 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:45.891 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72140 2024-09-18T04:58:45.891 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.891 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:45.891 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.891 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:45.891 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:58:45.891 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.892 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:45.892 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954818+0000", 2024-09-18T04:58:45.892 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.892 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.892 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:45.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954579+0000", 2024-09-18T04:58:45.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997611 2024-09-18T04:58:45.893 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.893 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.893 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:45.893 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954821+0000", 2024-09-18T04:58:45.893 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024241000000000001 2024-09-18T04:58:45.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:45.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954831+0000", 2024-09-18T04:58:45.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.8770000000000005e-06 2024-09-18T04:58:45.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.894 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.894 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:45.894 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954838+0000", 2024-09-18T04:58:45.894 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0690000000000004e-06 2024-09-18T04:58:45.894 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:45.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003317+0000", 2024-09-18T04:58:45.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048478796999999997 2024-09-18T04:58:45.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:45.895 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003392+0000", 2024-09-18T04:58:45.895 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4411000000000003e-05 2024-09-18T04:58:45.895 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.895 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.895 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:45.896 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003488+0000", 2024-09-18T04:58:45.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6927000000000002e-05 2024-09-18T04:58:45.896 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.896 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:45.896 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.896 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.896 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.896 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72141 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 486400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:45.896 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:58:45.896 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.90895588799999, 2024-09-18T04:58:45.896 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048808114, 2024-09-18T04:58:45.897 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:45.897 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:45.897 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:45.897 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:45.897 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:45.897 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72141 2024-09-18T04:58:45.897 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.897 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:45.897 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.897 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:45.897 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:58:45.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:45.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:58:45.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.898 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.898 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:45.898 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954845+0000", 2024-09-18T04:58:45.898 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:45.898 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:45.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954847+0000", 2024-09-18T04:58:45.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3800000000000001e-06 2024-09-18T04:58:45.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:45.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954854+0000", 2024-09-18T04:58:45.899 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6080000000000001e-06 2024-09-18T04:58:45.899 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.899 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.899 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:45.899 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954859+0000", 2024-09-18T04:58:45.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:58:45.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:45.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003526+0000", 2024-09-18T04:58:45.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048666433000000002 2024-09-18T04:58:45.900 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.900 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.900 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:45.900 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003599+0000", 2024-09-18T04:58:45.900 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2854000000000004e-05 2024-09-18T04:58:45.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.901 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:45.901 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003653+0000", 2024-09-18T04:58:45.901 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4511e-05 2024-09-18T04:58:45.901 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.901 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:45.901 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.901 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.901 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.901 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72142 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 501760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:45.901 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:58:45.902 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.908917086, 2024-09-18T04:58:45.902 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048885500999999998, 2024-09-18T04:58:45.902 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:45.902 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:45.902 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:45.902 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:45.902 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:45.902 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72142 2024-09-18T04:58:45.902 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.902 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:45.902 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.902 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:45.902 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:58:45.903 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.903 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.903 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.903 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:45.903 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954884+0000", 2024-09-18T04:58:45.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.904 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.904 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.904 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:45.904 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954864+0000", 2024-09-18T04:58:45.904 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999804 2024-09-18T04:58:45.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:45.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954886+0000", 2024-09-18T04:58:45.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2042999999999999e-05 2024-09-18T04:58:45.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:45.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954893+0000", 2024-09-18T04:58:45.905 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.255e-06 2024-09-18T04:58:45.905 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.905 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.905 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:45.905 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954899+0000", 2024-09-18T04:58:45.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8780000000000003e-06 2024-09-18T04:58:45.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:45.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003678+0000", 2024-09-18T04:58:45.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048778834 2024-09-18T04:58:45.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.906 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:45.906 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003717+0000", 2024-09-18T04:58:45.906 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8860999999999999e-05 2024-09-18T04:58:45.906 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.906 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.907 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:45.907 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003769+0000", 2024-09-18T04:58:45.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2389000000000003e-05 2024-09-18T04:58:45.907 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.907 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:45.907 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.907 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.907 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.907 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72143 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 517120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:45.907 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:58:45.907 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.90879146, 2024-09-18T04:58:45.907 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048951738000000002, 2024-09-18T04:58:45.907 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:45.907 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:45.907 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:45.908 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:45.908 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:45.908 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72143 2024-09-18T04:58:45.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.908 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:45.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:45.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:58:45.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.908 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.908 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.908 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:45.908 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955010+0000", 2024-09-18T04:58:45.908 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:45.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.954904+0000", 2024-09-18T04:58:45.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998946 2024-09-18T04:58:45.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:45.909 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955013+0000", 2024-09-18T04:58:45.909 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000108977 2024-09-18T04:58:45.909 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.909 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.909 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:45.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955021+0000", 2024-09-18T04:58:45.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7570000000000004e-06 2024-09-18T04:58:45.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:45.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955025+0000", 2024-09-18T04:58:45.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1219999999999997e-06 2024-09-18T04:58:45.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.910 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:45.910 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003792+0000", 2024-09-18T04:58:45.910 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048767379 2024-09-18T04:58:45.910 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.910 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:45.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003860+0000", 2024-09-18T04:58:45.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8033000000000004e-05 2024-09-18T04:58:45.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.911 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:45.911 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.003961+0000", 2024-09-18T04:58:45.911 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010103300000000001 2024-09-18T04:58:45.911 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.911 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:45.911 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.911 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.911 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.911 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72161 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 13312~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:45.912 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:58:45.912 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.908527823, 2024-09-18T04:58:45.912 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048891539999999997, 2024-09-18T04:58:45.912 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:45.912 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:45.912 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:45.912 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:45.912 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:45.912 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72161 2024-09-18T04:58:45.912 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.912 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:45.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:45.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:58:45.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:45.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955273+0000", 2024-09-18T04:58:45.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.913 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.913 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.913 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:45.913 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955028+0000", 2024-09-18T04:58:45.913 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997549 2024-09-18T04:58:45.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:45.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955278+0000", 2024-09-18T04:58:45.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00024973299999999998 2024-09-18T04:58:45.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:45.914 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955289+0000", 2024-09-18T04:58:45.914 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0812000000000001e-05 2024-09-18T04:58:45.914 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.914 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.914 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:45.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955296+0000", 2024-09-18T04:58:45.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2570000000000001e-06 2024-09-18T04:58:45.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:45.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004002+0000", 2024-09-18T04:58:45.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048705904000000001 2024-09-18T04:58:45.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.915 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:45.915 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004076+0000", 2024-09-18T04:58:45.915 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4086999999999997e-05 2024-09-18T04:58:45.915 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.915 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.916 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:45.916 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004165+0000", 2024-09-18T04:58:45.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8712e-05 2024-09-18T04:58:45.916 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.916 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:45.916 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.916 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.916 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.916 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72163 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 44032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:45.916 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:58:45.916 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.907902385, 2024-09-18T04:58:45.916 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048567011, 2024-09-18T04:58:45.916 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:45.917 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:45.917 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:45.917 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:45.917 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:45.917 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72163 2024-09-18T04:58:45.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.917 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:45.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:45.917 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:58:45.917 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.917 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.917 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.917 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:45.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955899+0000", 2024-09-18T04:58:45.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:45.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955898+0000", 2024-09-18T04:58:45.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:45.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.918 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:45.918 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955901+0000", 2024-09-18T04:58:45.918 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.706e-06 2024-09-18T04:58:45.918 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.918 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:45.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955907+0000", 2024-09-18T04:58:45.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3559999999999998e-06 2024-09-18T04:58:45.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:45.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955912+0000", 2024-09-18T04:58:45.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9640000000000002e-06 2024-09-18T04:58:45.919 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.919 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.919 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:45.919 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004377+0000", 2024-09-18T04:58:45.919 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048465293999999999 2024-09-18T04:58:45.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:45.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004418+0000", 2024-09-18T04:58:45.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0163000000000002e-05 2024-09-18T04:58:45.920 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.920 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.920 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:45.920 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004466+0000", 2024-09-18T04:58:45.920 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8004000000000003e-05 2024-09-18T04:58:45.920 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.920 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:45.920 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.921 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.921 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72164 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:45.921 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:58:45.921 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.907884112, 2024-09-18T04:58:45.921 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048707253999999998, 2024-09-18T04:58:45.921 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:45.921 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:45.921 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:45.921 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:45.921 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:45.921 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72164 2024-09-18T04:58:45.921 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.921 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:45.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:45.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:58:45.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:45.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:58:45.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.922 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.922 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.922 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:45.922 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955917+0000", 2024-09-18T04:58:45.922 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:45.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:45.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955919+0000", 2024-09-18T04:58:45.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2520000000000002e-06 2024-09-18T04:58:45.923 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.923 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.923 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:45.923 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955925+0000", 2024-09-18T04:58:45.923 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3740000000000003e-06 2024-09-18T04:58:45.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:45.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955930+0000", 2024-09-18T04:58:45.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3009999999999997e-06 2024-09-18T04:58:45.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:45.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004489+0000", 2024-09-18T04:58:45.924 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048558581000000003 2024-09-18T04:58:45.924 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.924 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.924 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:45.924 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004529+0000", 2024-09-18T04:58:45.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9722999999999999e-05 2024-09-18T04:58:45.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.925 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:45.925 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004624+0000", 2024-09-18T04:58:45.925 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5402999999999994e-05 2024-09-18T04:58:45.925 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.925 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:45.925 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.925 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.925 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.925 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72165 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 74752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:45.925 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:58:45.925 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.907609253, 2024-09-18T04:58:45.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048633223000000003, 2024-09-18T04:58:45.926 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:45.926 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:45.926 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:45.926 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:45.926 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:45.926 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72165 2024-09-18T04:58:45.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.926 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:45.926 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.926 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:45.926 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:58:45.926 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.926 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:45.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956192+0000", 2024-09-18T04:58:45.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:45.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.955935+0000", 2024-09-18T04:58:45.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999743 2024-09-18T04:58:45.927 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.927 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.927 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:45.927 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956199+0000", 2024-09-18T04:58:45.927 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000263906 2024-09-18T04:58:45.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:45.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956207+0000", 2024-09-18T04:58:45.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2519999999999999e-06 2024-09-18T04:58:45.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:45.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956214+0000", 2024-09-18T04:58:45.928 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7610000000000002e-06 2024-09-18T04:58:45.928 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.928 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.928 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:45.928 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004661+0000", 2024-09-18T04:58:45.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048447967000000002 2024-09-18T04:58:45.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:45.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004735+0000", 2024-09-18T04:58:45.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3789999999999997e-05 2024-09-18T04:58:45.929 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.929 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.929 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:45.929 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004825+0000", 2024-09-18T04:58:45.929 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9724999999999999e-05 2024-09-18T04:58:45.929 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.929 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:45.930 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.930 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.930 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72166 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 90112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:45.930 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:58:45.930 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.90735571899999, 2024-09-18T04:58:45.930 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048595737999999999, 2024-09-18T04:58:45.930 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:45.930 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:45.930 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:45.930 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:45.930 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:45.930 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72166 2024-09-18T04:58:45.930 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.931 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:45.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:45.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:58:45.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.931 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.931 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.931 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:45.931 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956445+0000", 2024-09-18T04:58:45.931 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:45.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956221+0000", 2024-09-18T04:58:45.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9997759 2024-09-18T04:58:45.932 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.932 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.932 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:45.932 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956448+0000", 2024-09-18T04:58:45.932 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00022685500000000001 2024-09-18T04:58:45.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:45.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956456+0000", 2024-09-18T04:58:45.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0099999999999995e-06 2024-09-18T04:58:45.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:45.933 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956462+0000", 2024-09-18T04:58:45.933 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7200000000000003e-06 2024-09-18T04:58:45.933 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.933 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.933 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:45.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004862+0000", 2024-09-18T04:58:45.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048399795000000002 2024-09-18T04:58:45.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:45.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.004941+0000", 2024-09-18T04:58:45.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9414999999999998e-05 2024-09-18T04:58:45.934 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.934 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.934 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:45.934 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005041+0000", 2024-09-18T04:58:45.934 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9942e-05 2024-09-18T04:58:45.934 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.934 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:45.935 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.935 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.935 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72167 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:45.935 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:58:45.935 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.90726868100001, 2024-09-18T04:58:45.935 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048622894999999999, 2024-09-18T04:58:45.935 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:45.935 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:45.935 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:45.935 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:45.935 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:45.935 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72167 2024-09-18T04:58:45.935 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.935 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:45.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:45.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:58:45.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:45.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956532+0000", 2024-09-18T04:58:45.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.936 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.936 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.936 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:45.936 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956468+0000", 2024-09-18T04:58:45.936 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999352 2024-09-18T04:58:45.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:45.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956535+0000", 2024-09-18T04:58:45.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7440999999999993e-05 2024-09-18T04:58:45.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:45.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956544+0000", 2024-09-18T04:58:45.937 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1039999999999996e-06 2024-09-18T04:58:45.937 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.937 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.937 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:45.937 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956550+0000", 2024-09-18T04:58:45.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9850000000000003e-06 2024-09-18T04:58:45.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:45.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005066+0000", 2024-09-18T04:58:45.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048515614999999998 2024-09-18T04:58:45.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:45.938 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005107+0000", 2024-09-18T04:58:45.938 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0843999999999997e-05 2024-09-18T04:58:45.938 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.938 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.938 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:45.939 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005155+0000", 2024-09-18T04:58:45.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8686000000000001e-05 2024-09-18T04:58:45.939 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.939 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:45.939 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.939 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.939 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.939 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72168 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:45.939 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:58:45.939 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.907107618, 2024-09-18T04:58:45.939 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048585179999999999, 2024-09-18T04:58:45.939 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:45.939 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:45.939 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:45.939 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:45.940 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:45.940 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72168 2024-09-18T04:58:45.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.940 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:45.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:45.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:58:45.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.940 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:45.940 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956693+0000", 2024-09-18T04:58:45.940 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.940 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.940 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:45.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956555+0000", 2024-09-18T04:58:45.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9998612 2024-09-18T04:58:45.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:45.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956696+0000", 2024-09-18T04:58:45.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000140922 2024-09-18T04:58:45.941 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.941 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.941 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:45.941 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956707+0000", 2024-09-18T04:58:45.941 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1555e-05 2024-09-18T04:58:45.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:45.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.956714+0000", 2024-09-18T04:58:45.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2709999999999998e-06 2024-09-18T04:58:45.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:45.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005177+0000", 2024-09-18T04:58:45.942 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048463421 2024-09-18T04:58:45.942 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.942 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.942 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:45.942 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005214+0000", 2024-09-18T04:58:45.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.676e-05 2024-09-18T04:58:45.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.943 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:45.943 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.005279+0000", 2024-09-18T04:58:45.943 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4899000000000003e-05 2024-09-18T04:58:45.943 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.943 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:45.943 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.943 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.943 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.943 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72199 3.9 3:95be3a3b:::rbd_data.11acf2b3fa50.0000000000000006:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:45.943 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:58:45.943 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.904575726, 2024-09-18T04:58:45.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048626057, 2024-09-18T04:58:45.944 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:45.944 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:45.944 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:45.944 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:45.944 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:45.944 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72199 2024-09-18T04:58:45.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.944 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:45.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.944 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:45.944 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:58:45.944 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.944 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.944 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:45.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959225+0000", 2024-09-18T04:58:45.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:45.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959195+0000", 2024-09-18T04:58:45.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999969 2024-09-18T04:58:45.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.945 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.945 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:45.945 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959228+0000", 2024-09-18T04:58:45.945 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3609000000000003e-05 2024-09-18T04:58:45.945 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:45.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959236+0000", 2024-09-18T04:58:45.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.157e-06 2024-09-18T04:58:45.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:45.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.959243+0000", 2024-09-18T04:58:45.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5910000000000001e-06 2024-09-18T04:58:45.946 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.946 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.946 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:45.946 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007684+0000", 2024-09-18T04:58:45.946 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048440745 2024-09-18T04:58:45.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:45.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007762+0000", 2024-09-18T04:58:45.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8360000000000002e-05 2024-09-18T04:58:45.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.947 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.947 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:45.947 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.007851+0000", 2024-09-18T04:58:45.947 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.9388000000000005e-05 2024-09-18T04:58:45.947 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.947 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:45.947 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.947 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.948 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:72240 3.9 3:9247b2ec:::rbd_data.11acf2b3fa50.0000000000000000:17 [read 428032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-09-18T04:58:45.948 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:58:45.948 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 108.900482416, 2024-09-18T04:58:45.948 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048643767999999997, 2024-09-18T04:58:45.948 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:45.948 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:45.948 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:45.948 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:45.948 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:45.948 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 72240 2024-09-18T04:58:45.948 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.948 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:45.948 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:45.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:58:45.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:45.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963319+0000", 2024-09-18T04:58:45.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.949 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.949 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.949 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:45.949 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963240+0000", 2024-09-18T04:58:45.949 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999218 2024-09-18T04:58:45.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:45.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963323+0000", 2024-09-18T04:58:45.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2375999999999999e-05 2024-09-18T04:58:45.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:45.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963332+0000", 2024-09-18T04:58:45.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.1700000000000003e-06 2024-09-18T04:58:45.950 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.950 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:45.950 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:56.963338+0000", 2024-09-18T04:58:45.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.2149999999999998e-06 2024-09-18T04:58:45.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:45.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011785+0000", 2024-09-18T04:58:45.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.048446493 2024-09-18T04:58:45.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.951 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:45.951 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011861+0000", 2024-09-18T04:58:45.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6472999999999998e-05 2024-09-18T04:58:45.951 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.951 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.952 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:45.952 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:57.011962+0000", 2024-09-18T04:58:45.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000101167 2024-09-18T04:58:45.952 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.952 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:45.952 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.952 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.952 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:117375 3.1 3:80b18d38:::rbd_header.11acae5d5365:head [watch reconnect cookie 140406307615440 gen 3] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T04:58:45.952 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:58:45.952 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 82.479407878999993, 2024-09-18T04:58:45.952 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.018908156, 2024-09-18T04:58:45.952 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:45.952 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:45.953 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:45.953 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:45.953 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:45.953 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117375 2024-09-18T04:58:45.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.953 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:45.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:45.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:58:45.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:45.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384393+0000", 2024-09-18T04:58:45.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:45.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384391+0000", 2024-09-18T04:58:45.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:58:45.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:45.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384401+0000", 2024-09-18T04:58:45.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4299999999999995e-06 2024-09-18T04:58:45.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:45.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384418+0000", 2024-09-18T04:58:45.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.6745e-05 2024-09-18T04:58:45.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:45.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384429+0000", 2024-09-18T04:58:45.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1122e-05 2024-09-18T04:58:45.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:45.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384496+0000", 2024-09-18T04:58:45.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6841000000000006e-05 2024-09-18T04:58:45.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-18T04:58:45.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.384505+0000", 2024-09-18T04:58:45.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4080000000000004e-06 2024-09-18T04:58:45.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:45.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.388971+0000", 2024-09-18T04:58:45.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0044660000000000004 2024-09-18T04:58:45.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:45.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.388981+0000", 2024-09-18T04:58:45.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0118e-05 2024-09-18T04:58:45.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:45.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.399433+0000", 2024-09-18T04:58:45.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.010452055 2024-09-18T04:58:45.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:45.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.399594+0000", 2024-09-18T04:58:45.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000160669 2024-09-18T04:58:45.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:45.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.403301+0000", 2024-09-18T04:58:45.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0037074370000000001 2024-09-18T04:58:45.958 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.958 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:45.958 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.958 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:117376 3.1 3:80b18d38:::rbd_header.11acae5d5365:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-09-18T04:58:45.958 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:58:45.958 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 82.477356228999994, 2024-09-18T04:58:45.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0196134539999999, 2024-09-18T04:58:45.958 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:45.958 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:45.958 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:45.958 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:45.959 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:45.959 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 117376 2024-09-18T04:58:45.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.959 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:45.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:45.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:58:45.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.959 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:45.959 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386445+0000", 2024-09-18T04:58:45.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:45.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386443+0000", 2024-09-18T04:58:45.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:58:45.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:45.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386457+0000", 2024-09-18T04:58:45.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4207e-05 2024-09-18T04:58:45.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:45.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386478+0000", 2024-09-18T04:58:45.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0349000000000001e-05 2024-09-18T04:58:45.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:45.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386503+0000", 2024-09-18T04:58:45.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5582e-05 2024-09-18T04:58:45.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:45.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386558+0000", 2024-09-18T04:58:45.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4892999999999998e-05 2024-09-18T04:58:45.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-18T04:58:45.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.386562+0000", 2024-09-18T04:58:45.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.313e-06 2024-09-18T04:58:45.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:45.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.389010+0000", 2024-09-18T04:58:45.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024480890000000001 2024-09-18T04:58:45.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:45.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:23.389018+0000", 2024-09-18T04:58:45.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.8469999999999996e-06 2024-09-18T04:58:45.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:45.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400073+0000", 2024-09-18T04:58:45.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.011054286 2024-09-18T04:58:45.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:45.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400124+0000", 2024-09-18T04:58:45.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1332999999999998e-05 2024-09-18T04:58:45.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:58:45.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.400477+0000", 2024-09-18T04:58:45.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00035329300000000002 2024-09-18T04:58:45.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:45.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.401353+0000", 2024-09-18T04:58:45.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00087584200000000005 2024-09-18T04:58:45.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:45.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406006+0000", 2024-09-18T04:58:45.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0046534250000000001 2024-09-18T04:58:45.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:45.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406037+0000", 2024-09-18T04:58:45.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0559000000000003e-05 2024-09-18T04:58:45.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:45.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:24.406058+0000", 2024-09-18T04:58:45.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1260999999999999e-05 2024-09-18T04:58:45.965 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.965 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:45.965 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.965 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167963 3.1a 3:5cea7035:::rbd_info:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e147)", 2024-09-18T04:58:45.965 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:37.082364+0000", 2024-09-18T04:58:45.965 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 8.7814372160000005, 2024-09-18T04:58:45.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94325145700000002, 2024-09-18T04:58:45.966 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:45.966 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:45.966 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:45.966 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:45.966 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:45.966 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167963 2024-09-18T04:58:45.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.966 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:45.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:45.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:37.082364+0000", 2024-09-18T04:58:45.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:45.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:37.082364+0000", 2024-09-18T04:58:45.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:45.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:45.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:37.082362+0000", 2024-09-18T04:58:45.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:58:45.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:45.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:37.082368+0000", 2024-09-18T04:58:45.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8289999999999996e-06 2024-09-18T04:58:45.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:45.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:37.082379+0000", 2024-09-18T04:58:45.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0975000000000001e-05 2024-09-18T04:58:45.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:45.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:37.082403+0000", 2024-09-18T04:58:45.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4029e-05 2024-09-18T04:58:45.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:45.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:37.082443+0000", 2024-09-18T04:58:45.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9947999999999998e-05 2024-09-18T04:58:45.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:45.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:37.082450+0000", 2024-09-18T04:58:45.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0350000000000004e-06 2024-09-18T04:58:45.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:45.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:38.025368+0000", 2024-09-18T04:58:45.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94291755399999999 2024-09-18T04:58:45.969 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.969 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.969 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:45.969 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:38.025504+0000", 2024-09-18T04:58:45.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013585100000000001 2024-09-18T04:58:45.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:45.970 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:45.970 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:45.970 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:38.025615+0000", 2024-09-18T04:58:45.970 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000111618 2024-09-18T04:58:45.970 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.970 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:45.970 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.970 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:45.970 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:45.970 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:45.971 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:58:45.972 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:58:45.972 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:58:45.973 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:58:45.973 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:58:45.974 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:45.969+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:46.072 INFO:tasks.ceph.osd.6.smithi099.stderr:2024-09-18T04:58:46.070+0000 7fb4ae62a640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 33006) UID: 0 2024-09-18T04:58:46.112 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-09-18T04:58:46.120 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:46.120 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:46.120 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:58:46.121 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:46.129 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:58:46.173 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:46.170+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:46.274 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:46.271+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:46.274 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:46.274 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:46.275 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:58:46.275 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:58:46.275 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:46.283 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:58:46.375 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:46.373+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:46.386 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:46.386 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:58:46.386 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:58:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60345 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1471488~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:58:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 115.271262871, 2024-09-18T04:58:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044180090999999998, 2024-09-18T04:58:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60345 2024-09-18T04:58:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.387 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:58:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112239+0000", 2024-09-18T04:58:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112238+0000", 2024-09-18T04:58:46.388 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112240+0000", 2024-09-18T04:58:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9910000000000001e-06 2024-09-18T04:58:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112246+0000", 2024-09-18T04:58:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0859999999999998e-06 2024-09-18T04:58:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.389 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112251+0000", 2024-09-18T04:58:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5020000000000003e-06 2024-09-18T04:58:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156274+0000", 2024-09-18T04:58:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044023063000000001 2024-09-18T04:58:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156337+0000", 2024-09-18T04:58:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3077000000000006e-05 2024-09-18T04:58:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.390 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156419+0000", 2024-09-18T04:58:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1699000000000005e-05 2024-09-18T04:58:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60346 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1486848~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:58:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 115.27124508999999, 2024-09-18T04:58:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044336331999999999, 2024-09-18T04:58:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.391 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:46.392 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.392 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.392 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.392 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60346 2024-09-18T04:58:46.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.392 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.392 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:58:46.392 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.392 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.392 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.392 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112257+0000", 2024-09-18T04:58:46.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.393 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.393 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.393 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112256+0000", 2024-09-18T04:58:46.393 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:46.393 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112258+0000", 2024-09-18T04:58:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1720000000000001e-06 2024-09-18T04:58:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112265+0000", 2024-09-18T04:58:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4280000000000001e-06 2024-09-18T04:58:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.394 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112269+0000", 2024-09-18T04:58:46.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.741e-06 2024-09-18T04:58:46.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156445+0000", 2024-09-18T04:58:46.395 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044175341 2024-09-18T04:58:46.395 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.395 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.395 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.395 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156512+0000", 2024-09-18T04:58:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6723000000000005e-05 2024-09-18T04:58:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156593+0000", 2024-09-18T04:58:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1284000000000003e-05 2024-09-18T04:58:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.396 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60347 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1502208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:46.397 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:58:46.397 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 115.27121667900001, 2024-09-18T04:58:46.397 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044482560999999997, 2024-09-18T04:58:46.397 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.397 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:46.397 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60347 2024-09-18T04:58:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:58:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112285+0000", 2024-09-18T04:58:46.398 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112284+0000", 2024-09-18T04:58:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112286+0000", 2024-09-18T04:58:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9709999999999998e-06 2024-09-18T04:58:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.399 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112290+0000", 2024-09-18T04:58:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9199999999999997e-06 2024-09-18T04:58:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112293+0000", 2024-09-18T04:58:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8870000000000001e-06 2024-09-18T04:58:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156617+0000", 2024-09-18T04:58:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044323960000000003 2024-09-18T04:58:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.400 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156687+0000", 2024-09-18T04:58:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0141000000000005e-05 2024-09-18T04:58:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156767+0000", 2024-09-18T04:58:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0140000000000002e-05 2024-09-18T04:58:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.401 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60348 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1517568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:46.402 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:58:46.402 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 115.271205675, 2024-09-18T04:58:46.402 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044645562, 2024-09-18T04:58:46.402 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.402 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:46.402 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.402 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.402 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.402 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60348 2024-09-18T04:58:46.402 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.403 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.403 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.403 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.403 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:58:46.403 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.403 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.404 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.404 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.404 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:58:46.404 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.404 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112296+0000", 2024-09-18T04:58:46.405 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:46.405 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.405 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.405 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.405 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112297+0000", 2024-09-18T04:58:46.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.384e-06 2024-09-18T04:58:46.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112301+0000", 2024-09-18T04:58:46.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0570000000000003e-06 2024-09-18T04:58:46.406 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.406 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.406 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112304+0000", 2024-09-18T04:58:46.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6000000000000001e-06 2024-09-18T04:58:46.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156793+0000", 2024-09-18T04:58:46.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044489494999999997 2024-09-18T04:58:46.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156859+0000", 2024-09-18T04:58:46.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.5483999999999998e-05 2024-09-18T04:58:46.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156941+0000", 2024-09-18T04:58:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2856999999999996e-05 2024-09-18T04:58:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60349 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1532928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:58:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 115.271195699, 2024-09-18T04:58:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044832167999999999, 2024-09-18T04:58:46.410 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60349 2024-09-18T04:58:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:58:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:58:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112306+0000", 2024-09-18T04:58:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112307+0000", 2024-09-18T04:58:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.308e-06 2024-09-18T04:58:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112310+0000", 2024-09-18T04:58:46.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-18T04:58:46.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112312+0000", 2024-09-18T04:58:46.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4889999999999998e-06 2024-09-18T04:58:46.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.156977+0000", 2024-09-18T04:58:46.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044664598999999999 2024-09-18T04:58:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157053+0000", 2024-09-18T04:58:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6014999999999996e-05 2024-09-18T04:58:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157138+0000", 2024-09-18T04:58:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5162999999999998e-05 2024-09-18T04:58:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.415 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60350 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1548288~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:46.415 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:58:46.415 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 115.271186785, 2024-09-18T04:58:46.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045011858000000002, 2024-09-18T04:58:46.415 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.415 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:46.415 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.416 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.416 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.416 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60350 2024-09-18T04:58:46.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.416 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.416 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.416 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:58:46.416 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.416 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:58:46.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112315+0000", 2024-09-18T04:58:46.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:46.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112316+0000", 2024-09-18T04:58:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7579999999999999e-06 2024-09-18T04:58:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112321+0000", 2024-09-18T04:58:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1649999999999997e-06 2024-09-18T04:58:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112325+0000", 2024-09-18T04:58:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.726e-06 2024-09-18T04:58:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157172+0000", 2024-09-18T04:58:46.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.04484718 2024-09-18T04:58:46.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157242+0000", 2024-09-18T04:58:46.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9675e-05 2024-09-18T04:58:46.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157327+0000", 2024-09-18T04:58:46.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4614e-05 2024-09-18T04:58:46.420 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60351 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1563648~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:46.421 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:58:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 115.271173369, 2024-09-18T04:58:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045132113000000001, 2024-09-18T04:58:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.422 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.423 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60351 2024-09-18T04:58:46.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.423 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.423 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.423 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.423 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:58:46.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:58:46.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112328+0000", 2024-09-18T04:58:46.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:46.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112330+0000", 2024-09-18T04:58:46.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1660000000000001e-06 2024-09-18T04:58:46.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112336+0000", 2024-09-18T04:58:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5940000000000001e-06 2024-09-18T04:58:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.112340+0000", 2024-09-18T04:58:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5589999999999999e-06 2024-09-18T04:58:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157365+0000", 2024-09-18T04:58:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.045024675 2024-09-18T04:58:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157409+0000", 2024-09-18T04:58:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4356e-05 2024-09-18T04:58:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.157460+0000", 2024-09-18T04:58:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1072000000000002e-05 2024-09-18T04:58:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.431 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60502 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1509376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:58:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 115.265097196, 2024-09-18T04:58:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044092438999999997, 2024-09-18T04:58:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60502 2024-09-18T04:58:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.432 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:58:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:58:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.433 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118404+0000", 2024-09-18T04:58:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118406+0000", 2024-09-18T04:58:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:58:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118412+0000", 2024-09-18T04:58:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.524e-06 2024-09-18T04:58:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118416+0000", 2024-09-18T04:58:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:58:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162418+0000", 2024-09-18T04:58:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044001508000000002 2024-09-18T04:58:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162451+0000", 2024-09-18T04:58:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3662999999999999e-05 2024-09-18T04:58:46.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162497+0000", 2024-09-18T04:58:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5550000000000003e-05 2024-09-18T04:58:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60503 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1524736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:58:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 115.26508103800001, 2024-09-18T04:58:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044178780000000001, 2024-09-18T04:58:46.436 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60503 2024-09-18T04:58:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:58:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118421+0000", 2024-09-18T04:58:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118420+0000", 2024-09-18T04:58:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118422+0000", 2024-09-18T04:58:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.08e-06 2024-09-18T04:58:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118428+0000", 2024-09-18T04:58:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.964e-06 2024-09-18T04:58:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118433+0000", 2024-09-18T04:58:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.425e-06 2024-09-18T04:58:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162519+0000", 2024-09-18T04:58:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044086082999999998 2024-09-18T04:58:46.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162552+0000", 2024-09-18T04:58:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3719e-05 2024-09-18T04:58:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162599+0000", 2024-09-18T04:58:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6873999999999999e-05 2024-09-18T04:58:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60504 3.5 3:a1b26b6d:::rbd_data.11acf2b3fa50.0000000000000004:17 [read 1540096~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:58:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 115.265064933, 2024-09-18T04:58:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044261921000000003, 2024-09-18T04:58:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60504 2024-09-18T04:58:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:58:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118437+0000", 2024-09-18T04:58:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118436+0000", 2024-09-18T04:58:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118438+0000", 2024-09-18T04:58:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0250000000000001e-06 2024-09-18T04:58:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118444+0000", 2024-09-18T04:58:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4360000000000001e-06 2024-09-18T04:58:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.118448+0000", 2024-09-18T04:58:46.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.357e-06 2024-09-18T04:58:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162620+0000", 2024-09-18T04:58:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044171821 2024-09-18T04:58:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162653+0000", 2024-09-18T04:58:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3479999999999998e-05 2024-09-18T04:58:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.162699+0000", 2024-09-18T04:58:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5113e-05 2024-09-18T04:58:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60585 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1211392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:58:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 115.263255808, 2024-09-18T04:58:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044389814, 2024-09-18T04:58:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.445 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60585 2024-09-18T04:58:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:58:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120246+0000", 2024-09-18T04:58:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120245+0000", 2024-09-18T04:58:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120247+0000", 2024-09-18T04:58:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0150000000000002e-06 2024-09-18T04:58:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120255+0000", 2024-09-18T04:58:46.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.858e-06 2024-09-18T04:58:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120260+0000", 2024-09-18T04:58:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6530000000000002e-06 2024-09-18T04:58:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164489+0000", 2024-09-18T04:58:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044229483999999999 2024-09-18T04:58:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.448 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164555+0000", 2024-09-18T04:58:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.533e-05 2024-09-18T04:58:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164636+0000", 2024-09-18T04:58:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.0819e-05 2024-09-18T04:58:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60586 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1226752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:46.449 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:58:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 115.26323735699999, 2024-09-18T04:58:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044578214999999997, 2024-09-18T04:58:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60586 2024-09-18T04:58:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:58:46.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:58:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120264+0000", 2024-09-18T04:58:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.451 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120266+0000", 2024-09-18T04:58:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0660000000000002e-06 2024-09-18T04:58:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120272+0000", 2024-09-18T04:58:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9959999999999999e-06 2024-09-18T04:58:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120276+0000", 2024-09-18T04:58:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.476e-06 2024-09-18T04:58:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164673+0000", 2024-09-18T04:58:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044396672999999998 2024-09-18T04:58:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164749+0000", 2024-09-18T04:58:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5811999999999999e-05 2024-09-18T04:58:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164842+0000", 2024-09-18T04:58:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.3560000000000005e-05 2024-09-18T04:58:46.453 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60587 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1242112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:58:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 115.263188258, 2024-09-18T04:58:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044634794999999998, 2024-09-18T04:58:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.454 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60587 2024-09-18T04:58:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:58:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120313+0000", 2024-09-18T04:58:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120280+0000", 2024-09-18T04:58:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999671 2024-09-18T04:58:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120316+0000", 2024-09-18T04:58:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5728000000000001e-05 2024-09-18T04:58:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120329+0000", 2024-09-18T04:58:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2914e-05 2024-09-18T04:58:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.120335+0000", 2024-09-18T04:58:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8289999999999996e-06 2024-09-18T04:58:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164858+0000", 2024-09-18T04:58:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044523417000000003 2024-09-18T04:58:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164896+0000", 2024-09-18T04:58:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7969999999999997e-05 2024-09-18T04:58:46.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.164948+0000", 2024-09-18T04:58:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1996000000000002e-05 2024-09-18T04:58:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60596 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1380352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:58:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 115.261328721, 2024-09-18T04:58:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044120800000000002, 2024-09-18T04:58:46.458 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60596 2024-09-18T04:58:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:58:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.459 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:58:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122173+0000", 2024-09-18T04:58:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122175+0000", 2024-09-18T04:58:46.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1270000000000001e-06 2024-09-18T04:58:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122182+0000", 2024-09-18T04:58:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3509999999999996e-06 2024-09-18T04:58:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122187+0000", 2024-09-18T04:58:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7940000000000002e-06 2024-09-18T04:58:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166184+0000", 2024-09-18T04:58:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.043997236000000002 2024-09-18T04:58:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166227+0000", 2024-09-18T04:58:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2561000000000001e-05 2024-09-18T04:58:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166294+0000", 2024-09-18T04:58:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.7068000000000002e-05 2024-09-18T04:58:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.462 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:60597 3.1d 3:bf184944:::rbd_data.11acf2b3fa50.0000000000000007:17 [read 1395712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:58:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 115.261310693, 2024-09-18T04:58:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044242166999999999, 2024-09-18T04:58:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 60597 2024-09-18T04:58:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.463 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:58:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:58:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122191+0000", 2024-09-18T04:58:46.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122193+0000", 2024-09-18T04:58:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.1289999999999999e-06 2024-09-18T04:58:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122199+0000", 2024-09-18T04:58:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6449999999999999e-06 2024-09-18T04:58:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.122204+0000", 2024-09-18T04:58:46.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4819999999999996e-06 2024-09-18T04:58:46.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166329+0000", 2024-09-18T04:58:46.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.044125059000000001 2024-09-18T04:58:46.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166370+0000", 2024-09-18T04:58:46.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1012e-05 2024-09-18T04:58:46.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.166433+0000", 2024-09-18T04:58:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3178999999999999e-05 2024-09-18T04:58:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167074 3.5 3:a712c57a:::rbd_data.11ac138032e7.0000000000000000:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.163703+0000", 2024-09-18T04:58:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.219798512000001, 2024-09-18T04:58:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.90992387799999996, 2024-09-18T04:58:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:46.467 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167074 2024-09-18T04:58:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163703+0000", 2024-09-18T04:58:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163703+0000", 2024-09-18T04:58:46.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163702+0000", 2024-09-18T04:58:46.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:58:46.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.469 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163707+0000", 2024-09-18T04:58:46.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5159999999999996e-06 2024-09-18T04:58:46.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163716+0000", 2024-09-18T04:58:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.5270000000000004e-06 2024-09-18T04:58:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163723+0000", 2024-09-18T04:58:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9809999999999997e-06 2024-09-18T04:58:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163781+0000", 2024-09-18T04:58:46.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7494999999999998e-05 2024-09-18T04:58:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163791+0000", 2024-09-18T04:58:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0614999999999999e-05 2024-09-18T04:58:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067213+0000", 2024-09-18T04:58:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.90342179700000003 2024-09-18T04:58:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067327+0000", 2024-09-18T04:58:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000114023 2024-09-18T04:58:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-18T04:58:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067432+0000", 2024-09-18T04:58:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000104775 2024-09-18T04:58:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.071972+0000", 2024-09-18T04:58:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0045401219999999997 2024-09-18T04:58:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073530+0000", 2024-09-18T04:58:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001558341 2024-09-18T04:58:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073582+0000", 2024-09-18T04:58:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1924e-05 2024-09-18T04:58:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073627+0000", 2024-09-18T04:58:46.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4647000000000001e-05 2024-09-18T04:58:46.474 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.474 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.474 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.474 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167075 3.1 3:864982b2:::rbd_data.11ac138032e7.0000000000000001:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:46.474 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.163740+0000", 2024-09-18T04:58:46.474 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.219761244000001, 2024-09-18T04:58:46.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.90703181499999996, 2024-09-18T04:58:46.474 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.474 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167075 2024-09-18T04:58:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163740+0000", 2024-09-18T04:58:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.475 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163740+0000", 2024-09-18T04:58:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163740+0000", 2024-09-18T04:58:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163743+0000", 2024-09-18T04:58:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0230000000000001e-06 2024-09-18T04:58:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163750+0000", 2024-09-18T04:58:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2710000000000003e-06 2024-09-18T04:58:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163756+0000", 2024-09-18T04:58:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7559999999999996e-06 2024-09-18T04:58:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163803+0000", 2024-09-18T04:58:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6576999999999998e-05 2024-09-18T04:58:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163813+0000", 2024-09-18T04:58:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0336000000000001e-05 2024-09-18T04:58:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.066573+0000", 2024-09-18T04:58:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.90276046799999998 2024-09-18T04:58:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.066694+0000", 2024-09-18T04:58:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012100299999999999 2024-09-18T04:58:46.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:58:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.066791+0000", 2024-09-18T04:58:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.7141999999999999e-05 2024-09-18T04:58:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.068686+0000", 2024-09-18T04:58:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0018949570000000001 2024-09-18T04:58:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070676+0000", 2024-09-18T04:58:46.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001989158 2024-09-18T04:58:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070733+0000", 2024-09-18T04:58:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7642000000000001e-05 2024-09-18T04:58:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070772+0000", 2024-09-18T04:58:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8949000000000003e-05 2024-09-18T04:58:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.480 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167093 3.1 3:82b38587:::rbd_data.11ac138032e7.0000000000000025:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.173880+0000", 2024-09-18T04:58:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.209621217999999, 2024-09-18T04:58:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.89950706999999996, 2024-09-18T04:58:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167093 2024-09-18T04:58:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173880+0000", 2024-09-18T04:58:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173880+0000", 2024-09-18T04:58:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173880+0000", 2024-09-18T04:58:46.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173884+0000", 2024-09-18T04:58:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9910000000000004e-06 2024-09-18T04:58:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173895+0000", 2024-09-18T04:58:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1238e-05 2024-09-18T04:58:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173903+0000", 2024-09-18T04:58:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.2989999999999997e-06 2024-09-18T04:58:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173954+0000", 2024-09-18T04:58:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0862999999999997e-05 2024-09-18T04:58:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173965+0000", 2024-09-18T04:58:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0658e-05 2024-09-18T04:58:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067554+0000", 2024-09-18T04:58:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.89358902200000001 2024-09-18T04:58:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067664+0000", 2024-09-18T04:58:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010977200000000001 2024-09-18T04:58:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:58:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067754+0000", 2024-09-18T04:58:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.0484999999999998e-05 2024-09-18T04:58:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070563+0000", 2024-09-18T04:58:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0028086650000000001 2024-09-18T04:58:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073294+0000", 2024-09-18T04:58:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027316599999999999 2024-09-18T04:58:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073351+0000", 2024-09-18T04:58:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6873999999999998e-05 2024-09-18T04:58:46.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073387+0000", 2024-09-18T04:58:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6254000000000002e-05 2024-09-18T04:58:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167095 3.5 3:a340fea6:::rbd_data.11ac138032e7.0000000000000027:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.173930+0000", 2024-09-18T04:58:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.209571484000001, 2024-09-18T04:58:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.89985602499999995, 2024-09-18T04:58:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.487 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:46.488 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.488 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.488 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.488 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167095 2024-09-18T04:58:46.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.488 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173930+0000", 2024-09-18T04:58:46.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173930+0000", 2024-09-18T04:58:46.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173929+0000", 2024-09-18T04:58:46.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:46.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173933+0000", 2024-09-18T04:58:46.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4860000000000002e-06 2024-09-18T04:58:46.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173940+0000", 2024-09-18T04:58:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4499999999999998e-06 2024-09-18T04:58:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173945+0000", 2024-09-18T04:58:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5290000000000002e-06 2024-09-18T04:58:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173981+0000", 2024-09-18T04:58:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6548999999999999e-05 2024-09-18T04:58:46.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.173990+0000", 2024-09-18T04:58:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.1810000000000001e-06 2024-09-18T04:58:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.069067+0000", 2024-09-18T04:58:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.89507692699999997 2024-09-18T04:58:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.069196+0000", 2024-09-18T04:58:46.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000129572 2024-09-18T04:58:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-18T04:58:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.069316+0000", 2024-09-18T04:58:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000120165 2024-09-18T04:58:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.072036+0000", 2024-09-18T04:58:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027195489999999999 2024-09-18T04:58:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073703+0000", 2024-09-18T04:58:46.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001667555 2024-09-18T04:58:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073749+0000", 2024-09-18T04:58:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5636000000000001e-05 2024-09-18T04:58:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073786+0000", 2024-09-18T04:58:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7070000000000003e-05 2024-09-18T04:58:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.493 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167097 3.c 3:3137739a:::rbd_data.11ac138032e7.0000000000000029:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.178900+0000", 2024-09-18T04:58:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.204601698000001, 2024-09-18T04:58:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.89454723999999997, 2024-09-18T04:58:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167097 2024-09-18T04:58:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.494 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.495 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.495 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178900+0000", 2024-09-18T04:58:46.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178900+0000", 2024-09-18T04:58:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178899+0000", 2024-09-18T04:58:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:58:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178903+0000", 2024-09-18T04:58:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7699999999999999e-06 2024-09-18T04:58:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178913+0000", 2024-09-18T04:58:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0032e-05 2024-09-18T04:58:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178918+0000", 2024-09-18T04:58:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9740000000000001e-06 2024-09-18T04:58:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178959+0000", 2024-09-18T04:58:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1257999999999997e-05 2024-09-18T04:58:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178966+0000", 2024-09-18T04:58:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.4139999999999999e-06 2024-09-18T04:58:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.066273+0000", 2024-09-18T04:58:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.88730742299999998 2024-09-18T04:58:46.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.066394+0000", 2024-09-18T04:58:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00012086099999999999 2024-09-18T04:58:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 6", 2024-09-18T04:58:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.066503+0000", 2024-09-18T04:58:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000109218 2024-09-18T04:58:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:46.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070854+0000", 2024-09-18T04:58:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0043508219999999998 2024-09-18T04:58:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073373+0000", 2024-09-18T04:58:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0025193519999999999 2024-09-18T04:58:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073413+0000", 2024-09-18T04:58:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9592999999999999e-05 2024-09-18T04:58:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.501 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073447+0000", 2024-09-18T04:58:46.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4239999999999997e-05 2024-09-18T04:58:46.501 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.501 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.501 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.501 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.501 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.501 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:46.503 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:58:46.503 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:46.500+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:46.603 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:46.600+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:46.646 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:46.646 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:46.646 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 0 2024-09-18T04:58:46.646 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:46.654 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:58:46.703 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:46.701+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:46.797 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:46.797 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:46.797 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:58:46.797 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:58:46.797 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:46.804 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:46.802+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:46.805 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:58:46.905 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:46.903+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:46.950 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:46.950 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:58:46.950 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:58:46.950 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:46.950 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.951 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26200 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 32768~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:46.951 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:58:46.951 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 126.306446335, 2024-09-18T04:58:46.951 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022501118000000001, 2024-09-18T04:58:46.951 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.952 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:46.952 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.952 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.952 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.952 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26200 2024-09-18T04:58:46.952 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.952 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.953 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:58:46.953 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.953 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.953 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.953 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:58:46.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.954 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.954 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639820+0000", 2024-09-18T04:58:46.954 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:46.954 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.954 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639822+0000", 2024-09-18T04:58:46.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.007e-06 2024-09-18T04:58:46.955 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.955 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.955 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.955 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639827+0000", 2024-09-18T04:58:46.955 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5709999999999997e-06 2024-09-18T04:58:46.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.956 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.956 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639831+0000", 2024-09-18T04:58:46.956 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2819999999999998e-06 2024-09-18T04:58:46.956 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.956 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.957 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662181+0000", 2024-09-18T04:58:46.957 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022349878 2024-09-18T04:58:46.957 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.957 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.957 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662252+0000", 2024-09-18T04:58:46.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0542000000000003e-05 2024-09-18T04:58:46.958 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.958 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.958 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.958 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662321+0000", 2024-09-18T04:58:46.958 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9167999999999999e-05 2024-09-18T04:58:46.958 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.958 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26201 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 61440~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:58:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 126.30643104399999, 2024-09-18T04:58:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022569783, 2024-09-18T04:58:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26201 2024-09-18T04:58:46.959 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.960 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:58:46.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.960 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:58:46.960 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.960 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.960 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.960 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639835+0000", 2024-09-18T04:58:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639837+0000", 2024-09-18T04:58:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.928e-06 2024-09-18T04:58:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639843+0000", 2024-09-18T04:58:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8130000000000001e-06 2024-09-18T04:58:46.961 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639847+0000", 2024-09-18T04:58:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0999999999999997e-06 2024-09-18T04:58:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662335+0000", 2024-09-18T04:58:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022488207999999999 2024-09-18T04:58:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.962 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662370+0000", 2024-09-18T04:58:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5292e-05 2024-09-18T04:58:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662405+0000", 2024-09-18T04:58:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4765999999999999e-05 2024-09-18T04:58:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26202 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 90112~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:58:46.963 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 126.30641562700001, 2024-09-18T04:58:46.964 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022655937000000001, 2024-09-18T04:58:46.964 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.964 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:46.964 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.964 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.964 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.964 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26202 2024-09-18T04:58:46.964 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.964 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.964 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.964 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.964 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:58:46.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639851+0000", 2024-09-18T04:58:46.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.965 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.965 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.965 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.965 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639850+0000", 2024-09-18T04:58:46.965 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:46.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639852+0000", 2024-09-18T04:58:46.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.9700000000000002e-06 2024-09-18T04:58:46.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.966 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639858+0000", 2024-09-18T04:58:46.966 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.434e-06 2024-09-18T04:58:46.966 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.966 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.966 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639862+0000", 2024-09-18T04:58:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3499999999999999e-06 2024-09-18T04:58:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662429+0000", 2024-09-18T04:58:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022567409 2024-09-18T04:58:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662469+0000", 2024-09-18T04:58:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9801000000000001e-05 2024-09-18T04:58:46.967 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.968 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.968 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662507+0000", 2024-09-18T04:58:46.968 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7311000000000002e-05 2024-09-18T04:58:46.968 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.968 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.968 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.968 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.968 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.968 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26203 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 118784~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:46.969 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:58:46.969 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 126.30640047200001, 2024-09-18T04:58:46.969 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022730358999999999, 2024-09-18T04:58:46.969 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.969 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:46.970 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.970 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.970 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.970 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26203 2024-09-18T04:58:46.970 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.971 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.971 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.971 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.971 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:58:46.971 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.971 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.972 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639866+0000", 2024-09-18T04:58:46.972 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.972 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.972 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.972 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639865+0000", 2024-09-18T04:58:46.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:46.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.973 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639867+0000", 2024-09-18T04:58:46.973 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0889999999999998e-06 2024-09-18T04:58:46.973 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.973 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.973 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639873+0000", 2024-09-18T04:58:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4979999999999997e-06 2024-09-18T04:58:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639877+0000", 2024-09-18T04:58:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4510000000000002e-06 2024-09-18T04:58:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662528+0000", 2024-09-18T04:58:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022650791999999999 2024-09-18T04:58:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.974 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662563+0000", 2024-09-18T04:58:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4530000000000003e-05 2024-09-18T04:58:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662596+0000", 2024-09-18T04:58:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3463000000000001e-05 2024-09-18T04:58:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.975 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26204 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 147456~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:58:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 126.30638508, 2024-09-18T04:58:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022803233999999999, 2024-09-18T04:58:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26204 2024-09-18T04:58:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.976 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:58:46.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:58:46.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.977 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.977 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.977 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.977 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639881+0000", 2024-09-18T04:58:46.977 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639883+0000", 2024-09-18T04:58:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6520000000000002e-06 2024-09-18T04:58:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639888+0000", 2024-09-18T04:58:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2870000000000004e-06 2024-09-18T04:58:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.978 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639893+0000", 2024-09-18T04:58:46.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3839999999999999e-06 2024-09-18T04:58:46.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662617+0000", 2024-09-18T04:58:46.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022723876 2024-09-18T04:58:46.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.979 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.979 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662651+0000", 2024-09-18T04:58:46.979 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3797000000000002e-05 2024-09-18T04:58:46.979 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.979 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.980 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.980 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662684+0000", 2024-09-18T04:58:46.980 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.3828999999999997e-05 2024-09-18T04:58:46.980 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.980 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.980 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.980 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.980 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.980 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26205 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 176128~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:46.980 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:58:46.980 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 126.306362119, 2024-09-18T04:58:46.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022919108000000001, 2024-09-18T04:58:46.981 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.981 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:46.981 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.981 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.981 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.981 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26205 2024-09-18T04:58:46.981 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.981 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.981 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.981 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.981 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:58:46.981 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:58:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639904+0000", 2024-09-18T04:58:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.982 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639906+0000", 2024-09-18T04:58:46.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.531e-06 2024-09-18T04:58:46.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639913+0000", 2024-09-18T04:58:46.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.331e-06 2024-09-18T04:58:46.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.983 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.983 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.983 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639920+0000", 2024-09-18T04:58:46.983 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5399999999999998e-06 2024-09-18T04:58:46.983 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662706+0000", 2024-09-18T04:58:46.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.02278587 2024-09-18T04:58:46.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.984 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662764+0000", 2024-09-18T04:58:46.984 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7630000000000002e-05 2024-09-18T04:58:46.984 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.984 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.984 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.985 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662823+0000", 2024-09-18T04:58:46.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9648000000000002e-05 2024-09-18T04:58:46.985 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.985 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.985 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.985 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.985 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.985 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26206 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 204800~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:46.985 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:58:46.985 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 126.30634172000001, 2024-09-18T04:58:46.985 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023047517, 2024-09-18T04:58:46.985 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.985 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:46.986 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.986 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.986 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.986 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26206 2024-09-18T04:58:46.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.986 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.986 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:58:46.986 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.986 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.986 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.986 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:58:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639924+0000", 2024-09-18T04:58:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639926+0000", 2024-09-18T04:58:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3120000000000001e-06 2024-09-18T04:58:46.987 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639932+0000", 2024-09-18T04:58:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4890000000000003e-06 2024-09-18T04:58:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639936+0000", 2024-09-18T04:58:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.07e-06 2024-09-18T04:58:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662850+0000", 2024-09-18T04:58:46.988 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.022913579999999999 2024-09-18T04:58:46.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662903+0000", 2024-09-18T04:58:46.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3176e-05 2024-09-18T04:58:46.989 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.989 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.989 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.989 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.662972+0000", 2024-09-18T04:58:46.989 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9270999999999995e-05 2024-09-18T04:58:46.989 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.989 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.989 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.990 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.990 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.990 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26207 3.a 3:532cf7f3:::rbd_data.11ac82b416ac.0000000000000014:f [read 233472~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:46.990 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:58:46.990 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 126.306326576, 2024-09-18T04:58:46.990 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023139541999999999, 2024-09-18T04:58:46.990 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.990 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:46.990 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.990 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.991 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.991 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26207 2024-09-18T04:58:46.991 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.991 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.991 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.991 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:58:46.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.992 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639940+0000", 2024-09-18T04:58:46.992 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.992 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.992 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.992 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639939+0000", 2024-09-18T04:58:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639941+0000", 2024-09-18T04:58:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0080000000000001e-06 2024-09-18T04:58:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639947+0000", 2024-09-18T04:58:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.485e-06 2024-09-18T04:58:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.993 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639951+0000", 2024-09-18T04:58:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4549999999999997e-06 2024-09-18T04:58:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663005+0000", 2024-09-18T04:58:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023053569999999999 2024-09-18T04:58:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663042+0000", 2024-09-18T04:58:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7245e-05 2024-09-18T04:58:46.994 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663079+0000", 2024-09-18T04:58:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7119000000000002e-05 2024-09-18T04:58:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26208 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 524288~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:58:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 126.306309828, 2024-09-18T04:58:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023283564999999999, 2024-09-18T04:58:46.995 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26208 2024-09-18T04:58:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:58:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.996 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:46.997 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:58:46.997 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:46.997 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:46.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639956+0000", 2024-09-18T04:58:46.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:46.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:46.998 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639958+0000", 2024-09-18T04:58:46.998 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0899999999999999e-06 2024-09-18T04:58:46.998 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.998 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.998 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:46.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639963+0000", 2024-09-18T04:58:46.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2209999999999996e-06 2024-09-18T04:58:46.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:46.999 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:46.999 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:46.999 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639967+0000", 2024-09-18T04:58:46.999 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1169999999999997e-06 2024-09-18T04:58:46.999 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663114+0000", 2024-09-18T04:58:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023146102000000002 2024-09-18T04:58:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663183+0000", 2024-09-18T04:58:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9686999999999999e-05 2024-09-18T04:58:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.000 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663240+0000", 2024-09-18T04:58:47.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6728000000000003e-05 2024-09-18T04:58:47.001 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.001 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.001 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.001 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.001 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.001 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26209 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 552960~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:47.001 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:58:47.001 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 126.306292052, 2024-09-18T04:58:47.001 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023440966000000001, 2024-09-18T04:58:47.001 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.002 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:47.002 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.002 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.002 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.002 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26209 2024-09-18T04:58:47.002 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.002 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.002 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.003 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:58:47.003 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.003 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.003 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.003 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.004 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:58:47.004 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.004 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.004 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.004 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639974+0000", 2024-09-18T04:58:47.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.005 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.005 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.005 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.005 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639976+0000", 2024-09-18T04:58:47.005 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4940000000000002e-06 2024-09-18T04:58:47.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.006 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.006 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639982+0000", 2024-09-18T04:58:47.006 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0560000000000001e-06 2024-09-18T04:58:47.006 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.006 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639987+0000", 2024-09-18T04:58:47.007 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.605e-06 2024-09-18T04:58:47.007 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.007 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.007 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.007 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663278+0000", 2024-09-18T04:58:47.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023290789999999999 2024-09-18T04:58:47.008 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.008 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.008 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.008 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663348+0000", 2024-09-18T04:58:47.008 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0837e-05 2024-09-18T04:58:47.009 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.009 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.009 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.009 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663415+0000", 2024-09-18T04:58:47.009 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6736000000000006e-05 2024-09-18T04:58:47.009 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.009 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26210 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 581632~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:58:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 126.306275044, 2024-09-18T04:58:47.010 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023616437000000001, 2024-09-18T04:58:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26210 2024-09-18T04:58:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.011 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:58:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:58:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.012 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639991+0000", 2024-09-18T04:58:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.639993+0000", 2024-09-18T04:58:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.419e-06 2024-09-18T04:58:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640006+0000", 2024-09-18T04:58:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2918000000000001e-05 2024-09-18T04:58:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.013 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640011+0000", 2024-09-18T04:58:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9320000000000004e-06 2024-09-18T04:58:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663454+0000", 2024-09-18T04:58:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023443035000000001 2024-09-18T04:58:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663532+0000", 2024-09-18T04:58:47.014 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.7443999999999999e-05 2024-09-18T04:58:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663608+0000", 2024-09-18T04:58:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.6093000000000005e-05 2024-09-18T04:58:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26211 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 610304~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:58:47.015 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 126.306250492, 2024-09-18T04:58:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023774337999999999, 2024-09-18T04:58:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26211 2024-09-18T04:58:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.016 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:58:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640016+0000", 2024-09-18T04:58:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640015+0000", 2024-09-18T04:58:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.017 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640023+0000", 2024-09-18T04:58:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3900000000000004e-06 2024-09-18T04:58:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640036+0000", 2024-09-18T04:58:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3091999999999999e-05 2024-09-18T04:58:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640041+0000", 2024-09-18T04:58:47.018 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6219999999999997e-06 2024-09-18T04:58:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663644+0000", 2024-09-18T04:58:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023602643999999999 2024-09-18T04:58:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663718+0000", 2024-09-18T04:58:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4481999999999996e-05 2024-09-18T04:58:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.019 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663790+0000", 2024-09-18T04:58:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1589999999999997e-05 2024-09-18T04:58:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26212 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 638976~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:58:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 126.306217452, 2024-09-18T04:58:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023924748999999999, 2024-09-18T04:58:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.020 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26212 2024-09-18T04:58:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:58:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640049+0000", 2024-09-18T04:58:47.021 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640048+0000", 2024-09-18T04:58:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640051+0000", 2024-09-18T04:58:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.095e-06 2024-09-18T04:58:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640060+0000", 2024-09-18T04:58:47.022 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8929999999999994e-06 2024-09-18T04:58:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640065+0000", 2024-09-18T04:58:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0270000000000003e-06 2024-09-18T04:58:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663824+0000", 2024-09-18T04:58:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023758873999999999 2024-09-18T04:58:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.023 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663904+0000", 2024-09-18T04:58:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.9856000000000003e-05 2024-09-18T04:58:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.663974+0000", 2024-09-18T04:58:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.9450999999999996e-05 2024-09-18T04:58:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.024 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26213 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 667648~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:58:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 126.306196281, 2024-09-18T04:58:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024059812999999999, 2024-09-18T04:58:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26213 2024-09-18T04:58:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.025 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:58:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:58:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640070+0000", 2024-09-18T04:58:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.026 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640072+0000", 2024-09-18T04:58:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4200000000000001e-06 2024-09-18T04:58:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640078+0000", 2024-09-18T04:58:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0179999999999998e-06 2024-09-18T04:58:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640083+0000", 2024-09-18T04:58:47.027 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:58:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664008+0000", 2024-09-18T04:58:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023925287 2024-09-18T04:58:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664059+0000", 2024-09-18T04:58:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.0730999999999999e-05 2024-09-18T04:58:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.028 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664130+0000", 2024-09-18T04:58:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0801000000000002e-05 2024-09-18T04:58:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26214 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 696320~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:58:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 126.30617931099999, 2024-09-18T04:58:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024223738000000002, 2024-09-18T04:58:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.029 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26214 2024-09-18T04:58:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:58:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.030 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:58:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640087+0000", 2024-09-18T04:58:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640089+0000", 2024-09-18T04:58:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.0849999999999999e-06 2024-09-18T04:58:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.031 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640095+0000", 2024-09-18T04:58:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9279999999999998e-06 2024-09-18T04:58:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640099+0000", 2024-09-18T04:58:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9490000000000004e-06 2024-09-18T04:58:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664166+0000", 2024-09-18T04:58:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024066513000000001 2024-09-18T04:58:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.032 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664239+0000", 2024-09-18T04:58:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3201999999999995e-05 2024-09-18T04:58:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664311+0000", 2024-09-18T04:58:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1434999999999996e-05 2024-09-18T04:58:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.033 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26215 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 724992~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:58:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 126.30616255, 2024-09-18T04:58:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024302284, 2024-09-18T04:58:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26215 2024-09-18T04:58:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.034 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:58:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640104+0000", 2024-09-18T04:58:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640103+0000", 2024-09-18T04:58:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.035 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640106+0000", 2024-09-18T04:58:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.2120000000000002e-06 2024-09-18T04:58:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640111+0000", 2024-09-18T04:58:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.7880000000000003e-06 2024-09-18T04:58:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640116+0000", 2024-09-18T04:58:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6099999999999999e-06 2024-09-18T04:58:47.036 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664327+0000", 2024-09-18T04:58:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024210941999999999 2024-09-18T04:58:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664367+0000", 2024-09-18T04:58:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9666000000000003e-05 2024-09-18T04:58:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.037 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664406+0000", 2024-09-18T04:58:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.9402e-05 2024-09-18T04:58:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26216 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 753664~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:47.038 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:58:47.039 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 126.306146056, 2024-09-18T04:58:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024452854999999999, 2024-09-18T04:58:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26216 2024-09-18T04:58:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:58:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.040 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:58:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640120+0000", 2024-09-18T04:58:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.041 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640122+0000", 2024-09-18T04:58:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.3599999999999999e-06 2024-09-18T04:58:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640128+0000", 2024-09-18T04:58:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6339999999999998e-06 2024-09-18T04:58:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640132+0000", 2024-09-18T04:58:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.3669999999999999e-06 2024-09-18T04:58:47.042 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664436+0000", 2024-09-18T04:58:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024303585999999999 2024-09-18T04:58:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664507+0000", 2024-09-18T04:58:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1046999999999999e-05 2024-09-18T04:58:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.043 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664573+0000", 2024-09-18T04:58:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.6217000000000007e-05 2024-09-18T04:58:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:26217 3.a 3:512c4223:::rbd_data.11ac82b416ac.0000000000000013:f [read 782336~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e36)", 2024-09-18T04:58:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:58:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 126.306129822, 2024-09-18T04:58:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.024526287000000001, 2024-09-18T04:58:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:47.044 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 26217 2024-09-18T04:58:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:58:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:58:47.045 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640136+0000", 2024-09-18T04:58:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640140+0000", 2024-09-18T04:58:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2989999999999998e-06 2024-09-18T04:58:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.046 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640146+0000", 2024-09-18T04:58:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9569999999999999e-06 2024-09-18T04:58:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.640151+0000", 2024-09-18T04:58:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:58:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664590+0000", 2024-09-18T04:58:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0244389 2024-09-18T04:58:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.047 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664628+0000", 2024-09-18T04:58:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7910999999999997e-05 2024-09-18T04:58:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:40.664663+0000", 2024-09-18T04:58:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5024999999999998e-05 2024-09-18T04:58:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.048 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167100 3.1e 3:7bd358cc:::rbd_data.11ac138032e7.000000000000004f:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.178944+0000", 2024-09-18T04:58:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.767322543999999, 2024-09-18T04:58:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.89428526100000005, 2024-09-18T04:58:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167100 2024-09-18T04:58:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.049 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178944+0000", 2024-09-18T04:58:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178944+0000", 2024-09-18T04:58:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178942+0000", 2024-09-18T04:58:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:58:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.050 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178948+0000", 2024-09-18T04:58:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8100000000000003e-06 2024-09-18T04:58:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178962+0000", 2024-09-18T04:58:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.399e-05 2024-09-18T04:58:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.178970+0000", 2024-09-18T04:58:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.537e-06 2024-09-18T04:58:47.051 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.179015+0000", 2024-09-18T04:58:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5652999999999998e-05 2024-09-18T04:58:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.179021+0000", 2024-09-18T04:58:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5280000000000003e-06 2024-09-18T04:58:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.065479+0000", 2024-09-18T04:58:47.052 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.88645855799999995 2024-09-18T04:58:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.065788+0000", 2024-09-18T04:58:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00030816099999999999 2024-09-18T04:58:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-18T04:58:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.065904+0000", 2024-09-18T04:58:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000116554 2024-09-18T04:58:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.053 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.067684+0000", 2024-09-18T04:58:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001780195 2024-09-18T04:58:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073137+0000", 2024-09-18T04:58:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0054525210000000001 2024-09-18T04:58:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073193+0000", 2024-09-18T04:58:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6492000000000001e-05 2024-09-18T04:58:47.054 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073229+0000", 2024-09-18T04:58:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.5664000000000003e-05 2024-09-18T04:58:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167111 3.19 3:99c9092f:::rbd_data.11ac138032e7.000000000000005a:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.201980+0000", 2024-09-18T04:58:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 27.744285942000001, 2024-09-18T04:58:47.055 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.87136403900000003, 2024-09-18T04:58:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167111 2024-09-18T04:58:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.201980+0000", 2024-09-18T04:58:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.056 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.201980+0000", 2024-09-18T04:58:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.201980+0000", 2024-09-18T04:58:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.201983+0000", 2024-09-18T04:58:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8949999999999998e-06 2024-09-18T04:58:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.057 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.201990+0000", 2024-09-18T04:58:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5129999999999999e-06 2024-09-18T04:58:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.201999+0000", 2024-09-18T04:58:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5490000000000006e-06 2024-09-18T04:58:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.202033+0000", 2024-09-18T04:58:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4076999999999999e-05 2024-09-18T04:58:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.058 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.202038+0000", 2024-09-18T04:58:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3310000000000003e-06 2024-09-18T04:58:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.068314+0000", 2024-09-18T04:58:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.86627544899999998 2024-09-18T04:58:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.068426+0000", 2024-09-18T04:58:47.059 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00011262199999999999 2024-09-18T04:58:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 7", 2024-09-18T04:58:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.068531+0000", 2024-09-18T04:58:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00010510600000000001 2024-09-18T04:58:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.071305+0000", 2024-09-18T04:58:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0027742539999999999 2024-09-18T04:58:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.060 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073258+0000", 2024-09-18T04:58:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.001952264 2024-09-18T04:58:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073303+0000", 2024-09-18T04:58:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5286e-05 2024-09-18T04:58:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073344+0000", 2024-09-18T04:58:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1269000000000001e-05 2024-09-18T04:58:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.061 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.062 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.062 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:47.063 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:58:47.064 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:47.061+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:47.109 INFO:teuthology.orchestra.run.smithi002.stderr:set 1.5 pg_upmap mapping to [1,4] 2024-09-18T04:58:47.109 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is set 2024-09-18T04:58:47.174 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:47.172+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:47.180 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:169271 3.f 3:f65fed3f:::rbd_header.11acdc0eab99:head [call rbd.get_size in=19b,call rbd.get_object_prefix in=20b,call rbd.get_features in=24b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e157)", 2024-09-18T04:58:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:47.177856+0000", 2024-09-18T04:58:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 0.00023875799999999999, 2024-09-18T04:58:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000257292, 2024-09-18T04:58:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.180 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 169271 2024-09-18T04:58:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:47.177856+0000", 2024-09-18T04:58:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:47.177856+0000", 2024-09-18T04:58:47.181 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:47.177855+0000", 2024-09-18T04:58:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:58:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:47.177860+0000", 2024-09-18T04:58:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1710000000000001e-06 2024-09-18T04:58:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:47.177872+0000", 2024-09-18T04:58:47.182 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1812e-05 2024-09-18T04:58:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:47.177878+0000", 2024-09-18T04:58:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6579999999999999e-06 2024-09-18T04:58:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:47.177919+0000", 2024-09-18T04:58:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1548000000000003e-05 2024-09-18T04:58:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:47.178014+0000", 2024-09-18T04:58:47.183 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.4602e-05 2024-09-18T04:58:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: ], 2024-09-18T04:58:47.184 INFO:teuthology.orchestra.run.smithi099.stdout: "num_ops": 1 2024-09-18T04:58:47.184 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:47.189 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:58:47.265 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:47.263+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:47.294 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:47.294 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [], 2024-09-18T04:58:47.294 INFO:teuthology.orchestra.run.smithi099.stdout: "complaint_time": 30, 2024-09-18T04:58:47.294 INFO:teuthology.orchestra.run.smithi099.stdout: "num_blocked_ops": 0 2024-09-18T04:58:47.294 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:47.302 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:58:47.367 INFO:tasks.ceph.osd.2.smithi002.stderr:2024-09-18T04:58:47.364+0000 7f576a1f6640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 33036) UID: 0 2024-09-18T04:58:47.405 INFO:teuthology.orchestra.run.smithi099.stdout:{ 2024-09-18T04:58:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: "size": 20, 2024-09-18T04:58:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 600, 2024-09-18T04:58:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: "ops": [ 2024-09-18T04:58:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4204.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-09-18T04:58:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:58:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 156.00925548399999, 2024-09-18T04:58:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037718511000000003, 2024-09-18T04:58:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:47.406 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4204", 2024-09-18T04:58:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.99:0/3155863658", 2024-09-18T04:58:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 1 2024-09-18T04:58:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:58:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.407 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393126+0000", 2024-09-18T04:58:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393125+0000", 2024-09-18T04:58:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999986 2024-09-18T04:58:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393133+0000", 2024-09-18T04:58:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.225e-06 2024-09-18T04:58:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.408 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393144+0000", 2024-09-18T04:58:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1279e-05 2024-09-18T04:58:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393153+0000", 2024-09-18T04:58:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.4540000000000007e-06 2024-09-18T04:58:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.409 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.410 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.410 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393203+0000", 2024-09-18T04:58:47.410 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9963000000000003e-05 2024-09-18T04:58:47.410 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.410 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:47.411 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.393210+0000", 2024-09-18T04:58:47.411 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5689999999999999e-06 2024-09-18T04:58:47.411 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.411 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.411 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.412 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427399+0000", 2024-09-18T04:58:47.412 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.034188784999999999 2024-09-18T04:58:47.412 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.412 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.412 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427532+0000", 2024-09-18T04:58:47.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013264700000000001 2024-09-18T04:58:47.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 1", 2024-09-18T04:58:47.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.427685+0000", 2024-09-18T04:58:47.413 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00015300599999999999 2024-09-18T04:58:47.413 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.413 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.413 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:47.413 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.428520+0000", 2024-09-18T04:58:47.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00083502499999999998 2024-09-18T04:58:47.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:47.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430772+0000", 2024-09-18T04:58:47.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022518880000000001 2024-09-18T04:58:47.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:47.414 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430817+0000", 2024-09-18T04:58:47.414 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5077000000000002e-05 2024-09-18T04:58:47.414 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.414 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.414 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.415 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:11.430845+0000", 2024-09-18T04:58:47.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8036999999999999e-05 2024-09-18T04:58:47.415 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.415 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.415 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.415 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.415 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.415 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:62520 3.13 3.84cdb893 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-09-18T04:58:47.415 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:47.415 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 116.104945601, 2024-09-18T04:58:47.415 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94452134799999998, 2024-09-18T04:58:47.415 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.415 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "reached pg", 2024-09-18T04:58:47.415 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.416 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.416 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.416 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 62520 2024-09-18T04:58:47.416 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.416 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:47.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:47.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.417 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.417 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.417 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.417 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297436+0000", 2024-09-18T04:58:47.417 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297441+0000", 2024-09-18T04:58:47.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9840000000000001e-06 2024-09-18T04:58:47.418 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.418 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.418 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.418 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297449+0000", 2024-09-18T04:58:47.418 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3999999999999992e-06 2024-09-18T04:58:47.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297455+0000", 2024-09-18T04:58:47.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.3350000000000003e-06 2024-09-18T04:58:47.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.419 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.419 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.419 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297552+0000", 2024-09-18T04:58:47.419 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.6602999999999996e-05 2024-09-18T04:58:47.419 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:47.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:51.297559+0000", 2024-09-18T04:58:47.420 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.52e-06 2024-09-18T04:58:47.420 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.420 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.420 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.420 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241943+0000", 2024-09-18T04:58:47.421 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.94438357100000003 2024-09-18T04:58:47.421 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.421 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.421 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.421 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:56:52.241958+0000", 2024-09-18T04:58:47.422 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.4598e-05 2024-09-18T04:58:47.422 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.422 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.422 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.422 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.422 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.423 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:113179 3.13 3:cf138017:::rbd_data.11ac5f86a3f9.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-09-18T04:58:47.423 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:58:47.423 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 88.715155190000004, 2024-09-18T04:58:47.423 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036508304999999998, 2024-09-18T04:58:47.423 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.423 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:47.423 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.423 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.423 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.423 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 113179 2024-09-18T04:58:47.423 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.424 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.424 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:58:47.424 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.424 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.424 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.424 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.425 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687227+0000", 2024-09-18T04:58:47.425 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.425 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.425 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.425 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.687226+0000", 2024-09-18T04:58:47.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:58:47.426 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.426 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.426 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.426 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688169+0000", 2024-09-18T04:58:47.426 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00094328200000000004 2024-09-18T04:58:47.427 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.427 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.427 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.427 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688182+0000", 2024-09-18T04:58:47.427 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2756e-05 2024-09-18T04:58:47.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688188+0000", 2024-09-18T04:58:47.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.0290000000000002e-06 2024-09-18T04:58:47.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.428 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688225+0000", 2024-09-18T04:58:47.428 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.7301000000000001e-05 2024-09-18T04:58:47.428 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.428 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.428 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.688324+0000", 2024-09-18T04:58:47.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.9419999999999993e-05 2024-09-18T04:58:47.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 4", 2024-09-18T04:58:47.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.700483+0000", 2024-09-18T04:58:47.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012158771000000001 2024-09-18T04:58:47.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.429 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:47.429 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723586+0000", 2024-09-18T04:58:47.429 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.023102952 2024-09-18T04:58:47.429 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.429 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:47.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723634+0000", 2024-09-18T04:58:47.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7468e-05 2024-09-18T04:58:47.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:47.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723693+0000", 2024-09-18T04:58:47.430 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.9539e-05 2024-09-18T04:58:47.430 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.430 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.430 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.430 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:18.723735+0000", 2024-09-18T04:58:47.432 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.1829000000000002e-05 2024-09-18T04:58:47.432 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.432 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.432 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.432 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.432 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.432 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146078 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1449984~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:47.432 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:47.433 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.809077346999999, 2024-09-18T04:58:47.433 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035495325000000001, 2024-09-18T04:58:47.433 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.433 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:47.433 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.433 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.433 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.433 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146078 2024-09-18T04:58:47.433 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.433 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.433 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.433 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593304+0000", 2024-09-18T04:58:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.434 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593306+0000", 2024-09-18T04:58:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.686e-06 2024-09-18T04:58:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593311+0000", 2024-09-18T04:58:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8500000000000002e-06 2024-09-18T04:58:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.435 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593314+0000", 2024-09-18T04:58:47.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0290000000000001e-06 2024-09-18T04:58:47.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628674+0000", 2024-09-18T04:58:47.436 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035360346000000001 2024-09-18T04:58:47.436 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.436 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.436 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.436 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628726+0000", 2024-09-18T04:58:47.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2219999999999998e-05 2024-09-18T04:58:47.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.437 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.437 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628800+0000", 2024-09-18T04:58:47.437 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.3613000000000001e-05 2024-09-18T04:58:47.437 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.437 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.437 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.437 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.437 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.437 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146079 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1499136~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:47.438 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:47.438 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.809065942000004, 2024-09-18T04:58:47.438 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035604581000000003, 2024-09-18T04:58:47.438 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.438 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:47.438 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.438 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.438 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.438 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146079 2024-09-18T04:58:47.438 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.438 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.438 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.438 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:47.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:47.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.439 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.439 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.439 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593316+0000", 2024-09-18T04:58:47.439 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:47.439 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593317+0000", 2024-09-18T04:58:47.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1599999999999999e-06 2024-09-18T04:58:47.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.440 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593320+0000", 2024-09-18T04:58:47.440 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0130000000000002e-06 2024-09-18T04:58:47.440 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.440 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.440 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593322+0000", 2024-09-18T04:58:47.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.6079999999999998e-06 2024-09-18T04:58:47.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628825+0000", 2024-09-18T04:58:47.441 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035502747000000001 2024-09-18T04:58:47.441 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.441 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.441 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.441 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628868+0000", 2024-09-18T04:58:47.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.2719999999999998e-05 2024-09-18T04:58:47.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.442 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.442 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628920+0000", 2024-09-18T04:58:47.442 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2533e-05 2024-09-18T04:58:47.442 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.442 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.442 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.442 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.442 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.442 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146080 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1548288~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:47.443 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:47.443 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.809050443999993, 2024-09-18T04:58:47.443 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035755146000000002, 2024-09-18T04:58:47.443 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.443 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:47.443 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.443 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.443 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.443 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146080 2024-09-18T04:58:47.443 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.444 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:47.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.444 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.444 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:47.444 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.444 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.444 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593331+0000", 2024-09-18T04:58:47.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593333+0000", 2024-09-18T04:58:47.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.858e-06 2024-09-18T04:58:47.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.445 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.445 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.445 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593336+0000", 2024-09-18T04:58:47.445 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.4910000000000001e-06 2024-09-18T04:58:47.445 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593340+0000", 2024-09-18T04:58:47.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2590000000000001e-06 2024-09-18T04:58:47.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.446 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.446 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.446 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.628956+0000", 2024-09-18T04:58:47.446 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035616039000000002 2024-09-18T04:58:47.446 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629031+0000", 2024-09-18T04:58:47.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5504000000000001e-05 2024-09-18T04:58:47.447 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.447 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.447 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.447 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629086+0000", 2024-09-18T04:58:47.447 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5429000000000001e-05 2024-09-18T04:58:47.447 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.447 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.447 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.448 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.448 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.448 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146081 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1597440~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:47.448 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:47.448 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.809039872, 2024-09-18T04:58:47.448 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035873223000000003, 2024-09-18T04:58:47.448 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.448 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:47.448 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.448 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.448 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.448 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146081 2024-09-18T04:58:47.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.449 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:47.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.449 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.449 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.449 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:47.449 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.449 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593342+0000", 2024-09-18T04:58:47.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.450 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.450 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593343+0000", 2024-09-18T04:58:47.450 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3319999999999999e-06 2024-09-18T04:58:47.450 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.450 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593346+0000", 2024-09-18T04:58:47.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9059999999999998e-06 2024-09-18T04:58:47.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.452 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.452 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593348+0000", 2024-09-18T04:58:47.452 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4530000000000001e-06 2024-09-18T04:58:47.452 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.452 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629110+0000", 2024-09-18T04:58:47.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035761814000000003 2024-09-18T04:58:47.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.453 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.453 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629159+0000", 2024-09-18T04:58:47.453 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8971000000000003e-05 2024-09-18T04:58:47.453 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.453 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.454 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.454 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629215+0000", 2024-09-18T04:58:47.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6107000000000003e-05 2024-09-18T04:58:47.454 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.454 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.454 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.454 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.454 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.454 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146082 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1646592~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:47.454 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:47.454 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.809023822, 2024-09-18T04:58:47.454 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035987007000000001, 2024-09-18T04:58:47.454 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.454 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:47.455 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.455 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.455 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.455 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146082 2024-09-18T04:58:47.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.455 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.455 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:47.455 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.455 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.455 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.455 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:47.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593358+0000", 2024-09-18T04:58:47.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.456 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.456 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593360+0000", 2024-09-18T04:58:47.456 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.8560000000000001e-06 2024-09-18T04:58:47.456 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.456 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593365+0000", 2024-09-18T04:58:47.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3410000000000002e-06 2024-09-18T04:58:47.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593369+0000", 2024-09-18T04:58:47.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4850000000000003e-06 2024-09-18T04:58:47.457 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.457 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.457 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.457 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629242+0000", 2024-09-18T04:58:47.457 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035872464999999999 2024-09-18T04:58:47.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629288+0000", 2024-09-18T04:58:47.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6592999999999999e-05 2024-09-18T04:58:47.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.458 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.458 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.458 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629345+0000", 2024-09-18T04:58:47.458 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6575e-05 2024-09-18T04:58:47.458 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.458 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.458 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.458 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.459 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146083 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1695744~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:47.459 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:58:47.459 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.809007975, 2024-09-18T04:58:47.459 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036100587000000003, 2024-09-18T04:58:47.459 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.459 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:47.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.459 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.459 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146083 2024-09-18T04:58:47.459 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.459 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.459 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:58:47.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593374+0000", 2024-09-18T04:58:47.460 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.460 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.460 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.460 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.460 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593373+0000", 2024-09-18T04:58:47.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593376+0000", 2024-09-18T04:58:47.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.244e-06 2024-09-18T04:58:47.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.461 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.461 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.461 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593379+0000", 2024-09-18T04:58:47.461 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.619e-06 2024-09-18T04:58:47.461 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593382+0000", 2024-09-18T04:58:47.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.7199999999999998e-06 2024-09-18T04:58:47.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.462 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.462 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.462 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629368+0000", 2024-09-18T04:58:47.462 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.035986281000000002 2024-09-18T04:58:47.462 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629416+0000", 2024-09-18T04:58:47.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8055e-05 2024-09-18T04:58:47.463 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.463 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.463 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.463 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629474+0000", 2024-09-18T04:58:47.463 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8176e-05 2024-09-18T04:58:47.463 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.464 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.464 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.464 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.464 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.464 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146084 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1744896~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:47.464 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:47.464 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.808994845000001, 2024-09-18T04:58:47.464 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036214349, 2024-09-18T04:58:47.464 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.464 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:47.464 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.465 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.465 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.465 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146084 2024-09-18T04:58:47.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.465 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:47.465 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.465 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.465 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.465 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.465 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:47.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593387+0000", 2024-09-18T04:58:47.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.466 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.466 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.466 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.466 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593388+0000", 2024-09-18T04:58:47.466 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.564e-06 2024-09-18T04:58:47.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593392+0000", 2024-09-18T04:58:47.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0579999999999999e-06 2024-09-18T04:58:47.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593395+0000", 2024-09-18T04:58:47.467 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8890000000000002e-06 2024-09-18T04:58:47.467 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.467 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.467 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.467 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629500+0000", 2024-09-18T04:58:47.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036105343999999998 2024-09-18T04:58:47.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.468 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629547+0000", 2024-09-18T04:58:47.468 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7036000000000002e-05 2024-09-18T04:58:47.468 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.468 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.468 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.469 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629601+0000", 2024-09-18T04:58:47.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3866999999999998e-05 2024-09-18T04:58:47.469 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.469 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.469 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.469 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.469 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.469 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146085 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1794048~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:47.469 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:47.469 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.808984457999998, 2024-09-18T04:58:47.469 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036331186000000001, 2024-09-18T04:58:47.469 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.470 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:47.470 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.470 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.470 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.470 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146085 2024-09-18T04:58:47.470 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.470 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.470 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.470 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.470 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:47.470 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:47.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593397+0000", 2024-09-18T04:58:47.471 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:47.471 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.471 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.471 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.471 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593398+0000", 2024-09-18T04:58:47.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2270000000000001e-06 2024-09-18T04:58:47.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593401+0000", 2024-09-18T04:58:47.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.999e-06 2024-09-18T04:58:47.472 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.472 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.472 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.472 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593404+0000", 2024-09-18T04:58:47.472 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.486e-06 2024-09-18T04:58:47.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629624+0000", 2024-09-18T04:58:47.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036219983999999997 2024-09-18T04:58:47.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.473 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.473 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629672+0000", 2024-09-18T04:58:47.473 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8368000000000002e-05 2024-09-18T04:58:47.473 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.473 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.474 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.474 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629729+0000", 2024-09-18T04:58:47.474 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.6359e-05 2024-09-18T04:58:47.474 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.474 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.474 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.474 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.474 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.474 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146086 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1843200~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:47.474 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:47.475 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.808975894, 2024-09-18T04:58:47.475 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036449098999999999, 2024-09-18T04:58:47.475 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.475 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:47.475 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.475 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.475 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.475 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146086 2024-09-18T04:58:47.475 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.475 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:47.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:47.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.476 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.476 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.476 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.476 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593406+0000", 2024-09-18T04:58:47.476 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:47.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593407+0000", 2024-09-18T04:58:47.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1799999999999999e-06 2024-09-18T04:58:47.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.477 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.477 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593410+0000", 2024-09-18T04:58:47.477 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.926e-06 2024-09-18T04:58:47.477 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.477 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593413+0000", 2024-09-18T04:58:47.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8250000000000001e-06 2024-09-18T04:58:47.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629755+0000", 2024-09-18T04:58:47.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036342212999999998 2024-09-18T04:58:47.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.478 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.478 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.478 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629803+0000", 2024-09-18T04:58:47.478 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.8619000000000003e-05 2024-09-18T04:58:47.478 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.479 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.479 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629855+0000", 2024-09-18T04:58:47.479 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1536000000000003e-05 2024-09-18T04:58:47.479 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.479 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.479 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.479 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.479 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.479 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146087 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1892352~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:47.479 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:47.479 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.808966968999997, 2024-09-18T04:58:47.480 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036564608999999998, 2024-09-18T04:58:47.480 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.480 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:47.480 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.480 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.480 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.480 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146087 2024-09-18T04:58:47.480 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.480 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.480 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.480 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:47.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:47.481 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.481 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.481 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.481 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.481 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593415+0000", 2024-09-18T04:58:47.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:47.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593416+0000", 2024-09-18T04:58:47.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.218e-06 2024-09-18T04:58:47.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.482 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.482 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593419+0000", 2024-09-18T04:58:47.482 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.8700000000000001e-06 2024-09-18T04:58:47.482 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.482 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593421+0000", 2024-09-18T04:58:47.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.4329999999999998e-06 2024-09-18T04:58:47.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.483 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.483 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629878+0000", 2024-09-18T04:58:47.483 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036456578000000003 2024-09-18T04:58:47.483 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.483 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629925+0000", 2024-09-18T04:58:47.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7595000000000001e-05 2024-09-18T04:58:47.484 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.484 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.484 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.484 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.629979+0000", 2024-09-18T04:58:47.484 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4117999999999999e-05 2024-09-18T04:58:47.484 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.485 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.485 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.485 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.485 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.485 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146088 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1941504~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:47.485 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:47.485 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.808944804999996, 2024-09-18T04:58:47.485 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036676121999999999, 2024-09-18T04:58:47.485 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.485 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:47.485 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.485 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.485 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.486 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146088 2024-09-18T04:58:47.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.486 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.486 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:47.486 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.486 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.486 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.486 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:47.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593437+0000", 2024-09-18T04:58:47.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.487 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.487 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.487 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593438+0000", 2024-09-18T04:58:47.487 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7570000000000001e-06 2024-09-18T04:58:47.487 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593442+0000", 2024-09-18T04:58:47.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.2880000000000001e-06 2024-09-18T04:58:47.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.488 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593444+0000", 2024-09-18T04:58:47.488 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.593e-06 2024-09-18T04:58:47.488 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.488 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.488 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630015+0000", 2024-09-18T04:58:47.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036571125000000003 2024-09-18T04:58:47.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.489 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630061+0000", 2024-09-18T04:58:47.489 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5445000000000003e-05 2024-09-18T04:58:47.489 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.489 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.489 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.490 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630113+0000", 2024-09-18T04:58:47.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.2358000000000003e-05 2024-09-18T04:58:47.490 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.490 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.490 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.490 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.490 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.490 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146089 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 1990656~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:47.490 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:58:47.490 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.808935223999995, 2024-09-18T04:58:47.490 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036791483999999999, 2024-09-18T04:58:47.490 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.490 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:47.490 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.491 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.491 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.491 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146089 2024-09-18T04:58:47.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.491 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:58:47.491 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.491 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.491 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.491 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.491 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593447+0000", 2024-09-18T04:58:47.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593446+0000", 2024-09-18T04:58:47.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:47.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.492 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593448+0000", 2024-09-18T04:58:47.492 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1939999999999999e-06 2024-09-18T04:58:47.492 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.492 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.492 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593451+0000", 2024-09-18T04:58:47.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.766e-06 2024-09-18T04:58:47.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.493 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593458+0000", 2024-09-18T04:58:47.493 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 6.8539999999999999e-06 2024-09-18T04:58:47.493 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.493 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.493 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630136+0000", 2024-09-18T04:58:47.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.03667757 2024-09-18T04:58:47.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630183+0000", 2024-09-18T04:58:47.494 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.7289000000000001e-05 2024-09-18T04:58:47.494 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.494 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.494 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.494 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630238+0000", 2024-09-18T04:58:47.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.5013000000000003e-05 2024-09-18T04:58:47.495 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.495 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.495 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.495 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.495 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.495 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146090 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 2039808~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:47.495 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:47.495 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.808921343999998, 2024-09-18T04:58:47.495 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036904477999999998, 2024-09-18T04:58:47.495 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.495 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:47.495 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.496 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.496 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146090 2024-09-18T04:58:47.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.496 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.496 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:47.496 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.496 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.496 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.496 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:47.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.497 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593460+0000", 2024-09-18T04:58:47.497 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.497 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.497 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.497 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593461+0000", 2024-09-18T04:58:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3680000000000001e-06 2024-09-18T04:58:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593465+0000", 2024-09-18T04:58:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.8120000000000001e-06 2024-09-18T04:58:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593468+0000", 2024-09-18T04:58:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.582e-06 2024-09-18T04:58:47.498 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.499 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630268+0000", 2024-09-18T04:58:47.499 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036800449999999998 2024-09-18T04:58:47.499 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.499 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.499 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630313+0000", 2024-09-18T04:58:47.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.4836000000000002e-05 2024-09-18T04:58:47.500 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.500 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.500 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.500 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630365+0000", 2024-09-18T04:58:47.500 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.1839999999999998e-05 2024-09-18T04:58:47.500 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.501 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.501 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.501 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.501 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.501 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:146091 3.11 3:88dc690a:::rbd_data.11ac6967455f.0000000000000002:2d [read 2088960~8192] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e84)", 2024-09-18T04:58:47.501 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:47.501 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 73.808910882000006, 2024-09-18T04:58:47.501 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.037011534999999998, 2024-09-18T04:58:47.501 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.501 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:47.501 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.501 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 146091 2024-09-18T04:58:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.502 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593471+0000", 2024-09-18T04:58:47.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967296 2024-09-18T04:58:47.503 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.503 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.503 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.503 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593472+0000", 2024-09-18T04:58:47.503 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.1620000000000001e-06 2024-09-18T04:58:47.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593475+0000", 2024-09-18T04:58:47.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.0019999999999998e-06 2024-09-18T04:58:47.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.504 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.504 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.504 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.593477+0000", 2024-09-18T04:58:47.504 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.5009999999999999e-06 2024-09-18T04:58:47.504 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630381+0000", 2024-09-18T04:58:47.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.036904014999999998 2024-09-18T04:58:47.505 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.505 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.505 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.505 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630428+0000", 2024-09-18T04:58:47.505 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.6931000000000002e-05 2024-09-18T04:58:47.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.506 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.506 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:33.630482+0000", 2024-09-18T04:58:47.506 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.4138999999999998e-05 2024-09-18T04:58:47.506 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.506 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.506 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.506 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.506 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.507 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:162075 3.13 3:cf3f6048:::rbd_header.11ac6967455f:head [watch reconnect cookie 140406508629968 gen 1] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T04:58:47.507 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:58:47.507 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 64.911938077000002, 2024-09-18T04:58:47.507 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.53184266599999996, 2024-09-18T04:58:47.507 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.507 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "started", 2024-09-18T04:58:47.507 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.507 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.507 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.507 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 162075 2024-09-18T04:58:47.507 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.507 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.507 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.507 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:58:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490444+0000", 2024-09-18T04:58:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490442+0000", 2024-09-18T04:58:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999981 2024-09-18T04:58:47.508 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490453+0000", 2024-09-18T04:58:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.0613e-05 2024-09-18T04:58:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490466+0000", 2024-09-18T04:58:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.3702e-05 2024-09-18T04:58:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490475+0000", 2024-09-18T04:58:47.509 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 9.2690000000000005e-06 2024-09-18T04:58:47.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.510 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.510 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490612+0000", 2024-09-18T04:58:47.510 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013620900000000001 2024-09-18T04:58:47.510 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.510 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op must wait for map", 2024-09-18T04:58:47.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490620+0000", 2024-09-18T04:58:47.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.3089999999999996e-06 2024-09-18T04:58:47.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.511 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.511 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.511 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503041+0000", 2024-09-18T04:58:47.511 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012420512 2024-09-18T04:58:47.511 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:47.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503048+0000", 2024-09-18T04:58:47.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.0389999999999999e-06 2024-09-18T04:58:47.512 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.512 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.512 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.512 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.016122+0000", 2024-09-18T04:58:47.512 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.51307432900000005 2024-09-18T04:58:47.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.016289+0000", 2024-09-18T04:58:47.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000166875 2024-09-18T04:58:47.513 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.513 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.513 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.513 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.022286+0000", 2024-09-18T04:58:47.513 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0059976049999999996 2024-09-18T04:58:47.513 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.513 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.514 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.514 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.514 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.514 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:162076 3.13 3:cf3f6048:::rbd_header.11ac6967455f:head [call rbd.snapshot_remove in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e92)", 2024-09-18T04:58:47.514 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:47.514 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 64.911815403000006, 2024-09-18T04:58:47.514 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.53464117600000005, 2024-09-18T04:58:47.514 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.514 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:47.514 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.515 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.515 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.515 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 162076 2024-09-18T04:58:47.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.515 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.515 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.515 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.515 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:47.515 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.515 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:47.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.516 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490566+0000", 2024-09-18T04:58:47.516 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.999999 2024-09-18T04:58:47.516 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.516 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.516 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490571+0000", 2024-09-18T04:58:47.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.8259999999999998e-06 2024-09-18T04:58:47.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490584+0000", 2024-09-18T04:58:47.517 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.2391999999999999e-05 2024-09-18T04:58:47.517 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.517 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.517 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.517 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490591+0000", 2024-09-18T04:58:47.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.5460000000000003e-06 2024-09-18T04:58:47.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490714+0000", 2024-09-18T04:58:47.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.000122543 2024-09-18T04:58:47.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting_for_map not empty", 2024-09-18T04:58:47.518 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.490731+0000", 2024-09-18T04:58:47.518 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 1.7405e-05 2024-09-18T04:58:47.518 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.518 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.518 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503069+0000", 2024-09-18T04:58:47.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.012337836 2024-09-18T04:58:47.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:47.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:42.503078+0000", 2024-09-18T04:58:47.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.8289999999999997e-06 2024-09-18T04:58:47.519 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.519 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.519 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.519 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019018+0000", 2024-09-18T04:58:47.519 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.51594035900000002 2024-09-18T04:58:47.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.520 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019091+0000", 2024-09-18T04:58:47.520 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.2608999999999995e-05 2024-09-18T04:58:47.520 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.520 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.520 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 2", 2024-09-18T04:58:47.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.019421+0000", 2024-09-18T04:58:47.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00033004300000000002 2024-09-18T04:58:47.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:47.521 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.021644+0000", 2024-09-18T04:58:47.521 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0022229749999999999 2024-09-18T04:58:47.521 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.521 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.521 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025113+0000", 2024-09-18T04:58:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.003469228 2024-09-18T04:58:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025167+0000", 2024-09-18T04:58:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 5.3569999999999997e-05 2024-09-18T04:58:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:57:43.025208+0000", 2024-09-18T04:58:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.0821000000000001e-05 2024-09-18T04:58:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.522 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: "description": "osd_op(client.4524.0:167082 3.15 3:a80f5cd7:::rbd_data.11ac138032e7.0000000000000008:head [delete] snapc 11=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-18T04:58:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: "initiated_at": "2024-09-18T04:58:19.163826+0000", 2024-09-18T04:58:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: "age": 28.238555656999999, 2024-09-18T04:58:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.91230620200000001, 2024-09-18T04:58:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: "type_data": { 2024-09-18T04:58:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-18T04:58:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: "client_info": { 2024-09-18T04:58:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: "client": "client.4524", 2024-09-18T04:58:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: "tid": 167082 2024-09-18T04:58:47.523 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.524 INFO:teuthology.orchestra.run.smithi099.stdout: "events": [ 2024-09-18T04:58:47.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "initiated", 2024-09-18T04:58:47.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163826+0000", 2024-09-18T04:58:47.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.524 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.524 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "throttled", 2024-09-18T04:58:47.524 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163826+0000", 2024-09-18T04:58:47.524 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0 2024-09-18T04:58:47.524 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "header_read", 2024-09-18T04:58:47.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163826+0000", 2024-09-18T04:58:47.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.525 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.525 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "all_read", 2024-09-18T04:58:47.525 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163829+0000", 2024-09-18T04:58:47.525 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 2.9069999999999999e-06 2024-09-18T04:58:47.525 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "dispatched", 2024-09-18T04:58:47.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163836+0000", 2024-09-18T04:58:47.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.4780000000000002e-06 2024-09-18T04:58:47.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163841+0000", 2024-09-18T04:58:47.526 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.9389999999999996e-06 2024-09-18T04:58:47.526 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.526 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.526 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.526 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163886+0000", 2024-09-18T04:58:47.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 4.5037999999999998e-05 2024-09-18T04:58:47.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for peered", 2024-09-18T04:58:47.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:19.163895+0000", 2024-09-18T04:58:47.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.7819999999999999e-06 2024-09-18T04:58:47.527 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.527 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.527 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "reached_pg", 2024-09-18T04:58:47.527 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070784+0000", 2024-09-18T04:58:47.527 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.90688966400000004 2024-09-18T04:58:47.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "started", 2024-09-18T04:58:47.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.070917+0000", 2024-09-18T04:58:47.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.00013228799999999999 2024-09-18T04:58:47.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.528 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.528 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "waiting for subops from 0", 2024-09-18T04:58:47.528 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.071003+0000", 2024-09-18T04:58:47.528 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 8.5915000000000007e-05 2024-09-18T04:58:47.528 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "sub_op_commit_rec", 2024-09-18T04:58:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.073598+0000", 2024-09-18T04:58:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0025949329999999998 2024-09-18T04:58:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "op_commit", 2024-09-18T04:58:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.076024+0000", 2024-09-18T04:58:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 0.0024267310000000001 2024-09-18T04:58:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "commit_sent", 2024-09-18T04:58:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.076096+0000", 2024-09-18T04:58:47.529 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 7.1512000000000002e-05 2024-09-18T04:58:47.530 INFO:teuthology.orchestra.run.smithi099.stdout: }, 2024-09-18T04:58:47.530 INFO:teuthology.orchestra.run.smithi099.stdout: { 2024-09-18T04:58:47.530 INFO:teuthology.orchestra.run.smithi099.stdout: "event": "done", 2024-09-18T04:58:47.530 INFO:teuthology.orchestra.run.smithi099.stdout: "time": "2024-09-18T04:58:20.076132+0000", 2024-09-18T04:58:47.530 INFO:teuthology.orchestra.run.smithi099.stdout: "duration": 3.6520999999999998e-05 2024-09-18T04:58:47.530 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.530 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.530 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.530 INFO:teuthology.orchestra.run.smithi099.stdout: } 2024-09-18T04:58:47.530 INFO:teuthology.orchestra.run.smithi099.stdout: ] 2024-09-18T04:58:47.530 INFO:teuthology.orchestra.run.smithi099.stdout:} 2024-09-18T04:58:47.532 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:47.529+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:47.533 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-18T04:58:47.633 INFO:tasks.ceph.osd.3.smithi002.stderr:2024-09-18T04:58:47.631+0000 7f2a67a41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 33038) UID: 0 2024-09-18T04:58:47.653 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:47.653 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:47.653 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:58:47.653 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:47.662 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-09-18T04:58:47.746 INFO:tasks.ceph.osd.0.smithi002.stderr:2024-09-18T04:58:47.744+0000 7f36fbc41640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 33021) UID: 0 2024-09-18T04:58:47.810 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:47.810 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:47.810 INFO:teuthology.orchestra.run.smithi002.stdout: "complaint_time": 30, 2024-09-18T04:58:47.811 INFO:teuthology.orchestra.run.smithi002.stdout: "num_blocked_ops": 0 2024-09-18T04:58:47.811 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:47.820 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-09-18T04:58:47.835 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:47.833+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:47.937 INFO:tasks.ceph.osd.4.smithi099.stderr:2024-09-18T04:58:47.934+0000 7f861bd4d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32999) UID: 0 2024-09-18T04:58:47.968 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:47.968 INFO:teuthology.orchestra.run.smithi002.stdout: "size": 20, 2024-09-18T04:58:47.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 600, 2024-09-18T04:58:47.968 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [ 2024-09-18T04:58:47.968 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.968 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9316 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1679360~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:47.968 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:58:47.968 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 132.19763693199999, 2024-09-18T04:58:47.968 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044027208999999998, 2024-09-18T04:58:47.968 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:47.968 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:47.969 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:47.969 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:47.969 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.969 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9316 2024-09-18T04:58:47.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.969 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:47.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:47.969 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:58:47.969 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:47.969 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.969 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.969 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:47.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:58:47.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:47.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:47.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767142+0000", 2024-09-18T04:58:47.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.970 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:47.970 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767144+0000", 2024-09-18T04:58:47.970 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.3070000000000001e-06 2024-09-18T04:58:47.970 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.970 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:47.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767150+0000", 2024-09-18T04:58:47.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1700000000000002e-06 2024-09-18T04:58:47.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767163+0000", 2024-09-18T04:58:47.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.2670999999999999e-05 2024-09-18T04:58:47.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.971 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.971 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:47.971 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.810995+0000", 2024-09-18T04:58:47.971 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043832310999999999 2024-09-18T04:58:47.971 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:47.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811087+0000", 2024-09-18T04:58:47.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.1345e-05 2024-09-18T04:58:47.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.972 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.972 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:47.972 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811169+0000", 2024-09-18T04:58:47.972 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2848000000000004e-05 2024-09-18T04:58:47.972 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:47.972 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:47.972 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:47.972 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.973 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9317 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1728512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:47.973 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:58:47.973 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 132.19761129700001, 2024-09-18T04:58:47.973 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044190170000000001, 2024-09-18T04:58:47.973 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:47.973 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:47.973 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:47.973 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:47.973 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.973 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9317 2024-09-18T04:58:47.973 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.973 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:47.973 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:47.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:58:47.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:47.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:47.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767168+0000", 2024-09-18T04:58:47.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:47.974 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.974 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.974 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:47.974 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767167+0000", 2024-09-18T04:58:47.974 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:47.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767170+0000", 2024-09-18T04:58:47.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.7259999999999998e-06 2024-09-18T04:58:47.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:47.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767176+0000", 2024-09-18T04:58:47.975 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1460000000000001e-06 2024-09-18T04:58:47.975 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.975 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.975 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.975 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767182+0000", 2024-09-18T04:58:47.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.1020000000000002e-06 2024-09-18T04:58:47.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:47.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811204+0000", 2024-09-18T04:58:47.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044022039999999998 2024-09-18T04:58:47.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.976 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:47.976 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811286+0000", 2024-09-18T04:58:47.976 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.1623000000000001e-05 2024-09-18T04:58:47.976 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.976 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.977 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:47.977 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811358+0000", 2024-09-18T04:58:47.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2125000000000005e-05 2024-09-18T04:58:47.977 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:47.977 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:47.977 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:47.977 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.977 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.977 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9318 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1777664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:47.977 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:58:47.977 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 132.19759275199999, 2024-09-18T04:58:47.977 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044380181999999997, 2024-09-18T04:58:47.977 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:47.977 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:47.978 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:47.978 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:47.978 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.978 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9318 2024-09-18T04:58:47.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.978 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:47.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:47.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:58:47.978 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:47.978 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.978 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.978 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:47.978 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:58:47.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:47.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:47.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767186+0000", 2024-09-18T04:58:47.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:47.979 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767189+0000", 2024-09-18T04:58:47.979 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.5909999999999998e-06 2024-09-18T04:58:47.979 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.979 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.979 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:47.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767194+0000", 2024-09-18T04:58:47.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7690000000000002e-06 2024-09-18T04:58:47.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.767199+0000", 2024-09-18T04:58:47.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.8189999999999999e-06 2024-09-18T04:58:47.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.980 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.980 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:47.980 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811394+0000", 2024-09-18T04:58:47.980 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044195350000000001 2024-09-18T04:58:47.980 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:47.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811483+0000", 2024-09-18T04:58:47.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8282999999999995e-05 2024-09-18T04:58:47.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.981 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.981 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:47.981 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.811567+0000", 2024-09-18T04:58:47.981 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.3855000000000003e-05 2024-09-18T04:58:47.981 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:47.981 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:47.981 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:47.981 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.982 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.982 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9325 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 2088960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:47.982 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:58:47.982 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 132.19586221399999, 2024-09-18T04:58:47.982 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044063989999999997, 2024-09-18T04:58:47.982 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:47.982 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:47.982 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:47.982 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:47.982 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.982 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9325 2024-09-18T04:58:47.982 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.982 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:47.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:47.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:58:47.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:47.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:47.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:58:47.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:47.983 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.983 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.983 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:47.983 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768917+0000", 2024-09-18T04:58:47.983 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:47.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768919+0000", 2024-09-18T04:58:47.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.9630000000000001e-06 2024-09-18T04:58:47.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:47.984 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768925+0000", 2024-09-18T04:58:47.984 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2310000000000001e-06 2024-09-18T04:58:47.984 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.984 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.984 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.768929+0000", 2024-09-18T04:58:47.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4469999999999999e-06 2024-09-18T04:58:47.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:47.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812823+0000", 2024-09-18T04:58:47.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043893975000000002 2024-09-18T04:58:47.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.985 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:47.985 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812908+0000", 2024-09-18T04:58:47.985 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.4862000000000002e-05 2024-09-18T04:58:47.985 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.985 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.986 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:47.986 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.812981+0000", 2024-09-18T04:58:47.986 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.2892000000000006e-05 2024-09-18T04:58:47.986 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:47.986 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:47.986 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:47.986 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.986 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.986 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9334 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 372736~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:47.986 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:58:47.986 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 132.19384342800001, 2024-09-18T04:58:47.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044042853999999999, 2024-09-18T04:58:47.987 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:47.987 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:47.987 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:47.987 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:47.987 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.987 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9334 2024-09-18T04:58:47.987 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.987 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:47.987 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.987 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:47.987 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:58:47.987 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:47.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:47.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770936+0000", 2024-09-18T04:58:47.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:47.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.988 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:47.988 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770935+0000", 2024-09-18T04:58:47.988 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.988 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.988 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:47.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770937+0000", 2024-09-18T04:58:47.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1489999999999999e-06 2024-09-18T04:58:47.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:47.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770940+0000", 2024-09-18T04:58:47.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.9569999999999998e-06 2024-09-18T04:58:47.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.989 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.989 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.989 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.770945+0000", 2024-09-18T04:58:47.989 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.823e-06 2024-09-18T04:58:47.989 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:47.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814838+0000", 2024-09-18T04:58:47.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043892635999999999 2024-09-18T04:58:47.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:47.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814920+0000", 2024-09-18T04:58:47.990 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2175000000000005e-05 2024-09-18T04:58:47.990 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.990 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.990 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:47.990 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.814979+0000", 2024-09-18T04:58:47.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8372000000000003e-05 2024-09-18T04:58:47.991 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:47.991 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:47.991 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:47.991 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.991 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.991 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9352 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1159168~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:47.991 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:58:47.991 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 132.19162225100001, 2024-09-18T04:58:47.991 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044011269999999998, 2024-09-18T04:58:47.991 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:47.991 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:47.991 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:47.991 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:47.992 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.992 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9352 2024-09-18T04:58:47.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.992 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:47.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:47.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:58:47.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:47.992 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.992 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.992 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:47.992 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773157+0000", 2024-09-18T04:58:47.992 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:47.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:47.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773156+0000", 2024-09-18T04:58:47.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:47.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773159+0000", 2024-09-18T04:58:47.993 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2919999999999998e-06 2024-09-18T04:58:47.993 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.993 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.993 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:47.993 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773165+0000", 2024-09-18T04:58:47.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0580000000000003e-06 2024-09-18T04:58:47.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773177+0000", 2024-09-18T04:58:47.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.204e-05 2024-09-18T04:58:47.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.994 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:47.994 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817082+0000", 2024-09-18T04:58:47.994 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043905528999999999 2024-09-18T04:58:47.994 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.994 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:47.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817122+0000", 2024-09-18T04:58:47.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.9258999999999998e-05 2024-09-18T04:58:47.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.995 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:47.995 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817168+0000", 2024-09-18T04:58:47.995 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.6545000000000002e-05 2024-09-18T04:58:47.995 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:47.995 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:47.995 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:47.995 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.995 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.996 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9353 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1208320~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:47.996 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:58:47.996 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 132.19159795300001, 2024-09-18T04:58:47.996 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044105537, 2024-09-18T04:58:47.996 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:47.996 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:47.996 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:47.996 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:47.996 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:47.996 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9353 2024-09-18T04:58:47.996 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.996 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:47.996 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.996 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:47.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:58:47.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:47.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:47.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:58:47.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:47.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.997 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.997 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:47.997 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773181+0000", 2024-09-18T04:58:47.997 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:47.997 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:47.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773183+0000", 2024-09-18T04:58:47.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2929999999999999e-06 2024-09-18T04:58:47.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:47.998 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773190+0000", 2024-09-18T04:58:47.998 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4330000000000001e-06 2024-09-18T04:58:47.998 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.998 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.998 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:47.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773194+0000", 2024-09-18T04:58:47.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.865e-06 2024-09-18T04:58:47.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:47.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817196+0000", 2024-09-18T04:58:47.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044001708 2024-09-18T04:58:47.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:47.999 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:47.999 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:47.999 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817236+0000", 2024-09-18T04:58:47.999 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.0043999999999998e-05 2024-09-18T04:58:47.999 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.000 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:48.000 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817287+0000", 2024-09-18T04:58:48.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.0605000000000001e-05 2024-09-18T04:58:48.000 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.000 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:48.000 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.000 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.000 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.000 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:9354 3.12 3:49d2ef69:::rbd_data.11acf292c51a.000000000000000e:d [read 1257472~4096] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32)", 2024-09-18T04:58:48.000 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:58:48.000 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 132.191580442, 2024-09-18T04:58:48.000 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044199796, 2024-09-18T04:58:48.001 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:48.001 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:48.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:48.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:48.001 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:48.001 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 9354 2024-09-18T04:58:48.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.001 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:48.001 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.001 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:48.001 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:58:48.001 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:48.001 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:48.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773199+0000", 2024-09-18T04:58:48.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:48.002 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.002 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.002 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:48.002 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773198+0000", 2024-09-18T04:58:48.002 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:48.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:48.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773200+0000", 2024-09-18T04:58:48.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.4159999999999998e-06 2024-09-18T04:58:48.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:48.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773206+0000", 2024-09-18T04:58:48.003 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.789e-06 2024-09-18T04:58:48.003 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.003 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.003 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:48.003 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.773210+0000", 2024-09-18T04:58:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2679999999999996e-06 2024-09-18T04:58:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817314+0000", 2024-09-18T04:58:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044103187000000002 2024-09-18T04:58:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817353+0000", 2024-09-18T04:58:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.8973000000000001e-05 2024-09-18T04:58:48.004 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:35.817399+0000", 2024-09-18T04:58:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5877999999999997e-05 2024-09-18T04:58:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46591 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2097152~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:58:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 118.266724025, 2024-09-18T04:58:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043987310000000002, 2024-09-18T04:58:48.005 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46591 2024-09-18T04:58:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:58:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.006 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:58:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698055+0000", 2024-09-18T04:58:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698057+0000", 2024-09-18T04:58:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.531e-06 2024-09-18T04:58:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.007 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698064+0000", 2024-09-18T04:58:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.0110000000000003e-06 2024-09-18T04:58:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698069+0000", 2024-09-18T04:58:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.091e-06 2024-09-18T04:58:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741843+0000", 2024-09-18T04:58:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043773470000000002 2024-09-18T04:58:48.008 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.741928+0000", 2024-09-18T04:58:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5113999999999999e-05 2024-09-18T04:58:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742042+0000", 2024-09-18T04:58:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000114527 2024-09-18T04:58:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.009 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46592 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2121728~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:58:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 118.266705598, 2024-09-18T04:58:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044077948999999998, 2024-09-18T04:58:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46592 2024-09-18T04:58:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:48.010 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:58:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698074+0000", 2024-09-18T04:58:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698073+0000", 2024-09-18T04:58:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.011 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698076+0000", 2024-09-18T04:58:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.481e-06 2024-09-18T04:58:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698082+0000", 2024-09-18T04:58:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8730000000000004e-06 2024-09-18T04:58:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:48.012 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698086+0000", 2024-09-18T04:58:48.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7539999999999997e-06 2024-09-18T04:58:48.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:48.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742055+0000", 2024-09-18T04:58:48.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043969170000000002 2024-09-18T04:58:48.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:48.013 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742093+0000", 2024-09-18T04:58:48.013 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7707000000000003e-05 2024-09-18T04:58:48.013 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.013 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.013 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:48.014 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742152+0000", 2024-09-18T04:58:48.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.8323999999999999e-05 2024-09-18T04:58:48.014 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.014 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:48.014 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.014 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.014 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.014 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46593 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2211840~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:48.014 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:58:48.014 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 118.266688695, 2024-09-18T04:58:48.014 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044259917000000003, 2024-09-18T04:58:48.014 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:48.014 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:48.014 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:48.015 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:48.015 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:48.015 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46593 2024-09-18T04:58:48.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.015 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:48.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:48.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:58:48.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:48.015 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.015 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.015 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:48.015 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:58:48.015 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:48.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:48.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698090+0000", 2024-09-18T04:58:48.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:48.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.016 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:48.016 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698092+0000", 2024-09-18T04:58:48.016 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2249999999999999e-06 2024-09-18T04:58:48.016 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.016 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:48.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698098+0000", 2024-09-18T04:58:48.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:58:48.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:48.017 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698103+0000", 2024-09-18T04:58:48.017 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.5700000000000003e-06 2024-09-18T04:58:48.017 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.017 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.017 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:48.018 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742182+0000", 2024-09-18T04:58:48.018 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044079508000000003 2024-09-18T04:58:48.018 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.018 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.018 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:48.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742257+0000", 2024-09-18T04:58:48.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.4678000000000006e-05 2024-09-18T04:58:48.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.019 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:48.019 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742350+0000", 2024-09-18T04:58:48.019 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.3531999999999998e-05 2024-09-18T04:58:48.019 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.019 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:48.019 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.019 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.019 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.019 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46594 3.6 3:632b64da:::rbd_data.11ac728e6e9a.0000000000000009:15 [read 2236416~8192] snapc 0=[] RETRY=3 ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:48.019 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:58:48.020 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 118.266672998, 2024-09-18T04:58:48.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044378984000000003, 2024-09-18T04:58:48.020 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:48.020 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:48.020 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:48.020 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:48.020 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:48.020 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46594 2024-09-18T04:58:48.020 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.020 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:48.020 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.020 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:48.020 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:58:48.020 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:48.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:48.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:58:48.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:48.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:48.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698106+0000", 2024-09-18T04:58:48.021 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:48.021 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.021 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.021 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:48.021 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698108+0000", 2024-09-18T04:58:48.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.0880000000000002e-06 2024-09-18T04:58:48.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:48.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698114+0000", 2024-09-18T04:58:48.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6269999999999997e-06 2024-09-18T04:58:48.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.022 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:48.022 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698118+0000", 2024-09-18T04:58:48.022 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.4050000000000002e-06 2024-09-18T04:58:48.022 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.022 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:48.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742365+0000", 2024-09-18T04:58:48.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044246799000000003 2024-09-18T04:58:48.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:48.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742407+0000", 2024-09-18T04:58:48.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.2008000000000002e-05 2024-09-18T04:58:48.023 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.023 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.023 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:48.023 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742485+0000", 2024-09-18T04:58:48.023 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.8405999999999994e-05 2024-09-18T04:58:48.023 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.024 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:48.024 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.024 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.024 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.024 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46595 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:48.024 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:58:48.024 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 118.26665692500001, 2024-09-18T04:58:48.024 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.04440463, 2024-09-18T04:58:48.024 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:48.024 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:48.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:48.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:48.024 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:48.025 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46595 2024-09-18T04:58:48.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.025 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:48.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:48.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:58:48.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:48.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.025 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:48.025 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:58:48.025 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:48.025 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.025 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:48.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698122+0000", 2024-09-18T04:58:48.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:48.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:48.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698124+0000", 2024-09-18T04:58:48.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.1840000000000002e-06 2024-09-18T04:58:48.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.026 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.026 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:48.026 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698130+0000", 2024-09-18T04:58:48.026 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.947e-06 2024-09-18T04:58:48.026 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:48.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698136+0000", 2024-09-18T04:58:48.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.6520000000000003e-06 2024-09-18T04:58:48.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:48.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742521+0000", 2024-09-18T04:58:48.027 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044385446000000002 2024-09-18T04:58:48.027 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.027 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.027 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:48.027 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742527+0000", 2024-09-18T04:58:48.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.7849999999999997e-06 2024-09-18T04:58:48.028 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.028 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:48.028 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.028 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.028 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.028 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46596 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:48.028 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:58:48.028 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 118.266638681, 2024-09-18T04:58:48.028 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044418066999999999, 2024-09-18T04:58:48.028 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:48.028 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:48.028 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:48.029 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:48.029 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:48.029 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46596 2024-09-18T04:58:48.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.029 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:48.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:48.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:58:48.029 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:48.029 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.029 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.029 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:48.029 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:58:48.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:48.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:48.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698140+0000", 2024-09-18T04:58:48.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:48.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:48.030 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698146+0000", 2024-09-18T04:58:48.030 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2779999999999999e-06 2024-09-18T04:58:48.030 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.030 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.030 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:48.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698153+0000", 2024-09-18T04:58:48.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.3849999999999999e-06 2024-09-18T04:58:48.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:48.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698164+0000", 2024-09-18T04:58:48.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.1375999999999999e-05 2024-09-18T04:58:48.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.031 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.031 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:48.031 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742554+0000", 2024-09-18T04:58:48.031 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044389482000000001 2024-09-18T04:58:48.031 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.032 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:48.032 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742559+0000", 2024-09-18T04:58:48.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9540000000000003e-06 2024-09-18T04:58:48.032 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.032 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:48.032 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.032 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.032 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.032 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46597 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:48.032 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:58:48.032 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 118.266610316, 2024-09-18T04:58:48.032 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044416513999999997, 2024-09-18T04:58:48.032 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:48.033 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:48.033 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:48.033 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:48.033 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:48.033 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46597 2024-09-18T04:58:48.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.033 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:48.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:48.033 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:58:48.033 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:48.033 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.033 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.033 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:48.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698169+0000", 2024-09-18T04:58:48.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:48.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:48.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698168+0000", 2024-09-18T04:58:48.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:48.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.034 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.034 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:48.034 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698175+0000", 2024-09-18T04:58:48.034 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4520000000000002e-06 2024-09-18T04:58:48.034 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:48.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698181+0000", 2024-09-18T04:58:48.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.4150000000000004e-06 2024-09-18T04:58:48.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:48.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698186+0000", 2024-09-18T04:58:48.035 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.7489999999999998e-06 2024-09-18T04:58:48.035 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.035 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.035 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:48.035 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742580+0000", 2024-09-18T04:58:48.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044394325999999998 2024-09-18T04:58:48.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.036 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:48.036 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742585+0000", 2024-09-18T04:58:48.036 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9849999999999997e-06 2024-09-18T04:58:48.036 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.036 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:48.036 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.036 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.036 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.036 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:46598 3.6 3.5b26d4c6 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e44)", 2024-09-18T04:58:48.036 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:58:48.036 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 118.26650284, 2024-09-18T04:58:48.037 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044335316999999999, 2024-09-18T04:58:48.037 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:48.037 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "reached pg", 2024-09-18T04:58:48.037 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:48.037 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:48.037 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:48.037 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 46598 2024-09-18T04:58:48.037 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.037 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:48.037 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.037 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:48.037 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:58:48.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:48.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:48.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698276+0000", 2024-09-18T04:58:48.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:48.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:48.038 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698190+0000", 2024-09-18T04:58:48.038 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999137 2024-09-18T04:58:48.038 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.038 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.038 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:48.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698280+0000", 2024-09-18T04:58:48.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0409999999999997e-05 2024-09-18T04:58:48.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:48.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698289+0000", 2024-09-18T04:58:48.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.9490000000000002e-06 2024-09-18T04:58:48.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.039 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.039 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:48.039 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.698296+0000", 2024-09-18T04:58:48.039 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.2670000000000003e-06 2024-09-18T04:58:48.039 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:48.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742607+0000", 2024-09-18T04:58:48.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044311072999999999 2024-09-18T04:58:48.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.040 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.040 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:48.040 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:49.742612+0000", 2024-09-18T04:58:48.040 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9400000000000001e-06 2024-09-18T04:58:48.040 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.040 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:48.040 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.040 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.041 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.041 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57986 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3086336~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:48.041 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:58:48.041 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.279657117, 2024-09-18T04:58:48.041 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044047731999999999, 2024-09-18T04:58:48.041 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:48.041 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:48.041 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:48.041 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:48.041 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:48.041 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57986 2024-09-18T04:58:48.041 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.041 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:48.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:48.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:58:48.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:48.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:48.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:58:48.042 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:48.042 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.042 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.042 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:48.042 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685122+0000", 2024-09-18T04:58:48.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:48.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:48.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685124+0000", 2024-09-18T04:58:48.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.2859999999999998e-06 2024-09-18T04:58:48.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.043 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:48.043 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685130+0000", 2024-09-18T04:58:48.043 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.5400000000000001e-06 2024-09-18T04:58:48.043 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.043 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:48.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685136+0000", 2024-09-18T04:58:48.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.152e-06 2024-09-18T04:58:48.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:48.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.728976+0000", 2024-09-18T04:58:48.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.043840037999999998 2024-09-18T04:58:48.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.044 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.044 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:48.044 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729064+0000", 2024-09-18T04:58:48.044 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8709999999999996e-05 2024-09-18T04:58:48.044 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.045 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:48.045 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729170+0000", 2024-09-18T04:58:48.045 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000105386 2024-09-18T04:58:48.045 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.045 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:48.045 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.045 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.045 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.045 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57987 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3101696~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:48.045 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:58:48.045 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.279608142, 2024-09-18T04:58:48.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044238123999999997, 2024-09-18T04:58:48.046 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:48.046 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:48.046 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:48.046 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:48.046 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:48.046 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57987 2024-09-18T04:58:48.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.046 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:48.046 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.046 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:48.046 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:58:48.046 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:48.046 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:48.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685171+0000", 2024-09-18T04:58:48.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:48.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:48.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685170+0000", 2024-09-18T04:58:48.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.999999 2024-09-18T04:58:48.047 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.047 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.047 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:48.047 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685175+0000", 2024-09-18T04:58:48.047 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4.9069999999999998e-06 2024-09-18T04:58:48.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:48.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685187+0000", 2024-09-18T04:58:48.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 1.189e-05 2024-09-18T04:58:48.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:48.048 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685195+0000", 2024-09-18T04:58:48.048 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 7.9459999999999998e-06 2024-09-18T04:58:48.048 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.048 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.048 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:48.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729212+0000", 2024-09-18T04:58:48.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044017239 2024-09-18T04:58:48.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:48.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729301+0000", 2024-09-18T04:58:48.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.8887999999999993e-05 2024-09-18T04:58:48.049 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.049 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.049 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:48.049 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729409+0000", 2024-09-18T04:58:48.049 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000108119 2024-09-18T04:58:48.049 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.050 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:48.050 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.050 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.050 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.050 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57988 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3117056~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:48.050 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:58:48.050 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.279566023, 2024-09-18T04:58:48.050 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044429835000000001, 2024-09-18T04:58:48.050 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:48.050 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:48.050 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:48.050 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:48.050 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:48.050 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57988 2024-09-18T04:58:48.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.051 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:48.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:48.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:58:48.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:48.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.051 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:48.051 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685213+0000", 2024-09-18T04:58:48.051 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:48.051 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.051 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:48.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685212+0000", 2024-09-18T04:58:48.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:48.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:48.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685216+0000", 2024-09-18T04:58:48.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 3.7390000000000001e-06 2024-09-18T04:58:48.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.052 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.052 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:48.052 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685224+0000", 2024-09-18T04:58:48.052 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.2260000000000005e-06 2024-09-18T04:58:48.052 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:48.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685230+0000", 2024-09-18T04:58:48.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.0299999999999999e-06 2024-09-18T04:58:48.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:48.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729454+0000", 2024-09-18T04:58:48.053 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044223248999999999 2024-09-18T04:58:48.053 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.053 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.053 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:48.053 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729539+0000", 2024-09-18T04:58:48.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 8.5029000000000002e-05 2024-09-18T04:58:48.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.054 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:48.054 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729643+0000", 2024-09-18T04:58:48.054 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.000104267 2024-09-18T04:58:48.054 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.054 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:48.054 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.054 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.054 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.054 INFO:teuthology.orchestra.run.smithi002.stdout: "description": "osd_op(client.4524.0:57989 3.1 3:815103ee:::rbd_data.11acf2b3fa50.0000000000000002:17 [read 3132416~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e45)", 2024-09-18T04:58:48.054 INFO:teuthology.orchestra.run.smithi002.stdout: "initiated_at": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:58:48.054 INFO:teuthology.orchestra.run.smithi002.stdout: "age": 117.27954327099999, 2024-09-18T04:58:48.055 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044642881000000002, 2024-09-18T04:58:48.055 INFO:teuthology.orchestra.run.smithi002.stdout: "type_data": { 2024-09-18T04:58:48.055 INFO:teuthology.orchestra.run.smithi002.stdout: "flag_point": "started", 2024-09-18T04:58:48.055 INFO:teuthology.orchestra.run.smithi002.stdout: "client_info": { 2024-09-18T04:58:48.055 INFO:teuthology.orchestra.run.smithi002.stdout: "client": "client.4524", 2024-09-18T04:58:48.055 INFO:teuthology.orchestra.run.smithi002.stdout: "client_addr": "172.21.15.2:0/3919698436", 2024-09-18T04:58:48.055 INFO:teuthology.orchestra.run.smithi002.stdout: "tid": 57989 2024-09-18T04:58:48.055 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.055 INFO:teuthology.orchestra.run.smithi002.stdout: "events": [ 2024-09-18T04:58:48.055 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.055 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "initiated", 2024-09-18T04:58:48.055 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:58:48.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:48.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "throttled", 2024-09-18T04:58:48.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685236+0000", 2024-09-18T04:58:48.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0 2024-09-18T04:58:48.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.056 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "header_read", 2024-09-18T04:58:48.056 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685235+0000", 2024-09-18T04:58:48.056 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 4294967295.9999995 2024-09-18T04:58:48.056 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.056 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "all_read", 2024-09-18T04:58:48.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685238+0000", 2024-09-18T04:58:48.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 2.667e-06 2024-09-18T04:58:48.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "dispatched", 2024-09-18T04:58:48.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685245+0000", 2024-09-18T04:58:48.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 6.7909999999999999e-06 2024-09-18T04:58:48.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.057 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.057 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "queued_for_pg", 2024-09-18T04:58:48.057 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.685250+0000", 2024-09-18T04:58:48.057 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 5.3730000000000001e-06 2024-09-18T04:58:48.057 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "reached_pg", 2024-09-18T04:58:48.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729685+0000", 2024-09-18T04:58:48.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.044434552000000002 2024-09-18T04:58:48.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "started", 2024-09-18T04:58:48.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729775+0000", 2024-09-18T04:58:48.058 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 9.0395000000000004e-05 2024-09-18T04:58:48.058 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2024-09-18T04:58:48.058 INFO:teuthology.orchestra.run.smithi002.stdout: { 2024-09-18T04:58:48.058 INFO:teuthology.orchestra.run.smithi002.stdout: "event": "done", 2024-09-18T04:58:48.058 INFO:teuthology.orchestra.run.smithi002.stdout: "time": "2024-09-18T04:56:50.729879+0000", 2024-09-18T04:58:48.059 INFO:teuthology.orchestra.run.smithi002.stdout: "duration": 0.00010361099999999999 2024-09-18T04:58:48.059 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.059 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:48.059 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.059 INFO:teuthology.orchestra.run.smithi002.stdout: } 2024-09-18T04:58:48.059 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2024-09-18T04:58:48.059 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:48.059 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-18T04:58:48.061 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-18T04:58:48.061 INFO:tasks.ceph.osd.7.smithi099.stderr:2024-09-18T04:58:48.058+0000 7fae956ca640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 33005) UID: 0 2024-09-18T04:58:48.123 INFO:teuthology.orchestra.run.smithi002.stdout:All operations completed A-OK! 2024-09-18T04:58:48.133 DEBUG:teuthology.parallel:result is None 2024-09-18T04:58:48.134 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-18T04:58:48.142 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-18T04:58:48.186 INFO:tasks.thrashosds:joining thrashosds 2024-09-18T04:58:48.214 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:58:48.214 INFO:teuthology.orchestra.run.smithi002.stdout: "ops": [], 2024-09-18T04:58:48.214 INFO:teuthology.orchestra.run.smithi002.stdout: "num_ops": 0 2024-09-18T04:58:48.214 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:58:48.224 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-09-18T04:58:48.229 INFO:teuthology.orchestra.run.smithi002.stdout:osd.0: {} 2024-09-18T04:58:48.230 INFO:teuthology.orchestra.run.smithi002.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-18T04:58:48.247 INFO:teuthology.orchestra.run.smithi002.stdout:osd.1: {} 2024-09-18T04:58:48.247 INFO:teuthology.orchestra.run.smithi002.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-18T04:58:48.264 INFO:teuthology.orchestra.run.smithi002.stdout:osd.2: {} 2024-09-18T04:58:48.265 INFO:teuthology.orchestra.run.smithi002.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-18T04:58:48.281 INFO:teuthology.orchestra.run.smithi002.stdout:osd.3: {} 2024-09-18T04:58:48.281 INFO:teuthology.orchestra.run.smithi002.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-18T04:58:48.298 INFO:teuthology.orchestra.run.smithi002.stdout:osd.4: {} 2024-09-18T04:58:48.298 INFO:teuthology.orchestra.run.smithi002.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-18T04:58:48.315 INFO:teuthology.orchestra.run.smithi002.stdout:osd.5: {} 2024-09-18T04:58:48.316 INFO:teuthology.orchestra.run.smithi002.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-18T04:58:48.332 INFO:teuthology.orchestra.run.smithi002.stdout:osd.6: {} 2024-09-18T04:58:48.332 INFO:teuthology.orchestra.run.smithi002.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-18T04:58:48.333 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:48.333 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:48.342 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-09-18T04:58:48.349 INFO:teuthology.orchestra.run.smithi002.stdout:osd.7: {} 2024-09-18T04:58:48.349 INFO:teuthology.orchestra.run.smithi002.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-18T04:58:48.457 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:48.457 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:48.464 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-18T04:58:48.571 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:48.572 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:48.580 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-09-18T04:58:48.687 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:48.687 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:48.696 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-09-18T04:58:48.802 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:48.802 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:48.811 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-09-18T04:58:48.917 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:48.918 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:48.927 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-09-18T04:58:49.034 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:49.034 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:49.043 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-09-18T04:58:49.132 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-09-18T04:58:49.151 INFO:teuthology.orchestra.run.smithi002.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:49.151 INFO:teuthology.orchestra.run.smithi002.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:49.160 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-09-18T04:58:49.276 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:49.276 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:49.286 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-09-18T04:58:49.389 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:49.389 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:49.398 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-09-18T04:58:49.500 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:49.500 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:49.508 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-09-18T04:58:49.610 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:49.611 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:49.619 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-09-18T04:58:49.720 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:49.721 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:49.729 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-09-18T04:58:49.831 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:49.832 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:49.840 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-09-18T04:58:49.942 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:49.942 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:49.951 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-09-18T04:58:50.052 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:50.052 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:50.060 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-09-18T04:58:50.119 INFO:teuthology.orchestra.run.smithi002.stderr:noscrub is unset 2024-09-18T04:58:50.133 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-09-18T04:58:50.163 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:50.163 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:50.171 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-09-18T04:58:50.275 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:50.276 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:50.284 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-09-18T04:58:50.386 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:50.386 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:50.395 DEBUG:teuthology.orchestra.run.smithi099:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-09-18T04:58:50.498 INFO:teuthology.orchestra.run.smithi099.stdout:ERROR: (22) Invalid argument 2024-09-18T04:58:50.498 INFO:teuthology.orchestra.run.smithi099.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-18T04:58:51.125 INFO:teuthology.orchestra.run.smithi002.stderr:nodeep-scrub is unset 2024-09-18T04:58:52.133 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-18T04:58:52.259 INFO:teuthology.orchestra.run.smithi002.stdout:mon.a: {} 2024-09-18T04:58:52.259 INFO:teuthology.orchestra.run.smithi002.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T04:58:52.269 INFO:teuthology.orchestra.run.smithi002.stdout:mon.b: {} 2024-09-18T04:58:52.269 INFO:teuthology.orchestra.run.smithi002.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T04:58:52.278 INFO:teuthology.orchestra.run.smithi002.stdout:mon.c: {} 2024-09-18T04:58:52.278 INFO:teuthology.orchestra.run.smithi002.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-18T04:58:52.289 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-09-18T04:58:53.025 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.0 to 1 (10000) 2024-09-18T04:58:53.041 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-09-18T04:58:54.122 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-18T04:58:54.141 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-09-18T04:58:55.124 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.1 to 1 (10000) 2024-09-18T04:58:55.145 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-09-18T04:58:56.136 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-18T04:58:56.152 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-09-18T04:58:57.144 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.2 to 1 (10000) 2024-09-18T04:58:57.166 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-09-18T04:58:58.145 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-18T04:58:58.162 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-09-18T04:58:59.154 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.3 to 1 (10000) 2024-09-18T04:58:59.170 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-09-18T04:59:00.156 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-18T04:59:00.173 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-09-18T04:59:01.167 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.4 to 1 (10000) 2024-09-18T04:59:01.183 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-09-18T04:59:02.167 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-18T04:59:02.184 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-09-18T04:59:03.177 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.5 to 1 (10000) 2024-09-18T04:59:03.195 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-09-18T04:59:04.179 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-18T04:59:04.197 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-09-18T04:59:05.179 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.6 to 1 (10000) 2024-09-18T04:59:05.201 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-09-18T04:59:06.192 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-18T04:59:06.210 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-09-18T04:59:07.198 INFO:teuthology.orchestra.run.smithi002.stderr:reweighted osd.7 to 1 (10000) 2024-09-18T04:59:07.216 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-09-18T04:59:08.199 INFO:teuthology.orchestra.run.smithi002.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-18T04:59:08.215 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-18T04:59:08.216 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-18T04:59:08.216 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-18T04:59:08.216 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-18T04:59:08.216 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-18T04:59:08.216 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T04:59:08.507 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:59:08.507 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":176,"fsid":"c0761d0c-69d5-4b64-b8b9-c7d098b2d06d","created":"2024-09-18T04:55:58.216538+0000","modified":"2024-09-18T04:59:08.192977+0000","last_up_change":"2024-09-18T04:56:08.369924+0000","last_in_change":"2024-09-18T04:58:34.032750+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-18T04:56:08.983344+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":2.630000114440918,"raw_score_stable":2.630000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-18T04:56:10.313988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-18T04:56:19.562579+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":93,"snap_epoch":158,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c43f8024-3a3e-424e-aa54-398310ceb70d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4106704527},{"type":"v1","addr":"172.21.15.2:6825","nonce":4106704527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":4106704527},{"type":"v1","addr":"172.21.15.2:6827","nonce":4106704527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":4106704527},{"type":"v1","addr":"172.21.15.2:6831","nonce":4106704527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":4106704527},{"type":"v1","addr":"172.21.15.2:6829","nonce":4106704527}]},"public_addr":"172.21.15.2:6825/4106704527","cluster_addr":"172.21.15.2:6827/4106704527","heartbeat_back_addr":"172.21.15.2:6831/4106704527","heartbeat_front_addr":"172.21.15.2:6829/4106704527","state":["exists","up"]},{"osd":1,"uuid":"c664b970-145c-4b6f-8852-2f6dcf1fa30d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1781959259},{"type":"v1","addr":"172.21.15.2:6809","nonce":1781959259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1781959259},{"type":"v1","addr":"172.21.15.2:6811","nonce":1781959259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1781959259},{"type":"v1","addr":"172.21.15.2:6815","nonce":1781959259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1781959259},{"type":"v1","addr":"172.21.15.2:6813","nonce":1781959259}]},"public_addr":"172.21.15.2:6809/1781959259","cluster_addr":"172.21.15.2:6811/1781959259","heartbeat_back_addr":"172.21.15.2:6815/1781959259","heartbeat_front_addr":"172.21.15.2:6813/1781959259","state":["exists","up"]},{"osd":2,"uuid":"0609ea8c-d975-490e-8844-04a3e4512efd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":1153570802},{"type":"v1","addr":"172.21.15.2:6801","nonce":1153570802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1153570802},{"type":"v1","addr":"172.21.15.2:6803","nonce":1153570802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1153570802},{"type":"v1","addr":"172.21.15.2:6807","nonce":1153570802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1153570802},{"type":"v1","addr":"172.21.15.2:6805","nonce":1153570802}]},"public_addr":"172.21.15.2:6801/1153570802","cluster_addr":"172.21.15.2:6803/1153570802","heartbeat_back_addr":"172.21.15.2:6807/1153570802","heartbeat_front_addr":"172.21.15.2:6805/1153570802","state":["exists","up"]},{"osd":3,"uuid":"c6bd5ac8-ab89-4fa2-b843-3b166d1b238b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":2008321375},{"type":"v1","addr":"172.21.15.2:6817","nonce":2008321375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2008321375},{"type":"v1","addr":"172.21.15.2:6819","nonce":2008321375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2008321375},{"type":"v1","addr":"172.21.15.2:6823","nonce":2008321375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2008321375},{"type":"v1","addr":"172.21.15.2:6821","nonce":2008321375}]},"public_addr":"172.21.15.2:6817/2008321375","cluster_addr":"172.21.15.2:6819/2008321375","heartbeat_back_addr":"172.21.15.2:6823/2008321375","heartbeat_front_addr":"172.21.15.2:6821/2008321375","state":["exists","up"]},{"osd":4,"uuid":"89d6a371-4f4e-465e-a023-d5083cce6e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2939896598},{"type":"v1","addr":"172.21.15.99:6825","nonce":2939896598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":2939896598},{"type":"v1","addr":"172.21.15.99:6827","nonce":2939896598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":2939896598},{"type":"v1","addr":"172.21.15.99:6831","nonce":2939896598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":2939896598},{"type":"v1","addr":"172.21.15.99:6829","nonce":2939896598}]},"public_addr":"172.21.15.99:6825/2939896598","cluster_addr":"172.21.15.99:6827/2939896598","heartbeat_back_addr":"172.21.15.99:6831/2939896598","heartbeat_front_addr":"172.21.15.99:6829/2939896598","state":["exists","up"]},{"osd":5,"uuid":"da99686b-d979-4753-b4aa-a3b26e9a0497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":3440879618},{"type":"v1","addr":"172.21.15.99:6817","nonce":3440879618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":3440879618},{"type":"v1","addr":"172.21.15.99:6819","nonce":3440879618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":3440879618},{"type":"v1","addr":"172.21.15.99:6823","nonce":3440879618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":3440879618},{"type":"v1","addr":"172.21.15.99:6821","nonce":3440879618}]},"public_addr":"172.21.15.99:6817/3440879618","cluster_addr":"172.21.15.99:6819/3440879618","heartbeat_back_addr":"172.21.15.99:6823/3440879618","heartbeat_front_addr":"172.21.15.99:6821/3440879618","state":["exists","up"]},{"osd":6,"uuid":"ceec6aa6-603d-4706-bfea-3c07dbdaed3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":2493830504},{"type":"v1","addr":"172.21.15.99:6801","nonce":2493830504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":2493830504},{"type":"v1","addr":"172.21.15.99:6803","nonce":2493830504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":2493830504},{"type":"v1","addr":"172.21.15.99:6807","nonce":2493830504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":2493830504},{"type":"v1","addr":"172.21.15.99:6805","nonce":2493830504}]},"public_addr":"172.21.15.99:6801/2493830504","cluster_addr":"172.21.15.99:6803/2493830504","heartbeat_back_addr":"172.21.15.99:6807/2493830504","heartbeat_front_addr":"172.21.15.99:6805/2493830504","state":["exists","up"]},{"osd":7,"uuid":"6779dfad-f60b-41f4-b666-a99db6dc1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":91038858},{"type":"v1","addr":"172.21.15.99:6809","nonce":91038858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":91038858},{"type":"v1","addr":"172.21.15.99:6811","nonce":91038858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":91038858},{"type":"v1","addr":"172.21.15.99:6815","nonce":91038858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":91038858},{"type":"v1","addr":"172.21.15.99:6813","nonce":91038858}]},"public_addr":"172.21.15.99:6809/91038858","cluster_addr":"172.21.15.99:6811/91038858","heartbeat_back_addr":"172.21.15.99:6815/91038858","heartbeat_front_addr":"172.21.15.99:6813/91038858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.468213+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.538610+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.500881+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.474179+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.483737+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.460693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.475970+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.519287+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"1.4","osds":[6,7]},{"pgid":"1.5","osds":[1,4]}],"pg_upmap_items":[{"pgid":"3.1","mappings":[{"from":4,"to":2}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.11","mappings":[{"from":4,"to":1}]},{"pgid":"3.1a","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T04:59:08.519 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-18T04:59:08.520 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T04:59:08.804 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:59:08.804 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":176,"fsid":"c0761d0c-69d5-4b64-b8b9-c7d098b2d06d","created":"2024-09-18T04:55:58.216538+0000","modified":"2024-09-18T04:59:08.192977+0000","last_up_change":"2024-09-18T04:56:08.369924+0000","last_in_change":"2024-09-18T04:58:34.032750+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-18T04:56:08.983344+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":2.630000114440918,"raw_score_stable":2.630000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-18T04:56:10.313988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-18T04:56:19.562579+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":93,"snap_epoch":158,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c43f8024-3a3e-424e-aa54-398310ceb70d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4106704527},{"type":"v1","addr":"172.21.15.2:6825","nonce":4106704527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":4106704527},{"type":"v1","addr":"172.21.15.2:6827","nonce":4106704527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":4106704527},{"type":"v1","addr":"172.21.15.2:6831","nonce":4106704527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":4106704527},{"type":"v1","addr":"172.21.15.2:6829","nonce":4106704527}]},"public_addr":"172.21.15.2:6825/4106704527","cluster_addr":"172.21.15.2:6827/4106704527","heartbeat_back_addr":"172.21.15.2:6831/4106704527","heartbeat_front_addr":"172.21.15.2:6829/4106704527","state":["exists","up"]},{"osd":1,"uuid":"c664b970-145c-4b6f-8852-2f6dcf1fa30d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1781959259},{"type":"v1","addr":"172.21.15.2:6809","nonce":1781959259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1781959259},{"type":"v1","addr":"172.21.15.2:6811","nonce":1781959259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1781959259},{"type":"v1","addr":"172.21.15.2:6815","nonce":1781959259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1781959259},{"type":"v1","addr":"172.21.15.2:6813","nonce":1781959259}]},"public_addr":"172.21.15.2:6809/1781959259","cluster_addr":"172.21.15.2:6811/1781959259","heartbeat_back_addr":"172.21.15.2:6815/1781959259","heartbeat_front_addr":"172.21.15.2:6813/1781959259","state":["exists","up"]},{"osd":2,"uuid":"0609ea8c-d975-490e-8844-04a3e4512efd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":1153570802},{"type":"v1","addr":"172.21.15.2:6801","nonce":1153570802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1153570802},{"type":"v1","addr":"172.21.15.2:6803","nonce":1153570802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1153570802},{"type":"v1","addr":"172.21.15.2:6807","nonce":1153570802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1153570802},{"type":"v1","addr":"172.21.15.2:6805","nonce":1153570802}]},"public_addr":"172.21.15.2:6801/1153570802","cluster_addr":"172.21.15.2:6803/1153570802","heartbeat_back_addr":"172.21.15.2:6807/1153570802","heartbeat_front_addr":"172.21.15.2:6805/1153570802","state":["exists","up"]},{"osd":3,"uuid":"c6bd5ac8-ab89-4fa2-b843-3b166d1b238b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":2008321375},{"type":"v1","addr":"172.21.15.2:6817","nonce":2008321375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2008321375},{"type":"v1","addr":"172.21.15.2:6819","nonce":2008321375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2008321375},{"type":"v1","addr":"172.21.15.2:6823","nonce":2008321375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2008321375},{"type":"v1","addr":"172.21.15.2:6821","nonce":2008321375}]},"public_addr":"172.21.15.2:6817/2008321375","cluster_addr":"172.21.15.2:6819/2008321375","heartbeat_back_addr":"172.21.15.2:6823/2008321375","heartbeat_front_addr":"172.21.15.2:6821/2008321375","state":["exists","up"]},{"osd":4,"uuid":"89d6a371-4f4e-465e-a023-d5083cce6e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2939896598},{"type":"v1","addr":"172.21.15.99:6825","nonce":2939896598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":2939896598},{"type":"v1","addr":"172.21.15.99:6827","nonce":2939896598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":2939896598},{"type":"v1","addr":"172.21.15.99:6831","nonce":2939896598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":2939896598},{"type":"v1","addr":"172.21.15.99:6829","nonce":2939896598}]},"public_addr":"172.21.15.99:6825/2939896598","cluster_addr":"172.21.15.99:6827/2939896598","heartbeat_back_addr":"172.21.15.99:6831/2939896598","heartbeat_front_addr":"172.21.15.99:6829/2939896598","state":["exists","up"]},{"osd":5,"uuid":"da99686b-d979-4753-b4aa-a3b26e9a0497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":3440879618},{"type":"v1","addr":"172.21.15.99:6817","nonce":3440879618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":3440879618},{"type":"v1","addr":"172.21.15.99:6819","nonce":3440879618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":3440879618},{"type":"v1","addr":"172.21.15.99:6823","nonce":3440879618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":3440879618},{"type":"v1","addr":"172.21.15.99:6821","nonce":3440879618}]},"public_addr":"172.21.15.99:6817/3440879618","cluster_addr":"172.21.15.99:6819/3440879618","heartbeat_back_addr":"172.21.15.99:6823/3440879618","heartbeat_front_addr":"172.21.15.99:6821/3440879618","state":["exists","up"]},{"osd":6,"uuid":"ceec6aa6-603d-4706-bfea-3c07dbdaed3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":2493830504},{"type":"v1","addr":"172.21.15.99:6801","nonce":2493830504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":2493830504},{"type":"v1","addr":"172.21.15.99:6803","nonce":2493830504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":2493830504},{"type":"v1","addr":"172.21.15.99:6807","nonce":2493830504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":2493830504},{"type":"v1","addr":"172.21.15.99:6805","nonce":2493830504}]},"public_addr":"172.21.15.99:6801/2493830504","cluster_addr":"172.21.15.99:6803/2493830504","heartbeat_back_addr":"172.21.15.99:6807/2493830504","heartbeat_front_addr":"172.21.15.99:6805/2493830504","state":["exists","up"]},{"osd":7,"uuid":"6779dfad-f60b-41f4-b666-a99db6dc1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":91038858},{"type":"v1","addr":"172.21.15.99:6809","nonce":91038858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":91038858},{"type":"v1","addr":"172.21.15.99:6811","nonce":91038858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":91038858},{"type":"v1","addr":"172.21.15.99:6815","nonce":91038858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":91038858},{"type":"v1","addr":"172.21.15.99:6813","nonce":91038858}]},"public_addr":"172.21.15.99:6809/91038858","cluster_addr":"172.21.15.99:6811/91038858","heartbeat_back_addr":"172.21.15.99:6815/91038858","heartbeat_front_addr":"172.21.15.99:6813/91038858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.468213+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.538610+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.500881+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.474179+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.483737+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.460693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.475970+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.519287+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"1.4","osds":[6,7]},{"pgid":"1.5","osds":[1,4]}],"pg_upmap_items":[{"pgid":"3.1","mappings":[{"from":4,"to":2}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.11","mappings":[{"from":4,"to":1}]},{"pgid":"3.1a","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T04:59:08.817 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-18T04:59:08.818 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-18T04:59:08.818 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-18T04:59:08.818 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-18T04:59:08.819 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-18T04:59:08.819 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-18T04:59:08.819 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-18T04:59:08.820 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-18T04:59:09.065 INFO:teuthology.orchestra.run.smithi002.stdout:55834574888 2024-09-18T04:59:09.065 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T04:59:09.070 INFO:teuthology.orchestra.run.smithi002.stdout:55834574888 2024-09-18T04:59:09.071 INFO:teuthology.orchestra.run.smithi002.stdout:55834574888 2024-09-18T04:59:09.071 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T04:59:09.072 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T04:59:09.073 INFO:teuthology.orchestra.run.smithi002.stdout:55834574887 2024-09-18T04:59:09.073 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T04:59:09.074 INFO:teuthology.orchestra.run.smithi002.stdout:55834574887 2024-09-18T04:59:09.074 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T04:59:09.075 INFO:teuthology.orchestra.run.smithi002.stdout:55834574887 2024-09-18T04:59:09.075 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T04:59:09.076 INFO:teuthology.orchestra.run.smithi002.stdout:55834574887 2024-09-18T04:59:09.077 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T04:59:09.078 INFO:teuthology.orchestra.run.smithi002.stdout:55834574887 2024-09-18T04:59:09.078 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T04:59:09.630 INFO:teuthology.orchestra.run.smithi002.stdout:55834574887 2024-09-18T04:59:09.647 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574888 got 55834574887 for osd.1 2024-09-18T04:59:09.679 INFO:teuthology.orchestra.run.smithi002.stdout:55834574886 2024-09-18T04:59:09.682 INFO:teuthology.orchestra.run.smithi002.stdout:55834574887 2024-09-18T04:59:09.694 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574887 got 55834574886 for osd.5 2024-09-18T04:59:09.701 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574888 got 55834574887 for osd.0 2024-09-18T04:59:09.702 INFO:teuthology.orchestra.run.smithi002.stdout:55834574886 2024-09-18T04:59:09.707 INFO:teuthology.orchestra.run.smithi002.stdout:55834574887 2024-09-18T04:59:09.717 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574887 got 55834574886 for osd.4 2024-09-18T04:59:09.719 INFO:teuthology.orchestra.run.smithi002.stdout:55834574886 2024-09-18T04:59:09.725 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574888 got 55834574887 for osd.7 2024-09-18T04:59:09.727 INFO:teuthology.orchestra.run.smithi002.stdout:55834574886 2024-09-18T04:59:09.730 INFO:teuthology.orchestra.run.smithi002.stdout:55834574886 2024-09-18T04:59:09.734 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574887 got 55834574886 for osd.2 2024-09-18T04:59:09.741 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574887 got 55834574886 for osd.6 2024-09-18T04:59:09.745 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574887 got 55834574886 for osd.3 2024-09-18T04:59:10.648 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T04:59:10.695 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T04:59:10.702 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T04:59:10.718 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T04:59:10.725 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T04:59:10.735 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T04:59:10.741 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T04:59:10.746 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T04:59:11.174 INFO:teuthology.orchestra.run.smithi002.stdout:55834574887 2024-09-18T04:59:11.193 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574888 got 55834574887 for osd.1 2024-09-18T04:59:11.254 INFO:teuthology.orchestra.run.smithi002.stdout:55834574888 2024-09-18T04:59:11.259 INFO:teuthology.orchestra.run.smithi002.stdout:55834574888 2024-09-18T04:59:11.273 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574887 got 55834574888 for osd.2 2024-09-18T04:59:11.273 DEBUG:teuthology.parallel:result is None 2024-09-18T04:59:11.274 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574887 got 55834574888 for osd.5 2024-09-18T04:59:11.275 DEBUG:teuthology.parallel:result is None 2024-09-18T04:59:11.301 INFO:teuthology.orchestra.run.smithi002.stdout:55834574888 2024-09-18T04:59:11.314 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574887 got 55834574888 for osd.4 2024-09-18T04:59:11.315 INFO:teuthology.orchestra.run.smithi002.stdout:55834574889 2024-09-18T04:59:11.315 DEBUG:teuthology.parallel:result is None 2024-09-18T04:59:11.327 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574888 got 55834574889 for osd.7 2024-09-18T04:59:11.328 DEBUG:teuthology.parallel:result is None 2024-09-18T04:59:11.328 INFO:teuthology.orchestra.run.smithi002.stdout:55834574889 2024-09-18T04:59:11.334 INFO:teuthology.orchestra.run.smithi002.stdout:55834574888 2024-09-18T04:59:11.343 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574888 got 55834574889 for osd.0 2024-09-18T04:59:11.343 DEBUG:teuthology.parallel:result is None 2024-09-18T04:59:11.348 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574887 got 55834574888 for osd.6 2024-09-18T04:59:11.348 DEBUG:teuthology.parallel:result is None 2024-09-18T04:59:11.356 INFO:teuthology.orchestra.run.smithi002.stdout:55834574888 2024-09-18T04:59:11.369 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574887 got 55834574888 for osd.3 2024-09-18T04:59:11.369 DEBUG:teuthology.parallel:result is None 2024-09-18T04:59:12.195 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T04:59:12.497 INFO:teuthology.orchestra.run.smithi002.stdout:55834574889 2024-09-18T04:59:12.509 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574888 got 55834574889 for osd.1 2024-09-18T04:59:12.510 DEBUG:teuthology.parallel:result is None 2024-09-18T04:59:12.510 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-09-18T04:59:12.510 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T04:59:12.797 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:59:12.798 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-18T04:59:12.807 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2024-09-18T04:59:12.299589+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":47,"num_read":220900,"num_read_kb":683302,"num_write":30057,"num_write_kb":380196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9215,"num_bytes_recovered":185732404,"num_keys_recovered":562,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24084,"ondisk_log_size":24084,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":440292,"kb_used_data":16620,"kb_used_omap":12,"kb_used_meta":423539,"kb_avail":754534428,"statfs":{"total":773094113280,"available":772643254272,"internally_reserved":0,"allocated":17018880,"data_stored":9417476,"data_compressed":18433,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":12742,"internal_metadata":433704506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":524,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.110496"},"pg_stats":[{"pgid":"3.1f","version":"158'766","reported_seq":5157,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.197217+0000","last_change":"2024-09-18T04:58:55.128107+0000","last_active":"2024-09-18T04:59:07.197217+0000","last_peered":"2024-09-18T04:59:07.197217+0000","last_clean":"2024-09-18T04:59:07.197217+0000","last_became_active":"2024-09-18T04:58:55.126111+0000","last_became_peered":"2024-09-18T04:58:55.126111+0000","last_unstale":"2024-09-18T04:59:07.197217+0000","last_undegraded":"2024-09-18T04:59:07.197217+0000","last_fullsized":"2024-09-18T04:59:07.197217+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:56:05.136229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001156916,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6348,"num_read_kb":20852,"num_write":870,"num_write_kb":9417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"157'817","reported_seq":2783,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731844+0000","last_change":"2024-09-18T04:58:48.096250+0000","last_active":"2024-09-18T04:59:07.731844+0000","last_peered":"2024-09-18T04:59:07.731844+0000","last_clean":"2024-09-18T04:59:07.731844+0000","last_became_active":"2024-09-18T04:58:32.029310+0000","last_became_peered":"2024-09-18T04:58:32.029310+0000","last_unstale":"2024-09-18T04:59:07.731844+0000","last_undegraded":"2024-09-18T04:59:07.731844+0000","last_fullsized":"2024-09-18T04:59:07.731844+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:13:02.688292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001168749,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3351,"num_read_kb":18023,"num_write":1120,"num_write_kb":8161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":3252478,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"156'500","reported_seq":10759,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195673+0000","last_change":"2024-09-18T04:58:48.023738+0000","last_active":"2024-09-18T04:59:07.195673+0000","last_peered":"2024-09-18T04:59:07.195673+0000","last_clean":"2024-09-18T04:59:07.195673+0000","last_became_active":"2024-09-18T04:58:27.115104+0000","last_became_peered":"2024-09-18T04:58:27.115104+0000","last_unstale":"2024-09-18T04:59:07.195673+0000","last_undegraded":"2024-09-18T04:59:07.195673+0000","last_fullsized":"2024-09-18T04:59:07.195673+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:33:08.572507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082741200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9761,"num_read_kb":28168,"num_write":548,"num_write_kb":23097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7293282,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"158'840","reported_seq":7984,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195584+0000","last_change":"2024-09-18T04:58:48.025083+0000","last_active":"2024-09-18T04:59:07.195584+0000","last_peered":"2024-09-18T04:59:07.195584+0000","last_clean":"2024-09-18T04:59:07.195584+0000","last_became_active":"2024-09-18T04:58:28.028905+0000","last_became_peered":"2024-09-18T04:58:28.028905+0000","last_unstale":"2024-09-18T04:59:07.195584+0000","last_undegraded":"2024-09-18T04:59:07.195584+0000","last_fullsized":"2024-09-18T04:59:07.195584+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:15:17.569793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014698000000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":8594,"num_read_kb":9084,"num_write":1169,"num_write_kb":5309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":1412856,"num_keys_recovered":106,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"155'473","reported_seq":4267,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.202696+0000","last_change":"2024-09-18T04:59:07.202189+0000","last_active":"2024-09-18T04:59:07.202696+0000","last_peered":"2024-09-18T04:59:07.202696+0000","last_clean":"2024-09-18T04:59:07.202696+0000","last_became_active":"2024-09-18T04:59:07.199588+0000","last_became_peered":"2024-09-18T04:59:07.199588+0000","last_unstale":"2024-09-18T04:59:07.202696+0000","last_undegraded":"2024-09-18T04:59:07.202696+0000","last_fullsized":"2024-09-18T04:59:07.202696+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:53:25.971943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012312829999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2677,"num_read_kb":15514,"num_write":503,"num_write_kb":9112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":912,"num_bytes_recovered":7005168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"157'455","reported_seq":3935,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:08.199477+0000","last_change":"2024-09-18T04:59:07.200974+0000","last_active":"2024-09-18T04:59:08.199477+0000","last_peered":"2024-09-18T04:59:08.199477+0000","last_clean":"2024-09-18T04:59:08.199477+0000","last_became_active":"2024-09-18T04:59:07.199276+0000","last_became_peered":"2024-09-18T04:59:07.199276+0000","last_unstale":"2024-09-18T04:59:08.199477+0000","last_undegraded":"2024-09-18T04:59:08.199477+0000","last_fullsized":"2024-09-18T04:59:08.199477+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:42:10.766349+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.000757346,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":2981,"num_read_kb":11740,"num_write":512,"num_write_kb":18285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":519,"num_bytes_recovered":11494100,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"156'520","reported_seq":5473,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198149+0000","last_change":"2024-09-18T04:58:57.148756+0000","last_active":"2024-09-18T04:59:07.198149+0000","last_peered":"2024-09-18T04:59:07.198149+0000","last_clean":"2024-09-18T04:59:07.198149+0000","last_became_active":"2024-09-18T04:58:57.145900+0000","last_became_peered":"2024-09-18T04:58:57.145900+0000","last_unstale":"2024-09-18T04:59:07.198149+0000","last_undegraded":"2024-09-18T04:59:07.198149+0000","last_fullsized":"2024-09-18T04:59:07.198149+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:03:40.179686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018405870000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4543,"num_read_kb":26564,"num_write":538,"num_write_kb":13069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":23811007,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"156'576","reported_seq":3426,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731747+0000","last_change":"2024-09-18T04:58:48.095394+0000","last_active":"2024-09-18T04:59:07.731747+0000","last_peered":"2024-09-18T04:59:07.731747+0000","last_clean":"2024-09-18T04:59:07.731747+0000","last_became_active":"2024-09-18T04:58:32.029074+0000","last_became_peered":"2024-09-18T04:58:32.029074+0000","last_unstale":"2024-09-18T04:59:07.731747+0000","last_undegraded":"2024-09-18T04:59:07.731747+0000","last_fullsized":"2024-09-18T04:59:07.731747+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:15:38.950060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053623899999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3403,"num_read_kb":16958,"num_write":699,"num_write_kb":12986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":4650417,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"153'506","reported_seq":4239,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.197394+0000","last_change":"2024-09-18T04:58:55.128037+0000","last_active":"2024-09-18T04:59:07.197394+0000","last_peered":"2024-09-18T04:59:07.197394+0000","last_clean":"2024-09-18T04:59:07.197394+0000","last_became_active":"2024-09-18T04:58:55.125721+0000","last_became_peered":"2024-09-18T04:58:55.125721+0000","last_unstale":"2024-09-18T04:59:07.197394+0000","last_undegraded":"2024-09-18T04:59:07.197394+0000","last_fullsized":"2024-09-18T04:59:07.197394+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:06:23.656568+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001441276,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4899,"num_read_kb":13699,"num_write":541,"num_write_kb":6077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"154'885","reported_seq":5042,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195463+0000","last_change":"2024-09-18T04:58:48.024658+0000","last_active":"2024-09-18T04:59:07.195463+0000","last_peered":"2024-09-18T04:59:07.195463+0000","last_clean":"2024-09-18T04:59:07.195463+0000","last_became_active":"2024-09-18T04:56:21.131133+0000","last_became_peered":"2024-09-18T04:56:21.131133+0000","last_unstale":"2024-09-18T04:59:07.195463+0000","last_undegraded":"2024-09-18T04:59:07.195463+0000","last_fullsized":"2024-09-18T04:59:07.195463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:01:57.036904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012610480000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6340,"num_read_kb":26579,"num_write":1141,"num_write_kb":12621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"157'637","reported_seq":5292,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198169+0000","last_change":"2024-09-18T04:58:48.028934+0000","last_active":"2024-09-18T04:59:07.198169+0000","last_peered":"2024-09-18T04:59:07.198169+0000","last_clean":"2024-09-18T04:59:07.198169+0000","last_became_active":"2024-09-18T04:58:32.032762+0000","last_became_peered":"2024-09-18T04:58:32.032762+0000","last_unstale":"2024-09-18T04:59:07.198169+0000","last_undegraded":"2024-09-18T04:59:07.198169+0000","last_fullsized":"2024-09-18T04:59:07.198169+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:33:25.010425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066811099999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":5292,"num_read_kb":26013,"num_write":596,"num_write_kb":7300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":447,"num_bytes_recovered":1594801,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"155'502","reported_seq":2817,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-18T04:59:06.190587+0000","last_change":"2024-09-18T04:58:48.023734+0000","last_active":"2024-09-18T04:59:06.190587+0000","last_peered":"2024-09-18T04:59:06.190587+0000","last_clean":"2024-09-18T04:59:06.190587+0000","last_became_active":"2024-09-18T04:58:30.025558+0000","last_became_peered":"2024-09-18T04:58:30.025558+0000","last_unstale":"2024-09-18T04:59:06.190587+0000","last_undegraded":"2024-09-18T04:59:06.190587+0000","last_fullsized":"2024-09-18T04:59:06.190587+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:57:31.113016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087474099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3131,"num_read_kb":27325,"num_write":609,"num_write_kb":11672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":12427258,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":192,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195662+0000","last_change":"2024-09-18T04:58:29.025271+0000","last_active":"2024-09-18T04:59:07.195662+0000","last_peered":"2024-09-18T04:59:07.195662+0000","last_clean":"2024-09-18T04:59:07.195662+0000","last_became_active":"2024-09-18T04:58:29.024911+0000","last_became_peered":"2024-09-18T04:58:29.024911+0000","last_unstale":"2024-09-18T04:59:07.195662+0000","last_undegraded":"2024-09-18T04:59:07.195662+0000","last_fullsized":"2024-09-18T04:59:07.195662+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:53:21.863837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063436799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"155'2480","reported_seq":9899,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195724+0000","last_change":"2024-09-18T04:58:48.024643+0000","last_active":"2024-09-18T04:59:07.195724+0000","last_peered":"2024-09-18T04:59:07.195724+0000","last_clean":"2024-09-18T04:59:07.195724+0000","last_became_active":"2024-09-18T04:58:33.027340+0000","last_became_peered":"2024-09-18T04:58:33.027340+0000","last_unstale":"2024-09-18T04:59:07.195724+0000","last_undegraded":"2024-09-18T04:59:07.195724+0000","last_fullsized":"2024-09-18T04:59:07.195724+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":2480,"log_dups_size":0,"ondisk_log_size":2480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:47:47.826132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000952045,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":18529,"num_read_kb":32990,"num_write":4379,"num_write_kb":20206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":5439803,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":176,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195585+0000","last_change":"2024-09-18T04:56:12.389971+0000","last_active":"2024-09-18T04:59:07.195585+0000","last_peered":"2024-09-18T04:59:07.195585+0000","last_clean":"2024-09-18T04:59:07.195585+0000","last_became_active":"2024-09-18T04:56:10.412595+0000","last_became_peered":"2024-09-18T04:56:10.412595+0000","last_unstale":"2024-09-18T04:59:07.195585+0000","last_undegraded":"2024-09-18T04:59:07.195585+0000","last_fullsized":"2024-09-18T04:59:07.195585+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:45.638420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038576799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"157'807","reported_seq":7861,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198219+0000","last_change":"2024-09-18T04:58:57.151001+0000","last_active":"2024-09-18T04:59:07.198219+0000","last_peered":"2024-09-18T04:59:07.198219+0000","last_clean":"2024-09-18T04:59:07.198219+0000","last_became_active":"2024-09-18T04:58:57.147959+0000","last_became_peered":"2024-09-18T04:58:57.147959+0000","last_unstale":"2024-09-18T04:59:07.198219+0000","last_undegraded":"2024-09-18T04:59:07.198219+0000","last_fullsized":"2024-09-18T04:59:07.198219+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:30:19.725543+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00065254400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8940,"num_read_kb":28417,"num_write":1051,"num_write_kb":11406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":5706670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198307+0000","last_change":"2024-09-18T04:58:57.150288+0000","last_active":"2024-09-18T04:59:07.198307+0000","last_peered":"2024-09-18T04:59:07.198307+0000","last_clean":"2024-09-18T04:59:07.198307+0000","last_became_active":"2024-09-18T04:58:57.149907+0000","last_became_peered":"2024-09-18T04:58:57.149907+0000","last_unstale":"2024-09-18T04:59:07.198307+0000","last_undegraded":"2024-09-18T04:59:07.198307+0000","last_fullsized":"2024-09-18T04:59:07.198307+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:12:39.499969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"157'757","reported_seq":6181,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731891+0000","last_change":"2024-09-18T04:58:48.096507+0000","last_active":"2024-09-18T04:59:07.731891+0000","last_peered":"2024-09-18T04:59:07.731891+0000","last_clean":"2024-09-18T04:59:07.731891+0000","last_became_active":"2024-09-18T04:58:33.026806+0000","last_became_peered":"2024-09-18T04:58:33.026806+0000","last_unstale":"2024-09-18T04:59:07.731891+0000","last_undegraded":"2024-09-18T04:59:07.731891+0000","last_fullsized":"2024-09-18T04:59:07.731891+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:59:41.673907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013736829999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6896,"num_read_kb":21261,"num_write":943,"num_write_kb":5208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":571544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":145,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195438+0000","last_change":"2024-09-18T04:56:46.254241+0000","last_active":"2024-09-18T04:59:07.195438+0000","last_peered":"2024-09-18T04:59:07.195438+0000","last_clean":"2024-09-18T04:59:07.195438+0000","last_became_active":"2024-09-18T04:56:46.253894+0000","last_became_peered":"2024-09-18T04:56:46.253894+0000","last_unstale":"2024-09-18T04:59:07.195438+0000","last_undegraded":"2024-09-18T04:59:07.195438+0000","last_fullsized":"2024-09-18T04:59:07.195438+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:58:42.098099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045144100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"158'505","reported_seq":5028,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.197318+0000","last_change":"2024-09-18T04:58:48.061503+0000","last_active":"2024-09-18T04:59:07.197318+0000","last_peered":"2024-09-18T04:59:07.197318+0000","last_clean":"2024-09-18T04:59:07.197318+0000","last_became_active":"2024-09-18T04:58:41.078435+0000","last_became_peered":"2024-09-18T04:58:41.078435+0000","last_unstale":"2024-09-18T04:59:07.197318+0000","last_undegraded":"2024-09-18T04:59:07.197318+0000","last_fullsized":"2024-09-18T04:59:07.197318+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:34:50.214360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056043399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4221,"num_read_kb":33044,"num_write":472,"num_write_kb":37350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":25349011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"155'485","reported_seq":9536,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.201259+0000","last_change":"2024-09-18T04:59:07.200564+0000","last_active":"2024-09-18T04:59:07.201259+0000","last_peered":"2024-09-18T04:59:07.201259+0000","last_clean":"2024-09-18T04:59:07.201259+0000","last_became_active":"2024-09-18T04:59:07.199056+0000","last_became_peered":"2024-09-18T04:59:07.199056+0000","last_unstale":"2024-09-18T04:59:07.201259+0000","last_undegraded":"2024-09-18T04:59:07.201259+0000","last_fullsized":"2024-09-18T04:59:07.201259+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:30:30.980376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049796699999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":9360,"num_read_kb":28051,"num_write":499,"num_write_kb":12488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":472,"num_bytes_recovered":11362815,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":362,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198082+0000","last_change":"2024-09-18T04:58:38.025236+0000","last_active":"2024-09-18T04:59:07.198082+0000","last_peered":"2024-09-18T04:59:07.198082+0000","last_clean":"2024-09-18T04:59:07.198082+0000","last_became_active":"2024-09-18T04:58:38.024740+0000","last_became_peered":"2024-09-18T04:58:38.024740+0000","last_unstale":"2024-09-18T04:59:07.198082+0000","last_undegraded":"2024-09-18T04:59:07.198082+0000","last_fullsized":"2024-09-18T04:59:07.198082+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_clean_scrub_stamp":"2024-09-18T04:56:10.373785+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:10:45.251654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":37,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198355+0000","last_change":"2024-09-18T04:58:57.148289+0000","last_active":"2024-09-18T04:59:07.198355+0000","last_peered":"2024-09-18T04:59:07.198355+0000","last_clean":"2024-09-18T04:59:07.198355+0000","last_became_active":"2024-09-18T04:58:57.147915+0000","last_became_peered":"2024-09-18T04:58:57.147915+0000","last_unstale":"2024-09-18T04:59:07.198355+0000","last_undegraded":"2024-09-18T04:59:07.198355+0000","last_fullsized":"2024-09-18T04:59:07.198355+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:31:21.699401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"157'811","reported_seq":13428,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198153+0000","last_change":"2024-09-18T04:58:57.152156+0000","last_active":"2024-09-18T04:59:07.198153+0000","last_peered":"2024-09-18T04:59:07.198153+0000","last_clean":"2024-09-18T04:59:07.198153+0000","last_became_active":"2024-09-18T04:58:57.148221+0000","last_became_peered":"2024-09-18T04:58:57.148221+0000","last_unstale":"2024-09-18T04:59:07.198153+0000","last_undegraded":"2024-09-18T04:59:07.198153+0000","last_fullsized":"2024-09-18T04:59:07.198153+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:45:57.434891+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0033124909999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":14837,"num_read_kb":27169,"num_write":1047,"num_write_kb":7785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":1280827,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":178,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198076+0000","last_change":"2024-09-18T04:56:12.398631+0000","last_active":"2024-09-18T04:59:07.198076+0000","last_peered":"2024-09-18T04:59:07.198076+0000","last_clean":"2024-09-18T04:59:07.198076+0000","last_became_active":"2024-09-18T04:56:10.412614+0000","last_became_peered":"2024-09-18T04:56:10.412614+0000","last_unstale":"2024-09-18T04:59:07.198076+0000","last_undegraded":"2024-09-18T04:59:07.198076+0000","last_fullsized":"2024-09-18T04:59:07.198076+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:44:13.401320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075126299999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"158'601","reported_seq":2977,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-18T04:59:06.191350+0000","last_change":"2024-09-18T04:58:48.024976+0000","last_active":"2024-09-18T04:59:06.191350+0000","last_peered":"2024-09-18T04:59:06.191350+0000","last_clean":"2024-09-18T04:59:06.191350+0000","last_became_active":"2024-09-18T04:58:30.027757+0000","last_became_peered":"2024-09-18T04:58:30.027757+0000","last_unstale":"2024-09-18T04:59:06.191350+0000","last_undegraded":"2024-09-18T04:59:06.191350+0000","last_fullsized":"2024-09-18T04:59:06.191350+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:29:00.106938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014951890000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":2216,"num_read_kb":23967,"num_write":593,"num_write_kb":16097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":10562840,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":176,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195410+0000","last_change":"2024-09-18T04:56:12.390044+0000","last_active":"2024-09-18T04:59:07.195410+0000","last_peered":"2024-09-18T04:59:07.195410+0000","last_clean":"2024-09-18T04:59:07.195410+0000","last_became_active":"2024-09-18T04:56:10.413042+0000","last_became_peered":"2024-09-18T04:56:10.413042+0000","last_unstale":"2024-09-18T04:59:07.195410+0000","last_undegraded":"2024-09-18T04:59:07.195410+0000","last_fullsized":"2024-09-18T04:59:07.195410+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:46:11.074243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"157'525","reported_seq":4818,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731809+0000","last_change":"2024-09-18T04:58:48.095298+0000","last_active":"2024-09-18T04:59:07.731809+0000","last_peered":"2024-09-18T04:59:07.731809+0000","last_clean":"2024-09-18T04:59:07.731809+0000","last_became_active":"2024-09-18T04:58:32.027104+0000","last_became_peered":"2024-09-18T04:58:32.027104+0000","last_unstale":"2024-09-18T04:59:07.731809+0000","last_undegraded":"2024-09-18T04:59:07.731809+0000","last_fullsized":"2024-09-18T04:59:07.731809+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:41:37.134684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056931099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5904,"num_read_kb":13840,"num_write":503,"num_write_kb":5042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":2187784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198027+0000","last_change":"2024-09-18T04:57:40.472681+0000","last_active":"2024-09-18T04:59:07.198027+0000","last_peered":"2024-09-18T04:59:07.198027+0000","last_clean":"2024-09-18T04:59:07.198027+0000","last_became_active":"2024-09-18T04:57:40.472217+0000","last_became_peered":"2024-09-18T04:57:40.472217+0000","last_unstale":"2024-09-18T04:59:07.198027+0000","last_undegraded":"2024-09-18T04:59:07.198027+0000","last_fullsized":"2024-09-18T04:59:07.198027+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:27:11.849311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"157'482","reported_seq":3615,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731740+0000","last_change":"2024-09-18T04:58:48.096157+0000","last_active":"2024-09-18T04:59:07.731740+0000","last_peered":"2024-09-18T04:59:07.731740+0000","last_clean":"2024-09-18T04:59:07.731740+0000","last_became_active":"2024-09-18T04:58:38.024916+0000","last_became_peered":"2024-09-18T04:58:38.024916+0000","last_unstale":"2024-09-18T04:59:07.731740+0000","last_undegraded":"2024-09-18T04:59:07.731740+0000","last_fullsized":"2024-09-18T04:59:07.731740+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:32:42.202542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001087188,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2758,"num_read_kb":7938,"num_write":473,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":458,"num_bytes_recovered":2219067,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"155'714","reported_seq":12009,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-18T04:59:06.191085+0000","last_change":"2024-09-18T04:58:48.024809+0000","last_active":"2024-09-18T04:59:06.191085+0000","last_peered":"2024-09-18T04:59:06.191085+0000","last_clean":"2024-09-18T04:59:06.191085+0000","last_became_active":"2024-09-18T04:58:26.101490+0000","last_became_peered":"2024-09-18T04:58:26.101490+0000","last_unstale":"2024-09-18T04:59:06.191085+0000","last_undegraded":"2024-09-18T04:59:06.191085+0000","last_fullsized":"2024-09-18T04:59:06.191085+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:30:15.538062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015101680000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":12956,"num_read_kb":26590,"num_write":848,"num_write_kb":15777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":5822953,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"157'470","reported_seq":6073,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195769+0000","last_change":"2024-09-18T04:58:48.136195+0000","last_active":"2024-09-18T04:59:07.195769+0000","last_peered":"2024-09-18T04:59:07.195769+0000","last_clean":"2024-09-18T04:59:07.195769+0000","last_became_active":"2024-09-18T04:58:27.115137+0000","last_became_peered":"2024-09-18T04:58:27.115137+0000","last_unstale":"2024-09-18T04:59:07.195769+0000","last_undegraded":"2024-09-18T04:59:07.195769+0000","last_fullsized":"2024-09-18T04:59:07.195769+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:57:42.359624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039325500000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":4725,"num_read_kb":14773,"num_write":478,"num_write_kb":8139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":6055124,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"155'1400","reported_seq":5772,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731780+0000","last_change":"2024-09-18T04:58:48.095758+0000","last_active":"2024-09-18T04:59:07.731780+0000","last_peered":"2024-09-18T04:59:07.731780+0000","last_clean":"2024-09-18T04:59:07.731780+0000","last_became_active":"2024-09-18T04:58:31.026041+0000","last_became_peered":"2024-09-18T04:58:31.026041+0000","last_unstale":"2024-09-18T04:59:07.731780+0000","last_undegraded":"2024-09-18T04:59:07.731780+0000","last_fullsized":"2024-09-18T04:59:07.731780+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":1400,"log_dups_size":0,"ondisk_log_size":1400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:55:29.526466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083671099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8851,"num_read_kb":15183,"num_write":2049,"num_write_kb":8862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":1966128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"155'908","reported_seq":7203,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195547+0000","last_change":"2024-09-18T04:58:48.023609+0000","last_active":"2024-09-18T04:59:07.195547+0000","last_peered":"2024-09-18T04:59:07.195547+0000","last_clean":"2024-09-18T04:59:07.195547+0000","last_became_active":"2024-09-18T04:58:33.027525+0000","last_became_peered":"2024-09-18T04:58:33.027525+0000","last_unstale":"2024-09-18T04:59:07.195547+0000","last_undegraded":"2024-09-18T04:59:07.195547+0000","last_fullsized":"2024-09-18T04:59:07.195547+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:27:34.074204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062770000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9983,"num_read_kb":24174,"num_write":928,"num_write_kb":8107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":86198,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"155'617","reported_seq":7056,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.194912+0000","last_change":"2024-09-18T04:58:48.021010+0000","last_active":"2024-09-18T04:59:07.194912+0000","last_peered":"2024-09-18T04:59:07.194912+0000","last_clean":"2024-09-18T04:59:07.194912+0000","last_became_active":"2024-09-18T04:58:07.017827+0000","last_became_peered":"2024-09-18T04:58:07.017827+0000","last_unstale":"2024-09-18T04:59:07.194912+0000","last_undegraded":"2024-09-18T04:59:07.194912+0000","last_fullsized":"2024-09-18T04:59:07.194912+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:42:31.897588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050117499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8566,"num_read_kb":18769,"num_write":595,"num_write_kb":12467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":2872264,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"155'1050","reported_seq":5261,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198243+0000","last_change":"2024-09-18T04:58:48.029466+0000","last_active":"2024-09-18T04:59:07.198243+0000","last_peered":"2024-09-18T04:59:07.198243+0000","last_clean":"2024-09-18T04:59:07.198243+0000","last_became_active":"2024-09-18T04:58:31.028189+0000","last_became_peered":"2024-09-18T04:58:31.028189+0000","last_unstale":"2024-09-18T04:59:07.198243+0000","last_undegraded":"2024-09-18T04:59:07.198243+0000","last_fullsized":"2024-09-18T04:59:07.198243+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":1050,"log_dups_size":0,"ondisk_log_size":1050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:26:10.430573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001035219,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7660,"num_read_kb":17955,"num_write":1663,"num_write_kb":5543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":2749742,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"157'556","reported_seq":5095,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198207+0000","last_change":"2024-09-18T04:58:48.029669+0000","last_active":"2024-09-18T04:59:07.198207+0000","last_peered":"2024-09-18T04:59:07.198207+0000","last_clean":"2024-09-18T04:59:07.198207+0000","last_became_active":"2024-09-18T04:58:29.029625+0000","last_became_peered":"2024-09-18T04:58:29.029625+0000","last_unstale":"2024-09-18T04:59:07.198207+0000","last_undegraded":"2024-09-18T04:59:07.198207+0000","last_fullsized":"2024-09-18T04:59:07.198207+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:51:09.257320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011607150000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4690,"num_read_kb":26470,"num_write":636,"num_write_kb":8108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":5504104,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"157'809","reported_seq":4779,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195411+0000","last_change":"2024-09-18T04:58:48.136237+0000","last_active":"2024-09-18T04:59:07.195411+0000","last_peered":"2024-09-18T04:59:07.195411+0000","last_clean":"2024-09-18T04:59:07.195411+0000","last_became_active":"2024-09-18T04:56:21.131202+0000","last_became_peered":"2024-09-18T04:56:21.131202+0000","last_unstale":"2024-09-18T04:59:07.195411+0000","last_undegraded":"2024-09-18T04:59:07.195411+0000","last_fullsized":"2024-09-18T04:59:07.195411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:17:35.438723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041769600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5877,"num_read_kb":14669,"num_write":993,"num_write_kb":10354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"157'920","reported_seq":8474,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.202504+0000","last_change":"2024-09-18T04:59:07.201833+0000","last_active":"2024-09-18T04:59:07.202504+0000","last_peered":"2024-09-18T04:59:07.202504+0000","last_clean":"2024-09-18T04:59:07.202504+0000","last_became_active":"2024-09-18T04:59:07.199844+0000","last_became_peered":"2024-09-18T04:59:07.199844+0000","last_unstale":"2024-09-18T04:59:07.202504+0000","last_undegraded":"2024-09-18T04:59:07.202504+0000","last_fullsized":"2024-09-18T04:59:07.202504+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":920,"log_dups_size":0,"ondisk_log_size":920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:40:08.684972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087268100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10766,"num_read_kb":19560,"num_write":908,"num_write_kb":18676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":9230703,"num_keys_recovered":59,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"144'809","reported_seq":5825,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.197043+0000","last_change":"2024-09-18T04:58:55.130930+0000","last_active":"2024-09-18T04:59:07.197043+0000","last_peered":"2024-09-18T04:59:07.197043+0000","last_clean":"2024-09-18T04:59:07.197043+0000","last_became_active":"2024-09-18T04:58:55.129007+0000","last_became_peered":"2024-09-18T04:58:55.129007+0000","last_unstale":"2024-09-18T04:59:07.197043+0000","last_undegraded":"2024-09-18T04:59:07.197043+0000","last_fullsized":"2024-09-18T04:59:07.197043+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:21:11.287190+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.00099438999999999994,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7388,"num_read_kb":33089,"num_write":1110,"num_write_kb":12855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"158'781","reported_seq":2932,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198095+0000","last_change":"2024-09-18T04:58:48.030265+0000","last_active":"2024-09-18T04:59:07.198095+0000","last_peered":"2024-09-18T04:59:07.198095+0000","last_clean":"2024-09-18T04:59:07.198095+0000","last_became_active":"2024-09-18T04:58:31.028633+0000","last_became_peered":"2024-09-18T04:58:31.028633+0000","last_unstale":"2024-09-18T04:59:07.198095+0000","last_undegraded":"2024-09-18T04:59:07.198095+0000","last_fullsized":"2024-09-18T04:59:07.198095+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:04:25.747259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00140489,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4341,"num_read_kb":14774,"num_write":882,"num_write_kb":9994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":12642724,"num_keys_recovered":71,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":47,"num_read":220784,"num_read_kb":683202,"num_write":29896,"num_write_kb":377778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9208,"num_bytes_recovered":184551668,"num_keys_recovered":562,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23964,"ondisk_log_size":23964,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28167,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51292,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320548,"statfs":{"total":96636764160,"available":96584241152,"internally_reserved":0,"allocated":2170880,"data_stored":1617954,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":50330059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.45600000000000002,"15min":0.46400000000000002},"min":{"1min":0.23799999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53100000000000003,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.437,"5min":0.41499999999999998,"15min":0.41099999999999998},"min":{"1min":0.312,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.60299999999999998}]},{"osd":1,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.46899999999999997,"15min":0.47799999999999998},"min":{"1min":0.249,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.55100000000000005,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.436,"15min":0.434},"min":{"1min":0.28399999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.621}]},{"osd":2,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.44700000000000001,"15min":0.44500000000000001},"min":{"1min":0.35399999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45200000000000001,"15min":0.45000000000000001},"min":{"1min":0.34200000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.56200000000000006,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.67500000000000004}]},{"osd":3,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52600000000000002,"15min":0.51900000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.4399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.34100000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.4809999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":0.72799999999999998}]},{"osd":4,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.42199999999999999,"15min":0.43099999999999999},"min":{"1min":0.26500000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.63700000000000001,"5min":1.954,"15min":1.954},"last":0.502},{"interface":"front","average":{"1min":0.375,"5min":0.41699999999999998,"15min":0.42399999999999999},"min":{"1min":0.246,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.629,"5min":1.478,"15min":1.478},"last":0.45800000000000002}]},{"osd":5,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.50800000000000001,"15min":0.52200000000000002},"min":{"1min":0.30299999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.71799999999999997,"5min":3.145,"15min":3.145},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.443,"15min":0.441},"min":{"1min":0.27700000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.65800000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.68999999999999995}]},{"osd":6,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.47899999999999998,"15min":0.48299999999999998},"min":{"1min":0.33300000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.69299999999999995,"5min":0.746,"15min":0.746},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.48899999999999999,"15min":0.49299999999999999},"min":{"1min":0.35199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.623,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.71399999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55336,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316504,"statfs":{"total":96636764160,"available":96580100096,"internally_reserved":0,"allocated":2121728,"data_stored":1032642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.71899999999999997,"15min":0.76900000000000002},"min":{"1min":0.23899999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.56799999999999995,"5min":6.1319999999999997,"15min":6.1319999999999997},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.75700000000000001,"15min":0.80500000000000005},"min":{"1min":0.35799999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.81100000000000005,"5min":6.1429999999999998,"15min":6.1429999999999998},"last":0.41099999999999998}]},{"osd":1,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.70999999999999996,"15min":0.76200000000000001},"min":{"1min":0.29299999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.44900000000000001,"5min":6.0410000000000004,"15min":6.0410000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.70899999999999996,"15min":0.753},"min":{"1min":0.33000000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.83599999999999997,"5min":5.5380000000000003,"15min":5.5380000000000003},"last":0.34599999999999997}]},{"osd":2,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.41999999999999998,"15min":0.42599999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.54400000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.315},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.46100000000000002,"15min":0.46800000000000003},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.55900000000000005,"5min":1.04,"15min":1.04},"last":0.33400000000000002}]},{"osd":3,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46000000000000002,"15min":0.46100000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.378},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.47799999999999998,"15min":0.48099999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.625,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.38700000000000001}]},{"osd":4,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.36799999999999999,"15min":0.374},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.47599999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.20799999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.43099999999999999,"15min":0.437},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.30399999999999999}]},{"osd":5,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.372,"15min":0.378},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.42499999999999999,"15min":0.435},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.53000000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.28699999999999998}]},{"osd":7,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.46800000000000003,"15min":0.46600000000000003},"min":{"1min":0.28100000000000003,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.48599999999999999,"15min":0.47699999999999998},"min":{"1min":0.34399999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.35599999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65188,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306652,"statfs":{"total":96636764160,"available":96570011648,"internally_reserved":0,"allocated":2121728,"data_stored":1033583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.54600000000000004,"15min":0.57099999999999995},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.59199999999999997,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.504,"15min":0.51000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71599999999999997,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.55700000000000005}]},{"osd":1,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.53300000000000003,"15min":0.55400000000000005},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.55600000000000005,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.32500000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63800000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.57499999999999996}]},{"osd":2,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47699999999999998,"15min":0.47899999999999998},"min":{"1min":0.28699999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.629,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.57399999999999995,"15min":0.59999999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.626,"5min":3.1070000000000002,"15min":3.1070000000000002},"last":0.504}]},{"osd":3,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.496,"15min":0.502},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.77000000000000002,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.504,"15min":0.50800000000000001},"min":{"1min":0.35099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.63,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.54600000000000004}]},{"osd":4,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.46800000000000003,"15min":0.47399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.59499999999999997,"5min":1.599,"15min":1.599},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.56299999999999994,"15min":0.59299999999999997},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.53500000000000003,"5min":3.379,"15min":3.379},"last":0.432}]},{"osd":6,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.45500000000000002,"15min":0.46600000000000003},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54200000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.56200000000000006,"15min":0.58499999999999996},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.085,"5min":3.1669999999999998,"15min":3.1669999999999998},"last":0.378}]},{"osd":7,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.48399999999999999,"15min":0.495},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80200000000000005,"5min":1.224,"15min":1.224},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.54200000000000004,"15min":0.55900000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.58599999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.45700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63512,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94308328,"statfs":{"total":96636764160,"available":96571727872,"internally_reserved":0,"allocated":2109440,"data_stored":1027567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.46000000000000002,"15min":0.44400000000000001},"min":{"1min":0.23499999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":3.7069999999999999,"5min":3.7069999999999999,"15min":3.7069999999999999},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.46100000000000002,"15min":0.441},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":3.3700000000000001,"5min":3.3700000000000001,"15min":3.3700000000000001},"last":0.219}]},{"osd":1,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.47099999999999997,"15min":0.45500000000000002},"min":{"1min":0.28199999999999997,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":3.2879999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.375},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.46600000000000003,"15min":0.44600000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.661,"5min":3.661,"15min":3.661},"last":0.38200000000000001}]},{"osd":2,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.52200000000000002,"15min":0.50600000000000001},"min":{"1min":0.29999999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.1890000000000001,"5min":3.1890000000000001,"15min":3.1890000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.51600000000000001,"15min":0.502},"min":{"1min":0.32300000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.3860000000000001,"5min":3.3860000000000001,"15min":3.3860000000000001},"last":0.35899999999999999}]},{"osd":3,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.504,"15min":0.48899999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":3.4500000000000002,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.48299999999999998,"15min":0.46100000000000002},"min":{"1min":0.372,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.3210000000000002,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.433}]},{"osd":5,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.42599999999999999,"15min":0.40200000000000002},"min":{"1min":0.25,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":3.9319999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.43099999999999999,"15min":0.40699999999999997},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":3.9449999999999998,"5min":3.9449999999999998,"15min":3.9449999999999998},"last":0.39700000000000002}]},{"osd":6,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.45100000000000001,"15min":0.42899999999999999},"min":{"1min":0.29299999999999998,"5min":0.251,"15min":0.251},"max":{"1min":3.3530000000000002,"5min":3.3530000000000002,"15min":3.3530000000000002},"last":0.312},{"interface":"front","average":{"1min":1.29,"5min":0.626,"15min":0.51600000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":17.422999999999998,"5min":17.422999999999998,"15min":17.422999999999998},"last":0.33500000000000002}]},{"osd":7,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.48399999999999999,"15min":0.46400000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":3.7330000000000001,"5min":3.7330000000000001,"15min":3.7330000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.52100000000000002,"15min":0.499},"min":{"1min":0.33300000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":3.6930000000000001,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":0.40500000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87320,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284520,"statfs":{"total":96636764160,"available":96547348480,"internally_reserved":0,"allocated":2109440,"data_stored":1027567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":87292360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.39500000000000002,"15min":0.40500000000000003},"min":{"1min":0.23899999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.45200000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.51800000000000002,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.38400000000000001}]},{"osd":1,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.376,"15min":0.375},"min":{"1min":0.249,"5min":0.245,"15min":0.245},"max":{"1min":0.51800000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.37},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.40500000000000003,"15min":0.41499999999999998},"min":{"1min":0.249,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.54400000000000004,"5min":0.755,"15min":0.755},"last":0.438}]},{"osd":2,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.41599999999999998,"15min":0.41999999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.56999999999999995,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.45200000000000001,"15min":0.45300000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.59399999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.47299999999999998}]},{"osd":4,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.52000000000000002,"15min":0.52900000000000003},"min":{"1min":0.31,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71999999999999997,"5min":0.755,"15min":0.755},"last":0.505},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.76100000000000001,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.63900000000000001}]},{"osd":5,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.55900000000000005,"15min":0.56899999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68600000000000005,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.66900000000000004,"5min":0.874,"15min":0.874},"last":0.60399999999999998}]},{"osd":6,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58699999999999997,"15min":0.59599999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80400000000000005,"5min":0.874,"15min":0.874},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72399999999999998,"5min":1.095,"15min":1.095},"last":0.51600000000000001}]},{"osd":7,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.65600000000000003,"15min":0.67200000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84499999999999997,"5min":1.335,"15min":1.335},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.61599999999999999,"15min":0.624},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.753,"5min":1.087,"15min":1.087},"last":0.58999999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37292,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":2121728,"data_stored":1033583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.33700000000000002,"15min":0.33600000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.39600000000000002,"15min":0.39800000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.63800000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.35899999999999999}]},{"osd":1,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.56999999999999995,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.36299999999999999,"15min":0.36099999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.29099999999999998}]},{"osd":3,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.40799999999999997,"15min":0.40400000000000003},"min":{"1min":0.26200000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.40999999999999998,"15min":0.40400000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.373}]},{"osd":4,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.58899999999999997,"15min":0.57899999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.4220000000000002,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":0.59999999999999998}]},{"osd":5,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.61199999999999999,"15min":0.56399999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":7.4480000000000004,"5min":7.4480000000000004,"15min":7.4480000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54800000000000004,"15min":0.54700000000000004},"min":{"1min":0.377,"5min":0.375,"15min":0.375},"max":{"1min":0.73699999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.64800000000000002}]},{"osd":6,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.84599999999999997,"5min":1.171,"15min":1.171},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.57799999999999996}]},{"osd":7,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80800000000000005,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":1.1719999999999999,"5min":0.69999999999999996,"15min":0.622},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":11.138,"5min":11.138,"15min":11.138},"last":0.61399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29124,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342716,"statfs":{"total":96636764160,"available":96606941184,"internally_reserved":0,"allocated":2154496,"data_stored":1617954,"data_compressed":9734,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.34999999999999998,"15min":0.35799999999999998},"min":{"1min":0.20899999999999999,"5min":0.185,"15min":0.185},"max":{"1min":0.622,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.63100000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":1.6579999999999999}]},{"osd":2,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.371,"15min":0.373},"min":{"1min":0.222,"5min":0.221,"15min":0.221},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":1.7270000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41599999999999998,"15min":0.41399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.6839999999999999}]},{"osd":3,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.38800000000000001,"15min":0.38400000000000001},"min":{"1min":0.23899999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":1.754},{"interface":"front","average":{"1min":0.434,"5min":0.40600000000000003,"15min":0.40100000000000002},"min":{"1min":0.29599999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":1.71}]},{"osd":4,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.50800000000000001,"15min":0.51700000000000002},"min":{"1min":0.32400000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.77000000000000002,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.496,"5min":0.50700000000000001,"15min":0.50900000000000001},"min":{"1min":0.315,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.86099999999999999,"5min":0.999,"15min":0.999},"last":0.51500000000000001}]},{"osd":5,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50900000000000001,"15min":0.51300000000000001},"min":{"1min":0.33600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80400000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":1.625},{"interface":"front","average":{"1min":0.5,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.373,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.83899999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":1.597}]},{"osd":6,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82799999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.74299999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67400000000000004}]},{"osd":7,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.56999999999999995,"15min":0.57799999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78200000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.40999999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0309999999999999,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.70299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51228,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320612,"statfs":{"total":96636764160,"available":96584306688,"internally_reserved":0,"allocated":2109440,"data_stored":1026626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50330055},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35999999999999999,"15min":0.36299999999999999},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.56799999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.247},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.34799999999999998,"15min":0.34599999999999997},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.53700000000000003,"5min":0.621,"15min":0.621},"last":0.28399999999999997}]},{"osd":2,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.39900000000000002,"15min":0.40100000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.42599999999999999,"15min":0.42399999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.46200000000000002}]},{"osd":3,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.434,"15min":0.42399999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.40400000000000003,"15min":0.39700000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.5}]},{"osd":4,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.63400000000000001,"15min":0.65600000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.89400000000000002,"5min":3.4710000000000001,"15min":3.4710000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.504,"15min":0.502},"min":{"1min":0.35899999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.376}]},{"osd":5,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.81699999999999995,"15min":0.84099999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":4.2640000000000002,"5min":4.2640000000000002,"15min":4.2640000000000002},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.54700000000000004}]},{"osd":6,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.66000000000000003,"15min":0.68000000000000005},"min":{"1min":0.39500000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81699999999999995,"5min":3.536,"15min":3.536},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57099999999999995,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.57999999999999996}]},{"osd":7,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55200000000000005,"15min":0.54700000000000004},"min":{"1min":0.41999999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.68400000000000005,"15min":0.70599999999999996},"min":{"1min":0.39100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.85699999999999998,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.48099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9734,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9734,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T04:59:12.810 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T04:59:13.098 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:59:13.099 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-18T04:59:13.110 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2024-09-18T04:59:12.299589+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":47,"num_read":220900,"num_read_kb":683302,"num_write":30057,"num_write_kb":380196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9215,"num_bytes_recovered":185732404,"num_keys_recovered":562,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24084,"ondisk_log_size":24084,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":440292,"kb_used_data":16620,"kb_used_omap":12,"kb_used_meta":423539,"kb_avail":754534428,"statfs":{"total":773094113280,"available":772643254272,"internally_reserved":0,"allocated":17018880,"data_stored":9417476,"data_compressed":18433,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":12742,"internal_metadata":433704506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":524,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.110496"},"pg_stats":[{"pgid":"3.1f","version":"158'766","reported_seq":5157,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.197217+0000","last_change":"2024-09-18T04:58:55.128107+0000","last_active":"2024-09-18T04:59:07.197217+0000","last_peered":"2024-09-18T04:59:07.197217+0000","last_clean":"2024-09-18T04:59:07.197217+0000","last_became_active":"2024-09-18T04:58:55.126111+0000","last_became_peered":"2024-09-18T04:58:55.126111+0000","last_unstale":"2024-09-18T04:59:07.197217+0000","last_undegraded":"2024-09-18T04:59:07.197217+0000","last_fullsized":"2024-09-18T04:59:07.197217+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:56:05.136229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001156916,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6348,"num_read_kb":20852,"num_write":870,"num_write_kb":9417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"157'817","reported_seq":2783,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731844+0000","last_change":"2024-09-18T04:58:48.096250+0000","last_active":"2024-09-18T04:59:07.731844+0000","last_peered":"2024-09-18T04:59:07.731844+0000","last_clean":"2024-09-18T04:59:07.731844+0000","last_became_active":"2024-09-18T04:58:32.029310+0000","last_became_peered":"2024-09-18T04:58:32.029310+0000","last_unstale":"2024-09-18T04:59:07.731844+0000","last_undegraded":"2024-09-18T04:59:07.731844+0000","last_fullsized":"2024-09-18T04:59:07.731844+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:13:02.688292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001168749,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3351,"num_read_kb":18023,"num_write":1120,"num_write_kb":8161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":3252478,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"156'500","reported_seq":10759,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195673+0000","last_change":"2024-09-18T04:58:48.023738+0000","last_active":"2024-09-18T04:59:07.195673+0000","last_peered":"2024-09-18T04:59:07.195673+0000","last_clean":"2024-09-18T04:59:07.195673+0000","last_became_active":"2024-09-18T04:58:27.115104+0000","last_became_peered":"2024-09-18T04:58:27.115104+0000","last_unstale":"2024-09-18T04:59:07.195673+0000","last_undegraded":"2024-09-18T04:59:07.195673+0000","last_fullsized":"2024-09-18T04:59:07.195673+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:33:08.572507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082741200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9761,"num_read_kb":28168,"num_write":548,"num_write_kb":23097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7293282,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"158'840","reported_seq":7984,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195584+0000","last_change":"2024-09-18T04:58:48.025083+0000","last_active":"2024-09-18T04:59:07.195584+0000","last_peered":"2024-09-18T04:59:07.195584+0000","last_clean":"2024-09-18T04:59:07.195584+0000","last_became_active":"2024-09-18T04:58:28.028905+0000","last_became_peered":"2024-09-18T04:58:28.028905+0000","last_unstale":"2024-09-18T04:59:07.195584+0000","last_undegraded":"2024-09-18T04:59:07.195584+0000","last_fullsized":"2024-09-18T04:59:07.195584+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:15:17.569793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014698000000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":8594,"num_read_kb":9084,"num_write":1169,"num_write_kb":5309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":1412856,"num_keys_recovered":106,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"155'473","reported_seq":4267,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.202696+0000","last_change":"2024-09-18T04:59:07.202189+0000","last_active":"2024-09-18T04:59:07.202696+0000","last_peered":"2024-09-18T04:59:07.202696+0000","last_clean":"2024-09-18T04:59:07.202696+0000","last_became_active":"2024-09-18T04:59:07.199588+0000","last_became_peered":"2024-09-18T04:59:07.199588+0000","last_unstale":"2024-09-18T04:59:07.202696+0000","last_undegraded":"2024-09-18T04:59:07.202696+0000","last_fullsized":"2024-09-18T04:59:07.202696+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:53:25.971943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012312829999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2677,"num_read_kb":15514,"num_write":503,"num_write_kb":9112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":912,"num_bytes_recovered":7005168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"157'455","reported_seq":3935,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:08.199477+0000","last_change":"2024-09-18T04:59:07.200974+0000","last_active":"2024-09-18T04:59:08.199477+0000","last_peered":"2024-09-18T04:59:08.199477+0000","last_clean":"2024-09-18T04:59:08.199477+0000","last_became_active":"2024-09-18T04:59:07.199276+0000","last_became_peered":"2024-09-18T04:59:07.199276+0000","last_unstale":"2024-09-18T04:59:08.199477+0000","last_undegraded":"2024-09-18T04:59:08.199477+0000","last_fullsized":"2024-09-18T04:59:08.199477+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:42:10.766349+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.000757346,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":2981,"num_read_kb":11740,"num_write":512,"num_write_kb":18285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":519,"num_bytes_recovered":11494100,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"156'520","reported_seq":5473,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198149+0000","last_change":"2024-09-18T04:58:57.148756+0000","last_active":"2024-09-18T04:59:07.198149+0000","last_peered":"2024-09-18T04:59:07.198149+0000","last_clean":"2024-09-18T04:59:07.198149+0000","last_became_active":"2024-09-18T04:58:57.145900+0000","last_became_peered":"2024-09-18T04:58:57.145900+0000","last_unstale":"2024-09-18T04:59:07.198149+0000","last_undegraded":"2024-09-18T04:59:07.198149+0000","last_fullsized":"2024-09-18T04:59:07.198149+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:03:40.179686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018405870000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4543,"num_read_kb":26564,"num_write":538,"num_write_kb":13069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":23811007,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"156'576","reported_seq":3426,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731747+0000","last_change":"2024-09-18T04:58:48.095394+0000","last_active":"2024-09-18T04:59:07.731747+0000","last_peered":"2024-09-18T04:59:07.731747+0000","last_clean":"2024-09-18T04:59:07.731747+0000","last_became_active":"2024-09-18T04:58:32.029074+0000","last_became_peered":"2024-09-18T04:58:32.029074+0000","last_unstale":"2024-09-18T04:59:07.731747+0000","last_undegraded":"2024-09-18T04:59:07.731747+0000","last_fullsized":"2024-09-18T04:59:07.731747+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:15:38.950060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053623899999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3403,"num_read_kb":16958,"num_write":699,"num_write_kb":12986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":4650417,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"153'506","reported_seq":4239,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.197394+0000","last_change":"2024-09-18T04:58:55.128037+0000","last_active":"2024-09-18T04:59:07.197394+0000","last_peered":"2024-09-18T04:59:07.197394+0000","last_clean":"2024-09-18T04:59:07.197394+0000","last_became_active":"2024-09-18T04:58:55.125721+0000","last_became_peered":"2024-09-18T04:58:55.125721+0000","last_unstale":"2024-09-18T04:59:07.197394+0000","last_undegraded":"2024-09-18T04:59:07.197394+0000","last_fullsized":"2024-09-18T04:59:07.197394+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:06:23.656568+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001441276,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4899,"num_read_kb":13699,"num_write":541,"num_write_kb":6077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"154'885","reported_seq":5042,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195463+0000","last_change":"2024-09-18T04:58:48.024658+0000","last_active":"2024-09-18T04:59:07.195463+0000","last_peered":"2024-09-18T04:59:07.195463+0000","last_clean":"2024-09-18T04:59:07.195463+0000","last_became_active":"2024-09-18T04:56:21.131133+0000","last_became_peered":"2024-09-18T04:56:21.131133+0000","last_unstale":"2024-09-18T04:59:07.195463+0000","last_undegraded":"2024-09-18T04:59:07.195463+0000","last_fullsized":"2024-09-18T04:59:07.195463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:01:57.036904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012610480000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6340,"num_read_kb":26579,"num_write":1141,"num_write_kb":12621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"157'637","reported_seq":5292,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198169+0000","last_change":"2024-09-18T04:58:48.028934+0000","last_active":"2024-09-18T04:59:07.198169+0000","last_peered":"2024-09-18T04:59:07.198169+0000","last_clean":"2024-09-18T04:59:07.198169+0000","last_became_active":"2024-09-18T04:58:32.032762+0000","last_became_peered":"2024-09-18T04:58:32.032762+0000","last_unstale":"2024-09-18T04:59:07.198169+0000","last_undegraded":"2024-09-18T04:59:07.198169+0000","last_fullsized":"2024-09-18T04:59:07.198169+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:33:25.010425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066811099999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":5292,"num_read_kb":26013,"num_write":596,"num_write_kb":7300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":447,"num_bytes_recovered":1594801,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"155'502","reported_seq":2817,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-18T04:59:06.190587+0000","last_change":"2024-09-18T04:58:48.023734+0000","last_active":"2024-09-18T04:59:06.190587+0000","last_peered":"2024-09-18T04:59:06.190587+0000","last_clean":"2024-09-18T04:59:06.190587+0000","last_became_active":"2024-09-18T04:58:30.025558+0000","last_became_peered":"2024-09-18T04:58:30.025558+0000","last_unstale":"2024-09-18T04:59:06.190587+0000","last_undegraded":"2024-09-18T04:59:06.190587+0000","last_fullsized":"2024-09-18T04:59:06.190587+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:57:31.113016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087474099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3131,"num_read_kb":27325,"num_write":609,"num_write_kb":11672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":12427258,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":192,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195662+0000","last_change":"2024-09-18T04:58:29.025271+0000","last_active":"2024-09-18T04:59:07.195662+0000","last_peered":"2024-09-18T04:59:07.195662+0000","last_clean":"2024-09-18T04:59:07.195662+0000","last_became_active":"2024-09-18T04:58:29.024911+0000","last_became_peered":"2024-09-18T04:58:29.024911+0000","last_unstale":"2024-09-18T04:59:07.195662+0000","last_undegraded":"2024-09-18T04:59:07.195662+0000","last_fullsized":"2024-09-18T04:59:07.195662+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:53:21.863837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063436799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"155'2480","reported_seq":9899,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195724+0000","last_change":"2024-09-18T04:58:48.024643+0000","last_active":"2024-09-18T04:59:07.195724+0000","last_peered":"2024-09-18T04:59:07.195724+0000","last_clean":"2024-09-18T04:59:07.195724+0000","last_became_active":"2024-09-18T04:58:33.027340+0000","last_became_peered":"2024-09-18T04:58:33.027340+0000","last_unstale":"2024-09-18T04:59:07.195724+0000","last_undegraded":"2024-09-18T04:59:07.195724+0000","last_fullsized":"2024-09-18T04:59:07.195724+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":2480,"log_dups_size":0,"ondisk_log_size":2480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:47:47.826132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000952045,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":18529,"num_read_kb":32990,"num_write":4379,"num_write_kb":20206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":5439803,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":176,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195585+0000","last_change":"2024-09-18T04:56:12.389971+0000","last_active":"2024-09-18T04:59:07.195585+0000","last_peered":"2024-09-18T04:59:07.195585+0000","last_clean":"2024-09-18T04:59:07.195585+0000","last_became_active":"2024-09-18T04:56:10.412595+0000","last_became_peered":"2024-09-18T04:56:10.412595+0000","last_unstale":"2024-09-18T04:59:07.195585+0000","last_undegraded":"2024-09-18T04:59:07.195585+0000","last_fullsized":"2024-09-18T04:59:07.195585+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:45.638420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038576799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"157'807","reported_seq":7861,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198219+0000","last_change":"2024-09-18T04:58:57.151001+0000","last_active":"2024-09-18T04:59:07.198219+0000","last_peered":"2024-09-18T04:59:07.198219+0000","last_clean":"2024-09-18T04:59:07.198219+0000","last_became_active":"2024-09-18T04:58:57.147959+0000","last_became_peered":"2024-09-18T04:58:57.147959+0000","last_unstale":"2024-09-18T04:59:07.198219+0000","last_undegraded":"2024-09-18T04:59:07.198219+0000","last_fullsized":"2024-09-18T04:59:07.198219+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:30:19.725543+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00065254400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8940,"num_read_kb":28417,"num_write":1051,"num_write_kb":11406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":5706670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198307+0000","last_change":"2024-09-18T04:58:57.150288+0000","last_active":"2024-09-18T04:59:07.198307+0000","last_peered":"2024-09-18T04:59:07.198307+0000","last_clean":"2024-09-18T04:59:07.198307+0000","last_became_active":"2024-09-18T04:58:57.149907+0000","last_became_peered":"2024-09-18T04:58:57.149907+0000","last_unstale":"2024-09-18T04:59:07.198307+0000","last_undegraded":"2024-09-18T04:59:07.198307+0000","last_fullsized":"2024-09-18T04:59:07.198307+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:12:39.499969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"157'757","reported_seq":6181,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731891+0000","last_change":"2024-09-18T04:58:48.096507+0000","last_active":"2024-09-18T04:59:07.731891+0000","last_peered":"2024-09-18T04:59:07.731891+0000","last_clean":"2024-09-18T04:59:07.731891+0000","last_became_active":"2024-09-18T04:58:33.026806+0000","last_became_peered":"2024-09-18T04:58:33.026806+0000","last_unstale":"2024-09-18T04:59:07.731891+0000","last_undegraded":"2024-09-18T04:59:07.731891+0000","last_fullsized":"2024-09-18T04:59:07.731891+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:59:41.673907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013736829999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6896,"num_read_kb":21261,"num_write":943,"num_write_kb":5208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":571544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":145,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195438+0000","last_change":"2024-09-18T04:56:46.254241+0000","last_active":"2024-09-18T04:59:07.195438+0000","last_peered":"2024-09-18T04:59:07.195438+0000","last_clean":"2024-09-18T04:59:07.195438+0000","last_became_active":"2024-09-18T04:56:46.253894+0000","last_became_peered":"2024-09-18T04:56:46.253894+0000","last_unstale":"2024-09-18T04:59:07.195438+0000","last_undegraded":"2024-09-18T04:59:07.195438+0000","last_fullsized":"2024-09-18T04:59:07.195438+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:58:42.098099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045144100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"158'505","reported_seq":5028,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.197318+0000","last_change":"2024-09-18T04:58:48.061503+0000","last_active":"2024-09-18T04:59:07.197318+0000","last_peered":"2024-09-18T04:59:07.197318+0000","last_clean":"2024-09-18T04:59:07.197318+0000","last_became_active":"2024-09-18T04:58:41.078435+0000","last_became_peered":"2024-09-18T04:58:41.078435+0000","last_unstale":"2024-09-18T04:59:07.197318+0000","last_undegraded":"2024-09-18T04:59:07.197318+0000","last_fullsized":"2024-09-18T04:59:07.197318+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:34:50.214360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056043399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4221,"num_read_kb":33044,"num_write":472,"num_write_kb":37350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":25349011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"155'485","reported_seq":9536,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.201259+0000","last_change":"2024-09-18T04:59:07.200564+0000","last_active":"2024-09-18T04:59:07.201259+0000","last_peered":"2024-09-18T04:59:07.201259+0000","last_clean":"2024-09-18T04:59:07.201259+0000","last_became_active":"2024-09-18T04:59:07.199056+0000","last_became_peered":"2024-09-18T04:59:07.199056+0000","last_unstale":"2024-09-18T04:59:07.201259+0000","last_undegraded":"2024-09-18T04:59:07.201259+0000","last_fullsized":"2024-09-18T04:59:07.201259+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:30:30.980376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049796699999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":9360,"num_read_kb":28051,"num_write":499,"num_write_kb":12488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":472,"num_bytes_recovered":11362815,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":362,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198082+0000","last_change":"2024-09-18T04:58:38.025236+0000","last_active":"2024-09-18T04:59:07.198082+0000","last_peered":"2024-09-18T04:59:07.198082+0000","last_clean":"2024-09-18T04:59:07.198082+0000","last_became_active":"2024-09-18T04:58:38.024740+0000","last_became_peered":"2024-09-18T04:58:38.024740+0000","last_unstale":"2024-09-18T04:59:07.198082+0000","last_undegraded":"2024-09-18T04:59:07.198082+0000","last_fullsized":"2024-09-18T04:59:07.198082+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_clean_scrub_stamp":"2024-09-18T04:56:10.373785+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:10:45.251654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":37,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198355+0000","last_change":"2024-09-18T04:58:57.148289+0000","last_active":"2024-09-18T04:59:07.198355+0000","last_peered":"2024-09-18T04:59:07.198355+0000","last_clean":"2024-09-18T04:59:07.198355+0000","last_became_active":"2024-09-18T04:58:57.147915+0000","last_became_peered":"2024-09-18T04:58:57.147915+0000","last_unstale":"2024-09-18T04:59:07.198355+0000","last_undegraded":"2024-09-18T04:59:07.198355+0000","last_fullsized":"2024-09-18T04:59:07.198355+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:31:21.699401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"157'811","reported_seq":13428,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198153+0000","last_change":"2024-09-18T04:58:57.152156+0000","last_active":"2024-09-18T04:59:07.198153+0000","last_peered":"2024-09-18T04:59:07.198153+0000","last_clean":"2024-09-18T04:59:07.198153+0000","last_became_active":"2024-09-18T04:58:57.148221+0000","last_became_peered":"2024-09-18T04:58:57.148221+0000","last_unstale":"2024-09-18T04:59:07.198153+0000","last_undegraded":"2024-09-18T04:59:07.198153+0000","last_fullsized":"2024-09-18T04:59:07.198153+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:45:57.434891+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0033124909999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":14837,"num_read_kb":27169,"num_write":1047,"num_write_kb":7785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":1280827,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":178,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198076+0000","last_change":"2024-09-18T04:56:12.398631+0000","last_active":"2024-09-18T04:59:07.198076+0000","last_peered":"2024-09-18T04:59:07.198076+0000","last_clean":"2024-09-18T04:59:07.198076+0000","last_became_active":"2024-09-18T04:56:10.412614+0000","last_became_peered":"2024-09-18T04:56:10.412614+0000","last_unstale":"2024-09-18T04:59:07.198076+0000","last_undegraded":"2024-09-18T04:59:07.198076+0000","last_fullsized":"2024-09-18T04:59:07.198076+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:44:13.401320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075126299999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"158'601","reported_seq":2977,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-18T04:59:06.191350+0000","last_change":"2024-09-18T04:58:48.024976+0000","last_active":"2024-09-18T04:59:06.191350+0000","last_peered":"2024-09-18T04:59:06.191350+0000","last_clean":"2024-09-18T04:59:06.191350+0000","last_became_active":"2024-09-18T04:58:30.027757+0000","last_became_peered":"2024-09-18T04:58:30.027757+0000","last_unstale":"2024-09-18T04:59:06.191350+0000","last_undegraded":"2024-09-18T04:59:06.191350+0000","last_fullsized":"2024-09-18T04:59:06.191350+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:29:00.106938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014951890000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":2216,"num_read_kb":23967,"num_write":593,"num_write_kb":16097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":10562840,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":176,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195410+0000","last_change":"2024-09-18T04:56:12.390044+0000","last_active":"2024-09-18T04:59:07.195410+0000","last_peered":"2024-09-18T04:59:07.195410+0000","last_clean":"2024-09-18T04:59:07.195410+0000","last_became_active":"2024-09-18T04:56:10.413042+0000","last_became_peered":"2024-09-18T04:56:10.413042+0000","last_unstale":"2024-09-18T04:59:07.195410+0000","last_undegraded":"2024-09-18T04:59:07.195410+0000","last_fullsized":"2024-09-18T04:59:07.195410+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:46:11.074243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"157'525","reported_seq":4818,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731809+0000","last_change":"2024-09-18T04:58:48.095298+0000","last_active":"2024-09-18T04:59:07.731809+0000","last_peered":"2024-09-18T04:59:07.731809+0000","last_clean":"2024-09-18T04:59:07.731809+0000","last_became_active":"2024-09-18T04:58:32.027104+0000","last_became_peered":"2024-09-18T04:58:32.027104+0000","last_unstale":"2024-09-18T04:59:07.731809+0000","last_undegraded":"2024-09-18T04:59:07.731809+0000","last_fullsized":"2024-09-18T04:59:07.731809+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:41:37.134684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056931099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5904,"num_read_kb":13840,"num_write":503,"num_write_kb":5042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":2187784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198027+0000","last_change":"2024-09-18T04:57:40.472681+0000","last_active":"2024-09-18T04:59:07.198027+0000","last_peered":"2024-09-18T04:59:07.198027+0000","last_clean":"2024-09-18T04:59:07.198027+0000","last_became_active":"2024-09-18T04:57:40.472217+0000","last_became_peered":"2024-09-18T04:57:40.472217+0000","last_unstale":"2024-09-18T04:59:07.198027+0000","last_undegraded":"2024-09-18T04:59:07.198027+0000","last_fullsized":"2024-09-18T04:59:07.198027+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:27:11.849311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"157'482","reported_seq":3615,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731740+0000","last_change":"2024-09-18T04:58:48.096157+0000","last_active":"2024-09-18T04:59:07.731740+0000","last_peered":"2024-09-18T04:59:07.731740+0000","last_clean":"2024-09-18T04:59:07.731740+0000","last_became_active":"2024-09-18T04:58:38.024916+0000","last_became_peered":"2024-09-18T04:58:38.024916+0000","last_unstale":"2024-09-18T04:59:07.731740+0000","last_undegraded":"2024-09-18T04:59:07.731740+0000","last_fullsized":"2024-09-18T04:59:07.731740+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:32:42.202542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001087188,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2758,"num_read_kb":7938,"num_write":473,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":458,"num_bytes_recovered":2219067,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"155'714","reported_seq":12009,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-18T04:59:06.191085+0000","last_change":"2024-09-18T04:58:48.024809+0000","last_active":"2024-09-18T04:59:06.191085+0000","last_peered":"2024-09-18T04:59:06.191085+0000","last_clean":"2024-09-18T04:59:06.191085+0000","last_became_active":"2024-09-18T04:58:26.101490+0000","last_became_peered":"2024-09-18T04:58:26.101490+0000","last_unstale":"2024-09-18T04:59:06.191085+0000","last_undegraded":"2024-09-18T04:59:06.191085+0000","last_fullsized":"2024-09-18T04:59:06.191085+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:30:15.538062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015101680000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":12956,"num_read_kb":26590,"num_write":848,"num_write_kb":15777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":5822953,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"157'470","reported_seq":6073,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195769+0000","last_change":"2024-09-18T04:58:48.136195+0000","last_active":"2024-09-18T04:59:07.195769+0000","last_peered":"2024-09-18T04:59:07.195769+0000","last_clean":"2024-09-18T04:59:07.195769+0000","last_became_active":"2024-09-18T04:58:27.115137+0000","last_became_peered":"2024-09-18T04:58:27.115137+0000","last_unstale":"2024-09-18T04:59:07.195769+0000","last_undegraded":"2024-09-18T04:59:07.195769+0000","last_fullsized":"2024-09-18T04:59:07.195769+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:57:42.359624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039325500000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":4725,"num_read_kb":14773,"num_write":478,"num_write_kb":8139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":6055124,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"155'1400","reported_seq":5772,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731780+0000","last_change":"2024-09-18T04:58:48.095758+0000","last_active":"2024-09-18T04:59:07.731780+0000","last_peered":"2024-09-18T04:59:07.731780+0000","last_clean":"2024-09-18T04:59:07.731780+0000","last_became_active":"2024-09-18T04:58:31.026041+0000","last_became_peered":"2024-09-18T04:58:31.026041+0000","last_unstale":"2024-09-18T04:59:07.731780+0000","last_undegraded":"2024-09-18T04:59:07.731780+0000","last_fullsized":"2024-09-18T04:59:07.731780+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":1400,"log_dups_size":0,"ondisk_log_size":1400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:55:29.526466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083671099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8851,"num_read_kb":15183,"num_write":2049,"num_write_kb":8862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":1966128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"155'908","reported_seq":7203,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195547+0000","last_change":"2024-09-18T04:58:48.023609+0000","last_active":"2024-09-18T04:59:07.195547+0000","last_peered":"2024-09-18T04:59:07.195547+0000","last_clean":"2024-09-18T04:59:07.195547+0000","last_became_active":"2024-09-18T04:58:33.027525+0000","last_became_peered":"2024-09-18T04:58:33.027525+0000","last_unstale":"2024-09-18T04:59:07.195547+0000","last_undegraded":"2024-09-18T04:59:07.195547+0000","last_fullsized":"2024-09-18T04:59:07.195547+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:27:34.074204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062770000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9983,"num_read_kb":24174,"num_write":928,"num_write_kb":8107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":86198,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"155'617","reported_seq":7056,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.194912+0000","last_change":"2024-09-18T04:58:48.021010+0000","last_active":"2024-09-18T04:59:07.194912+0000","last_peered":"2024-09-18T04:59:07.194912+0000","last_clean":"2024-09-18T04:59:07.194912+0000","last_became_active":"2024-09-18T04:58:07.017827+0000","last_became_peered":"2024-09-18T04:58:07.017827+0000","last_unstale":"2024-09-18T04:59:07.194912+0000","last_undegraded":"2024-09-18T04:59:07.194912+0000","last_fullsized":"2024-09-18T04:59:07.194912+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:42:31.897588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050117499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8566,"num_read_kb":18769,"num_write":595,"num_write_kb":12467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":2872264,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"155'1050","reported_seq":5261,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198243+0000","last_change":"2024-09-18T04:58:48.029466+0000","last_active":"2024-09-18T04:59:07.198243+0000","last_peered":"2024-09-18T04:59:07.198243+0000","last_clean":"2024-09-18T04:59:07.198243+0000","last_became_active":"2024-09-18T04:58:31.028189+0000","last_became_peered":"2024-09-18T04:58:31.028189+0000","last_unstale":"2024-09-18T04:59:07.198243+0000","last_undegraded":"2024-09-18T04:59:07.198243+0000","last_fullsized":"2024-09-18T04:59:07.198243+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":1050,"log_dups_size":0,"ondisk_log_size":1050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:26:10.430573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001035219,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7660,"num_read_kb":17955,"num_write":1663,"num_write_kb":5543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":2749742,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"157'556","reported_seq":5095,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198207+0000","last_change":"2024-09-18T04:58:48.029669+0000","last_active":"2024-09-18T04:59:07.198207+0000","last_peered":"2024-09-18T04:59:07.198207+0000","last_clean":"2024-09-18T04:59:07.198207+0000","last_became_active":"2024-09-18T04:58:29.029625+0000","last_became_peered":"2024-09-18T04:58:29.029625+0000","last_unstale":"2024-09-18T04:59:07.198207+0000","last_undegraded":"2024-09-18T04:59:07.198207+0000","last_fullsized":"2024-09-18T04:59:07.198207+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:51:09.257320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011607150000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4690,"num_read_kb":26470,"num_write":636,"num_write_kb":8108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":5504104,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"157'809","reported_seq":4779,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195411+0000","last_change":"2024-09-18T04:58:48.136237+0000","last_active":"2024-09-18T04:59:07.195411+0000","last_peered":"2024-09-18T04:59:07.195411+0000","last_clean":"2024-09-18T04:59:07.195411+0000","last_became_active":"2024-09-18T04:56:21.131202+0000","last_became_peered":"2024-09-18T04:56:21.131202+0000","last_unstale":"2024-09-18T04:59:07.195411+0000","last_undegraded":"2024-09-18T04:59:07.195411+0000","last_fullsized":"2024-09-18T04:59:07.195411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:17:35.438723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041769600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5877,"num_read_kb":14669,"num_write":993,"num_write_kb":10354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"157'920","reported_seq":8474,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.202504+0000","last_change":"2024-09-18T04:59:07.201833+0000","last_active":"2024-09-18T04:59:07.202504+0000","last_peered":"2024-09-18T04:59:07.202504+0000","last_clean":"2024-09-18T04:59:07.202504+0000","last_became_active":"2024-09-18T04:59:07.199844+0000","last_became_peered":"2024-09-18T04:59:07.199844+0000","last_unstale":"2024-09-18T04:59:07.202504+0000","last_undegraded":"2024-09-18T04:59:07.202504+0000","last_fullsized":"2024-09-18T04:59:07.202504+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":920,"log_dups_size":0,"ondisk_log_size":920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:40:08.684972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087268100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10766,"num_read_kb":19560,"num_write":908,"num_write_kb":18676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":9230703,"num_keys_recovered":59,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"144'809","reported_seq":5825,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.197043+0000","last_change":"2024-09-18T04:58:55.130930+0000","last_active":"2024-09-18T04:59:07.197043+0000","last_peered":"2024-09-18T04:59:07.197043+0000","last_clean":"2024-09-18T04:59:07.197043+0000","last_became_active":"2024-09-18T04:58:55.129007+0000","last_became_peered":"2024-09-18T04:58:55.129007+0000","last_unstale":"2024-09-18T04:59:07.197043+0000","last_undegraded":"2024-09-18T04:59:07.197043+0000","last_fullsized":"2024-09-18T04:59:07.197043+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:21:11.287190+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.00099438999999999994,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7388,"num_read_kb":33089,"num_write":1110,"num_write_kb":12855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"158'781","reported_seq":2932,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198095+0000","last_change":"2024-09-18T04:58:48.030265+0000","last_active":"2024-09-18T04:59:07.198095+0000","last_peered":"2024-09-18T04:59:07.198095+0000","last_clean":"2024-09-18T04:59:07.198095+0000","last_became_active":"2024-09-18T04:58:31.028633+0000","last_became_peered":"2024-09-18T04:58:31.028633+0000","last_unstale":"2024-09-18T04:59:07.198095+0000","last_undegraded":"2024-09-18T04:59:07.198095+0000","last_fullsized":"2024-09-18T04:59:07.198095+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:04:25.747259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00140489,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4341,"num_read_kb":14774,"num_write":882,"num_write_kb":9994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":12642724,"num_keys_recovered":71,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":47,"num_read":220784,"num_read_kb":683202,"num_write":29896,"num_write_kb":377778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9208,"num_bytes_recovered":184551668,"num_keys_recovered":562,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23964,"ondisk_log_size":23964,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28167,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51292,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320548,"statfs":{"total":96636764160,"available":96584241152,"internally_reserved":0,"allocated":2170880,"data_stored":1617954,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":50330059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.45600000000000002,"15min":0.46400000000000002},"min":{"1min":0.23799999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53100000000000003,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.437,"5min":0.41499999999999998,"15min":0.41099999999999998},"min":{"1min":0.312,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.60299999999999998}]},{"osd":1,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.46899999999999997,"15min":0.47799999999999998},"min":{"1min":0.249,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.55100000000000005,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.436,"15min":0.434},"min":{"1min":0.28399999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.621}]},{"osd":2,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.44700000000000001,"15min":0.44500000000000001},"min":{"1min":0.35399999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45200000000000001,"15min":0.45000000000000001},"min":{"1min":0.34200000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.56200000000000006,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.67500000000000004}]},{"osd":3,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52600000000000002,"15min":0.51900000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.4399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.34100000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.4809999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":0.72799999999999998}]},{"osd":4,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.42199999999999999,"15min":0.43099999999999999},"min":{"1min":0.26500000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.63700000000000001,"5min":1.954,"15min":1.954},"last":0.502},{"interface":"front","average":{"1min":0.375,"5min":0.41699999999999998,"15min":0.42399999999999999},"min":{"1min":0.246,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.629,"5min":1.478,"15min":1.478},"last":0.45800000000000002}]},{"osd":5,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.50800000000000001,"15min":0.52200000000000002},"min":{"1min":0.30299999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.71799999999999997,"5min":3.145,"15min":3.145},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.443,"15min":0.441},"min":{"1min":0.27700000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.65800000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.68999999999999995}]},{"osd":6,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.47899999999999998,"15min":0.48299999999999998},"min":{"1min":0.33300000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.69299999999999995,"5min":0.746,"15min":0.746},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.48899999999999999,"15min":0.49299999999999999},"min":{"1min":0.35199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.623,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.71399999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55336,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316504,"statfs":{"total":96636764160,"available":96580100096,"internally_reserved":0,"allocated":2121728,"data_stored":1032642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.71899999999999997,"15min":0.76900000000000002},"min":{"1min":0.23899999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.56799999999999995,"5min":6.1319999999999997,"15min":6.1319999999999997},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.75700000000000001,"15min":0.80500000000000005},"min":{"1min":0.35799999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.81100000000000005,"5min":6.1429999999999998,"15min":6.1429999999999998},"last":0.41099999999999998}]},{"osd":1,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.70999999999999996,"15min":0.76200000000000001},"min":{"1min":0.29299999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.44900000000000001,"5min":6.0410000000000004,"15min":6.0410000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.70899999999999996,"15min":0.753},"min":{"1min":0.33000000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.83599999999999997,"5min":5.5380000000000003,"15min":5.5380000000000003},"last":0.34599999999999997}]},{"osd":2,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.41999999999999998,"15min":0.42599999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.54400000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.315},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.46100000000000002,"15min":0.46800000000000003},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.55900000000000005,"5min":1.04,"15min":1.04},"last":0.33400000000000002}]},{"osd":3,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46000000000000002,"15min":0.46100000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.378},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.47799999999999998,"15min":0.48099999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.625,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.38700000000000001}]},{"osd":4,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.36799999999999999,"15min":0.374},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.47599999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.20799999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.43099999999999999,"15min":0.437},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.30399999999999999}]},{"osd":5,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.372,"15min":0.378},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.42499999999999999,"15min":0.435},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.53000000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.28699999999999998}]},{"osd":7,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.46800000000000003,"15min":0.46600000000000003},"min":{"1min":0.28100000000000003,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.48599999999999999,"15min":0.47699999999999998},"min":{"1min":0.34399999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.35599999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65188,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306652,"statfs":{"total":96636764160,"available":96570011648,"internally_reserved":0,"allocated":2121728,"data_stored":1033583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.54600000000000004,"15min":0.57099999999999995},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.59199999999999997,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.504,"15min":0.51000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71599999999999997,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.55700000000000005}]},{"osd":1,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.53300000000000003,"15min":0.55400000000000005},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.55600000000000005,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.32500000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63800000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.57499999999999996}]},{"osd":2,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47699999999999998,"15min":0.47899999999999998},"min":{"1min":0.28699999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.629,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.57399999999999995,"15min":0.59999999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.626,"5min":3.1070000000000002,"15min":3.1070000000000002},"last":0.504}]},{"osd":3,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.496,"15min":0.502},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.77000000000000002,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.504,"15min":0.50800000000000001},"min":{"1min":0.35099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.63,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.54600000000000004}]},{"osd":4,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.46800000000000003,"15min":0.47399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.59499999999999997,"5min":1.599,"15min":1.599},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.56299999999999994,"15min":0.59299999999999997},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.53500000000000003,"5min":3.379,"15min":3.379},"last":0.432}]},{"osd":6,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.45500000000000002,"15min":0.46600000000000003},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54200000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.56200000000000006,"15min":0.58499999999999996},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.085,"5min":3.1669999999999998,"15min":3.1669999999999998},"last":0.378}]},{"osd":7,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.48399999999999999,"15min":0.495},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80200000000000005,"5min":1.224,"15min":1.224},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.54200000000000004,"15min":0.55900000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.58599999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.45700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63512,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94308328,"statfs":{"total":96636764160,"available":96571727872,"internally_reserved":0,"allocated":2109440,"data_stored":1027567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.46000000000000002,"15min":0.44400000000000001},"min":{"1min":0.23499999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":3.7069999999999999,"5min":3.7069999999999999,"15min":3.7069999999999999},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.46100000000000002,"15min":0.441},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":3.3700000000000001,"5min":3.3700000000000001,"15min":3.3700000000000001},"last":0.219}]},{"osd":1,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.47099999999999997,"15min":0.45500000000000002},"min":{"1min":0.28199999999999997,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":3.2879999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.375},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.46600000000000003,"15min":0.44600000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.661,"5min":3.661,"15min":3.661},"last":0.38200000000000001}]},{"osd":2,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.52200000000000002,"15min":0.50600000000000001},"min":{"1min":0.29999999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.1890000000000001,"5min":3.1890000000000001,"15min":3.1890000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.51600000000000001,"15min":0.502},"min":{"1min":0.32300000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.3860000000000001,"5min":3.3860000000000001,"15min":3.3860000000000001},"last":0.35899999999999999}]},{"osd":3,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.504,"15min":0.48899999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":3.4500000000000002,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.48299999999999998,"15min":0.46100000000000002},"min":{"1min":0.372,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.3210000000000002,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.433}]},{"osd":5,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.42599999999999999,"15min":0.40200000000000002},"min":{"1min":0.25,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":3.9319999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.43099999999999999,"15min":0.40699999999999997},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":3.9449999999999998,"5min":3.9449999999999998,"15min":3.9449999999999998},"last":0.39700000000000002}]},{"osd":6,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.45100000000000001,"15min":0.42899999999999999},"min":{"1min":0.29299999999999998,"5min":0.251,"15min":0.251},"max":{"1min":3.3530000000000002,"5min":3.3530000000000002,"15min":3.3530000000000002},"last":0.312},{"interface":"front","average":{"1min":1.29,"5min":0.626,"15min":0.51600000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":17.422999999999998,"5min":17.422999999999998,"15min":17.422999999999998},"last":0.33500000000000002}]},{"osd":7,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.48399999999999999,"15min":0.46400000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":3.7330000000000001,"5min":3.7330000000000001,"15min":3.7330000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.52100000000000002,"15min":0.499},"min":{"1min":0.33300000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":3.6930000000000001,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":0.40500000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87320,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284520,"statfs":{"total":96636764160,"available":96547348480,"internally_reserved":0,"allocated":2109440,"data_stored":1027567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":87292360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.39500000000000002,"15min":0.40500000000000003},"min":{"1min":0.23899999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.45200000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.51800000000000002,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.38400000000000001}]},{"osd":1,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.376,"15min":0.375},"min":{"1min":0.249,"5min":0.245,"15min":0.245},"max":{"1min":0.51800000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.37},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.40500000000000003,"15min":0.41499999999999998},"min":{"1min":0.249,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.54400000000000004,"5min":0.755,"15min":0.755},"last":0.438}]},{"osd":2,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.41599999999999998,"15min":0.41999999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.56999999999999995,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.45200000000000001,"15min":0.45300000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.59399999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.47299999999999998}]},{"osd":4,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.52000000000000002,"15min":0.52900000000000003},"min":{"1min":0.31,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71999999999999997,"5min":0.755,"15min":0.755},"last":0.505},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.76100000000000001,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.63900000000000001}]},{"osd":5,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.55900000000000005,"15min":0.56899999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68600000000000005,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.66900000000000004,"5min":0.874,"15min":0.874},"last":0.60399999999999998}]},{"osd":6,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58699999999999997,"15min":0.59599999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80400000000000005,"5min":0.874,"15min":0.874},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72399999999999998,"5min":1.095,"15min":1.095},"last":0.51600000000000001}]},{"osd":7,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.65600000000000003,"15min":0.67200000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84499999999999997,"5min":1.335,"15min":1.335},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.61599999999999999,"15min":0.624},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.753,"5min":1.087,"15min":1.087},"last":0.58999999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37292,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":2121728,"data_stored":1033583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.33700000000000002,"15min":0.33600000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.39600000000000002,"15min":0.39800000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.63800000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.35899999999999999}]},{"osd":1,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.56999999999999995,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.36299999999999999,"15min":0.36099999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.29099999999999998}]},{"osd":3,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.40799999999999997,"15min":0.40400000000000003},"min":{"1min":0.26200000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.40999999999999998,"15min":0.40400000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.373}]},{"osd":4,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.58899999999999997,"15min":0.57899999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.4220000000000002,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":0.59999999999999998}]},{"osd":5,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.61199999999999999,"15min":0.56399999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":7.4480000000000004,"5min":7.4480000000000004,"15min":7.4480000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54800000000000004,"15min":0.54700000000000004},"min":{"1min":0.377,"5min":0.375,"15min":0.375},"max":{"1min":0.73699999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.64800000000000002}]},{"osd":6,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.84599999999999997,"5min":1.171,"15min":1.171},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.57799999999999996}]},{"osd":7,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80800000000000005,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":1.1719999999999999,"5min":0.69999999999999996,"15min":0.622},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":11.138,"5min":11.138,"15min":11.138},"last":0.61399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29124,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342716,"statfs":{"total":96636764160,"available":96606941184,"internally_reserved":0,"allocated":2154496,"data_stored":1617954,"data_compressed":9734,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.34999999999999998,"15min":0.35799999999999998},"min":{"1min":0.20899999999999999,"5min":0.185,"15min":0.185},"max":{"1min":0.622,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.63100000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":1.6579999999999999}]},{"osd":2,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.371,"15min":0.373},"min":{"1min":0.222,"5min":0.221,"15min":0.221},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":1.7270000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41599999999999998,"15min":0.41399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.6839999999999999}]},{"osd":3,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.38800000000000001,"15min":0.38400000000000001},"min":{"1min":0.23899999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":1.754},{"interface":"front","average":{"1min":0.434,"5min":0.40600000000000003,"15min":0.40100000000000002},"min":{"1min":0.29599999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":1.71}]},{"osd":4,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.50800000000000001,"15min":0.51700000000000002},"min":{"1min":0.32400000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.77000000000000002,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.496,"5min":0.50700000000000001,"15min":0.50900000000000001},"min":{"1min":0.315,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.86099999999999999,"5min":0.999,"15min":0.999},"last":0.51500000000000001}]},{"osd":5,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50900000000000001,"15min":0.51300000000000001},"min":{"1min":0.33600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80400000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":1.625},{"interface":"front","average":{"1min":0.5,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.373,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.83899999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":1.597}]},{"osd":6,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82799999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.74299999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67400000000000004}]},{"osd":7,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.56999999999999995,"15min":0.57799999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78200000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.40999999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0309999999999999,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.70299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51228,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320612,"statfs":{"total":96636764160,"available":96584306688,"internally_reserved":0,"allocated":2109440,"data_stored":1026626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50330055},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35999999999999999,"15min":0.36299999999999999},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.56799999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.247},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.34799999999999998,"15min":0.34599999999999997},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.53700000000000003,"5min":0.621,"15min":0.621},"last":0.28399999999999997}]},{"osd":2,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.39900000000000002,"15min":0.40100000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.42599999999999999,"15min":0.42399999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.46200000000000002}]},{"osd":3,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.434,"15min":0.42399999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.40400000000000003,"15min":0.39700000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.5}]},{"osd":4,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.63400000000000001,"15min":0.65600000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.89400000000000002,"5min":3.4710000000000001,"15min":3.4710000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.504,"15min":0.502},"min":{"1min":0.35899999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.376}]},{"osd":5,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.81699999999999995,"15min":0.84099999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":4.2640000000000002,"5min":4.2640000000000002,"15min":4.2640000000000002},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.54700000000000004}]},{"osd":6,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.66000000000000003,"15min":0.68000000000000005},"min":{"1min":0.39500000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81699999999999995,"5min":3.536,"15min":3.536},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57099999999999995,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.57999999999999996}]},{"osd":7,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55200000000000005,"15min":0.54700000000000004},"min":{"1min":0.41999999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.68400000000000005,"15min":0.70599999999999996},"min":{"1min":0.39100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.85699999999999998,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.48099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9734,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9734,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T04:59:13.113 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-09-18T04:59:13.113 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-18T04:59:13.419 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-18T04:59:14.092 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T04:59:14.092 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T04:59:14.395 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:59:14.396 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-18T04:59:14.406 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-09-18T04:59:14.300152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":47,"num_read":220900,"num_read_kb":683302,"num_write":30057,"num_write_kb":380196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9215,"num_bytes_recovered":185732404,"num_keys_recovered":562,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24084,"ondisk_log_size":24084,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":440292,"kb_used_data":16620,"kb_used_omap":12,"kb_used_meta":423539,"kb_avail":754534428,"statfs":{"total":773094113280,"available":772643254272,"internally_reserved":0,"allocated":17018880,"data_stored":9417476,"data_compressed":18433,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":12742,"internal_metadata":433704506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":524,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002051"},"pg_stats":[{"pgid":"3.1f","version":"158'766","reported_seq":5157,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.197217+0000","last_change":"2024-09-18T04:58:55.128107+0000","last_active":"2024-09-18T04:59:07.197217+0000","last_peered":"2024-09-18T04:59:07.197217+0000","last_clean":"2024-09-18T04:59:07.197217+0000","last_became_active":"2024-09-18T04:58:55.126111+0000","last_became_peered":"2024-09-18T04:58:55.126111+0000","last_unstale":"2024-09-18T04:59:07.197217+0000","last_undegraded":"2024-09-18T04:59:07.197217+0000","last_fullsized":"2024-09-18T04:59:07.197217+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:56:05.136229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001156916,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6348,"num_read_kb":20852,"num_write":870,"num_write_kb":9417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"157'817","reported_seq":2783,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731844+0000","last_change":"2024-09-18T04:58:48.096250+0000","last_active":"2024-09-18T04:59:07.731844+0000","last_peered":"2024-09-18T04:59:07.731844+0000","last_clean":"2024-09-18T04:59:07.731844+0000","last_became_active":"2024-09-18T04:58:32.029310+0000","last_became_peered":"2024-09-18T04:58:32.029310+0000","last_unstale":"2024-09-18T04:59:07.731844+0000","last_undegraded":"2024-09-18T04:59:07.731844+0000","last_fullsized":"2024-09-18T04:59:07.731844+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:13:02.688292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001168749,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3351,"num_read_kb":18023,"num_write":1120,"num_write_kb":8161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":3252478,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"156'500","reported_seq":10759,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195673+0000","last_change":"2024-09-18T04:58:48.023738+0000","last_active":"2024-09-18T04:59:07.195673+0000","last_peered":"2024-09-18T04:59:07.195673+0000","last_clean":"2024-09-18T04:59:07.195673+0000","last_became_active":"2024-09-18T04:58:27.115104+0000","last_became_peered":"2024-09-18T04:58:27.115104+0000","last_unstale":"2024-09-18T04:59:07.195673+0000","last_undegraded":"2024-09-18T04:59:07.195673+0000","last_fullsized":"2024-09-18T04:59:07.195673+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:33:08.572507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082741200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9761,"num_read_kb":28168,"num_write":548,"num_write_kb":23097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7293282,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"158'840","reported_seq":7984,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195584+0000","last_change":"2024-09-18T04:58:48.025083+0000","last_active":"2024-09-18T04:59:07.195584+0000","last_peered":"2024-09-18T04:59:07.195584+0000","last_clean":"2024-09-18T04:59:07.195584+0000","last_became_active":"2024-09-18T04:58:28.028905+0000","last_became_peered":"2024-09-18T04:58:28.028905+0000","last_unstale":"2024-09-18T04:59:07.195584+0000","last_undegraded":"2024-09-18T04:59:07.195584+0000","last_fullsized":"2024-09-18T04:59:07.195584+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:15:17.569793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014698000000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":8594,"num_read_kb":9084,"num_write":1169,"num_write_kb":5309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":1412856,"num_keys_recovered":106,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"155'473","reported_seq":4267,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.202696+0000","last_change":"2024-09-18T04:59:07.202189+0000","last_active":"2024-09-18T04:59:07.202696+0000","last_peered":"2024-09-18T04:59:07.202696+0000","last_clean":"2024-09-18T04:59:07.202696+0000","last_became_active":"2024-09-18T04:59:07.199588+0000","last_became_peered":"2024-09-18T04:59:07.199588+0000","last_unstale":"2024-09-18T04:59:07.202696+0000","last_undegraded":"2024-09-18T04:59:07.202696+0000","last_fullsized":"2024-09-18T04:59:07.202696+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:53:25.971943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012312829999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2677,"num_read_kb":15514,"num_write":503,"num_write_kb":9112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":912,"num_bytes_recovered":7005168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"157'455","reported_seq":3935,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:08.199477+0000","last_change":"2024-09-18T04:59:07.200974+0000","last_active":"2024-09-18T04:59:08.199477+0000","last_peered":"2024-09-18T04:59:08.199477+0000","last_clean":"2024-09-18T04:59:08.199477+0000","last_became_active":"2024-09-18T04:59:07.199276+0000","last_became_peered":"2024-09-18T04:59:07.199276+0000","last_unstale":"2024-09-18T04:59:08.199477+0000","last_undegraded":"2024-09-18T04:59:08.199477+0000","last_fullsized":"2024-09-18T04:59:08.199477+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:42:10.766349+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.000757346,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":2981,"num_read_kb":11740,"num_write":512,"num_write_kb":18285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":519,"num_bytes_recovered":11494100,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"156'520","reported_seq":5473,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198149+0000","last_change":"2024-09-18T04:58:57.148756+0000","last_active":"2024-09-18T04:59:07.198149+0000","last_peered":"2024-09-18T04:59:07.198149+0000","last_clean":"2024-09-18T04:59:07.198149+0000","last_became_active":"2024-09-18T04:58:57.145900+0000","last_became_peered":"2024-09-18T04:58:57.145900+0000","last_unstale":"2024-09-18T04:59:07.198149+0000","last_undegraded":"2024-09-18T04:59:07.198149+0000","last_fullsized":"2024-09-18T04:59:07.198149+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:03:40.179686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018405870000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4543,"num_read_kb":26564,"num_write":538,"num_write_kb":13069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":23811007,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"156'576","reported_seq":3426,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731747+0000","last_change":"2024-09-18T04:58:48.095394+0000","last_active":"2024-09-18T04:59:07.731747+0000","last_peered":"2024-09-18T04:59:07.731747+0000","last_clean":"2024-09-18T04:59:07.731747+0000","last_became_active":"2024-09-18T04:58:32.029074+0000","last_became_peered":"2024-09-18T04:58:32.029074+0000","last_unstale":"2024-09-18T04:59:07.731747+0000","last_undegraded":"2024-09-18T04:59:07.731747+0000","last_fullsized":"2024-09-18T04:59:07.731747+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:15:38.950060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053623899999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3403,"num_read_kb":16958,"num_write":699,"num_write_kb":12986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":4650417,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"153'506","reported_seq":4239,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.197394+0000","last_change":"2024-09-18T04:58:55.128037+0000","last_active":"2024-09-18T04:59:07.197394+0000","last_peered":"2024-09-18T04:59:07.197394+0000","last_clean":"2024-09-18T04:59:07.197394+0000","last_became_active":"2024-09-18T04:58:55.125721+0000","last_became_peered":"2024-09-18T04:58:55.125721+0000","last_unstale":"2024-09-18T04:59:07.197394+0000","last_undegraded":"2024-09-18T04:59:07.197394+0000","last_fullsized":"2024-09-18T04:59:07.197394+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:06:23.656568+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001441276,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4899,"num_read_kb":13699,"num_write":541,"num_write_kb":6077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"154'885","reported_seq":5042,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195463+0000","last_change":"2024-09-18T04:58:48.024658+0000","last_active":"2024-09-18T04:59:07.195463+0000","last_peered":"2024-09-18T04:59:07.195463+0000","last_clean":"2024-09-18T04:59:07.195463+0000","last_became_active":"2024-09-18T04:56:21.131133+0000","last_became_peered":"2024-09-18T04:56:21.131133+0000","last_unstale":"2024-09-18T04:59:07.195463+0000","last_undegraded":"2024-09-18T04:59:07.195463+0000","last_fullsized":"2024-09-18T04:59:07.195463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:01:57.036904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012610480000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6340,"num_read_kb":26579,"num_write":1141,"num_write_kb":12621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"157'637","reported_seq":5292,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198169+0000","last_change":"2024-09-18T04:58:48.028934+0000","last_active":"2024-09-18T04:59:07.198169+0000","last_peered":"2024-09-18T04:59:07.198169+0000","last_clean":"2024-09-18T04:59:07.198169+0000","last_became_active":"2024-09-18T04:58:32.032762+0000","last_became_peered":"2024-09-18T04:58:32.032762+0000","last_unstale":"2024-09-18T04:59:07.198169+0000","last_undegraded":"2024-09-18T04:59:07.198169+0000","last_fullsized":"2024-09-18T04:59:07.198169+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:33:25.010425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066811099999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":5292,"num_read_kb":26013,"num_write":596,"num_write_kb":7300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":447,"num_bytes_recovered":1594801,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"155'502","reported_seq":2817,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-18T04:59:06.190587+0000","last_change":"2024-09-18T04:58:48.023734+0000","last_active":"2024-09-18T04:59:06.190587+0000","last_peered":"2024-09-18T04:59:06.190587+0000","last_clean":"2024-09-18T04:59:06.190587+0000","last_became_active":"2024-09-18T04:58:30.025558+0000","last_became_peered":"2024-09-18T04:58:30.025558+0000","last_unstale":"2024-09-18T04:59:06.190587+0000","last_undegraded":"2024-09-18T04:59:06.190587+0000","last_fullsized":"2024-09-18T04:59:06.190587+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:57:31.113016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087474099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3131,"num_read_kb":27325,"num_write":609,"num_write_kb":11672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":12427258,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":192,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195662+0000","last_change":"2024-09-18T04:58:29.025271+0000","last_active":"2024-09-18T04:59:07.195662+0000","last_peered":"2024-09-18T04:59:07.195662+0000","last_clean":"2024-09-18T04:59:07.195662+0000","last_became_active":"2024-09-18T04:58:29.024911+0000","last_became_peered":"2024-09-18T04:58:29.024911+0000","last_unstale":"2024-09-18T04:59:07.195662+0000","last_undegraded":"2024-09-18T04:59:07.195662+0000","last_fullsized":"2024-09-18T04:59:07.195662+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:53:21.863837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063436799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"155'2480","reported_seq":9899,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195724+0000","last_change":"2024-09-18T04:58:48.024643+0000","last_active":"2024-09-18T04:59:07.195724+0000","last_peered":"2024-09-18T04:59:07.195724+0000","last_clean":"2024-09-18T04:59:07.195724+0000","last_became_active":"2024-09-18T04:58:33.027340+0000","last_became_peered":"2024-09-18T04:58:33.027340+0000","last_unstale":"2024-09-18T04:59:07.195724+0000","last_undegraded":"2024-09-18T04:59:07.195724+0000","last_fullsized":"2024-09-18T04:59:07.195724+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":2480,"log_dups_size":0,"ondisk_log_size":2480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:47:47.826132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000952045,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":18529,"num_read_kb":32990,"num_write":4379,"num_write_kb":20206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":5439803,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":176,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195585+0000","last_change":"2024-09-18T04:56:12.389971+0000","last_active":"2024-09-18T04:59:07.195585+0000","last_peered":"2024-09-18T04:59:07.195585+0000","last_clean":"2024-09-18T04:59:07.195585+0000","last_became_active":"2024-09-18T04:56:10.412595+0000","last_became_peered":"2024-09-18T04:56:10.412595+0000","last_unstale":"2024-09-18T04:59:07.195585+0000","last_undegraded":"2024-09-18T04:59:07.195585+0000","last_fullsized":"2024-09-18T04:59:07.195585+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:45.638420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038576799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"157'807","reported_seq":7861,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198219+0000","last_change":"2024-09-18T04:58:57.151001+0000","last_active":"2024-09-18T04:59:07.198219+0000","last_peered":"2024-09-18T04:59:07.198219+0000","last_clean":"2024-09-18T04:59:07.198219+0000","last_became_active":"2024-09-18T04:58:57.147959+0000","last_became_peered":"2024-09-18T04:58:57.147959+0000","last_unstale":"2024-09-18T04:59:07.198219+0000","last_undegraded":"2024-09-18T04:59:07.198219+0000","last_fullsized":"2024-09-18T04:59:07.198219+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:30:19.725543+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00065254400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8940,"num_read_kb":28417,"num_write":1051,"num_write_kb":11406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":5706670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198307+0000","last_change":"2024-09-18T04:58:57.150288+0000","last_active":"2024-09-18T04:59:07.198307+0000","last_peered":"2024-09-18T04:59:07.198307+0000","last_clean":"2024-09-18T04:59:07.198307+0000","last_became_active":"2024-09-18T04:58:57.149907+0000","last_became_peered":"2024-09-18T04:58:57.149907+0000","last_unstale":"2024-09-18T04:59:07.198307+0000","last_undegraded":"2024-09-18T04:59:07.198307+0000","last_fullsized":"2024-09-18T04:59:07.198307+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:12:39.499969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"157'757","reported_seq":6181,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731891+0000","last_change":"2024-09-18T04:58:48.096507+0000","last_active":"2024-09-18T04:59:07.731891+0000","last_peered":"2024-09-18T04:59:07.731891+0000","last_clean":"2024-09-18T04:59:07.731891+0000","last_became_active":"2024-09-18T04:58:33.026806+0000","last_became_peered":"2024-09-18T04:58:33.026806+0000","last_unstale":"2024-09-18T04:59:07.731891+0000","last_undegraded":"2024-09-18T04:59:07.731891+0000","last_fullsized":"2024-09-18T04:59:07.731891+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:59:41.673907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013736829999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6896,"num_read_kb":21261,"num_write":943,"num_write_kb":5208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":571544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":145,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195438+0000","last_change":"2024-09-18T04:56:46.254241+0000","last_active":"2024-09-18T04:59:07.195438+0000","last_peered":"2024-09-18T04:59:07.195438+0000","last_clean":"2024-09-18T04:59:07.195438+0000","last_became_active":"2024-09-18T04:56:46.253894+0000","last_became_peered":"2024-09-18T04:56:46.253894+0000","last_unstale":"2024-09-18T04:59:07.195438+0000","last_undegraded":"2024-09-18T04:59:07.195438+0000","last_fullsized":"2024-09-18T04:59:07.195438+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:58:42.098099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045144100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"158'505","reported_seq":5028,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.197318+0000","last_change":"2024-09-18T04:58:48.061503+0000","last_active":"2024-09-18T04:59:07.197318+0000","last_peered":"2024-09-18T04:59:07.197318+0000","last_clean":"2024-09-18T04:59:07.197318+0000","last_became_active":"2024-09-18T04:58:41.078435+0000","last_became_peered":"2024-09-18T04:58:41.078435+0000","last_unstale":"2024-09-18T04:59:07.197318+0000","last_undegraded":"2024-09-18T04:59:07.197318+0000","last_fullsized":"2024-09-18T04:59:07.197318+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:34:50.214360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056043399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4221,"num_read_kb":33044,"num_write":472,"num_write_kb":37350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":25349011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"155'485","reported_seq":9536,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.201259+0000","last_change":"2024-09-18T04:59:07.200564+0000","last_active":"2024-09-18T04:59:07.201259+0000","last_peered":"2024-09-18T04:59:07.201259+0000","last_clean":"2024-09-18T04:59:07.201259+0000","last_became_active":"2024-09-18T04:59:07.199056+0000","last_became_peered":"2024-09-18T04:59:07.199056+0000","last_unstale":"2024-09-18T04:59:07.201259+0000","last_undegraded":"2024-09-18T04:59:07.201259+0000","last_fullsized":"2024-09-18T04:59:07.201259+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:30:30.980376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049796699999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":9360,"num_read_kb":28051,"num_write":499,"num_write_kb":12488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":472,"num_bytes_recovered":11362815,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":362,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198082+0000","last_change":"2024-09-18T04:58:38.025236+0000","last_active":"2024-09-18T04:59:07.198082+0000","last_peered":"2024-09-18T04:59:07.198082+0000","last_clean":"2024-09-18T04:59:07.198082+0000","last_became_active":"2024-09-18T04:58:38.024740+0000","last_became_peered":"2024-09-18T04:58:38.024740+0000","last_unstale":"2024-09-18T04:59:07.198082+0000","last_undegraded":"2024-09-18T04:59:07.198082+0000","last_fullsized":"2024-09-18T04:59:07.198082+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_clean_scrub_stamp":"2024-09-18T04:56:10.373785+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:10:45.251654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":37,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198355+0000","last_change":"2024-09-18T04:58:57.148289+0000","last_active":"2024-09-18T04:59:07.198355+0000","last_peered":"2024-09-18T04:59:07.198355+0000","last_clean":"2024-09-18T04:59:07.198355+0000","last_became_active":"2024-09-18T04:58:57.147915+0000","last_became_peered":"2024-09-18T04:58:57.147915+0000","last_unstale":"2024-09-18T04:59:07.198355+0000","last_undegraded":"2024-09-18T04:59:07.198355+0000","last_fullsized":"2024-09-18T04:59:07.198355+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:31:21.699401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"157'811","reported_seq":13428,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198153+0000","last_change":"2024-09-18T04:58:57.152156+0000","last_active":"2024-09-18T04:59:07.198153+0000","last_peered":"2024-09-18T04:59:07.198153+0000","last_clean":"2024-09-18T04:59:07.198153+0000","last_became_active":"2024-09-18T04:58:57.148221+0000","last_became_peered":"2024-09-18T04:58:57.148221+0000","last_unstale":"2024-09-18T04:59:07.198153+0000","last_undegraded":"2024-09-18T04:59:07.198153+0000","last_fullsized":"2024-09-18T04:59:07.198153+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:45:57.434891+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0033124909999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":14837,"num_read_kb":27169,"num_write":1047,"num_write_kb":7785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":1280827,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":178,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198076+0000","last_change":"2024-09-18T04:56:12.398631+0000","last_active":"2024-09-18T04:59:07.198076+0000","last_peered":"2024-09-18T04:59:07.198076+0000","last_clean":"2024-09-18T04:59:07.198076+0000","last_became_active":"2024-09-18T04:56:10.412614+0000","last_became_peered":"2024-09-18T04:56:10.412614+0000","last_unstale":"2024-09-18T04:59:07.198076+0000","last_undegraded":"2024-09-18T04:59:07.198076+0000","last_fullsized":"2024-09-18T04:59:07.198076+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:44:13.401320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075126299999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"158'601","reported_seq":2977,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-18T04:59:06.191350+0000","last_change":"2024-09-18T04:58:48.024976+0000","last_active":"2024-09-18T04:59:06.191350+0000","last_peered":"2024-09-18T04:59:06.191350+0000","last_clean":"2024-09-18T04:59:06.191350+0000","last_became_active":"2024-09-18T04:58:30.027757+0000","last_became_peered":"2024-09-18T04:58:30.027757+0000","last_unstale":"2024-09-18T04:59:06.191350+0000","last_undegraded":"2024-09-18T04:59:06.191350+0000","last_fullsized":"2024-09-18T04:59:06.191350+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:29:00.106938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014951890000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":2216,"num_read_kb":23967,"num_write":593,"num_write_kb":16097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":10562840,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":176,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195410+0000","last_change":"2024-09-18T04:56:12.390044+0000","last_active":"2024-09-18T04:59:07.195410+0000","last_peered":"2024-09-18T04:59:07.195410+0000","last_clean":"2024-09-18T04:59:07.195410+0000","last_became_active":"2024-09-18T04:56:10.413042+0000","last_became_peered":"2024-09-18T04:56:10.413042+0000","last_unstale":"2024-09-18T04:59:07.195410+0000","last_undegraded":"2024-09-18T04:59:07.195410+0000","last_fullsized":"2024-09-18T04:59:07.195410+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:46:11.074243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"157'525","reported_seq":4818,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731809+0000","last_change":"2024-09-18T04:58:48.095298+0000","last_active":"2024-09-18T04:59:07.731809+0000","last_peered":"2024-09-18T04:59:07.731809+0000","last_clean":"2024-09-18T04:59:07.731809+0000","last_became_active":"2024-09-18T04:58:32.027104+0000","last_became_peered":"2024-09-18T04:58:32.027104+0000","last_unstale":"2024-09-18T04:59:07.731809+0000","last_undegraded":"2024-09-18T04:59:07.731809+0000","last_fullsized":"2024-09-18T04:59:07.731809+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:41:37.134684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056931099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5904,"num_read_kb":13840,"num_write":503,"num_write_kb":5042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":2187784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198027+0000","last_change":"2024-09-18T04:57:40.472681+0000","last_active":"2024-09-18T04:59:07.198027+0000","last_peered":"2024-09-18T04:59:07.198027+0000","last_clean":"2024-09-18T04:59:07.198027+0000","last_became_active":"2024-09-18T04:57:40.472217+0000","last_became_peered":"2024-09-18T04:57:40.472217+0000","last_unstale":"2024-09-18T04:59:07.198027+0000","last_undegraded":"2024-09-18T04:59:07.198027+0000","last_fullsized":"2024-09-18T04:59:07.198027+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:27:11.849311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"157'482","reported_seq":3615,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731740+0000","last_change":"2024-09-18T04:58:48.096157+0000","last_active":"2024-09-18T04:59:07.731740+0000","last_peered":"2024-09-18T04:59:07.731740+0000","last_clean":"2024-09-18T04:59:07.731740+0000","last_became_active":"2024-09-18T04:58:38.024916+0000","last_became_peered":"2024-09-18T04:58:38.024916+0000","last_unstale":"2024-09-18T04:59:07.731740+0000","last_undegraded":"2024-09-18T04:59:07.731740+0000","last_fullsized":"2024-09-18T04:59:07.731740+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:32:42.202542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001087188,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2758,"num_read_kb":7938,"num_write":473,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":458,"num_bytes_recovered":2219067,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"155'714","reported_seq":12009,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-18T04:59:06.191085+0000","last_change":"2024-09-18T04:58:48.024809+0000","last_active":"2024-09-18T04:59:06.191085+0000","last_peered":"2024-09-18T04:59:06.191085+0000","last_clean":"2024-09-18T04:59:06.191085+0000","last_became_active":"2024-09-18T04:58:26.101490+0000","last_became_peered":"2024-09-18T04:58:26.101490+0000","last_unstale":"2024-09-18T04:59:06.191085+0000","last_undegraded":"2024-09-18T04:59:06.191085+0000","last_fullsized":"2024-09-18T04:59:06.191085+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:30:15.538062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015101680000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":12956,"num_read_kb":26590,"num_write":848,"num_write_kb":15777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":5822953,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"157'470","reported_seq":6073,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195769+0000","last_change":"2024-09-18T04:58:48.136195+0000","last_active":"2024-09-18T04:59:07.195769+0000","last_peered":"2024-09-18T04:59:07.195769+0000","last_clean":"2024-09-18T04:59:07.195769+0000","last_became_active":"2024-09-18T04:58:27.115137+0000","last_became_peered":"2024-09-18T04:58:27.115137+0000","last_unstale":"2024-09-18T04:59:07.195769+0000","last_undegraded":"2024-09-18T04:59:07.195769+0000","last_fullsized":"2024-09-18T04:59:07.195769+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:57:42.359624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039325500000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":4725,"num_read_kb":14773,"num_write":478,"num_write_kb":8139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":6055124,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"155'1400","reported_seq":5772,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731780+0000","last_change":"2024-09-18T04:58:48.095758+0000","last_active":"2024-09-18T04:59:07.731780+0000","last_peered":"2024-09-18T04:59:07.731780+0000","last_clean":"2024-09-18T04:59:07.731780+0000","last_became_active":"2024-09-18T04:58:31.026041+0000","last_became_peered":"2024-09-18T04:58:31.026041+0000","last_unstale":"2024-09-18T04:59:07.731780+0000","last_undegraded":"2024-09-18T04:59:07.731780+0000","last_fullsized":"2024-09-18T04:59:07.731780+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":1400,"log_dups_size":0,"ondisk_log_size":1400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:55:29.526466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083671099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8851,"num_read_kb":15183,"num_write":2049,"num_write_kb":8862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":1966128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"155'908","reported_seq":7203,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195547+0000","last_change":"2024-09-18T04:58:48.023609+0000","last_active":"2024-09-18T04:59:07.195547+0000","last_peered":"2024-09-18T04:59:07.195547+0000","last_clean":"2024-09-18T04:59:07.195547+0000","last_became_active":"2024-09-18T04:58:33.027525+0000","last_became_peered":"2024-09-18T04:58:33.027525+0000","last_unstale":"2024-09-18T04:59:07.195547+0000","last_undegraded":"2024-09-18T04:59:07.195547+0000","last_fullsized":"2024-09-18T04:59:07.195547+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:27:34.074204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062770000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9983,"num_read_kb":24174,"num_write":928,"num_write_kb":8107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":86198,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"155'617","reported_seq":7056,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.194912+0000","last_change":"2024-09-18T04:58:48.021010+0000","last_active":"2024-09-18T04:59:07.194912+0000","last_peered":"2024-09-18T04:59:07.194912+0000","last_clean":"2024-09-18T04:59:07.194912+0000","last_became_active":"2024-09-18T04:58:07.017827+0000","last_became_peered":"2024-09-18T04:58:07.017827+0000","last_unstale":"2024-09-18T04:59:07.194912+0000","last_undegraded":"2024-09-18T04:59:07.194912+0000","last_fullsized":"2024-09-18T04:59:07.194912+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:42:31.897588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050117499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8566,"num_read_kb":18769,"num_write":595,"num_write_kb":12467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":2872264,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"155'1050","reported_seq":5261,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198243+0000","last_change":"2024-09-18T04:58:48.029466+0000","last_active":"2024-09-18T04:59:07.198243+0000","last_peered":"2024-09-18T04:59:07.198243+0000","last_clean":"2024-09-18T04:59:07.198243+0000","last_became_active":"2024-09-18T04:58:31.028189+0000","last_became_peered":"2024-09-18T04:58:31.028189+0000","last_unstale":"2024-09-18T04:59:07.198243+0000","last_undegraded":"2024-09-18T04:59:07.198243+0000","last_fullsized":"2024-09-18T04:59:07.198243+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":1050,"log_dups_size":0,"ondisk_log_size":1050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:26:10.430573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001035219,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7660,"num_read_kb":17955,"num_write":1663,"num_write_kb":5543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":2749742,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"157'556","reported_seq":5095,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198207+0000","last_change":"2024-09-18T04:58:48.029669+0000","last_active":"2024-09-18T04:59:07.198207+0000","last_peered":"2024-09-18T04:59:07.198207+0000","last_clean":"2024-09-18T04:59:07.198207+0000","last_became_active":"2024-09-18T04:58:29.029625+0000","last_became_peered":"2024-09-18T04:58:29.029625+0000","last_unstale":"2024-09-18T04:59:07.198207+0000","last_undegraded":"2024-09-18T04:59:07.198207+0000","last_fullsized":"2024-09-18T04:59:07.198207+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:51:09.257320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011607150000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4690,"num_read_kb":26470,"num_write":636,"num_write_kb":8108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":5504104,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"157'809","reported_seq":4779,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195411+0000","last_change":"2024-09-18T04:58:48.136237+0000","last_active":"2024-09-18T04:59:07.195411+0000","last_peered":"2024-09-18T04:59:07.195411+0000","last_clean":"2024-09-18T04:59:07.195411+0000","last_became_active":"2024-09-18T04:56:21.131202+0000","last_became_peered":"2024-09-18T04:56:21.131202+0000","last_unstale":"2024-09-18T04:59:07.195411+0000","last_undegraded":"2024-09-18T04:59:07.195411+0000","last_fullsized":"2024-09-18T04:59:07.195411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:17:35.438723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041769600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5877,"num_read_kb":14669,"num_write":993,"num_write_kb":10354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"157'920","reported_seq":8474,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.202504+0000","last_change":"2024-09-18T04:59:07.201833+0000","last_active":"2024-09-18T04:59:07.202504+0000","last_peered":"2024-09-18T04:59:07.202504+0000","last_clean":"2024-09-18T04:59:07.202504+0000","last_became_active":"2024-09-18T04:59:07.199844+0000","last_became_peered":"2024-09-18T04:59:07.199844+0000","last_unstale":"2024-09-18T04:59:07.202504+0000","last_undegraded":"2024-09-18T04:59:07.202504+0000","last_fullsized":"2024-09-18T04:59:07.202504+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":920,"log_dups_size":0,"ondisk_log_size":920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:40:08.684972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087268100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10766,"num_read_kb":19560,"num_write":908,"num_write_kb":18676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":9230703,"num_keys_recovered":59,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"144'809","reported_seq":5825,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.197043+0000","last_change":"2024-09-18T04:58:55.130930+0000","last_active":"2024-09-18T04:59:07.197043+0000","last_peered":"2024-09-18T04:59:07.197043+0000","last_clean":"2024-09-18T04:59:07.197043+0000","last_became_active":"2024-09-18T04:58:55.129007+0000","last_became_peered":"2024-09-18T04:58:55.129007+0000","last_unstale":"2024-09-18T04:59:07.197043+0000","last_undegraded":"2024-09-18T04:59:07.197043+0000","last_fullsized":"2024-09-18T04:59:07.197043+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:21:11.287190+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.00099438999999999994,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7388,"num_read_kb":33089,"num_write":1110,"num_write_kb":12855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"158'781","reported_seq":2932,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198095+0000","last_change":"2024-09-18T04:58:48.030265+0000","last_active":"2024-09-18T04:59:07.198095+0000","last_peered":"2024-09-18T04:59:07.198095+0000","last_clean":"2024-09-18T04:59:07.198095+0000","last_became_active":"2024-09-18T04:58:31.028633+0000","last_became_peered":"2024-09-18T04:58:31.028633+0000","last_unstale":"2024-09-18T04:59:07.198095+0000","last_undegraded":"2024-09-18T04:59:07.198095+0000","last_fullsized":"2024-09-18T04:59:07.198095+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:04:25.747259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00140489,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4341,"num_read_kb":14774,"num_write":882,"num_write_kb":9994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":12642724,"num_keys_recovered":71,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":47,"num_read":220784,"num_read_kb":683202,"num_write":29896,"num_write_kb":377778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9208,"num_bytes_recovered":184551668,"num_keys_recovered":562,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23964,"ondisk_log_size":23964,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28167,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51292,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320548,"statfs":{"total":96636764160,"available":96584241152,"internally_reserved":0,"allocated":2170880,"data_stored":1617954,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":50330059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.45600000000000002,"15min":0.46400000000000002},"min":{"1min":0.23799999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53100000000000003,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.437,"5min":0.41499999999999998,"15min":0.41099999999999998},"min":{"1min":0.312,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.60299999999999998}]},{"osd":1,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.46899999999999997,"15min":0.47799999999999998},"min":{"1min":0.249,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.55100000000000005,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.436,"15min":0.434},"min":{"1min":0.28399999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.621}]},{"osd":2,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.44700000000000001,"15min":0.44500000000000001},"min":{"1min":0.35399999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45200000000000001,"15min":0.45000000000000001},"min":{"1min":0.34200000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.56200000000000006,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.67500000000000004}]},{"osd":3,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52600000000000002,"15min":0.51900000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.4399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.34100000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.4809999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":0.72799999999999998}]},{"osd":4,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.42199999999999999,"15min":0.43099999999999999},"min":{"1min":0.26500000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.63700000000000001,"5min":1.954,"15min":1.954},"last":0.502},{"interface":"front","average":{"1min":0.375,"5min":0.41699999999999998,"15min":0.42399999999999999},"min":{"1min":0.246,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.629,"5min":1.478,"15min":1.478},"last":0.45800000000000002}]},{"osd":5,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.50800000000000001,"15min":0.52200000000000002},"min":{"1min":0.30299999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.71799999999999997,"5min":3.145,"15min":3.145},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.443,"15min":0.441},"min":{"1min":0.27700000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.65800000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.68999999999999995}]},{"osd":6,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.47899999999999998,"15min":0.48299999999999998},"min":{"1min":0.33300000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.69299999999999995,"5min":0.746,"15min":0.746},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.48899999999999999,"15min":0.49299999999999999},"min":{"1min":0.35199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.623,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.71399999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55336,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316504,"statfs":{"total":96636764160,"available":96580100096,"internally_reserved":0,"allocated":2121728,"data_stored":1032642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.71899999999999997,"15min":0.76900000000000002},"min":{"1min":0.23899999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.56799999999999995,"5min":6.1319999999999997,"15min":6.1319999999999997},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.75700000000000001,"15min":0.80500000000000005},"min":{"1min":0.35799999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.81100000000000005,"5min":6.1429999999999998,"15min":6.1429999999999998},"last":0.41099999999999998}]},{"osd":1,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.70999999999999996,"15min":0.76200000000000001},"min":{"1min":0.29299999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.44900000000000001,"5min":6.0410000000000004,"15min":6.0410000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.70899999999999996,"15min":0.753},"min":{"1min":0.33000000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.83599999999999997,"5min":5.5380000000000003,"15min":5.5380000000000003},"last":0.34599999999999997}]},{"osd":2,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.41999999999999998,"15min":0.42599999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.54400000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.315},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.46100000000000002,"15min":0.46800000000000003},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.55900000000000005,"5min":1.04,"15min":1.04},"last":0.33400000000000002}]},{"osd":3,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46000000000000002,"15min":0.46100000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.378},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.47799999999999998,"15min":0.48099999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.625,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.38700000000000001}]},{"osd":4,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.36799999999999999,"15min":0.374},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.47599999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.20799999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.43099999999999999,"15min":0.437},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.30399999999999999}]},{"osd":5,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.372,"15min":0.378},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.42499999999999999,"15min":0.435},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.53000000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.28699999999999998}]},{"osd":7,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.46800000000000003,"15min":0.46600000000000003},"min":{"1min":0.28100000000000003,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.48599999999999999,"15min":0.47699999999999998},"min":{"1min":0.34399999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.35599999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65188,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306652,"statfs":{"total":96636764160,"available":96570011648,"internally_reserved":0,"allocated":2121728,"data_stored":1033583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.54600000000000004,"15min":0.57099999999999995},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.59199999999999997,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.504,"15min":0.51000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71599999999999997,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.55700000000000005}]},{"osd":1,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.53300000000000003,"15min":0.55400000000000005},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.55600000000000005,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.32500000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63800000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.57499999999999996}]},{"osd":2,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47699999999999998,"15min":0.47899999999999998},"min":{"1min":0.28699999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.629,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.57399999999999995,"15min":0.59999999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.626,"5min":3.1070000000000002,"15min":3.1070000000000002},"last":0.504}]},{"osd":3,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.496,"15min":0.502},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.77000000000000002,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.504,"15min":0.50800000000000001},"min":{"1min":0.35099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.63,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.54600000000000004}]},{"osd":4,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.46800000000000003,"15min":0.47399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.59499999999999997,"5min":1.599,"15min":1.599},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.56299999999999994,"15min":0.59299999999999997},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.53500000000000003,"5min":3.379,"15min":3.379},"last":0.432}]},{"osd":6,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.45500000000000002,"15min":0.46600000000000003},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54200000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.56200000000000006,"15min":0.58499999999999996},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.085,"5min":3.1669999999999998,"15min":3.1669999999999998},"last":0.378}]},{"osd":7,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.48399999999999999,"15min":0.495},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80200000000000005,"5min":1.224,"15min":1.224},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.54200000000000004,"15min":0.55900000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.58599999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.45700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63512,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94308328,"statfs":{"total":96636764160,"available":96571727872,"internally_reserved":0,"allocated":2109440,"data_stored":1027567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.46000000000000002,"15min":0.44400000000000001},"min":{"1min":0.23499999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":3.7069999999999999,"5min":3.7069999999999999,"15min":3.7069999999999999},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.46100000000000002,"15min":0.441},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":3.3700000000000001,"5min":3.3700000000000001,"15min":3.3700000000000001},"last":0.219}]},{"osd":1,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.47099999999999997,"15min":0.45500000000000002},"min":{"1min":0.28199999999999997,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":3.2879999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.375},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.46600000000000003,"15min":0.44600000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.661,"5min":3.661,"15min":3.661},"last":0.38200000000000001}]},{"osd":2,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.52200000000000002,"15min":0.50600000000000001},"min":{"1min":0.29999999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.1890000000000001,"5min":3.1890000000000001,"15min":3.1890000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.51600000000000001,"15min":0.502},"min":{"1min":0.32300000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.3860000000000001,"5min":3.3860000000000001,"15min":3.3860000000000001},"last":0.35899999999999999}]},{"osd":3,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.504,"15min":0.48899999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":3.4500000000000002,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.48299999999999998,"15min":0.46100000000000002},"min":{"1min":0.372,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.3210000000000002,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.433}]},{"osd":5,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.42599999999999999,"15min":0.40200000000000002},"min":{"1min":0.25,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":3.9319999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.43099999999999999,"15min":0.40699999999999997},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":3.9449999999999998,"5min":3.9449999999999998,"15min":3.9449999999999998},"last":0.39700000000000002}]},{"osd":6,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.45100000000000001,"15min":0.42899999999999999},"min":{"1min":0.29299999999999998,"5min":0.251,"15min":0.251},"max":{"1min":3.3530000000000002,"5min":3.3530000000000002,"15min":3.3530000000000002},"last":0.312},{"interface":"front","average":{"1min":1.29,"5min":0.626,"15min":0.51600000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":17.422999999999998,"5min":17.422999999999998,"15min":17.422999999999998},"last":0.33500000000000002}]},{"osd":7,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.48399999999999999,"15min":0.46400000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":3.7330000000000001,"5min":3.7330000000000001,"15min":3.7330000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.52100000000000002,"15min":0.499},"min":{"1min":0.33300000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":3.6930000000000001,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":0.40500000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87320,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284520,"statfs":{"total":96636764160,"available":96547348480,"internally_reserved":0,"allocated":2109440,"data_stored":1027567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":87292360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.39500000000000002,"15min":0.40500000000000003},"min":{"1min":0.23899999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.45200000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.51800000000000002,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.38400000000000001}]},{"osd":1,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.376,"15min":0.375},"min":{"1min":0.249,"5min":0.245,"15min":0.245},"max":{"1min":0.51800000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.37},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.40500000000000003,"15min":0.41499999999999998},"min":{"1min":0.249,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.54400000000000004,"5min":0.755,"15min":0.755},"last":0.438}]},{"osd":2,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.41599999999999998,"15min":0.41999999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.56999999999999995,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.45200000000000001,"15min":0.45300000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.59399999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.47299999999999998}]},{"osd":4,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.52000000000000002,"15min":0.52900000000000003},"min":{"1min":0.31,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71999999999999997,"5min":0.755,"15min":0.755},"last":0.505},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.76100000000000001,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.63900000000000001}]},{"osd":5,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.55900000000000005,"15min":0.56899999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68600000000000005,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.66900000000000004,"5min":0.874,"15min":0.874},"last":0.60399999999999998}]},{"osd":6,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58699999999999997,"15min":0.59599999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80400000000000005,"5min":0.874,"15min":0.874},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72399999999999998,"5min":1.095,"15min":1.095},"last":0.51600000000000001}]},{"osd":7,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.65600000000000003,"15min":0.67200000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84499999999999997,"5min":1.335,"15min":1.335},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.61599999999999999,"15min":0.624},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.753,"5min":1.087,"15min":1.087},"last":0.58999999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37292,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":2121728,"data_stored":1033583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.33700000000000002,"15min":0.33600000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.39600000000000002,"15min":0.39800000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.63800000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.35899999999999999}]},{"osd":1,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.56999999999999995,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.36299999999999999,"15min":0.36099999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.29099999999999998}]},{"osd":3,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.40799999999999997,"15min":0.40400000000000003},"min":{"1min":0.26200000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.40999999999999998,"15min":0.40400000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.373}]},{"osd":4,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.58899999999999997,"15min":0.57899999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.4220000000000002,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":0.59999999999999998}]},{"osd":5,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.61199999999999999,"15min":0.56399999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":7.4480000000000004,"5min":7.4480000000000004,"15min":7.4480000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54800000000000004,"15min":0.54700000000000004},"min":{"1min":0.377,"5min":0.375,"15min":0.375},"max":{"1min":0.73699999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.64800000000000002}]},{"osd":6,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.84599999999999997,"5min":1.171,"15min":1.171},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.57799999999999996}]},{"osd":7,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80800000000000005,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":1.1719999999999999,"5min":0.69999999999999996,"15min":0.622},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":11.138,"5min":11.138,"15min":11.138},"last":0.61399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29124,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342716,"statfs":{"total":96636764160,"available":96606941184,"internally_reserved":0,"allocated":2154496,"data_stored":1617954,"data_compressed":9734,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.34999999999999998,"15min":0.35799999999999998},"min":{"1min":0.20899999999999999,"5min":0.185,"15min":0.185},"max":{"1min":0.622,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.63100000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":1.6579999999999999}]},{"osd":2,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.371,"15min":0.373},"min":{"1min":0.222,"5min":0.221,"15min":0.221},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":1.7270000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41599999999999998,"15min":0.41399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.6839999999999999}]},{"osd":3,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.38800000000000001,"15min":0.38400000000000001},"min":{"1min":0.23899999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":1.754},{"interface":"front","average":{"1min":0.434,"5min":0.40600000000000003,"15min":0.40100000000000002},"min":{"1min":0.29599999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":1.71}]},{"osd":4,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.50800000000000001,"15min":0.51700000000000002},"min":{"1min":0.32400000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.77000000000000002,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.496,"5min":0.50700000000000001,"15min":0.50900000000000001},"min":{"1min":0.315,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.86099999999999999,"5min":0.999,"15min":0.999},"last":0.51500000000000001}]},{"osd":5,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50900000000000001,"15min":0.51300000000000001},"min":{"1min":0.33600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80400000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":1.625},{"interface":"front","average":{"1min":0.5,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.373,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.83899999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":1.597}]},{"osd":6,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82799999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.74299999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67400000000000004}]},{"osd":7,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.56999999999999995,"15min":0.57799999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78200000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.40999999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0309999999999999,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.70299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51228,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320612,"statfs":{"total":96636764160,"available":96584306688,"internally_reserved":0,"allocated":2109440,"data_stored":1026626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50330055},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35999999999999999,"15min":0.36299999999999999},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.56799999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.247},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.34799999999999998,"15min":0.34599999999999997},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.53700000000000003,"5min":0.621,"15min":0.621},"last":0.28399999999999997}]},{"osd":2,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.39900000000000002,"15min":0.40100000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.42599999999999999,"15min":0.42399999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.46200000000000002}]},{"osd":3,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.434,"15min":0.42399999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.40400000000000003,"15min":0.39700000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.5}]},{"osd":4,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.63400000000000001,"15min":0.65600000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.89400000000000002,"5min":3.4710000000000001,"15min":3.4710000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.504,"15min":0.502},"min":{"1min":0.35899999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.376}]},{"osd":5,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.81699999999999995,"15min":0.84099999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":4.2640000000000002,"5min":4.2640000000000002,"15min":4.2640000000000002},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.54700000000000004}]},{"osd":6,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.66000000000000003,"15min":0.68000000000000005},"min":{"1min":0.39500000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81699999999999995,"5min":3.536,"15min":3.536},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57099999999999995,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.57999999999999996}]},{"osd":7,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55200000000000005,"15min":0.54700000000000004},"min":{"1min":0.41999999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.68400000000000005,"15min":0.70599999999999996},"min":{"1min":0.39100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.85699999999999998,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.48099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9734,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9734,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T04:59:14.409 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T04:59:14.697 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:59:14.697 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-18T04:59:14.708 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-09-18T04:59:14.300152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":47,"num_read":220900,"num_read_kb":683302,"num_write":30057,"num_write_kb":380196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9215,"num_bytes_recovered":185732404,"num_keys_recovered":562,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24084,"ondisk_log_size":24084,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":440292,"kb_used_data":16620,"kb_used_omap":12,"kb_used_meta":423539,"kb_avail":754534428,"statfs":{"total":773094113280,"available":772643254272,"internally_reserved":0,"allocated":17018880,"data_stored":9417476,"data_compressed":18433,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":12742,"internal_metadata":433704506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":524,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002051"},"pg_stats":[{"pgid":"3.1f","version":"158'766","reported_seq":5157,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.197217+0000","last_change":"2024-09-18T04:58:55.128107+0000","last_active":"2024-09-18T04:59:07.197217+0000","last_peered":"2024-09-18T04:59:07.197217+0000","last_clean":"2024-09-18T04:59:07.197217+0000","last_became_active":"2024-09-18T04:58:55.126111+0000","last_became_peered":"2024-09-18T04:58:55.126111+0000","last_unstale":"2024-09-18T04:59:07.197217+0000","last_undegraded":"2024-09-18T04:59:07.197217+0000","last_fullsized":"2024-09-18T04:59:07.197217+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:56:05.136229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001156916,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6348,"num_read_kb":20852,"num_write":870,"num_write_kb":9417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"157'817","reported_seq":2783,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731844+0000","last_change":"2024-09-18T04:58:48.096250+0000","last_active":"2024-09-18T04:59:07.731844+0000","last_peered":"2024-09-18T04:59:07.731844+0000","last_clean":"2024-09-18T04:59:07.731844+0000","last_became_active":"2024-09-18T04:58:32.029310+0000","last_became_peered":"2024-09-18T04:58:32.029310+0000","last_unstale":"2024-09-18T04:59:07.731844+0000","last_undegraded":"2024-09-18T04:59:07.731844+0000","last_fullsized":"2024-09-18T04:59:07.731844+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:13:02.688292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001168749,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3351,"num_read_kb":18023,"num_write":1120,"num_write_kb":8161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":3252478,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"156'500","reported_seq":10759,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195673+0000","last_change":"2024-09-18T04:58:48.023738+0000","last_active":"2024-09-18T04:59:07.195673+0000","last_peered":"2024-09-18T04:59:07.195673+0000","last_clean":"2024-09-18T04:59:07.195673+0000","last_became_active":"2024-09-18T04:58:27.115104+0000","last_became_peered":"2024-09-18T04:58:27.115104+0000","last_unstale":"2024-09-18T04:59:07.195673+0000","last_undegraded":"2024-09-18T04:59:07.195673+0000","last_fullsized":"2024-09-18T04:59:07.195673+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:33:08.572507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082741200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9761,"num_read_kb":28168,"num_write":548,"num_write_kb":23097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7293282,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"158'840","reported_seq":7984,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195584+0000","last_change":"2024-09-18T04:58:48.025083+0000","last_active":"2024-09-18T04:59:07.195584+0000","last_peered":"2024-09-18T04:59:07.195584+0000","last_clean":"2024-09-18T04:59:07.195584+0000","last_became_active":"2024-09-18T04:58:28.028905+0000","last_became_peered":"2024-09-18T04:58:28.028905+0000","last_unstale":"2024-09-18T04:59:07.195584+0000","last_undegraded":"2024-09-18T04:59:07.195584+0000","last_fullsized":"2024-09-18T04:59:07.195584+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:15:17.569793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014698000000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":8594,"num_read_kb":9084,"num_write":1169,"num_write_kb":5309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":1412856,"num_keys_recovered":106,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"155'473","reported_seq":4267,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.202696+0000","last_change":"2024-09-18T04:59:07.202189+0000","last_active":"2024-09-18T04:59:07.202696+0000","last_peered":"2024-09-18T04:59:07.202696+0000","last_clean":"2024-09-18T04:59:07.202696+0000","last_became_active":"2024-09-18T04:59:07.199588+0000","last_became_peered":"2024-09-18T04:59:07.199588+0000","last_unstale":"2024-09-18T04:59:07.202696+0000","last_undegraded":"2024-09-18T04:59:07.202696+0000","last_fullsized":"2024-09-18T04:59:07.202696+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:53:25.971943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012312829999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2677,"num_read_kb":15514,"num_write":503,"num_write_kb":9112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":912,"num_bytes_recovered":7005168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"157'455","reported_seq":3935,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:08.199477+0000","last_change":"2024-09-18T04:59:07.200974+0000","last_active":"2024-09-18T04:59:08.199477+0000","last_peered":"2024-09-18T04:59:08.199477+0000","last_clean":"2024-09-18T04:59:08.199477+0000","last_became_active":"2024-09-18T04:59:07.199276+0000","last_became_peered":"2024-09-18T04:59:07.199276+0000","last_unstale":"2024-09-18T04:59:08.199477+0000","last_undegraded":"2024-09-18T04:59:08.199477+0000","last_fullsized":"2024-09-18T04:59:08.199477+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:42:10.766349+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.000757346,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":2981,"num_read_kb":11740,"num_write":512,"num_write_kb":18285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":519,"num_bytes_recovered":11494100,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"156'520","reported_seq":5473,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198149+0000","last_change":"2024-09-18T04:58:57.148756+0000","last_active":"2024-09-18T04:59:07.198149+0000","last_peered":"2024-09-18T04:59:07.198149+0000","last_clean":"2024-09-18T04:59:07.198149+0000","last_became_active":"2024-09-18T04:58:57.145900+0000","last_became_peered":"2024-09-18T04:58:57.145900+0000","last_unstale":"2024-09-18T04:59:07.198149+0000","last_undegraded":"2024-09-18T04:59:07.198149+0000","last_fullsized":"2024-09-18T04:59:07.198149+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:03:40.179686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018405870000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4543,"num_read_kb":26564,"num_write":538,"num_write_kb":13069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":23811007,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"156'576","reported_seq":3426,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731747+0000","last_change":"2024-09-18T04:58:48.095394+0000","last_active":"2024-09-18T04:59:07.731747+0000","last_peered":"2024-09-18T04:59:07.731747+0000","last_clean":"2024-09-18T04:59:07.731747+0000","last_became_active":"2024-09-18T04:58:32.029074+0000","last_became_peered":"2024-09-18T04:58:32.029074+0000","last_unstale":"2024-09-18T04:59:07.731747+0000","last_undegraded":"2024-09-18T04:59:07.731747+0000","last_fullsized":"2024-09-18T04:59:07.731747+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:15:38.950060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053623899999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3403,"num_read_kb":16958,"num_write":699,"num_write_kb":12986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":4650417,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"153'506","reported_seq":4239,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.197394+0000","last_change":"2024-09-18T04:58:55.128037+0000","last_active":"2024-09-18T04:59:07.197394+0000","last_peered":"2024-09-18T04:59:07.197394+0000","last_clean":"2024-09-18T04:59:07.197394+0000","last_became_active":"2024-09-18T04:58:55.125721+0000","last_became_peered":"2024-09-18T04:58:55.125721+0000","last_unstale":"2024-09-18T04:59:07.197394+0000","last_undegraded":"2024-09-18T04:59:07.197394+0000","last_fullsized":"2024-09-18T04:59:07.197394+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:06:23.656568+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001441276,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4899,"num_read_kb":13699,"num_write":541,"num_write_kb":6077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"154'885","reported_seq":5042,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195463+0000","last_change":"2024-09-18T04:58:48.024658+0000","last_active":"2024-09-18T04:59:07.195463+0000","last_peered":"2024-09-18T04:59:07.195463+0000","last_clean":"2024-09-18T04:59:07.195463+0000","last_became_active":"2024-09-18T04:56:21.131133+0000","last_became_peered":"2024-09-18T04:56:21.131133+0000","last_unstale":"2024-09-18T04:59:07.195463+0000","last_undegraded":"2024-09-18T04:59:07.195463+0000","last_fullsized":"2024-09-18T04:59:07.195463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:01:57.036904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012610480000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6340,"num_read_kb":26579,"num_write":1141,"num_write_kb":12621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"157'637","reported_seq":5292,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198169+0000","last_change":"2024-09-18T04:58:48.028934+0000","last_active":"2024-09-18T04:59:07.198169+0000","last_peered":"2024-09-18T04:59:07.198169+0000","last_clean":"2024-09-18T04:59:07.198169+0000","last_became_active":"2024-09-18T04:58:32.032762+0000","last_became_peered":"2024-09-18T04:58:32.032762+0000","last_unstale":"2024-09-18T04:59:07.198169+0000","last_undegraded":"2024-09-18T04:59:07.198169+0000","last_fullsized":"2024-09-18T04:59:07.198169+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:33:25.010425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066811099999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":5292,"num_read_kb":26013,"num_write":596,"num_write_kb":7300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":447,"num_bytes_recovered":1594801,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"155'502","reported_seq":2817,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-18T04:59:06.190587+0000","last_change":"2024-09-18T04:58:48.023734+0000","last_active":"2024-09-18T04:59:06.190587+0000","last_peered":"2024-09-18T04:59:06.190587+0000","last_clean":"2024-09-18T04:59:06.190587+0000","last_became_active":"2024-09-18T04:58:30.025558+0000","last_became_peered":"2024-09-18T04:58:30.025558+0000","last_unstale":"2024-09-18T04:59:06.190587+0000","last_undegraded":"2024-09-18T04:59:06.190587+0000","last_fullsized":"2024-09-18T04:59:06.190587+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:57:31.113016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087474099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3131,"num_read_kb":27325,"num_write":609,"num_write_kb":11672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":12427258,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":192,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195662+0000","last_change":"2024-09-18T04:58:29.025271+0000","last_active":"2024-09-18T04:59:07.195662+0000","last_peered":"2024-09-18T04:59:07.195662+0000","last_clean":"2024-09-18T04:59:07.195662+0000","last_became_active":"2024-09-18T04:58:29.024911+0000","last_became_peered":"2024-09-18T04:58:29.024911+0000","last_unstale":"2024-09-18T04:59:07.195662+0000","last_undegraded":"2024-09-18T04:59:07.195662+0000","last_fullsized":"2024-09-18T04:59:07.195662+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:53:21.863837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063436799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"155'2480","reported_seq":9899,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195724+0000","last_change":"2024-09-18T04:58:48.024643+0000","last_active":"2024-09-18T04:59:07.195724+0000","last_peered":"2024-09-18T04:59:07.195724+0000","last_clean":"2024-09-18T04:59:07.195724+0000","last_became_active":"2024-09-18T04:58:33.027340+0000","last_became_peered":"2024-09-18T04:58:33.027340+0000","last_unstale":"2024-09-18T04:59:07.195724+0000","last_undegraded":"2024-09-18T04:59:07.195724+0000","last_fullsized":"2024-09-18T04:59:07.195724+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":2480,"log_dups_size":0,"ondisk_log_size":2480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:47:47.826132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000952045,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":18529,"num_read_kb":32990,"num_write":4379,"num_write_kb":20206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":5439803,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":176,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195585+0000","last_change":"2024-09-18T04:56:12.389971+0000","last_active":"2024-09-18T04:59:07.195585+0000","last_peered":"2024-09-18T04:59:07.195585+0000","last_clean":"2024-09-18T04:59:07.195585+0000","last_became_active":"2024-09-18T04:56:10.412595+0000","last_became_peered":"2024-09-18T04:56:10.412595+0000","last_unstale":"2024-09-18T04:59:07.195585+0000","last_undegraded":"2024-09-18T04:59:07.195585+0000","last_fullsized":"2024-09-18T04:59:07.195585+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:45.638420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038576799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"157'807","reported_seq":7861,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198219+0000","last_change":"2024-09-18T04:58:57.151001+0000","last_active":"2024-09-18T04:59:07.198219+0000","last_peered":"2024-09-18T04:59:07.198219+0000","last_clean":"2024-09-18T04:59:07.198219+0000","last_became_active":"2024-09-18T04:58:57.147959+0000","last_became_peered":"2024-09-18T04:58:57.147959+0000","last_unstale":"2024-09-18T04:59:07.198219+0000","last_undegraded":"2024-09-18T04:59:07.198219+0000","last_fullsized":"2024-09-18T04:59:07.198219+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:30:19.725543+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00065254400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8940,"num_read_kb":28417,"num_write":1051,"num_write_kb":11406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":5706670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198307+0000","last_change":"2024-09-18T04:58:57.150288+0000","last_active":"2024-09-18T04:59:07.198307+0000","last_peered":"2024-09-18T04:59:07.198307+0000","last_clean":"2024-09-18T04:59:07.198307+0000","last_became_active":"2024-09-18T04:58:57.149907+0000","last_became_peered":"2024-09-18T04:58:57.149907+0000","last_unstale":"2024-09-18T04:59:07.198307+0000","last_undegraded":"2024-09-18T04:59:07.198307+0000","last_fullsized":"2024-09-18T04:59:07.198307+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:12:39.499969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"157'757","reported_seq":6181,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731891+0000","last_change":"2024-09-18T04:58:48.096507+0000","last_active":"2024-09-18T04:59:07.731891+0000","last_peered":"2024-09-18T04:59:07.731891+0000","last_clean":"2024-09-18T04:59:07.731891+0000","last_became_active":"2024-09-18T04:58:33.026806+0000","last_became_peered":"2024-09-18T04:58:33.026806+0000","last_unstale":"2024-09-18T04:59:07.731891+0000","last_undegraded":"2024-09-18T04:59:07.731891+0000","last_fullsized":"2024-09-18T04:59:07.731891+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:59:41.673907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013736829999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6896,"num_read_kb":21261,"num_write":943,"num_write_kb":5208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":571544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":145,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195438+0000","last_change":"2024-09-18T04:56:46.254241+0000","last_active":"2024-09-18T04:59:07.195438+0000","last_peered":"2024-09-18T04:59:07.195438+0000","last_clean":"2024-09-18T04:59:07.195438+0000","last_became_active":"2024-09-18T04:56:46.253894+0000","last_became_peered":"2024-09-18T04:56:46.253894+0000","last_unstale":"2024-09-18T04:59:07.195438+0000","last_undegraded":"2024-09-18T04:59:07.195438+0000","last_fullsized":"2024-09-18T04:59:07.195438+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:58:42.098099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045144100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"158'505","reported_seq":5028,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.197318+0000","last_change":"2024-09-18T04:58:48.061503+0000","last_active":"2024-09-18T04:59:07.197318+0000","last_peered":"2024-09-18T04:59:07.197318+0000","last_clean":"2024-09-18T04:59:07.197318+0000","last_became_active":"2024-09-18T04:58:41.078435+0000","last_became_peered":"2024-09-18T04:58:41.078435+0000","last_unstale":"2024-09-18T04:59:07.197318+0000","last_undegraded":"2024-09-18T04:59:07.197318+0000","last_fullsized":"2024-09-18T04:59:07.197318+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:34:50.214360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056043399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4221,"num_read_kb":33044,"num_write":472,"num_write_kb":37350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":25349011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"155'485","reported_seq":9536,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.201259+0000","last_change":"2024-09-18T04:59:07.200564+0000","last_active":"2024-09-18T04:59:07.201259+0000","last_peered":"2024-09-18T04:59:07.201259+0000","last_clean":"2024-09-18T04:59:07.201259+0000","last_became_active":"2024-09-18T04:59:07.199056+0000","last_became_peered":"2024-09-18T04:59:07.199056+0000","last_unstale":"2024-09-18T04:59:07.201259+0000","last_undegraded":"2024-09-18T04:59:07.201259+0000","last_fullsized":"2024-09-18T04:59:07.201259+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:30:30.980376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049796699999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":9360,"num_read_kb":28051,"num_write":499,"num_write_kb":12488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":472,"num_bytes_recovered":11362815,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":362,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198082+0000","last_change":"2024-09-18T04:58:38.025236+0000","last_active":"2024-09-18T04:59:07.198082+0000","last_peered":"2024-09-18T04:59:07.198082+0000","last_clean":"2024-09-18T04:59:07.198082+0000","last_became_active":"2024-09-18T04:58:38.024740+0000","last_became_peered":"2024-09-18T04:58:38.024740+0000","last_unstale":"2024-09-18T04:59:07.198082+0000","last_undegraded":"2024-09-18T04:59:07.198082+0000","last_fullsized":"2024-09-18T04:59:07.198082+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_clean_scrub_stamp":"2024-09-18T04:56:10.373785+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:10:45.251654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":37,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198355+0000","last_change":"2024-09-18T04:58:57.148289+0000","last_active":"2024-09-18T04:59:07.198355+0000","last_peered":"2024-09-18T04:59:07.198355+0000","last_clean":"2024-09-18T04:59:07.198355+0000","last_became_active":"2024-09-18T04:58:57.147915+0000","last_became_peered":"2024-09-18T04:58:57.147915+0000","last_unstale":"2024-09-18T04:59:07.198355+0000","last_undegraded":"2024-09-18T04:59:07.198355+0000","last_fullsized":"2024-09-18T04:59:07.198355+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:31:21.699401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"157'811","reported_seq":13428,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198153+0000","last_change":"2024-09-18T04:58:57.152156+0000","last_active":"2024-09-18T04:59:07.198153+0000","last_peered":"2024-09-18T04:59:07.198153+0000","last_clean":"2024-09-18T04:59:07.198153+0000","last_became_active":"2024-09-18T04:58:57.148221+0000","last_became_peered":"2024-09-18T04:58:57.148221+0000","last_unstale":"2024-09-18T04:59:07.198153+0000","last_undegraded":"2024-09-18T04:59:07.198153+0000","last_fullsized":"2024-09-18T04:59:07.198153+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:45:57.434891+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0033124909999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":14837,"num_read_kb":27169,"num_write":1047,"num_write_kb":7785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":1280827,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":178,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198076+0000","last_change":"2024-09-18T04:56:12.398631+0000","last_active":"2024-09-18T04:59:07.198076+0000","last_peered":"2024-09-18T04:59:07.198076+0000","last_clean":"2024-09-18T04:59:07.198076+0000","last_became_active":"2024-09-18T04:56:10.412614+0000","last_became_peered":"2024-09-18T04:56:10.412614+0000","last_unstale":"2024-09-18T04:59:07.198076+0000","last_undegraded":"2024-09-18T04:59:07.198076+0000","last_fullsized":"2024-09-18T04:59:07.198076+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:44:13.401320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075126299999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"158'601","reported_seq":2977,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-18T04:59:06.191350+0000","last_change":"2024-09-18T04:58:48.024976+0000","last_active":"2024-09-18T04:59:06.191350+0000","last_peered":"2024-09-18T04:59:06.191350+0000","last_clean":"2024-09-18T04:59:06.191350+0000","last_became_active":"2024-09-18T04:58:30.027757+0000","last_became_peered":"2024-09-18T04:58:30.027757+0000","last_unstale":"2024-09-18T04:59:06.191350+0000","last_undegraded":"2024-09-18T04:59:06.191350+0000","last_fullsized":"2024-09-18T04:59:06.191350+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:29:00.106938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014951890000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":2216,"num_read_kb":23967,"num_write":593,"num_write_kb":16097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":10562840,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":176,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195410+0000","last_change":"2024-09-18T04:56:12.390044+0000","last_active":"2024-09-18T04:59:07.195410+0000","last_peered":"2024-09-18T04:59:07.195410+0000","last_clean":"2024-09-18T04:59:07.195410+0000","last_became_active":"2024-09-18T04:56:10.413042+0000","last_became_peered":"2024-09-18T04:56:10.413042+0000","last_unstale":"2024-09-18T04:59:07.195410+0000","last_undegraded":"2024-09-18T04:59:07.195410+0000","last_fullsized":"2024-09-18T04:59:07.195410+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:46:11.074243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"157'525","reported_seq":4818,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731809+0000","last_change":"2024-09-18T04:58:48.095298+0000","last_active":"2024-09-18T04:59:07.731809+0000","last_peered":"2024-09-18T04:59:07.731809+0000","last_clean":"2024-09-18T04:59:07.731809+0000","last_became_active":"2024-09-18T04:58:32.027104+0000","last_became_peered":"2024-09-18T04:58:32.027104+0000","last_unstale":"2024-09-18T04:59:07.731809+0000","last_undegraded":"2024-09-18T04:59:07.731809+0000","last_fullsized":"2024-09-18T04:59:07.731809+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:41:37.134684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056931099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5904,"num_read_kb":13840,"num_write":503,"num_write_kb":5042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":2187784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198027+0000","last_change":"2024-09-18T04:57:40.472681+0000","last_active":"2024-09-18T04:59:07.198027+0000","last_peered":"2024-09-18T04:59:07.198027+0000","last_clean":"2024-09-18T04:59:07.198027+0000","last_became_active":"2024-09-18T04:57:40.472217+0000","last_became_peered":"2024-09-18T04:57:40.472217+0000","last_unstale":"2024-09-18T04:59:07.198027+0000","last_undegraded":"2024-09-18T04:59:07.198027+0000","last_fullsized":"2024-09-18T04:59:07.198027+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:27:11.849311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"157'482","reported_seq":3615,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731740+0000","last_change":"2024-09-18T04:58:48.096157+0000","last_active":"2024-09-18T04:59:07.731740+0000","last_peered":"2024-09-18T04:59:07.731740+0000","last_clean":"2024-09-18T04:59:07.731740+0000","last_became_active":"2024-09-18T04:58:38.024916+0000","last_became_peered":"2024-09-18T04:58:38.024916+0000","last_unstale":"2024-09-18T04:59:07.731740+0000","last_undegraded":"2024-09-18T04:59:07.731740+0000","last_fullsized":"2024-09-18T04:59:07.731740+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:32:42.202542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001087188,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2758,"num_read_kb":7938,"num_write":473,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":458,"num_bytes_recovered":2219067,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"155'714","reported_seq":12009,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-18T04:59:06.191085+0000","last_change":"2024-09-18T04:58:48.024809+0000","last_active":"2024-09-18T04:59:06.191085+0000","last_peered":"2024-09-18T04:59:06.191085+0000","last_clean":"2024-09-18T04:59:06.191085+0000","last_became_active":"2024-09-18T04:58:26.101490+0000","last_became_peered":"2024-09-18T04:58:26.101490+0000","last_unstale":"2024-09-18T04:59:06.191085+0000","last_undegraded":"2024-09-18T04:59:06.191085+0000","last_fullsized":"2024-09-18T04:59:06.191085+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:30:15.538062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015101680000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":12956,"num_read_kb":26590,"num_write":848,"num_write_kb":15777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":5822953,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"157'470","reported_seq":6073,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195769+0000","last_change":"2024-09-18T04:58:48.136195+0000","last_active":"2024-09-18T04:59:07.195769+0000","last_peered":"2024-09-18T04:59:07.195769+0000","last_clean":"2024-09-18T04:59:07.195769+0000","last_became_active":"2024-09-18T04:58:27.115137+0000","last_became_peered":"2024-09-18T04:58:27.115137+0000","last_unstale":"2024-09-18T04:59:07.195769+0000","last_undegraded":"2024-09-18T04:59:07.195769+0000","last_fullsized":"2024-09-18T04:59:07.195769+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:57:42.359624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039325500000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":4725,"num_read_kb":14773,"num_write":478,"num_write_kb":8139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":6055124,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"155'1400","reported_seq":5772,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731780+0000","last_change":"2024-09-18T04:58:48.095758+0000","last_active":"2024-09-18T04:59:07.731780+0000","last_peered":"2024-09-18T04:59:07.731780+0000","last_clean":"2024-09-18T04:59:07.731780+0000","last_became_active":"2024-09-18T04:58:31.026041+0000","last_became_peered":"2024-09-18T04:58:31.026041+0000","last_unstale":"2024-09-18T04:59:07.731780+0000","last_undegraded":"2024-09-18T04:59:07.731780+0000","last_fullsized":"2024-09-18T04:59:07.731780+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":1400,"log_dups_size":0,"ondisk_log_size":1400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:55:29.526466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083671099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8851,"num_read_kb":15183,"num_write":2049,"num_write_kb":8862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":1966128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"155'908","reported_seq":7203,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195547+0000","last_change":"2024-09-18T04:58:48.023609+0000","last_active":"2024-09-18T04:59:07.195547+0000","last_peered":"2024-09-18T04:59:07.195547+0000","last_clean":"2024-09-18T04:59:07.195547+0000","last_became_active":"2024-09-18T04:58:33.027525+0000","last_became_peered":"2024-09-18T04:58:33.027525+0000","last_unstale":"2024-09-18T04:59:07.195547+0000","last_undegraded":"2024-09-18T04:59:07.195547+0000","last_fullsized":"2024-09-18T04:59:07.195547+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:27:34.074204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062770000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9983,"num_read_kb":24174,"num_write":928,"num_write_kb":8107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":86198,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"155'617","reported_seq":7056,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.194912+0000","last_change":"2024-09-18T04:58:48.021010+0000","last_active":"2024-09-18T04:59:07.194912+0000","last_peered":"2024-09-18T04:59:07.194912+0000","last_clean":"2024-09-18T04:59:07.194912+0000","last_became_active":"2024-09-18T04:58:07.017827+0000","last_became_peered":"2024-09-18T04:58:07.017827+0000","last_unstale":"2024-09-18T04:59:07.194912+0000","last_undegraded":"2024-09-18T04:59:07.194912+0000","last_fullsized":"2024-09-18T04:59:07.194912+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:42:31.897588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050117499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8566,"num_read_kb":18769,"num_write":595,"num_write_kb":12467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":2872264,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"155'1050","reported_seq":5261,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198243+0000","last_change":"2024-09-18T04:58:48.029466+0000","last_active":"2024-09-18T04:59:07.198243+0000","last_peered":"2024-09-18T04:59:07.198243+0000","last_clean":"2024-09-18T04:59:07.198243+0000","last_became_active":"2024-09-18T04:58:31.028189+0000","last_became_peered":"2024-09-18T04:58:31.028189+0000","last_unstale":"2024-09-18T04:59:07.198243+0000","last_undegraded":"2024-09-18T04:59:07.198243+0000","last_fullsized":"2024-09-18T04:59:07.198243+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":1050,"log_dups_size":0,"ondisk_log_size":1050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:26:10.430573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001035219,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7660,"num_read_kb":17955,"num_write":1663,"num_write_kb":5543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":2749742,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"157'556","reported_seq":5095,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198207+0000","last_change":"2024-09-18T04:58:48.029669+0000","last_active":"2024-09-18T04:59:07.198207+0000","last_peered":"2024-09-18T04:59:07.198207+0000","last_clean":"2024-09-18T04:59:07.198207+0000","last_became_active":"2024-09-18T04:58:29.029625+0000","last_became_peered":"2024-09-18T04:58:29.029625+0000","last_unstale":"2024-09-18T04:59:07.198207+0000","last_undegraded":"2024-09-18T04:59:07.198207+0000","last_fullsized":"2024-09-18T04:59:07.198207+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:51:09.257320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011607150000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4690,"num_read_kb":26470,"num_write":636,"num_write_kb":8108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":5504104,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"157'809","reported_seq":4779,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195411+0000","last_change":"2024-09-18T04:58:48.136237+0000","last_active":"2024-09-18T04:59:07.195411+0000","last_peered":"2024-09-18T04:59:07.195411+0000","last_clean":"2024-09-18T04:59:07.195411+0000","last_became_active":"2024-09-18T04:56:21.131202+0000","last_became_peered":"2024-09-18T04:56:21.131202+0000","last_unstale":"2024-09-18T04:59:07.195411+0000","last_undegraded":"2024-09-18T04:59:07.195411+0000","last_fullsized":"2024-09-18T04:59:07.195411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:17:35.438723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041769600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5877,"num_read_kb":14669,"num_write":993,"num_write_kb":10354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"157'920","reported_seq":8474,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.202504+0000","last_change":"2024-09-18T04:59:07.201833+0000","last_active":"2024-09-18T04:59:07.202504+0000","last_peered":"2024-09-18T04:59:07.202504+0000","last_clean":"2024-09-18T04:59:07.202504+0000","last_became_active":"2024-09-18T04:59:07.199844+0000","last_became_peered":"2024-09-18T04:59:07.199844+0000","last_unstale":"2024-09-18T04:59:07.202504+0000","last_undegraded":"2024-09-18T04:59:07.202504+0000","last_fullsized":"2024-09-18T04:59:07.202504+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":920,"log_dups_size":0,"ondisk_log_size":920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:40:08.684972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087268100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10766,"num_read_kb":19560,"num_write":908,"num_write_kb":18676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":9230703,"num_keys_recovered":59,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"144'809","reported_seq":5825,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.197043+0000","last_change":"2024-09-18T04:58:55.130930+0000","last_active":"2024-09-18T04:59:07.197043+0000","last_peered":"2024-09-18T04:59:07.197043+0000","last_clean":"2024-09-18T04:59:07.197043+0000","last_became_active":"2024-09-18T04:58:55.129007+0000","last_became_peered":"2024-09-18T04:58:55.129007+0000","last_unstale":"2024-09-18T04:59:07.197043+0000","last_undegraded":"2024-09-18T04:59:07.197043+0000","last_fullsized":"2024-09-18T04:59:07.197043+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:21:11.287190+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.00099438999999999994,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7388,"num_read_kb":33089,"num_write":1110,"num_write_kb":12855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"158'781","reported_seq":2932,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198095+0000","last_change":"2024-09-18T04:58:48.030265+0000","last_active":"2024-09-18T04:59:07.198095+0000","last_peered":"2024-09-18T04:59:07.198095+0000","last_clean":"2024-09-18T04:59:07.198095+0000","last_became_active":"2024-09-18T04:58:31.028633+0000","last_became_peered":"2024-09-18T04:58:31.028633+0000","last_unstale":"2024-09-18T04:59:07.198095+0000","last_undegraded":"2024-09-18T04:59:07.198095+0000","last_fullsized":"2024-09-18T04:59:07.198095+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:04:25.747259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00140489,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4341,"num_read_kb":14774,"num_write":882,"num_write_kb":9994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":12642724,"num_keys_recovered":71,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":47,"num_read":220784,"num_read_kb":683202,"num_write":29896,"num_write_kb":377778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9208,"num_bytes_recovered":184551668,"num_keys_recovered":562,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23964,"ondisk_log_size":23964,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28167,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51292,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320548,"statfs":{"total":96636764160,"available":96584241152,"internally_reserved":0,"allocated":2170880,"data_stored":1617954,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":50330059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.45600000000000002,"15min":0.46400000000000002},"min":{"1min":0.23799999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53100000000000003,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.437,"5min":0.41499999999999998,"15min":0.41099999999999998},"min":{"1min":0.312,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.60299999999999998}]},{"osd":1,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.46899999999999997,"15min":0.47799999999999998},"min":{"1min":0.249,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.55100000000000005,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.436,"15min":0.434},"min":{"1min":0.28399999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.621}]},{"osd":2,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.44700000000000001,"15min":0.44500000000000001},"min":{"1min":0.35399999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45200000000000001,"15min":0.45000000000000001},"min":{"1min":0.34200000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.56200000000000006,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.67500000000000004}]},{"osd":3,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52600000000000002,"15min":0.51900000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.4399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.34100000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.4809999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":0.72799999999999998}]},{"osd":4,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.42199999999999999,"15min":0.43099999999999999},"min":{"1min":0.26500000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.63700000000000001,"5min":1.954,"15min":1.954},"last":0.502},{"interface":"front","average":{"1min":0.375,"5min":0.41699999999999998,"15min":0.42399999999999999},"min":{"1min":0.246,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.629,"5min":1.478,"15min":1.478},"last":0.45800000000000002}]},{"osd":5,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.50800000000000001,"15min":0.52200000000000002},"min":{"1min":0.30299999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.71799999999999997,"5min":3.145,"15min":3.145},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.443,"15min":0.441},"min":{"1min":0.27700000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.65800000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.68999999999999995}]},{"osd":6,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.47899999999999998,"15min":0.48299999999999998},"min":{"1min":0.33300000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.69299999999999995,"5min":0.746,"15min":0.746},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.48899999999999999,"15min":0.49299999999999999},"min":{"1min":0.35199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.623,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.71399999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55336,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316504,"statfs":{"total":96636764160,"available":96580100096,"internally_reserved":0,"allocated":2121728,"data_stored":1032642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.71899999999999997,"15min":0.76900000000000002},"min":{"1min":0.23899999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.56799999999999995,"5min":6.1319999999999997,"15min":6.1319999999999997},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.75700000000000001,"15min":0.80500000000000005},"min":{"1min":0.35799999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.81100000000000005,"5min":6.1429999999999998,"15min":6.1429999999999998},"last":0.41099999999999998}]},{"osd":1,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.70999999999999996,"15min":0.76200000000000001},"min":{"1min":0.29299999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.44900000000000001,"5min":6.0410000000000004,"15min":6.0410000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.70899999999999996,"15min":0.753},"min":{"1min":0.33000000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.83599999999999997,"5min":5.5380000000000003,"15min":5.5380000000000003},"last":0.34599999999999997}]},{"osd":2,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.41999999999999998,"15min":0.42599999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.54400000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.315},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.46100000000000002,"15min":0.46800000000000003},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.55900000000000005,"5min":1.04,"15min":1.04},"last":0.33400000000000002}]},{"osd":3,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46000000000000002,"15min":0.46100000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.378},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.47799999999999998,"15min":0.48099999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.625,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.38700000000000001}]},{"osd":4,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.36799999999999999,"15min":0.374},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.47599999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.20799999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.43099999999999999,"15min":0.437},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.30399999999999999}]},{"osd":5,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.372,"15min":0.378},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.42499999999999999,"15min":0.435},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.53000000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.28699999999999998}]},{"osd":7,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.46800000000000003,"15min":0.46600000000000003},"min":{"1min":0.28100000000000003,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.48599999999999999,"15min":0.47699999999999998},"min":{"1min":0.34399999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.35599999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65188,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306652,"statfs":{"total":96636764160,"available":96570011648,"internally_reserved":0,"allocated":2121728,"data_stored":1033583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.54600000000000004,"15min":0.57099999999999995},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.59199999999999997,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.504,"15min":0.51000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71599999999999997,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.55700000000000005}]},{"osd":1,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.53300000000000003,"15min":0.55400000000000005},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.55600000000000005,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.32500000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63800000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.57499999999999996}]},{"osd":2,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47699999999999998,"15min":0.47899999999999998},"min":{"1min":0.28699999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.629,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.57399999999999995,"15min":0.59999999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.626,"5min":3.1070000000000002,"15min":3.1070000000000002},"last":0.504}]},{"osd":3,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.496,"15min":0.502},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.77000000000000002,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.504,"15min":0.50800000000000001},"min":{"1min":0.35099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.63,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.54600000000000004}]},{"osd":4,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.46800000000000003,"15min":0.47399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.59499999999999997,"5min":1.599,"15min":1.599},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.56299999999999994,"15min":0.59299999999999997},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.53500000000000003,"5min":3.379,"15min":3.379},"last":0.432}]},{"osd":6,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.45500000000000002,"15min":0.46600000000000003},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54200000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.56200000000000006,"15min":0.58499999999999996},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.085,"5min":3.1669999999999998,"15min":3.1669999999999998},"last":0.378}]},{"osd":7,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.48399999999999999,"15min":0.495},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80200000000000005,"5min":1.224,"15min":1.224},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.54200000000000004,"15min":0.55900000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.58599999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.45700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63512,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94308328,"statfs":{"total":96636764160,"available":96571727872,"internally_reserved":0,"allocated":2109440,"data_stored":1027567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.46000000000000002,"15min":0.44400000000000001},"min":{"1min":0.23499999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":3.7069999999999999,"5min":3.7069999999999999,"15min":3.7069999999999999},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.46100000000000002,"15min":0.441},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":3.3700000000000001,"5min":3.3700000000000001,"15min":3.3700000000000001},"last":0.219}]},{"osd":1,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.47099999999999997,"15min":0.45500000000000002},"min":{"1min":0.28199999999999997,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":3.2879999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.375},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.46600000000000003,"15min":0.44600000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.661,"5min":3.661,"15min":3.661},"last":0.38200000000000001}]},{"osd":2,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.52200000000000002,"15min":0.50600000000000001},"min":{"1min":0.29999999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.1890000000000001,"5min":3.1890000000000001,"15min":3.1890000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.51600000000000001,"15min":0.502},"min":{"1min":0.32300000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.3860000000000001,"5min":3.3860000000000001,"15min":3.3860000000000001},"last":0.35899999999999999}]},{"osd":3,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.504,"15min":0.48899999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":3.4500000000000002,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.48299999999999998,"15min":0.46100000000000002},"min":{"1min":0.372,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.3210000000000002,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.433}]},{"osd":5,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.42599999999999999,"15min":0.40200000000000002},"min":{"1min":0.25,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":3.9319999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.43099999999999999,"15min":0.40699999999999997},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":3.9449999999999998,"5min":3.9449999999999998,"15min":3.9449999999999998},"last":0.39700000000000002}]},{"osd":6,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.45100000000000001,"15min":0.42899999999999999},"min":{"1min":0.29299999999999998,"5min":0.251,"15min":0.251},"max":{"1min":3.3530000000000002,"5min":3.3530000000000002,"15min":3.3530000000000002},"last":0.312},{"interface":"front","average":{"1min":1.29,"5min":0.626,"15min":0.51600000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":17.422999999999998,"5min":17.422999999999998,"15min":17.422999999999998},"last":0.33500000000000002}]},{"osd":7,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.48399999999999999,"15min":0.46400000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":3.7330000000000001,"5min":3.7330000000000001,"15min":3.7330000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.52100000000000002,"15min":0.499},"min":{"1min":0.33300000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":3.6930000000000001,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":0.40500000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87320,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284520,"statfs":{"total":96636764160,"available":96547348480,"internally_reserved":0,"allocated":2109440,"data_stored":1027567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":87292360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.39500000000000002,"15min":0.40500000000000003},"min":{"1min":0.23899999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.45200000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.51800000000000002,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.38400000000000001}]},{"osd":1,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.376,"15min":0.375},"min":{"1min":0.249,"5min":0.245,"15min":0.245},"max":{"1min":0.51800000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.37},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.40500000000000003,"15min":0.41499999999999998},"min":{"1min":0.249,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.54400000000000004,"5min":0.755,"15min":0.755},"last":0.438}]},{"osd":2,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.41599999999999998,"15min":0.41999999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.56999999999999995,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.45200000000000001,"15min":0.45300000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.59399999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.47299999999999998}]},{"osd":4,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.52000000000000002,"15min":0.52900000000000003},"min":{"1min":0.31,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71999999999999997,"5min":0.755,"15min":0.755},"last":0.505},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.76100000000000001,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.63900000000000001}]},{"osd":5,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.55900000000000005,"15min":0.56899999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68600000000000005,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.66900000000000004,"5min":0.874,"15min":0.874},"last":0.60399999999999998}]},{"osd":6,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58699999999999997,"15min":0.59599999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80400000000000005,"5min":0.874,"15min":0.874},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72399999999999998,"5min":1.095,"15min":1.095},"last":0.51600000000000001}]},{"osd":7,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.65600000000000003,"15min":0.67200000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84499999999999997,"5min":1.335,"15min":1.335},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.61599999999999999,"15min":0.624},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.753,"5min":1.087,"15min":1.087},"last":0.58999999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37292,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":2121728,"data_stored":1033583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.33700000000000002,"15min":0.33600000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.39600000000000002,"15min":0.39800000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.63800000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.35899999999999999}]},{"osd":1,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.56999999999999995,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.36299999999999999,"15min":0.36099999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.29099999999999998}]},{"osd":3,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.40799999999999997,"15min":0.40400000000000003},"min":{"1min":0.26200000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.40999999999999998,"15min":0.40400000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.373}]},{"osd":4,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.58899999999999997,"15min":0.57899999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.4220000000000002,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":0.59999999999999998}]},{"osd":5,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.61199999999999999,"15min":0.56399999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":7.4480000000000004,"5min":7.4480000000000004,"15min":7.4480000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54800000000000004,"15min":0.54700000000000004},"min":{"1min":0.377,"5min":0.375,"15min":0.375},"max":{"1min":0.73699999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.64800000000000002}]},{"osd":6,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.84599999999999997,"5min":1.171,"15min":1.171},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.57799999999999996}]},{"osd":7,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80800000000000005,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":1.1719999999999999,"5min":0.69999999999999996,"15min":0.622},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":11.138,"5min":11.138,"15min":11.138},"last":0.61399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29124,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342716,"statfs":{"total":96636764160,"available":96606941184,"internally_reserved":0,"allocated":2154496,"data_stored":1617954,"data_compressed":9734,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.34999999999999998,"15min":0.35799999999999998},"min":{"1min":0.20899999999999999,"5min":0.185,"15min":0.185},"max":{"1min":0.622,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.63100000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":1.6579999999999999}]},{"osd":2,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.371,"15min":0.373},"min":{"1min":0.222,"5min":0.221,"15min":0.221},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":1.7270000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41599999999999998,"15min":0.41399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.6839999999999999}]},{"osd":3,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.38800000000000001,"15min":0.38400000000000001},"min":{"1min":0.23899999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":1.754},{"interface":"front","average":{"1min":0.434,"5min":0.40600000000000003,"15min":0.40100000000000002},"min":{"1min":0.29599999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":1.71}]},{"osd":4,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.50800000000000001,"15min":0.51700000000000002},"min":{"1min":0.32400000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.77000000000000002,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.496,"5min":0.50700000000000001,"15min":0.50900000000000001},"min":{"1min":0.315,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.86099999999999999,"5min":0.999,"15min":0.999},"last":0.51500000000000001}]},{"osd":5,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50900000000000001,"15min":0.51300000000000001},"min":{"1min":0.33600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80400000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":1.625},{"interface":"front","average":{"1min":0.5,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.373,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.83899999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":1.597}]},{"osd":6,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82799999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.74299999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67400000000000004}]},{"osd":7,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.56999999999999995,"15min":0.57799999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78200000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.40999999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0309999999999999,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.70299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51228,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320612,"statfs":{"total":96636764160,"available":96584306688,"internally_reserved":0,"allocated":2109440,"data_stored":1026626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50330055},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35999999999999999,"15min":0.36299999999999999},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.56799999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.247},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.34799999999999998,"15min":0.34599999999999997},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.53700000000000003,"5min":0.621,"15min":0.621},"last":0.28399999999999997}]},{"osd":2,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.39900000000000002,"15min":0.40100000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.42599999999999999,"15min":0.42399999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.46200000000000002}]},{"osd":3,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.434,"15min":0.42399999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.40400000000000003,"15min":0.39700000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.5}]},{"osd":4,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.63400000000000001,"15min":0.65600000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.89400000000000002,"5min":3.4710000000000001,"15min":3.4710000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.504,"15min":0.502},"min":{"1min":0.35899999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.376}]},{"osd":5,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.81699999999999995,"15min":0.84099999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":4.2640000000000002,"5min":4.2640000000000002,"15min":4.2640000000000002},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.54700000000000004}]},{"osd":6,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.66000000000000003,"15min":0.68000000000000005},"min":{"1min":0.39500000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81699999999999995,"5min":3.536,"15min":3.536},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57099999999999995,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.57999999999999996}]},{"osd":7,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55200000000000005,"15min":0.54700000000000004},"min":{"1min":0.41999999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.68400000000000005,"15min":0.70599999999999996},"min":{"1min":0.39100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.85699999999999998,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.48099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9734,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9734,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T04:59:14.711 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T04:59:14.711 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T04:59:14.998 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:59:14.999 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-18T04:59:15.009 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-09-18T04:59:14.300152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":47,"num_read":220900,"num_read_kb":683302,"num_write":30057,"num_write_kb":380196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9215,"num_bytes_recovered":185732404,"num_keys_recovered":562,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24084,"ondisk_log_size":24084,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":86,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":440292,"kb_used_data":16620,"kb_used_omap":12,"kb_used_meta":423539,"kb_avail":754534428,"statfs":{"total":773094113280,"available":772643254272,"internally_reserved":0,"allocated":17018880,"data_stored":9417476,"data_compressed":18433,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":12742,"internal_metadata":433704506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":524,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002051"},"pg_stats":[{"pgid":"3.1f","version":"158'766","reported_seq":5157,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.197217+0000","last_change":"2024-09-18T04:58:55.128107+0000","last_active":"2024-09-18T04:59:07.197217+0000","last_peered":"2024-09-18T04:59:07.197217+0000","last_clean":"2024-09-18T04:59:07.197217+0000","last_became_active":"2024-09-18T04:58:55.126111+0000","last_became_peered":"2024-09-18T04:58:55.126111+0000","last_unstale":"2024-09-18T04:59:07.197217+0000","last_undegraded":"2024-09-18T04:59:07.197217+0000","last_fullsized":"2024-09-18T04:59:07.197217+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:56:05.136229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001156916,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6348,"num_read_kb":20852,"num_write":870,"num_write_kb":9417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"157'817","reported_seq":2783,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731844+0000","last_change":"2024-09-18T04:58:48.096250+0000","last_active":"2024-09-18T04:59:07.731844+0000","last_peered":"2024-09-18T04:59:07.731844+0000","last_clean":"2024-09-18T04:59:07.731844+0000","last_became_active":"2024-09-18T04:58:32.029310+0000","last_became_peered":"2024-09-18T04:58:32.029310+0000","last_unstale":"2024-09-18T04:59:07.731844+0000","last_undegraded":"2024-09-18T04:59:07.731844+0000","last_fullsized":"2024-09-18T04:59:07.731844+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:13:02.688292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001168749,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3351,"num_read_kb":18023,"num_write":1120,"num_write_kb":8161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":3252478,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"156'500","reported_seq":10759,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195673+0000","last_change":"2024-09-18T04:58:48.023738+0000","last_active":"2024-09-18T04:59:07.195673+0000","last_peered":"2024-09-18T04:59:07.195673+0000","last_clean":"2024-09-18T04:59:07.195673+0000","last_became_active":"2024-09-18T04:58:27.115104+0000","last_became_peered":"2024-09-18T04:58:27.115104+0000","last_unstale":"2024-09-18T04:59:07.195673+0000","last_undegraded":"2024-09-18T04:59:07.195673+0000","last_fullsized":"2024-09-18T04:59:07.195673+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:33:08.572507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082741200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9761,"num_read_kb":28168,"num_write":548,"num_write_kb":23097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7293282,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"158'840","reported_seq":7984,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195584+0000","last_change":"2024-09-18T04:58:48.025083+0000","last_active":"2024-09-18T04:59:07.195584+0000","last_peered":"2024-09-18T04:59:07.195584+0000","last_clean":"2024-09-18T04:59:07.195584+0000","last_became_active":"2024-09-18T04:58:28.028905+0000","last_became_peered":"2024-09-18T04:58:28.028905+0000","last_unstale":"2024-09-18T04:59:07.195584+0000","last_undegraded":"2024-09-18T04:59:07.195584+0000","last_fullsized":"2024-09-18T04:59:07.195584+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:15:17.569793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014698000000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":8594,"num_read_kb":9084,"num_write":1169,"num_write_kb":5309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":1412856,"num_keys_recovered":106,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"155'473","reported_seq":4267,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.202696+0000","last_change":"2024-09-18T04:59:07.202189+0000","last_active":"2024-09-18T04:59:07.202696+0000","last_peered":"2024-09-18T04:59:07.202696+0000","last_clean":"2024-09-18T04:59:07.202696+0000","last_became_active":"2024-09-18T04:59:07.199588+0000","last_became_peered":"2024-09-18T04:59:07.199588+0000","last_unstale":"2024-09-18T04:59:07.202696+0000","last_undegraded":"2024-09-18T04:59:07.202696+0000","last_fullsized":"2024-09-18T04:59:07.202696+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:53:25.971943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012312829999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2677,"num_read_kb":15514,"num_write":503,"num_write_kb":9112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":912,"num_bytes_recovered":7005168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"157'455","reported_seq":3935,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:08.199477+0000","last_change":"2024-09-18T04:59:07.200974+0000","last_active":"2024-09-18T04:59:08.199477+0000","last_peered":"2024-09-18T04:59:08.199477+0000","last_clean":"2024-09-18T04:59:08.199477+0000","last_became_active":"2024-09-18T04:59:07.199276+0000","last_became_peered":"2024-09-18T04:59:07.199276+0000","last_unstale":"2024-09-18T04:59:08.199477+0000","last_undegraded":"2024-09-18T04:59:08.199477+0000","last_fullsized":"2024-09-18T04:59:08.199477+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:42:10.766349+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.000757346,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":2981,"num_read_kb":11740,"num_write":512,"num_write_kb":18285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":519,"num_bytes_recovered":11494100,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"156'520","reported_seq":5473,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198149+0000","last_change":"2024-09-18T04:58:57.148756+0000","last_active":"2024-09-18T04:59:07.198149+0000","last_peered":"2024-09-18T04:59:07.198149+0000","last_clean":"2024-09-18T04:59:07.198149+0000","last_became_active":"2024-09-18T04:58:57.145900+0000","last_became_peered":"2024-09-18T04:58:57.145900+0000","last_unstale":"2024-09-18T04:59:07.198149+0000","last_undegraded":"2024-09-18T04:59:07.198149+0000","last_fullsized":"2024-09-18T04:59:07.198149+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:03:40.179686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018405870000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4543,"num_read_kb":26564,"num_write":538,"num_write_kb":13069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":23811007,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"156'576","reported_seq":3426,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731747+0000","last_change":"2024-09-18T04:58:48.095394+0000","last_active":"2024-09-18T04:59:07.731747+0000","last_peered":"2024-09-18T04:59:07.731747+0000","last_clean":"2024-09-18T04:59:07.731747+0000","last_became_active":"2024-09-18T04:58:32.029074+0000","last_became_peered":"2024-09-18T04:58:32.029074+0000","last_unstale":"2024-09-18T04:59:07.731747+0000","last_undegraded":"2024-09-18T04:59:07.731747+0000","last_fullsized":"2024-09-18T04:59:07.731747+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:15:38.950060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053623899999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3403,"num_read_kb":16958,"num_write":699,"num_write_kb":12986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":4650417,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"153'506","reported_seq":4239,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.197394+0000","last_change":"2024-09-18T04:58:55.128037+0000","last_active":"2024-09-18T04:59:07.197394+0000","last_peered":"2024-09-18T04:59:07.197394+0000","last_clean":"2024-09-18T04:59:07.197394+0000","last_became_active":"2024-09-18T04:58:55.125721+0000","last_became_peered":"2024-09-18T04:58:55.125721+0000","last_unstale":"2024-09-18T04:59:07.197394+0000","last_undegraded":"2024-09-18T04:59:07.197394+0000","last_fullsized":"2024-09-18T04:59:07.197394+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:06:23.656568+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001441276,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4899,"num_read_kb":13699,"num_write":541,"num_write_kb":6077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"154'885","reported_seq":5042,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195463+0000","last_change":"2024-09-18T04:58:48.024658+0000","last_active":"2024-09-18T04:59:07.195463+0000","last_peered":"2024-09-18T04:59:07.195463+0000","last_clean":"2024-09-18T04:59:07.195463+0000","last_became_active":"2024-09-18T04:56:21.131133+0000","last_became_peered":"2024-09-18T04:56:21.131133+0000","last_unstale":"2024-09-18T04:59:07.195463+0000","last_undegraded":"2024-09-18T04:59:07.195463+0000","last_fullsized":"2024-09-18T04:59:07.195463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:01:57.036904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012610480000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6340,"num_read_kb":26579,"num_write":1141,"num_write_kb":12621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"157'637","reported_seq":5292,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198169+0000","last_change":"2024-09-18T04:58:48.028934+0000","last_active":"2024-09-18T04:59:07.198169+0000","last_peered":"2024-09-18T04:59:07.198169+0000","last_clean":"2024-09-18T04:59:07.198169+0000","last_became_active":"2024-09-18T04:58:32.032762+0000","last_became_peered":"2024-09-18T04:58:32.032762+0000","last_unstale":"2024-09-18T04:59:07.198169+0000","last_undegraded":"2024-09-18T04:59:07.198169+0000","last_fullsized":"2024-09-18T04:59:07.198169+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:33:25.010425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066811099999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":5292,"num_read_kb":26013,"num_write":596,"num_write_kb":7300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":447,"num_bytes_recovered":1594801,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"155'502","reported_seq":2817,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-18T04:59:06.190587+0000","last_change":"2024-09-18T04:58:48.023734+0000","last_active":"2024-09-18T04:59:06.190587+0000","last_peered":"2024-09-18T04:59:06.190587+0000","last_clean":"2024-09-18T04:59:06.190587+0000","last_became_active":"2024-09-18T04:58:30.025558+0000","last_became_peered":"2024-09-18T04:58:30.025558+0000","last_unstale":"2024-09-18T04:59:06.190587+0000","last_undegraded":"2024-09-18T04:59:06.190587+0000","last_fullsized":"2024-09-18T04:59:06.190587+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:57:31.113016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087474099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3131,"num_read_kb":27325,"num_write":609,"num_write_kb":11672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":12427258,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":192,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195662+0000","last_change":"2024-09-18T04:58:29.025271+0000","last_active":"2024-09-18T04:59:07.195662+0000","last_peered":"2024-09-18T04:59:07.195662+0000","last_clean":"2024-09-18T04:59:07.195662+0000","last_became_active":"2024-09-18T04:58:29.024911+0000","last_became_peered":"2024-09-18T04:58:29.024911+0000","last_unstale":"2024-09-18T04:59:07.195662+0000","last_undegraded":"2024-09-18T04:59:07.195662+0000","last_fullsized":"2024-09-18T04:59:07.195662+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:53:21.863837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063436799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"155'2480","reported_seq":9899,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195724+0000","last_change":"2024-09-18T04:58:48.024643+0000","last_active":"2024-09-18T04:59:07.195724+0000","last_peered":"2024-09-18T04:59:07.195724+0000","last_clean":"2024-09-18T04:59:07.195724+0000","last_became_active":"2024-09-18T04:58:33.027340+0000","last_became_peered":"2024-09-18T04:58:33.027340+0000","last_unstale":"2024-09-18T04:59:07.195724+0000","last_undegraded":"2024-09-18T04:59:07.195724+0000","last_fullsized":"2024-09-18T04:59:07.195724+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":2480,"log_dups_size":0,"ondisk_log_size":2480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:47:47.826132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000952045,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":18529,"num_read_kb":32990,"num_write":4379,"num_write_kb":20206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":5439803,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":176,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195585+0000","last_change":"2024-09-18T04:56:12.389971+0000","last_active":"2024-09-18T04:59:07.195585+0000","last_peered":"2024-09-18T04:59:07.195585+0000","last_clean":"2024-09-18T04:59:07.195585+0000","last_became_active":"2024-09-18T04:56:10.412595+0000","last_became_peered":"2024-09-18T04:56:10.412595+0000","last_unstale":"2024-09-18T04:59:07.195585+0000","last_undegraded":"2024-09-18T04:59:07.195585+0000","last_fullsized":"2024-09-18T04:59:07.195585+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:45.638420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038576799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"157'807","reported_seq":7861,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198219+0000","last_change":"2024-09-18T04:58:57.151001+0000","last_active":"2024-09-18T04:59:07.198219+0000","last_peered":"2024-09-18T04:59:07.198219+0000","last_clean":"2024-09-18T04:59:07.198219+0000","last_became_active":"2024-09-18T04:58:57.147959+0000","last_became_peered":"2024-09-18T04:58:57.147959+0000","last_unstale":"2024-09-18T04:59:07.198219+0000","last_undegraded":"2024-09-18T04:59:07.198219+0000","last_fullsized":"2024-09-18T04:59:07.198219+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:30:19.725543+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00065254400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8940,"num_read_kb":28417,"num_write":1051,"num_write_kb":11406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":5706670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198307+0000","last_change":"2024-09-18T04:58:57.150288+0000","last_active":"2024-09-18T04:59:07.198307+0000","last_peered":"2024-09-18T04:59:07.198307+0000","last_clean":"2024-09-18T04:59:07.198307+0000","last_became_active":"2024-09-18T04:58:57.149907+0000","last_became_peered":"2024-09-18T04:58:57.149907+0000","last_unstale":"2024-09-18T04:59:07.198307+0000","last_undegraded":"2024-09-18T04:59:07.198307+0000","last_fullsized":"2024-09-18T04:59:07.198307+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:12:39.499969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"157'757","reported_seq":6181,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731891+0000","last_change":"2024-09-18T04:58:48.096507+0000","last_active":"2024-09-18T04:59:07.731891+0000","last_peered":"2024-09-18T04:59:07.731891+0000","last_clean":"2024-09-18T04:59:07.731891+0000","last_became_active":"2024-09-18T04:58:33.026806+0000","last_became_peered":"2024-09-18T04:58:33.026806+0000","last_unstale":"2024-09-18T04:59:07.731891+0000","last_undegraded":"2024-09-18T04:59:07.731891+0000","last_fullsized":"2024-09-18T04:59:07.731891+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:59:41.673907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013736829999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6896,"num_read_kb":21261,"num_write":943,"num_write_kb":5208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":571544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":145,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195438+0000","last_change":"2024-09-18T04:56:46.254241+0000","last_active":"2024-09-18T04:59:07.195438+0000","last_peered":"2024-09-18T04:59:07.195438+0000","last_clean":"2024-09-18T04:59:07.195438+0000","last_became_active":"2024-09-18T04:56:46.253894+0000","last_became_peered":"2024-09-18T04:56:46.253894+0000","last_unstale":"2024-09-18T04:59:07.195438+0000","last_undegraded":"2024-09-18T04:59:07.195438+0000","last_fullsized":"2024-09-18T04:59:07.195438+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:58:42.098099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045144100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"158'505","reported_seq":5028,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.197318+0000","last_change":"2024-09-18T04:58:48.061503+0000","last_active":"2024-09-18T04:59:07.197318+0000","last_peered":"2024-09-18T04:59:07.197318+0000","last_clean":"2024-09-18T04:59:07.197318+0000","last_became_active":"2024-09-18T04:58:41.078435+0000","last_became_peered":"2024-09-18T04:58:41.078435+0000","last_unstale":"2024-09-18T04:59:07.197318+0000","last_undegraded":"2024-09-18T04:59:07.197318+0000","last_fullsized":"2024-09-18T04:59:07.197318+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:34:50.214360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056043399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4221,"num_read_kb":33044,"num_write":472,"num_write_kb":37350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":25349011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"155'485","reported_seq":9536,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.201259+0000","last_change":"2024-09-18T04:59:07.200564+0000","last_active":"2024-09-18T04:59:07.201259+0000","last_peered":"2024-09-18T04:59:07.201259+0000","last_clean":"2024-09-18T04:59:07.201259+0000","last_became_active":"2024-09-18T04:59:07.199056+0000","last_became_peered":"2024-09-18T04:59:07.199056+0000","last_unstale":"2024-09-18T04:59:07.201259+0000","last_undegraded":"2024-09-18T04:59:07.201259+0000","last_fullsized":"2024-09-18T04:59:07.201259+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:30:30.980376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049796699999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":9360,"num_read_kb":28051,"num_write":499,"num_write_kb":12488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":472,"num_bytes_recovered":11362815,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":362,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198082+0000","last_change":"2024-09-18T04:58:38.025236+0000","last_active":"2024-09-18T04:59:07.198082+0000","last_peered":"2024-09-18T04:59:07.198082+0000","last_clean":"2024-09-18T04:59:07.198082+0000","last_became_active":"2024-09-18T04:58:38.024740+0000","last_became_peered":"2024-09-18T04:58:38.024740+0000","last_unstale":"2024-09-18T04:59:07.198082+0000","last_undegraded":"2024-09-18T04:59:07.198082+0000","last_fullsized":"2024-09-18T04:59:07.198082+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_clean_scrub_stamp":"2024-09-18T04:56:10.373785+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:10:45.251654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":37,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198355+0000","last_change":"2024-09-18T04:58:57.148289+0000","last_active":"2024-09-18T04:59:07.198355+0000","last_peered":"2024-09-18T04:59:07.198355+0000","last_clean":"2024-09-18T04:59:07.198355+0000","last_became_active":"2024-09-18T04:58:57.147915+0000","last_became_peered":"2024-09-18T04:58:57.147915+0000","last_unstale":"2024-09-18T04:59:07.198355+0000","last_undegraded":"2024-09-18T04:59:07.198355+0000","last_fullsized":"2024-09-18T04:59:07.198355+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:31:21.699401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"157'811","reported_seq":13428,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198153+0000","last_change":"2024-09-18T04:58:57.152156+0000","last_active":"2024-09-18T04:59:07.198153+0000","last_peered":"2024-09-18T04:59:07.198153+0000","last_clean":"2024-09-18T04:59:07.198153+0000","last_became_active":"2024-09-18T04:58:57.148221+0000","last_became_peered":"2024-09-18T04:58:57.148221+0000","last_unstale":"2024-09-18T04:59:07.198153+0000","last_undegraded":"2024-09-18T04:59:07.198153+0000","last_fullsized":"2024-09-18T04:59:07.198153+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:45:57.434891+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0033124909999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":14837,"num_read_kb":27169,"num_write":1047,"num_write_kb":7785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":1280827,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":178,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198076+0000","last_change":"2024-09-18T04:56:12.398631+0000","last_active":"2024-09-18T04:59:07.198076+0000","last_peered":"2024-09-18T04:59:07.198076+0000","last_clean":"2024-09-18T04:59:07.198076+0000","last_became_active":"2024-09-18T04:56:10.412614+0000","last_became_peered":"2024-09-18T04:56:10.412614+0000","last_unstale":"2024-09-18T04:59:07.198076+0000","last_undegraded":"2024-09-18T04:59:07.198076+0000","last_fullsized":"2024-09-18T04:59:07.198076+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:44:13.401320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075126299999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"158'601","reported_seq":2977,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-18T04:59:06.191350+0000","last_change":"2024-09-18T04:58:48.024976+0000","last_active":"2024-09-18T04:59:06.191350+0000","last_peered":"2024-09-18T04:59:06.191350+0000","last_clean":"2024-09-18T04:59:06.191350+0000","last_became_active":"2024-09-18T04:58:30.027757+0000","last_became_peered":"2024-09-18T04:58:30.027757+0000","last_unstale":"2024-09-18T04:59:06.191350+0000","last_undegraded":"2024-09-18T04:59:06.191350+0000","last_fullsized":"2024-09-18T04:59:06.191350+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:29:00.106938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014951890000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":2216,"num_read_kb":23967,"num_write":593,"num_write_kb":16097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":10562840,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":176,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195410+0000","last_change":"2024-09-18T04:56:12.390044+0000","last_active":"2024-09-18T04:59:07.195410+0000","last_peered":"2024-09-18T04:59:07.195410+0000","last_clean":"2024-09-18T04:59:07.195410+0000","last_became_active":"2024-09-18T04:56:10.413042+0000","last_became_peered":"2024-09-18T04:56:10.413042+0000","last_unstale":"2024-09-18T04:59:07.195410+0000","last_undegraded":"2024-09-18T04:59:07.195410+0000","last_fullsized":"2024-09-18T04:59:07.195410+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:46:11.074243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"157'525","reported_seq":4818,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731809+0000","last_change":"2024-09-18T04:58:48.095298+0000","last_active":"2024-09-18T04:59:07.731809+0000","last_peered":"2024-09-18T04:59:07.731809+0000","last_clean":"2024-09-18T04:59:07.731809+0000","last_became_active":"2024-09-18T04:58:32.027104+0000","last_became_peered":"2024-09-18T04:58:32.027104+0000","last_unstale":"2024-09-18T04:59:07.731809+0000","last_undegraded":"2024-09-18T04:59:07.731809+0000","last_fullsized":"2024-09-18T04:59:07.731809+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:41:37.134684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056931099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5904,"num_read_kb":13840,"num_write":503,"num_write_kb":5042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":2187784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198027+0000","last_change":"2024-09-18T04:57:40.472681+0000","last_active":"2024-09-18T04:59:07.198027+0000","last_peered":"2024-09-18T04:59:07.198027+0000","last_clean":"2024-09-18T04:59:07.198027+0000","last_became_active":"2024-09-18T04:57:40.472217+0000","last_became_peered":"2024-09-18T04:57:40.472217+0000","last_unstale":"2024-09-18T04:59:07.198027+0000","last_undegraded":"2024-09-18T04:59:07.198027+0000","last_fullsized":"2024-09-18T04:59:07.198027+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:27:11.849311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"157'482","reported_seq":3615,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731740+0000","last_change":"2024-09-18T04:58:48.096157+0000","last_active":"2024-09-18T04:59:07.731740+0000","last_peered":"2024-09-18T04:59:07.731740+0000","last_clean":"2024-09-18T04:59:07.731740+0000","last_became_active":"2024-09-18T04:58:38.024916+0000","last_became_peered":"2024-09-18T04:58:38.024916+0000","last_unstale":"2024-09-18T04:59:07.731740+0000","last_undegraded":"2024-09-18T04:59:07.731740+0000","last_fullsized":"2024-09-18T04:59:07.731740+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:32:42.202542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001087188,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2758,"num_read_kb":7938,"num_write":473,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":458,"num_bytes_recovered":2219067,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"155'714","reported_seq":12009,"reported_epoch":174,"state":"active+clean","last_fresh":"2024-09-18T04:59:06.191085+0000","last_change":"2024-09-18T04:58:48.024809+0000","last_active":"2024-09-18T04:59:06.191085+0000","last_peered":"2024-09-18T04:59:06.191085+0000","last_clean":"2024-09-18T04:59:06.191085+0000","last_became_active":"2024-09-18T04:58:26.101490+0000","last_became_peered":"2024-09-18T04:58:26.101490+0000","last_unstale":"2024-09-18T04:59:06.191085+0000","last_undegraded":"2024-09-18T04:59:06.191085+0000","last_fullsized":"2024-09-18T04:59:06.191085+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:30:15.538062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015101680000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":12956,"num_read_kb":26590,"num_write":848,"num_write_kb":15777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":5822953,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"157'470","reported_seq":6073,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195769+0000","last_change":"2024-09-18T04:58:48.136195+0000","last_active":"2024-09-18T04:59:07.195769+0000","last_peered":"2024-09-18T04:59:07.195769+0000","last_clean":"2024-09-18T04:59:07.195769+0000","last_became_active":"2024-09-18T04:58:27.115137+0000","last_became_peered":"2024-09-18T04:58:27.115137+0000","last_unstale":"2024-09-18T04:59:07.195769+0000","last_undegraded":"2024-09-18T04:59:07.195769+0000","last_fullsized":"2024-09-18T04:59:07.195769+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:57:42.359624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039325500000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":4725,"num_read_kb":14773,"num_write":478,"num_write_kb":8139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":6055124,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"155'1400","reported_seq":5772,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731780+0000","last_change":"2024-09-18T04:58:48.095758+0000","last_active":"2024-09-18T04:59:07.731780+0000","last_peered":"2024-09-18T04:59:07.731780+0000","last_clean":"2024-09-18T04:59:07.731780+0000","last_became_active":"2024-09-18T04:58:31.026041+0000","last_became_peered":"2024-09-18T04:58:31.026041+0000","last_unstale":"2024-09-18T04:59:07.731780+0000","last_undegraded":"2024-09-18T04:59:07.731780+0000","last_fullsized":"2024-09-18T04:59:07.731780+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":1400,"log_dups_size":0,"ondisk_log_size":1400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:55:29.526466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083671099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8851,"num_read_kb":15183,"num_write":2049,"num_write_kb":8862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":1966128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"155'908","reported_seq":7203,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195547+0000","last_change":"2024-09-18T04:58:48.023609+0000","last_active":"2024-09-18T04:59:07.195547+0000","last_peered":"2024-09-18T04:59:07.195547+0000","last_clean":"2024-09-18T04:59:07.195547+0000","last_became_active":"2024-09-18T04:58:33.027525+0000","last_became_peered":"2024-09-18T04:58:33.027525+0000","last_unstale":"2024-09-18T04:59:07.195547+0000","last_undegraded":"2024-09-18T04:59:07.195547+0000","last_fullsized":"2024-09-18T04:59:07.195547+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:27:34.074204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062770000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9983,"num_read_kb":24174,"num_write":928,"num_write_kb":8107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":86198,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"155'617","reported_seq":7056,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.194912+0000","last_change":"2024-09-18T04:58:48.021010+0000","last_active":"2024-09-18T04:59:07.194912+0000","last_peered":"2024-09-18T04:59:07.194912+0000","last_clean":"2024-09-18T04:59:07.194912+0000","last_became_active":"2024-09-18T04:58:07.017827+0000","last_became_peered":"2024-09-18T04:58:07.017827+0000","last_unstale":"2024-09-18T04:59:07.194912+0000","last_undegraded":"2024-09-18T04:59:07.194912+0000","last_fullsized":"2024-09-18T04:59:07.194912+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:42:31.897588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050117499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8566,"num_read_kb":18769,"num_write":595,"num_write_kb":12467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":2872264,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"155'1050","reported_seq":5261,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198243+0000","last_change":"2024-09-18T04:58:48.029466+0000","last_active":"2024-09-18T04:59:07.198243+0000","last_peered":"2024-09-18T04:59:07.198243+0000","last_clean":"2024-09-18T04:59:07.198243+0000","last_became_active":"2024-09-18T04:58:31.028189+0000","last_became_peered":"2024-09-18T04:58:31.028189+0000","last_unstale":"2024-09-18T04:59:07.198243+0000","last_undegraded":"2024-09-18T04:59:07.198243+0000","last_fullsized":"2024-09-18T04:59:07.198243+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":1050,"log_dups_size":0,"ondisk_log_size":1050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:26:10.430573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001035219,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7660,"num_read_kb":17955,"num_write":1663,"num_write_kb":5543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":2749742,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"157'556","reported_seq":5095,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198207+0000","last_change":"2024-09-18T04:58:48.029669+0000","last_active":"2024-09-18T04:59:07.198207+0000","last_peered":"2024-09-18T04:59:07.198207+0000","last_clean":"2024-09-18T04:59:07.198207+0000","last_became_active":"2024-09-18T04:58:29.029625+0000","last_became_peered":"2024-09-18T04:58:29.029625+0000","last_unstale":"2024-09-18T04:59:07.198207+0000","last_undegraded":"2024-09-18T04:59:07.198207+0000","last_fullsized":"2024-09-18T04:59:07.198207+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:51:09.257320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011607150000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4690,"num_read_kb":26470,"num_write":636,"num_write_kb":8108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":5504104,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"157'809","reported_seq":4779,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195411+0000","last_change":"2024-09-18T04:58:48.136237+0000","last_active":"2024-09-18T04:59:07.195411+0000","last_peered":"2024-09-18T04:59:07.195411+0000","last_clean":"2024-09-18T04:59:07.195411+0000","last_became_active":"2024-09-18T04:56:21.131202+0000","last_became_peered":"2024-09-18T04:56:21.131202+0000","last_unstale":"2024-09-18T04:59:07.195411+0000","last_undegraded":"2024-09-18T04:59:07.195411+0000","last_fullsized":"2024-09-18T04:59:07.195411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:17:35.438723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041769600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5877,"num_read_kb":14669,"num_write":993,"num_write_kb":10354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"157'920","reported_seq":8474,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.202504+0000","last_change":"2024-09-18T04:59:07.201833+0000","last_active":"2024-09-18T04:59:07.202504+0000","last_peered":"2024-09-18T04:59:07.202504+0000","last_clean":"2024-09-18T04:59:07.202504+0000","last_became_active":"2024-09-18T04:59:07.199844+0000","last_became_peered":"2024-09-18T04:59:07.199844+0000","last_unstale":"2024-09-18T04:59:07.202504+0000","last_undegraded":"2024-09-18T04:59:07.202504+0000","last_fullsized":"2024-09-18T04:59:07.202504+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":920,"log_dups_size":0,"ondisk_log_size":920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:40:08.684972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087268100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10766,"num_read_kb":19560,"num_write":908,"num_write_kb":18676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":9230703,"num_keys_recovered":59,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"144'809","reported_seq":5825,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.197043+0000","last_change":"2024-09-18T04:58:55.130930+0000","last_active":"2024-09-18T04:59:07.197043+0000","last_peered":"2024-09-18T04:59:07.197043+0000","last_clean":"2024-09-18T04:59:07.197043+0000","last_became_active":"2024-09-18T04:58:55.129007+0000","last_became_peered":"2024-09-18T04:58:55.129007+0000","last_unstale":"2024-09-18T04:59:07.197043+0000","last_undegraded":"2024-09-18T04:59:07.197043+0000","last_fullsized":"2024-09-18T04:59:07.197043+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:21:11.287190+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.00099438999999999994,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7388,"num_read_kb":33089,"num_write":1110,"num_write_kb":12855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"158'781","reported_seq":2932,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198095+0000","last_change":"2024-09-18T04:58:48.030265+0000","last_active":"2024-09-18T04:59:07.198095+0000","last_peered":"2024-09-18T04:59:07.198095+0000","last_clean":"2024-09-18T04:59:07.198095+0000","last_became_active":"2024-09-18T04:58:31.028633+0000","last_became_peered":"2024-09-18T04:58:31.028633+0000","last_unstale":"2024-09-18T04:59:07.198095+0000","last_undegraded":"2024-09-18T04:59:07.198095+0000","last_fullsized":"2024-09-18T04:59:07.198095+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:04:25.747259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00140489,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4341,"num_read_kb":14774,"num_write":882,"num_write_kb":9994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":12642724,"num_keys_recovered":71,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":47,"num_read":220784,"num_read_kb":683202,"num_write":29896,"num_write_kb":377778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9208,"num_bytes_recovered":184551668,"num_keys_recovered":562,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23964,"ondisk_log_size":23964,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28167,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574889,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51292,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320548,"statfs":{"total":96636764160,"available":96584241152,"internally_reserved":0,"allocated":2170880,"data_stored":1617954,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":50330059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.45600000000000002,"15min":0.46400000000000002},"min":{"1min":0.23799999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53100000000000003,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.437,"5min":0.41499999999999998,"15min":0.41099999999999998},"min":{"1min":0.312,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.60299999999999998}]},{"osd":1,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.46899999999999997,"15min":0.47799999999999998},"min":{"1min":0.249,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.55100000000000005,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.436,"15min":0.434},"min":{"1min":0.28399999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.621}]},{"osd":2,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.44700000000000001,"15min":0.44500000000000001},"min":{"1min":0.35399999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45200000000000001,"15min":0.45000000000000001},"min":{"1min":0.34200000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.56200000000000006,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.67500000000000004}]},{"osd":3,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52600000000000002,"15min":0.51900000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.4399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.34100000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.4809999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":0.72799999999999998}]},{"osd":4,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.42199999999999999,"15min":0.43099999999999999},"min":{"1min":0.26500000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.63700000000000001,"5min":1.954,"15min":1.954},"last":0.502},{"interface":"front","average":{"1min":0.375,"5min":0.41699999999999998,"15min":0.42399999999999999},"min":{"1min":0.246,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.629,"5min":1.478,"15min":1.478},"last":0.45800000000000002}]},{"osd":5,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.50800000000000001,"15min":0.52200000000000002},"min":{"1min":0.30299999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.71799999999999997,"5min":3.145,"15min":3.145},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.443,"15min":0.441},"min":{"1min":0.27700000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.65800000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.68999999999999995}]},{"osd":6,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.47899999999999998,"15min":0.48299999999999998},"min":{"1min":0.33300000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.69299999999999995,"5min":0.746,"15min":0.746},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.48899999999999999,"15min":0.49299999999999999},"min":{"1min":0.35199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.623,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.71399999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55336,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316504,"statfs":{"total":96636764160,"available":96580100096,"internally_reserved":0,"allocated":2121728,"data_stored":1032642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.71899999999999997,"15min":0.76900000000000002},"min":{"1min":0.23899999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.56799999999999995,"5min":6.1319999999999997,"15min":6.1319999999999997},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.75700000000000001,"15min":0.80500000000000005},"min":{"1min":0.35799999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.81100000000000005,"5min":6.1429999999999998,"15min":6.1429999999999998},"last":0.41099999999999998}]},{"osd":1,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.70999999999999996,"15min":0.76200000000000001},"min":{"1min":0.29299999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.44900000000000001,"5min":6.0410000000000004,"15min":6.0410000000000004},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.70899999999999996,"15min":0.753},"min":{"1min":0.33000000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.83599999999999997,"5min":5.5380000000000003,"15min":5.5380000000000003},"last":0.34599999999999997}]},{"osd":2,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.41999999999999998,"15min":0.42599999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.54400000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.315},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.46100000000000002,"15min":0.46800000000000003},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.55900000000000005,"5min":1.04,"15min":1.04},"last":0.33400000000000002}]},{"osd":3,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46000000000000002,"15min":0.46100000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.378},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.47799999999999998,"15min":0.48099999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.625,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.38700000000000001}]},{"osd":4,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.36799999999999999,"15min":0.374},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.47599999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.20799999999999999},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.43099999999999999,"15min":0.437},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.30399999999999999}]},{"osd":5,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.372,"15min":0.378},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.42499999999999999,"15min":0.435},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.53000000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.28699999999999998}]},{"osd":7,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.46800000000000003,"15min":0.46600000000000003},"min":{"1min":0.28100000000000003,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.48599999999999999,"15min":0.47699999999999998},"min":{"1min":0.34399999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.35599999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574888,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65188,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306652,"statfs":{"total":96636764160,"available":96570011648,"internally_reserved":0,"allocated":2121728,"data_stored":1033583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.54600000000000004,"15min":0.57099999999999995},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.59199999999999997,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.504,"15min":0.51000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71599999999999997,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.55700000000000005}]},{"osd":1,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.53300000000000003,"15min":0.55400000000000005},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.55600000000000005,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.32500000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63800000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.57499999999999996}]},{"osd":2,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47699999999999998,"15min":0.47899999999999998},"min":{"1min":0.28699999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.629,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.57399999999999995,"15min":0.59999999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.626,"5min":3.1070000000000002,"15min":3.1070000000000002},"last":0.504}]},{"osd":3,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.496,"15min":0.502},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.77000000000000002,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.504,"15min":0.50800000000000001},"min":{"1min":0.35099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.63,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.54600000000000004}]},{"osd":4,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.46800000000000003,"15min":0.47399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.59499999999999997,"5min":1.599,"15min":1.599},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.56299999999999994,"15min":0.59299999999999997},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.53500000000000003,"5min":3.379,"15min":3.379},"last":0.432}]},{"osd":6,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.45500000000000002,"15min":0.46600000000000003},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54200000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.56200000000000006,"15min":0.58499999999999996},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.085,"5min":3.1669999999999998,"15min":3.1669999999999998},"last":0.378}]},{"osd":7,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.48399999999999999,"15min":0.495},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80200000000000005,"5min":1.224,"15min":1.224},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.54200000000000004,"15min":0.55900000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.58599999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.45700000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63512,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94308328,"statfs":{"total":96636764160,"available":96571727872,"internally_reserved":0,"allocated":2109440,"data_stored":1027567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.46000000000000002,"15min":0.44400000000000001},"min":{"1min":0.23499999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":3.7069999999999999,"5min":3.7069999999999999,"15min":3.7069999999999999},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.46100000000000002,"15min":0.441},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":3.3700000000000001,"5min":3.3700000000000001,"15min":3.3700000000000001},"last":0.219}]},{"osd":1,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.47099999999999997,"15min":0.45500000000000002},"min":{"1min":0.28199999999999997,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":3.2879999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.375},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.46600000000000003,"15min":0.44600000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.661,"5min":3.661,"15min":3.661},"last":0.38200000000000001}]},{"osd":2,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.52200000000000002,"15min":0.50600000000000001},"min":{"1min":0.29999999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.1890000000000001,"5min":3.1890000000000001,"15min":3.1890000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.51600000000000001,"15min":0.502},"min":{"1min":0.32300000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.3860000000000001,"5min":3.3860000000000001,"15min":3.3860000000000001},"last":0.35899999999999999}]},{"osd":3,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.504,"15min":0.48899999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":3.4500000000000002,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.48299999999999998,"15min":0.46100000000000002},"min":{"1min":0.372,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.3210000000000002,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.433}]},{"osd":5,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.42599999999999999,"15min":0.40200000000000002},"min":{"1min":0.25,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":3.9319999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.43099999999999999,"15min":0.40699999999999997},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":3.9449999999999998,"5min":3.9449999999999998,"15min":3.9449999999999998},"last":0.39700000000000002}]},{"osd":6,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.45100000000000001,"15min":0.42899999999999999},"min":{"1min":0.29299999999999998,"5min":0.251,"15min":0.251},"max":{"1min":3.3530000000000002,"5min":3.3530000000000002,"15min":3.3530000000000002},"last":0.312},{"interface":"front","average":{"1min":1.29,"5min":0.626,"15min":0.51600000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":17.422999999999998,"5min":17.422999999999998,"15min":17.422999999999998},"last":0.33500000000000002}]},{"osd":7,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.48399999999999999,"15min":0.46400000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":3.7330000000000001,"5min":3.7330000000000001,"15min":3.7330000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.52100000000000002,"15min":0.499},"min":{"1min":0.33300000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":3.6930000000000001,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":0.40500000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87320,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284520,"statfs":{"total":96636764160,"available":96547348480,"internally_reserved":0,"allocated":2109440,"data_stored":1027567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":87292360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.39500000000000002,"15min":0.40500000000000003},"min":{"1min":0.23899999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.45200000000000001,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.51800000000000002,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.38400000000000001}]},{"osd":1,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.376,"15min":0.375},"min":{"1min":0.249,"5min":0.245,"15min":0.245},"max":{"1min":0.51800000000000002,"5min":0.56799999999999995,"15min":0.56799999999999995},"last":0.37},{"interface":"front","average":{"1min":0.34300000000000003,"5min":0.40500000000000003,"15min":0.41499999999999998},"min":{"1min":0.249,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.54400000000000004,"5min":0.755,"15min":0.755},"last":0.438}]},{"osd":2,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.41599999999999998,"15min":0.41999999999999998},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.56999999999999995,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.45200000000000001,"15min":0.45300000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.59399999999999997,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.47299999999999998}]},{"osd":4,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.52000000000000002,"15min":0.52900000000000003},"min":{"1min":0.31,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71999999999999997,"5min":0.755,"15min":0.755},"last":0.505},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.76100000000000001,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.63900000000000001}]},{"osd":5,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.55900000000000005,"15min":0.56899999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68600000000000005,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.56599999999999995,"15min":0.57599999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.66900000000000004,"5min":0.874,"15min":0.874},"last":0.60399999999999998}]},{"osd":6,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.58699999999999997,"15min":0.59599999999999997},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.80400000000000005,"5min":0.874,"15min":0.874},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72399999999999998,"5min":1.095,"15min":1.095},"last":0.51600000000000001}]},{"osd":7,"last update":"Wed Sep 18 04:58:09 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.65600000000000003,"15min":0.67200000000000004},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84499999999999997,"5min":1.335,"15min":1.335},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.61599999999999999,"15min":0.624},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.753,"5min":1.087,"15min":1.087},"last":0.58999999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37292,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":2121728,"data_stored":1033583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.33700000000000002,"15min":0.33600000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.39600000000000002,"15min":0.39800000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.63800000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.35899999999999999}]},{"osd":1,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.56999999999999995,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.36299999999999999,"15min":0.36099999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.29099999999999998}]},{"osd":3,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.40799999999999997,"15min":0.40400000000000003},"min":{"1min":0.26200000000000001,"5min":0.253,"15min":0.253},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.40999999999999998,"15min":0.40400000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.373}]},{"osd":4,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.54900000000000004,"15min":0.53500000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.58899999999999997,"15min":0.57899999999999996},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":2.4220000000000002,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":0.59999999999999998}]},{"osd":5,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.89900000000000002,"5min":0.61199999999999999,"15min":0.56399999999999995},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":7.4480000000000004,"5min":7.4480000000000004,"15min":7.4480000000000004},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54800000000000004,"15min":0.54700000000000004},"min":{"1min":0.377,"5min":0.375,"15min":0.375},"max":{"1min":0.73699999999999999,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.64800000000000002}]},{"osd":6,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.55600000000000005,"15min":0.55200000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.84599999999999997,"5min":1.171,"15min":1.171},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56899999999999995,"15min":0.56799999999999995},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.57799999999999996}]},{"osd":7,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80800000000000005,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":1.1719999999999999,"5min":0.69999999999999996,"15min":0.622},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":11.138,"5min":11.138,"15min":11.138},"last":0.61399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574889,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29124,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342716,"statfs":{"total":96636764160,"available":96606941184,"internally_reserved":0,"allocated":2154496,"data_stored":1617954,"data_compressed":9734,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.34999999999999998,"15min":0.35799999999999998},"min":{"1min":0.20899999999999999,"5min":0.185,"15min":0.185},"max":{"1min":0.622,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.63100000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":1.6579999999999999}]},{"osd":2,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.371,"15min":0.373},"min":{"1min":0.222,"5min":0.221,"15min":0.221},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":1.7270000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41599999999999998,"15min":0.41399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.6839999999999999}]},{"osd":3,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.38800000000000001,"15min":0.38400000000000001},"min":{"1min":0.23899999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":1.754},{"interface":"front","average":{"1min":0.434,"5min":0.40600000000000003,"15min":0.40100000000000002},"min":{"1min":0.29599999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":1.71}]},{"osd":4,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.50800000000000001,"15min":0.51700000000000002},"min":{"1min":0.32400000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.77000000000000002,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.496,"5min":0.50700000000000001,"15min":0.50900000000000001},"min":{"1min":0.315,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.86099999999999999,"5min":0.999,"15min":0.999},"last":0.51500000000000001}]},{"osd":5,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50900000000000001,"15min":0.51300000000000001},"min":{"1min":0.33600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80400000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":1.625},{"interface":"front","average":{"1min":0.5,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.373,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.83899999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":1.597}]},{"osd":6,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82799999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.74299999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67400000000000004}]},{"osd":7,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.56999999999999995,"15min":0.57799999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78200000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.40999999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0309999999999999,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.70299999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51228,"kb_used_data":2060,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320612,"statfs":{"total":96636764160,"available":96584306688,"internally_reserved":0,"allocated":2109440,"data_stored":1026626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50330055},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.35999999999999999,"15min":0.36299999999999999},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.56799999999999995,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.247},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.34799999999999998,"15min":0.34599999999999997},"min":{"1min":0.188,"5min":0.188,"15min":0.188},"max":{"1min":0.53700000000000003,"5min":0.621,"15min":0.621},"last":0.28399999999999997}]},{"osd":2,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.39900000000000002,"15min":0.40100000000000002},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.435,"5min":0.42599999999999999,"15min":0.42399999999999999},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.46200000000000002}]},{"osd":3,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.434,"15min":0.42399999999999999},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.40400000000000003,"15min":0.39700000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.5}]},{"osd":4,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.63400000000000001,"15min":0.65600000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.89400000000000002,"5min":3.4710000000000001,"15min":3.4710000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.504,"15min":0.502},"min":{"1min":0.35899999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.376}]},{"osd":5,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.81699999999999995,"15min":0.84099999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":4.2640000000000002,"5min":4.2640000000000002,"15min":4.2640000000000002},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.38500000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.54700000000000004}]},{"osd":6,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.66000000000000003,"15min":0.68000000000000005},"min":{"1min":0.39500000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.81699999999999995,"5min":3.536,"15min":3.536},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57099999999999995,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.57999999999999996}]},{"osd":7,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.55200000000000005,"15min":0.54700000000000004},"min":{"1min":0.41999999999999998,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.68400000000000005,"15min":0.70599999999999996},"min":{"1min":0.39100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.85699999999999998,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.48099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9734,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9734,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T04:59:15.013 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T04:59:15.297 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:59:15.297 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":176,"fsid":"c0761d0c-69d5-4b64-b8b9-c7d098b2d06d","created":"2024-09-18T04:55:58.216538+0000","modified":"2024-09-18T04:59:08.192977+0000","last_up_change":"2024-09-18T04:56:08.369924+0000","last_in_change":"2024-09-18T04:58:34.032750+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-18T04:56:08.983344+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":2.630000114440918,"raw_score_stable":2.630000114440918,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-09-18T04:56:10.313988+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-18T04:56:19.562579+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":93,"snap_epoch":158,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c43f8024-3a3e-424e-aa54-398310ceb70d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6824","nonce":4106704527},{"type":"v1","addr":"172.21.15.2:6825","nonce":4106704527}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":4106704527},{"type":"v1","addr":"172.21.15.2:6827","nonce":4106704527}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":4106704527},{"type":"v1","addr":"172.21.15.2:6831","nonce":4106704527}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":4106704527},{"type":"v1","addr":"172.21.15.2:6829","nonce":4106704527}]},"public_addr":"172.21.15.2:6825/4106704527","cluster_addr":"172.21.15.2:6827/4106704527","heartbeat_back_addr":"172.21.15.2:6831/4106704527","heartbeat_front_addr":"172.21.15.2:6829/4106704527","state":["exists","up"]},{"osd":1,"uuid":"c664b970-145c-4b6f-8852-2f6dcf1fa30d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":1781959259},{"type":"v1","addr":"172.21.15.2:6809","nonce":1781959259}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":1781959259},{"type":"v1","addr":"172.21.15.2:6811","nonce":1781959259}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":1781959259},{"type":"v1","addr":"172.21.15.2:6815","nonce":1781959259}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":1781959259},{"type":"v1","addr":"172.21.15.2:6813","nonce":1781959259}]},"public_addr":"172.21.15.2:6809/1781959259","cluster_addr":"172.21.15.2:6811/1781959259","heartbeat_back_addr":"172.21.15.2:6815/1781959259","heartbeat_front_addr":"172.21.15.2:6813/1781959259","state":["exists","up"]},{"osd":2,"uuid":"0609ea8c-d975-490e-8844-04a3e4512efd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":1153570802},{"type":"v1","addr":"172.21.15.2:6801","nonce":1153570802}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":1153570802},{"type":"v1","addr":"172.21.15.2:6803","nonce":1153570802}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":1153570802},{"type":"v1","addr":"172.21.15.2:6807","nonce":1153570802}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":1153570802},{"type":"v1","addr":"172.21.15.2:6805","nonce":1153570802}]},"public_addr":"172.21.15.2:6801/1153570802","cluster_addr":"172.21.15.2:6803/1153570802","heartbeat_back_addr":"172.21.15.2:6807/1153570802","heartbeat_front_addr":"172.21.15.2:6805/1153570802","state":["exists","up"]},{"osd":3,"uuid":"c6bd5ac8-ab89-4fa2-b843-3b166d1b238b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":2008321375},{"type":"v1","addr":"172.21.15.2:6817","nonce":2008321375}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":2008321375},{"type":"v1","addr":"172.21.15.2:6819","nonce":2008321375}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2008321375},{"type":"v1","addr":"172.21.15.2:6823","nonce":2008321375}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":2008321375},{"type":"v1","addr":"172.21.15.2:6821","nonce":2008321375}]},"public_addr":"172.21.15.2:6817/2008321375","cluster_addr":"172.21.15.2:6819/2008321375","heartbeat_back_addr":"172.21.15.2:6823/2008321375","heartbeat_front_addr":"172.21.15.2:6821/2008321375","state":["exists","up"]},{"osd":4,"uuid":"89d6a371-4f4e-465e-a023-d5083cce6e6c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6824","nonce":2939896598},{"type":"v1","addr":"172.21.15.99:6825","nonce":2939896598}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6826","nonce":2939896598},{"type":"v1","addr":"172.21.15.99:6827","nonce":2939896598}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6830","nonce":2939896598},{"type":"v1","addr":"172.21.15.99:6831","nonce":2939896598}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6828","nonce":2939896598},{"type":"v1","addr":"172.21.15.99:6829","nonce":2939896598}]},"public_addr":"172.21.15.99:6825/2939896598","cluster_addr":"172.21.15.99:6827/2939896598","heartbeat_back_addr":"172.21.15.99:6831/2939896598","heartbeat_front_addr":"172.21.15.99:6829/2939896598","state":["exists","up"]},{"osd":5,"uuid":"da99686b-d979-4753-b4aa-a3b26e9a0497","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6816","nonce":3440879618},{"type":"v1","addr":"172.21.15.99:6817","nonce":3440879618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6818","nonce":3440879618},{"type":"v1","addr":"172.21.15.99:6819","nonce":3440879618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6822","nonce":3440879618},{"type":"v1","addr":"172.21.15.99:6823","nonce":3440879618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6820","nonce":3440879618},{"type":"v1","addr":"172.21.15.99:6821","nonce":3440879618}]},"public_addr":"172.21.15.99:6817/3440879618","cluster_addr":"172.21.15.99:6819/3440879618","heartbeat_back_addr":"172.21.15.99:6823/3440879618","heartbeat_front_addr":"172.21.15.99:6821/3440879618","state":["exists","up"]},{"osd":6,"uuid":"ceec6aa6-603d-4706-bfea-3c07dbdaed3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6800","nonce":2493830504},{"type":"v1","addr":"172.21.15.99:6801","nonce":2493830504}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6802","nonce":2493830504},{"type":"v1","addr":"172.21.15.99:6803","nonce":2493830504}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6806","nonce":2493830504},{"type":"v1","addr":"172.21.15.99:6807","nonce":2493830504}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6804","nonce":2493830504},{"type":"v1","addr":"172.21.15.99:6805","nonce":2493830504}]},"public_addr":"172.21.15.99:6801/2493830504","cluster_addr":"172.21.15.99:6803/2493830504","heartbeat_back_addr":"172.21.15.99:6807/2493830504","heartbeat_front_addr":"172.21.15.99:6805/2493830504","state":["exists","up"]},{"osd":7,"uuid":"6779dfad-f60b-41f4-b666-a99db6dc1814","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":174,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6808","nonce":91038858},{"type":"v1","addr":"172.21.15.99:6809","nonce":91038858}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6810","nonce":91038858},{"type":"v1","addr":"172.21.15.99:6811","nonce":91038858}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6814","nonce":91038858},{"type":"v1","addr":"172.21.15.99:6815","nonce":91038858}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.99:6812","nonce":91038858},{"type":"v1","addr":"172.21.15.99:6813","nonce":91038858}]},"public_addr":"172.21.15.99:6809/91038858","cluster_addr":"172.21.15.99:6811/91038858","heartbeat_back_addr":"172.21.15.99:6815/91038858","heartbeat_front_addr":"172.21.15.99:6813/91038858","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.468213+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.538610+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.500881+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.474179+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.483737+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.460693+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.475970+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T04:56:05.519287+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"1.4","osds":[6,7]},{"pgid":"1.5","osds":[1,4]}],"pg_upmap_items":[{"pgid":"3.1","mappings":[{"from":4,"to":2}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.11","mappings":[{"from":4,"to":1}]},{"pgid":"3.1a","mappings":[{"from":4,"to":6}]},{"pgid":"3.1b","mappings":[{"from":4,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T04:59:16.310 INFO:tasks.ceph:Scrubbing osd.0 2024-09-18T04:59:16.311 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-18T04:59:16.450 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:59:16.451 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T04:59:16.451 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:59:16.462 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-18T04:59:16.747 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 0 to deep-scrub 2024-09-18T04:59:16.759 INFO:tasks.ceph:Scrubbing osd.1 2024-09-18T04:59:16.759 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-18T04:59:16.885 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:59:16.885 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T04:59:16.885 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:59:16.895 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-18T04:59:17.181 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 1 to deep-scrub 2024-09-18T04:59:17.194 INFO:tasks.ceph:Scrubbing osd.2 2024-09-18T04:59:17.194 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-18T04:59:17.320 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:59:17.320 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T04:59:17.321 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:59:17.331 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-18T04:59:17.614 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 2 to deep-scrub 2024-09-18T04:59:17.627 INFO:tasks.ceph:Scrubbing osd.3 2024-09-18T04:59:17.627 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-18T04:59:17.752 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:59:17.753 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T04:59:17.753 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:59:17.763 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-18T04:59:18.047 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 3 to deep-scrub 2024-09-18T04:59:18.060 INFO:tasks.ceph:Scrubbing osd.4 2024-09-18T04:59:18.060 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-18T04:59:18.187 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:59:18.188 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T04:59:18.188 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:59:18.198 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-18T04:59:18.484 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 4 to deep-scrub 2024-09-18T04:59:18.497 INFO:tasks.ceph:Scrubbing osd.5 2024-09-18T04:59:18.497 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-18T04:59:18.623 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:59:18.624 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T04:59:18.624 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:59:18.634 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-18T04:59:18.919 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 5 to deep-scrub 2024-09-18T04:59:18.931 INFO:tasks.ceph:Scrubbing osd.6 2024-09-18T04:59:18.931 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-18T04:59:19.057 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:59:19.057 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T04:59:19.057 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:59:19.068 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-18T04:59:19.353 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 6 to deep-scrub 2024-09-18T04:59:19.367 INFO:tasks.ceph:Scrubbing osd.7 2024-09-18T04:59:19.367 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-18T04:59:19.494 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2024-09-18T04:59:19.494 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T04:59:19.494 INFO:teuthology.orchestra.run.smithi002.stdout:} 2024-09-18T04:59:19.504 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-18T04:59:19.787 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 7 to deep-scrub 2024-09-18T04:59:19.800 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T04:59:20.087 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:59:20.088 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-18T04:59:20.100 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-09-18T04:59:18.301052+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":47,"num_read":220900,"num_read_kb":683302,"num_write":30057,"num_write_kb":380196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9215,"num_bytes_recovered":185732404,"num_keys_recovered":562,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24084,"ondisk_log_size":24084,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":440352,"kb_used_data":16680,"kb_used_omap":12,"kb_used_meta":423539,"kb_avail":754534368,"statfs":{"total":773094113280,"available":772643192832,"internally_reserved":0,"allocated":17080320,"data_stored":9447556,"data_compressed":18433,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":12742,"internal_metadata":433704506},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":524,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.103044"},"pg_stats":[{"pgid":"3.1f","version":"158'766","reported_seq":5157,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.197217+0000","last_change":"2024-09-18T04:58:55.128107+0000","last_active":"2024-09-18T04:59:07.197217+0000","last_peered":"2024-09-18T04:59:07.197217+0000","last_clean":"2024-09-18T04:59:07.197217+0000","last_became_active":"2024-09-18T04:58:55.126111+0000","last_became_peered":"2024-09-18T04:58:55.126111+0000","last_unstale":"2024-09-18T04:59:07.197217+0000","last_undegraded":"2024-09-18T04:59:07.197217+0000","last_fullsized":"2024-09-18T04:59:07.197217+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:56:05.136229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001156916,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6348,"num_read_kb":20852,"num_write":870,"num_write_kb":9417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"157'817","reported_seq":2783,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731844+0000","last_change":"2024-09-18T04:58:48.096250+0000","last_active":"2024-09-18T04:59:07.731844+0000","last_peered":"2024-09-18T04:59:07.731844+0000","last_clean":"2024-09-18T04:59:07.731844+0000","last_became_active":"2024-09-18T04:58:32.029310+0000","last_became_peered":"2024-09-18T04:58:32.029310+0000","last_unstale":"2024-09-18T04:59:07.731844+0000","last_undegraded":"2024-09-18T04:59:07.731844+0000","last_fullsized":"2024-09-18T04:59:07.731844+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":817,"log_dups_size":0,"ondisk_log_size":817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:13:02.688292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001168749,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3351,"num_read_kb":18023,"num_write":1120,"num_write_kb":8161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":3252478,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"156'500","reported_seq":10759,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195673+0000","last_change":"2024-09-18T04:58:48.023738+0000","last_active":"2024-09-18T04:59:07.195673+0000","last_peered":"2024-09-18T04:59:07.195673+0000","last_clean":"2024-09-18T04:59:07.195673+0000","last_became_active":"2024-09-18T04:58:27.115104+0000","last_became_peered":"2024-09-18T04:58:27.115104+0000","last_unstale":"2024-09-18T04:59:07.195673+0000","last_undegraded":"2024-09-18T04:59:07.195673+0000","last_fullsized":"2024-09-18T04:59:07.195673+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:33:08.572507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082741200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9761,"num_read_kb":28168,"num_write":548,"num_write_kb":23097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7293282,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"158'840","reported_seq":7984,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195584+0000","last_change":"2024-09-18T04:58:48.025083+0000","last_active":"2024-09-18T04:59:07.195584+0000","last_peered":"2024-09-18T04:59:07.195584+0000","last_clean":"2024-09-18T04:59:07.195584+0000","last_became_active":"2024-09-18T04:58:28.028905+0000","last_became_peered":"2024-09-18T04:58:28.028905+0000","last_unstale":"2024-09-18T04:59:07.195584+0000","last_undegraded":"2024-09-18T04:59:07.195584+0000","last_fullsized":"2024-09-18T04:59:07.195584+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:15:17.569793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014698000000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":8594,"num_read_kb":9084,"num_write":1169,"num_write_kb":5309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":1412856,"num_keys_recovered":106,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"155'473","reported_seq":4267,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.202696+0000","last_change":"2024-09-18T04:59:07.202189+0000","last_active":"2024-09-18T04:59:07.202696+0000","last_peered":"2024-09-18T04:59:07.202696+0000","last_clean":"2024-09-18T04:59:07.202696+0000","last_became_active":"2024-09-18T04:59:07.199588+0000","last_became_peered":"2024-09-18T04:59:07.199588+0000","last_unstale":"2024-09-18T04:59:07.202696+0000","last_undegraded":"2024-09-18T04:59:07.202696+0000","last_fullsized":"2024-09-18T04:59:07.202696+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:53:25.971943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012312829999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2677,"num_read_kb":15514,"num_write":503,"num_write_kb":9112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":912,"num_bytes_recovered":7005168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"157'455","reported_seq":3935,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:08.199477+0000","last_change":"2024-09-18T04:59:07.200974+0000","last_active":"2024-09-18T04:59:08.199477+0000","last_peered":"2024-09-18T04:59:08.199477+0000","last_clean":"2024-09-18T04:59:08.199477+0000","last_became_active":"2024-09-18T04:59:07.199276+0000","last_became_peered":"2024-09-18T04:59:07.199276+0000","last_unstale":"2024-09-18T04:59:08.199477+0000","last_undegraded":"2024-09-18T04:59:08.199477+0000","last_fullsized":"2024-09-18T04:59:08.199477+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:42:10.766349+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.000757346,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":2981,"num_read_kb":11740,"num_write":512,"num_write_kb":18285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":519,"num_bytes_recovered":11494100,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"156'520","reported_seq":5473,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198149+0000","last_change":"2024-09-18T04:58:57.148756+0000","last_active":"2024-09-18T04:59:07.198149+0000","last_peered":"2024-09-18T04:59:07.198149+0000","last_clean":"2024-09-18T04:59:07.198149+0000","last_became_active":"2024-09-18T04:58:57.145900+0000","last_became_peered":"2024-09-18T04:58:57.145900+0000","last_unstale":"2024-09-18T04:59:07.198149+0000","last_undegraded":"2024-09-18T04:59:07.198149+0000","last_fullsized":"2024-09-18T04:59:07.198149+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":520,"log_dups_size":0,"ondisk_log_size":520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:03:40.179686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018405870000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4543,"num_read_kb":26564,"num_write":538,"num_write_kb":13069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":23811007,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"156'576","reported_seq":3426,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731747+0000","last_change":"2024-09-18T04:58:48.095394+0000","last_active":"2024-09-18T04:59:07.731747+0000","last_peered":"2024-09-18T04:59:07.731747+0000","last_clean":"2024-09-18T04:59:07.731747+0000","last_became_active":"2024-09-18T04:58:32.029074+0000","last_became_peered":"2024-09-18T04:58:32.029074+0000","last_unstale":"2024-09-18T04:59:07.731747+0000","last_undegraded":"2024-09-18T04:59:07.731747+0000","last_fullsized":"2024-09-18T04:59:07.731747+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":576,"log_dups_size":0,"ondisk_log_size":576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:15:38.950060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053623899999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3403,"num_read_kb":16958,"num_write":699,"num_write_kb":12986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":4650417,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"153'506","reported_seq":4239,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.197394+0000","last_change":"2024-09-18T04:58:55.128037+0000","last_active":"2024-09-18T04:59:07.197394+0000","last_peered":"2024-09-18T04:59:07.197394+0000","last_clean":"2024-09-18T04:59:07.197394+0000","last_became_active":"2024-09-18T04:58:55.125721+0000","last_became_peered":"2024-09-18T04:58:55.125721+0000","last_unstale":"2024-09-18T04:59:07.197394+0000","last_undegraded":"2024-09-18T04:59:07.197394+0000","last_fullsized":"2024-09-18T04:59:07.197394+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:06:23.656568+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001441276,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4899,"num_read_kb":13699,"num_write":541,"num_write_kb":6077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"154'885","reported_seq":5042,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195463+0000","last_change":"2024-09-18T04:58:48.024658+0000","last_active":"2024-09-18T04:59:07.195463+0000","last_peered":"2024-09-18T04:59:07.195463+0000","last_clean":"2024-09-18T04:59:07.195463+0000","last_became_active":"2024-09-18T04:56:21.131133+0000","last_became_peered":"2024-09-18T04:56:21.131133+0000","last_unstale":"2024-09-18T04:59:07.195463+0000","last_undegraded":"2024-09-18T04:59:07.195463+0000","last_fullsized":"2024-09-18T04:59:07.195463+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":885,"log_dups_size":0,"ondisk_log_size":885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:01:57.036904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012610480000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6340,"num_read_kb":26579,"num_write":1141,"num_write_kb":12621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"157'637","reported_seq":5292,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198169+0000","last_change":"2024-09-18T04:58:48.028934+0000","last_active":"2024-09-18T04:59:07.198169+0000","last_peered":"2024-09-18T04:59:07.198169+0000","last_clean":"2024-09-18T04:59:07.198169+0000","last_became_active":"2024-09-18T04:58:32.032762+0000","last_became_peered":"2024-09-18T04:58:32.032762+0000","last_unstale":"2024-09-18T04:59:07.198169+0000","last_undegraded":"2024-09-18T04:59:07.198169+0000","last_fullsized":"2024-09-18T04:59:07.198169+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":637,"log_dups_size":0,"ondisk_log_size":637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:33:25.010425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00066811099999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":5292,"num_read_kb":26013,"num_write":596,"num_write_kb":7300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":447,"num_bytes_recovered":1594801,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"155'502","reported_seq":2818,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.688094+0000","last_change":"2024-09-18T04:58:48.023734+0000","last_active":"2024-09-18T04:59:07.688094+0000","last_peered":"2024-09-18T04:59:07.688094+0000","last_clean":"2024-09-18T04:59:07.688094+0000","last_became_active":"2024-09-18T04:58:30.025558+0000","last_became_peered":"2024-09-18T04:58:30.025558+0000","last_unstale":"2024-09-18T04:59:07.688094+0000","last_undegraded":"2024-09-18T04:59:07.688094+0000","last_fullsized":"2024-09-18T04:59:07.688094+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:57:31.113016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087474099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3131,"num_read_kb":27325,"num_write":609,"num_write_kb":11672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":12427258,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":192,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195662+0000","last_change":"2024-09-18T04:58:29.025271+0000","last_active":"2024-09-18T04:59:07.195662+0000","last_peered":"2024-09-18T04:59:07.195662+0000","last_clean":"2024-09-18T04:59:07.195662+0000","last_became_active":"2024-09-18T04:58:29.024911+0000","last_became_peered":"2024-09-18T04:58:29.024911+0000","last_unstale":"2024-09-18T04:59:07.195662+0000","last_undegraded":"2024-09-18T04:59:07.195662+0000","last_fullsized":"2024-09-18T04:59:07.195662+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:53:21.863837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063436799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"155'2480","reported_seq":9899,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195724+0000","last_change":"2024-09-18T04:58:48.024643+0000","last_active":"2024-09-18T04:59:07.195724+0000","last_peered":"2024-09-18T04:59:07.195724+0000","last_clean":"2024-09-18T04:59:07.195724+0000","last_became_active":"2024-09-18T04:58:33.027340+0000","last_became_peered":"2024-09-18T04:58:33.027340+0000","last_unstale":"2024-09-18T04:59:07.195724+0000","last_undegraded":"2024-09-18T04:59:07.195724+0000","last_fullsized":"2024-09-18T04:59:07.195724+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":2480,"log_dups_size":0,"ondisk_log_size":2480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:47:47.826132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000952045,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":18529,"num_read_kb":32990,"num_write":4379,"num_write_kb":20206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":5439803,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":176,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195585+0000","last_change":"2024-09-18T04:56:12.389971+0000","last_active":"2024-09-18T04:59:07.195585+0000","last_peered":"2024-09-18T04:59:07.195585+0000","last_clean":"2024-09-18T04:59:07.195585+0000","last_became_active":"2024-09-18T04:56:10.412595+0000","last_became_peered":"2024-09-18T04:56:10.412595+0000","last_unstale":"2024-09-18T04:59:07.195585+0000","last_undegraded":"2024-09-18T04:59:07.195585+0000","last_fullsized":"2024-09-18T04:59:07.195585+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:22:45.638420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038576799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"157'807","reported_seq":7861,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198219+0000","last_change":"2024-09-18T04:58:57.151001+0000","last_active":"2024-09-18T04:59:07.198219+0000","last_peered":"2024-09-18T04:59:07.198219+0000","last_clean":"2024-09-18T04:59:07.198219+0000","last_became_active":"2024-09-18T04:58:57.147959+0000","last_became_peered":"2024-09-18T04:58:57.147959+0000","last_unstale":"2024-09-18T04:59:07.198219+0000","last_undegraded":"2024-09-18T04:59:07.198219+0000","last_fullsized":"2024-09-18T04:59:07.198219+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:30:19.725543+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00065254400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8940,"num_read_kb":28417,"num_write":1051,"num_write_kb":11406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":5706670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198307+0000","last_change":"2024-09-18T04:58:57.150288+0000","last_active":"2024-09-18T04:59:07.198307+0000","last_peered":"2024-09-18T04:59:07.198307+0000","last_clean":"2024-09-18T04:59:07.198307+0000","last_became_active":"2024-09-18T04:58:57.149907+0000","last_became_peered":"2024-09-18T04:58:57.149907+0000","last_unstale":"2024-09-18T04:59:07.198307+0000","last_undegraded":"2024-09-18T04:59:07.198307+0000","last_fullsized":"2024-09-18T04:59:07.198307+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:12:39.499969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"157'757","reported_seq":6181,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731891+0000","last_change":"2024-09-18T04:58:48.096507+0000","last_active":"2024-09-18T04:59:07.731891+0000","last_peered":"2024-09-18T04:59:07.731891+0000","last_clean":"2024-09-18T04:59:07.731891+0000","last_became_active":"2024-09-18T04:58:33.026806+0000","last_became_peered":"2024-09-18T04:58:33.026806+0000","last_unstale":"2024-09-18T04:59:07.731891+0000","last_undegraded":"2024-09-18T04:59:07.731891+0000","last_fullsized":"2024-09-18T04:59:07.731891+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":757,"log_dups_size":0,"ondisk_log_size":757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:59:41.673907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013736829999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6896,"num_read_kb":21261,"num_write":943,"num_write_kb":5208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":571544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":145,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195438+0000","last_change":"2024-09-18T04:56:46.254241+0000","last_active":"2024-09-18T04:59:07.195438+0000","last_peered":"2024-09-18T04:59:07.195438+0000","last_clean":"2024-09-18T04:59:07.195438+0000","last_became_active":"2024-09-18T04:56:46.253894+0000","last_became_peered":"2024-09-18T04:56:46.253894+0000","last_unstale":"2024-09-18T04:59:07.195438+0000","last_undegraded":"2024-09-18T04:59:07.195438+0000","last_fullsized":"2024-09-18T04:59:07.195438+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:58:42.098099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045144100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"158'505","reported_seq":5028,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.197318+0000","last_change":"2024-09-18T04:58:48.061503+0000","last_active":"2024-09-18T04:59:07.197318+0000","last_peered":"2024-09-18T04:59:07.197318+0000","last_clean":"2024-09-18T04:59:07.197318+0000","last_became_active":"2024-09-18T04:58:41.078435+0000","last_became_peered":"2024-09-18T04:58:41.078435+0000","last_unstale":"2024-09-18T04:59:07.197318+0000","last_undegraded":"2024-09-18T04:59:07.197318+0000","last_fullsized":"2024-09-18T04:59:07.197318+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:34:50.214360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056043399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4221,"num_read_kb":33044,"num_write":472,"num_write_kb":37350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":25349011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"155'485","reported_seq":9536,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.201259+0000","last_change":"2024-09-18T04:59:07.200564+0000","last_active":"2024-09-18T04:59:07.201259+0000","last_peered":"2024-09-18T04:59:07.201259+0000","last_clean":"2024-09-18T04:59:07.201259+0000","last_became_active":"2024-09-18T04:59:07.199056+0000","last_became_peered":"2024-09-18T04:59:07.199056+0000","last_unstale":"2024-09-18T04:59:07.201259+0000","last_undegraded":"2024-09-18T04:59:07.201259+0000","last_fullsized":"2024-09-18T04:59:07.201259+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":485,"log_dups_size":0,"ondisk_log_size":485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:30:30.980376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049796699999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":9360,"num_read_kb":28051,"num_write":499,"num_write_kb":12488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":472,"num_bytes_recovered":11362815,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":362,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198082+0000","last_change":"2024-09-18T04:58:38.025236+0000","last_active":"2024-09-18T04:59:07.198082+0000","last_peered":"2024-09-18T04:59:07.198082+0000","last_clean":"2024-09-18T04:59:07.198082+0000","last_became_active":"2024-09-18T04:58:38.024740+0000","last_became_peered":"2024-09-18T04:58:38.024740+0000","last_unstale":"2024-09-18T04:59:07.198082+0000","last_undegraded":"2024-09-18T04:59:07.198082+0000","last_fullsized":"2024-09-18T04:59:07.198082+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:10.373785+0000","last_clean_scrub_stamp":"2024-09-18T04:56:10.373785+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:10:45.251654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"15'1","reported_seq":37,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198355+0000","last_change":"2024-09-18T04:58:57.148289+0000","last_active":"2024-09-18T04:59:07.198355+0000","last_peered":"2024-09-18T04:59:07.198355+0000","last_clean":"2024-09-18T04:59:07.198355+0000","last_became_active":"2024-09-18T04:58:57.147915+0000","last_became_peered":"2024-09-18T04:58:57.147915+0000","last_unstale":"2024-09-18T04:59:07.198355+0000","last_undegraded":"2024-09-18T04:59:07.198355+0000","last_fullsized":"2024-09-18T04:59:07.198355+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:31:21.699401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"157'811","reported_seq":13428,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198153+0000","last_change":"2024-09-18T04:58:57.152156+0000","last_active":"2024-09-18T04:59:07.198153+0000","last_peered":"2024-09-18T04:59:07.198153+0000","last_clean":"2024-09-18T04:59:07.198153+0000","last_became_active":"2024-09-18T04:58:57.148221+0000","last_became_peered":"2024-09-18T04:58:57.148221+0000","last_unstale":"2024-09-18T04:59:07.198153+0000","last_undegraded":"2024-09-18T04:59:07.198153+0000","last_fullsized":"2024-09-18T04:59:07.198153+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":811,"log_dups_size":0,"ondisk_log_size":811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:45:57.434891+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0033124909999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":14837,"num_read_kb":27169,"num_write":1047,"num_write_kb":7785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":1280827,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"17'2","reported_seq":178,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198076+0000","last_change":"2024-09-18T04:56:12.398631+0000","last_active":"2024-09-18T04:59:07.198076+0000","last_peered":"2024-09-18T04:59:07.198076+0000","last_clean":"2024-09-18T04:59:07.198076+0000","last_became_active":"2024-09-18T04:56:10.412614+0000","last_became_peered":"2024-09-18T04:56:10.412614+0000","last_unstale":"2024-09-18T04:59:07.198076+0000","last_undegraded":"2024-09-18T04:59:07.198076+0000","last_fullsized":"2024-09-18T04:59:07.198076+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:44:13.401320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075126299999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"158'601","reported_seq":2978,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.688184+0000","last_change":"2024-09-18T04:58:48.024976+0000","last_active":"2024-09-18T04:59:07.688184+0000","last_peered":"2024-09-18T04:59:07.688184+0000","last_clean":"2024-09-18T04:59:07.688184+0000","last_became_active":"2024-09-18T04:58:30.027757+0000","last_became_peered":"2024-09-18T04:58:30.027757+0000","last_unstale":"2024-09-18T04:59:07.688184+0000","last_undegraded":"2024-09-18T04:59:07.688184+0000","last_fullsized":"2024-09-18T04:59:07.688184+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:29:00.106938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014951890000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":2216,"num_read_kb":23967,"num_write":593,"num_write_kb":16097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":10562840,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":176,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195410+0000","last_change":"2024-09-18T04:56:12.390044+0000","last_active":"2024-09-18T04:59:07.195410+0000","last_peered":"2024-09-18T04:59:07.195410+0000","last_clean":"2024-09-18T04:59:07.195410+0000","last_became_active":"2024-09-18T04:56:10.413042+0000","last_became_peered":"2024-09-18T04:56:10.413042+0000","last_unstale":"2024-09-18T04:59:07.195410+0000","last_undegraded":"2024-09-18T04:59:07.195410+0000","last_fullsized":"2024-09-18T04:59:07.195410+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:46:11.074243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000450999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"157'525","reported_seq":4818,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731809+0000","last_change":"2024-09-18T04:58:48.095298+0000","last_active":"2024-09-18T04:59:07.731809+0000","last_peered":"2024-09-18T04:59:07.731809+0000","last_clean":"2024-09-18T04:59:07.731809+0000","last_became_active":"2024-09-18T04:58:32.027104+0000","last_became_peered":"2024-09-18T04:58:32.027104+0000","last_unstale":"2024-09-18T04:59:07.731809+0000","last_undegraded":"2024-09-18T04:59:07.731809+0000","last_fullsized":"2024-09-18T04:59:07.731809+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:41:37.134684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056931099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5904,"num_read_kb":13840,"num_write":503,"num_write_kb":5042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":2187784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198027+0000","last_change":"2024-09-18T04:57:40.472681+0000","last_active":"2024-09-18T04:59:07.198027+0000","last_peered":"2024-09-18T04:59:07.198027+0000","last_clean":"2024-09-18T04:59:07.198027+0000","last_became_active":"2024-09-18T04:57:40.472217+0000","last_became_peered":"2024-09-18T04:57:40.472217+0000","last_unstale":"2024-09-18T04:59:07.198027+0000","last_undegraded":"2024-09-18T04:59:07.198027+0000","last_fullsized":"2024-09-18T04:59:07.198027+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:09.371788+0000","last_clean_scrub_stamp":"2024-09-18T04:56:09.371788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:27:11.849311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"157'482","reported_seq":3615,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731740+0000","last_change":"2024-09-18T04:58:48.096157+0000","last_active":"2024-09-18T04:59:07.731740+0000","last_peered":"2024-09-18T04:59:07.731740+0000","last_clean":"2024-09-18T04:59:07.731740+0000","last_became_active":"2024-09-18T04:58:38.024916+0000","last_became_peered":"2024-09-18T04:58:38.024916+0000","last_unstale":"2024-09-18T04:59:07.731740+0000","last_undegraded":"2024-09-18T04:59:07.731740+0000","last_fullsized":"2024-09-18T04:59:07.731740+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":482,"log_dups_size":0,"ondisk_log_size":482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:32:42.202542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001087188,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2758,"num_read_kb":7938,"num_write":473,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":458,"num_bytes_recovered":2219067,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"155'714","reported_seq":12010,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.688322+0000","last_change":"2024-09-18T04:58:48.024809+0000","last_active":"2024-09-18T04:59:07.688322+0000","last_peered":"2024-09-18T04:59:07.688322+0000","last_clean":"2024-09-18T04:59:07.688322+0000","last_became_active":"2024-09-18T04:58:26.101490+0000","last_became_peered":"2024-09-18T04:58:26.101490+0000","last_unstale":"2024-09-18T04:59:07.688322+0000","last_undegraded":"2024-09-18T04:59:07.688322+0000","last_fullsized":"2024-09-18T04:59:07.688322+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":714,"log_dups_size":0,"ondisk_log_size":714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:30:15.538062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015101680000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":12956,"num_read_kb":26590,"num_write":848,"num_write_kb":15777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":5822953,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"157'470","reported_seq":6073,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195769+0000","last_change":"2024-09-18T04:58:48.136195+0000","last_active":"2024-09-18T04:59:07.195769+0000","last_peered":"2024-09-18T04:59:07.195769+0000","last_clean":"2024-09-18T04:59:07.195769+0000","last_became_active":"2024-09-18T04:58:27.115137+0000","last_became_peered":"2024-09-18T04:58:27.115137+0000","last_unstale":"2024-09-18T04:59:07.195769+0000","last_undegraded":"2024-09-18T04:59:07.195769+0000","last_fullsized":"2024-09-18T04:59:07.195769+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":470,"log_dups_size":0,"ondisk_log_size":470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:57:42.359624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039325500000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":4725,"num_read_kb":14773,"num_write":478,"num_write_kb":8139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":6055124,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"155'1400","reported_seq":5772,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.731780+0000","last_change":"2024-09-18T04:58:48.095758+0000","last_active":"2024-09-18T04:59:07.731780+0000","last_peered":"2024-09-18T04:59:07.731780+0000","last_clean":"2024-09-18T04:59:07.731780+0000","last_became_active":"2024-09-18T04:58:31.026041+0000","last_became_peered":"2024-09-18T04:58:31.026041+0000","last_unstale":"2024-09-18T04:59:07.731780+0000","last_undegraded":"2024-09-18T04:59:07.731780+0000","last_fullsized":"2024-09-18T04:59:07.731780+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":1400,"log_dups_size":0,"ondisk_log_size":1400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:55:29.526466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083671099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8851,"num_read_kb":15183,"num_write":2049,"num_write_kb":8862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":1966128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"155'908","reported_seq":7203,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195547+0000","last_change":"2024-09-18T04:58:48.023609+0000","last_active":"2024-09-18T04:59:07.195547+0000","last_peered":"2024-09-18T04:59:07.195547+0000","last_clean":"2024-09-18T04:59:07.195547+0000","last_became_active":"2024-09-18T04:58:33.027525+0000","last_became_peered":"2024-09-18T04:58:33.027525+0000","last_unstale":"2024-09-18T04:59:07.195547+0000","last_undegraded":"2024-09-18T04:59:07.195547+0000","last_fullsized":"2024-09-18T04:59:07.195547+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:27:34.074204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062770000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9983,"num_read_kb":24174,"num_write":928,"num_write_kb":8107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":86198,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"155'617","reported_seq":7056,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.194912+0000","last_change":"2024-09-18T04:58:48.021010+0000","last_active":"2024-09-18T04:59:07.194912+0000","last_peered":"2024-09-18T04:59:07.194912+0000","last_clean":"2024-09-18T04:59:07.194912+0000","last_became_active":"2024-09-18T04:58:07.017827+0000","last_became_peered":"2024-09-18T04:58:07.017827+0000","last_unstale":"2024-09-18T04:59:07.194912+0000","last_undegraded":"2024-09-18T04:59:07.194912+0000","last_fullsized":"2024-09-18T04:59:07.194912+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:42:31.897588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050117499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8566,"num_read_kb":18769,"num_write":595,"num_write_kb":12467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":2872264,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"155'1050","reported_seq":5261,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198243+0000","last_change":"2024-09-18T04:58:48.029466+0000","last_active":"2024-09-18T04:59:07.198243+0000","last_peered":"2024-09-18T04:59:07.198243+0000","last_clean":"2024-09-18T04:59:07.198243+0000","last_became_active":"2024-09-18T04:58:31.028189+0000","last_became_peered":"2024-09-18T04:58:31.028189+0000","last_unstale":"2024-09-18T04:59:07.198243+0000","last_undegraded":"2024-09-18T04:59:07.198243+0000","last_fullsized":"2024-09-18T04:59:07.198243+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":1050,"log_dups_size":0,"ondisk_log_size":1050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:26:10.430573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001035219,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7660,"num_read_kb":17955,"num_write":1663,"num_write_kb":5543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":2749742,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"157'556","reported_seq":5095,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.198207+0000","last_change":"2024-09-18T04:58:48.029669+0000","last_active":"2024-09-18T04:59:07.198207+0000","last_peered":"2024-09-18T04:59:07.198207+0000","last_clean":"2024-09-18T04:59:07.198207+0000","last_became_active":"2024-09-18T04:58:29.029625+0000","last_became_peered":"2024-09-18T04:58:29.029625+0000","last_unstale":"2024-09-18T04:59:07.198207+0000","last_undegraded":"2024-09-18T04:59:07.198207+0000","last_fullsized":"2024-09-18T04:59:07.198207+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:51:09.257320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011607150000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4690,"num_read_kb":26470,"num_write":636,"num_write_kb":8108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":5504104,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"157'809","reported_seq":4779,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.195411+0000","last_change":"2024-09-18T04:58:48.136237+0000","last_active":"2024-09-18T04:59:07.195411+0000","last_peered":"2024-09-18T04:59:07.195411+0000","last_clean":"2024-09-18T04:59:07.195411+0000","last_became_active":"2024-09-18T04:56:21.131202+0000","last_became_peered":"2024-09-18T04:56:21.131202+0000","last_unstale":"2024-09-18T04:59:07.195411+0000","last_undegraded":"2024-09-18T04:59:07.195411+0000","last_fullsized":"2024-09-18T04:59:07.195411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:17:35.438723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041769600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5877,"num_read_kb":14669,"num_write":993,"num_write_kb":10354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"157'920","reported_seq":8474,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.202504+0000","last_change":"2024-09-18T04:59:07.201833+0000","last_active":"2024-09-18T04:59:07.202504+0000","last_peered":"2024-09-18T04:59:07.202504+0000","last_clean":"2024-09-18T04:59:07.202504+0000","last_became_active":"2024-09-18T04:59:07.199844+0000","last_became_peered":"2024-09-18T04:59:07.199844+0000","last_unstale":"2024-09-18T04:59:07.202504+0000","last_undegraded":"2024-09-18T04:59:07.202504+0000","last_fullsized":"2024-09-18T04:59:07.202504+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":920,"log_dups_size":0,"ondisk_log_size":920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:40:08.684972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087268100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10766,"num_read_kb":19560,"num_write":908,"num_write_kb":18676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":9230703,"num_keys_recovered":59,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"144'809","reported_seq":5825,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-18T04:59:07.197043+0000","last_change":"2024-09-18T04:58:55.130930+0000","last_active":"2024-09-18T04:59:07.197043+0000","last_peered":"2024-09-18T04:59:07.197043+0000","last_clean":"2024-09-18T04:59:07.197043+0000","last_became_active":"2024-09-18T04:58:55.129007+0000","last_became_peered":"2024-09-18T04:58:55.129007+0000","last_unstale":"2024-09-18T04:59:07.197043+0000","last_undegraded":"2024-09-18T04:59:07.197043+0000","last_fullsized":"2024-09-18T04:59:07.197043+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:21:11.287190+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.00099438999999999994,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7388,"num_read_kb":33089,"num_write":1110,"num_write_kb":12855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"158'781","reported_seq":2933,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:09.304030+0000","last_change":"2024-09-18T04:58:48.030265+0000","last_active":"2024-09-18T04:59:09.304030+0000","last_peered":"2024-09-18T04:59:09.304030+0000","last_clean":"2024-09-18T04:59:09.304030+0000","last_became_active":"2024-09-18T04:58:31.028633+0000","last_became_peered":"2024-09-18T04:58:31.028633+0000","last_unstale":"2024-09-18T04:59:09.304030+0000","last_undegraded":"2024-09-18T04:59:09.304030+0000","last_fullsized":"2024-09-18T04:59:09.304030+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:56:20.112128+0000","last_clean_scrub_stamp":"2024-09-18T04:56:20.112128+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:04:25.747259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00140489,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4341,"num_read_kb":14774,"num_write":882,"num_write_kb":9994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":12642724,"num_keys_recovered":71,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":47,"num_read":220784,"num_read_kb":683202,"num_write":29896,"num_write_kb":377778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9208,"num_bytes_recovered":184551668,"num_keys_recovered":562,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23964,"ondisk_log_size":23964,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28167,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51296,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320544,"statfs":{"total":96636764160,"available":96584237056,"internally_reserved":0,"allocated":2183168,"data_stored":1623970,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":50330059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.45600000000000002,"15min":0.46400000000000002},"min":{"1min":0.23799999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.53100000000000003,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.438},{"interface":"front","average":{"1min":0.437,"5min":0.41499999999999998,"15min":0.41099999999999998},"min":{"1min":0.312,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.41999999999999998}]},{"osd":1,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.46899999999999997,"15min":0.47799999999999998},"min":{"1min":0.249,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.55100000000000005,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.436,"15min":0.434},"min":{"1min":0.28399999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.495}]},{"osd":2,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.44700000000000001,"15min":0.44500000000000001},"min":{"1min":0.35399999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.45200000000000001,"15min":0.45000000000000001},"min":{"1min":0.34200000000000003,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.56200000000000006,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.45400000000000001}]},{"osd":3,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.52600000000000002,"15min":0.51900000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.4399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.51600000000000001,"15min":0.5},"min":{"1min":0.34100000000000003,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":2.4809999999999999,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":0.53500000000000003}]},{"osd":4,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.42199999999999999,"15min":0.43099999999999999},"min":{"1min":0.26500000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.63700000000000001,"5min":1.954,"15min":1.954},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.41699999999999998,"15min":0.42399999999999999},"min":{"1min":0.246,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.629,"5min":1.478,"15min":1.478},"last":0.38100000000000001}]},{"osd":5,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.50800000000000001,"15min":0.52200000000000002},"min":{"1min":0.30299999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.71799999999999997,"5min":3.145,"15min":3.145},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.443,"15min":0.441},"min":{"1min":0.27700000000000002,"5min":0.247,"15min":0.247},"max":{"1min":0.65800000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.58199999999999996}]},{"osd":6,"last update":"Wed Sep 18 04:58:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.47899999999999998,"15min":0.48299999999999998},"min":{"1min":0.33300000000000002,"5min":0.312,"15min":0.312},"max":{"1min":0.69299999999999995,"5min":0.746,"15min":0.746},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.48899999999999999,"15min":0.49299999999999999},"min":{"1min":0.35199999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.623,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.55000000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574889,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55336,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316504,"statfs":{"total":96636764160,"available":96580100096,"internally_reserved":0,"allocated":2121728,"data_stored":1032642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.71899999999999997,"15min":0.76900000000000002},"min":{"1min":0.23899999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.56799999999999995,"5min":6.1319999999999997,"15min":6.1319999999999997},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.75700000000000001,"15min":0.80500000000000005},"min":{"1min":0.35799999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.81100000000000005,"5min":6.1429999999999998,"15min":6.1429999999999998},"last":0.44800000000000001}]},{"osd":1,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.70999999999999996,"15min":0.76200000000000001},"min":{"1min":0.29299999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.44900000000000001,"5min":6.0410000000000004,"15min":6.0410000000000004},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.70899999999999996,"15min":0.753},"min":{"1min":0.33000000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.83599999999999997,"5min":5.5380000000000003,"15min":5.5380000000000003},"last":0.505}]},{"osd":2,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.41999999999999998,"15min":0.42599999999999999},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.54400000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.46100000000000002,"15min":0.46800000000000003},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.55900000000000005,"5min":1.04,"15min":1.04},"last":0.54600000000000004}]},{"osd":3,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46000000000000002,"15min":0.46100000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.47799999999999998,"15min":0.48099999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.625,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.53000000000000003}]},{"osd":4,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.36799999999999999,"15min":0.374},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.47599999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.43099999999999999,"15min":0.437},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.38900000000000001}]},{"osd":5,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.372,"15min":0.378},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.42499999999999999,"15min":0.435},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.53000000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.317}]},{"osd":7,"last update":"Wed Sep 18 04:58:14 2024","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.46800000000000003,"15min":0.46600000000000003},"min":{"1min":0.28100000000000003,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":2.6219999999999999,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.48599999999999999,"15min":0.47699999999999998},"min":{"1min":0.34399999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.43099999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574889,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65188,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94306652,"statfs":{"total":96636764160,"available":96570011648,"internally_reserved":0,"allocated":2121728,"data_stored":1033583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.54600000000000004,"15min":0.57099999999999995},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.59199999999999997,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.46800000000000003,"5min":0.504,"15min":0.51000000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.71599999999999997,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.499}]},{"osd":1,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.53300000000000003,"15min":0.55400000000000005},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.55600000000000005,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.32500000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.63800000000000001,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.52900000000000003}]},{"osd":2,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47699999999999998,"15min":0.47899999999999998},"min":{"1min":0.28699999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.629,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.57399999999999995,"15min":0.59999999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.626,"5min":3.1070000000000002,"15min":3.1070000000000002},"last":0.45600000000000002}]},{"osd":3,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.496,"15min":0.502},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.77000000000000002,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.504,"15min":0.50800000000000001},"min":{"1min":0.35099999999999998,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.63,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.47799999999999998}]},{"osd":4,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.46800000000000003,"15min":0.47399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.59499999999999997,"5min":1.599,"15min":1.599},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.56299999999999994,"15min":0.59299999999999997},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.53500000000000003,"5min":3.379,"15min":3.379},"last":0.64500000000000002}]},{"osd":6,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.45500000000000002,"15min":0.46600000000000003},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.54200000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.56200000000000006,"15min":0.58499999999999996},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":1.085,"5min":3.1669999999999998,"15min":3.1669999999999998},"last":0.59999999999999998}]},{"osd":7,"last update":"Wed Sep 18 04:58:17 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.48399999999999999,"15min":0.495},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.80200000000000005,"5min":1.224,"15min":1.224},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.439,"5min":0.54200000000000004,"15min":0.55900000000000005},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.58599999999999997,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.58399999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574889,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63528,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94308312,"statfs":{"total":96636764160,"available":96571711488,"internally_reserved":0,"allocated":2121728,"data_stored":1033583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.46000000000000002,"15min":0.44400000000000001},"min":{"1min":0.23499999999999999,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":3.7069999999999999,"5min":3.7069999999999999,"15min":3.7069999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.46100000000000002,"15min":0.441},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":3.3700000000000001,"5min":3.3700000000000001,"15min":3.3700000000000001},"last":0.498}]},{"osd":1,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.47099999999999997,"15min":0.45500000000000002},"min":{"1min":0.28199999999999997,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":3.2879999999999998,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.46600000000000003,"15min":0.44600000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":3.661,"5min":3.661,"15min":3.661},"last":0.47599999999999998}]},{"osd":2,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.52200000000000002,"15min":0.50600000000000001},"min":{"1min":0.29999999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.1890000000000001,"5min":3.1890000000000001,"15min":3.1890000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.51600000000000001,"15min":0.502},"min":{"1min":0.32300000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":3.3860000000000001,"5min":3.3860000000000001,"15min":3.3860000000000001},"last":0.58199999999999996}]},{"osd":3,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.504,"15min":0.48899999999999999},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":3.4500000000000002,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.48299999999999998,"15min":0.46100000000000002},"min":{"1min":0.372,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":3.3210000000000002,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.71799999999999997}]},{"osd":5,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.42599999999999999,"15min":0.40200000000000002},"min":{"1min":0.25,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":3.9319999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.43099999999999999,"15min":0.40699999999999997},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":3.9449999999999998,"5min":3.9449999999999998,"15min":3.9449999999999998},"last":0.70299999999999996}]},{"osd":6,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.45100000000000001,"15min":0.42899999999999999},"min":{"1min":0.29299999999999998,"5min":0.251,"15min":0.251},"max":{"1min":3.3530000000000002,"5min":3.3530000000000002,"15min":3.3530000000000002},"last":0.55000000000000004},{"interface":"front","average":{"1min":1.29,"5min":0.626,"15min":0.51600000000000001},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":17.422999999999998,"5min":17.422999999999998,"15min":17.422999999999998},"last":0.65600000000000003}]},{"osd":7,"last update":"Wed Sep 18 04:58:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.48399999999999999,"15min":0.46400000000000002},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":3.7330000000000001,"5min":3.7330000000000001,"15min":3.7330000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.52100000000000002,"15min":0.499},"min":{"1min":0.33300000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":3.6930000000000001,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":0.76700000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574889,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87336,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284504,"statfs":{"total":96636764160,"available":96547332096,"internally_reserved":0,"allocated":2121728,"data_stored":1033583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":87292360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.38,"15min":0.40000000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.48099999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.441},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.36199999999999999,"15min":0.36499999999999999},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.39200000000000002}]},{"osd":1,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.378,"15min":0.376},"min":{"1min":0.27200000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.377,"5min":0.39600000000000002,"15min":0.41199999999999998},"min":{"1min":0.248,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.68400000000000005,"5min":0.755,"15min":0.755},"last":0.495}]},{"osd":2,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.41099999999999998,"15min":0.41799999999999998},"min":{"1min":0.26300000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.57299999999999995,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.45100000000000001,"15min":0.45300000000000001},"min":{"1min":0.30299999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.747,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.52700000000000002}]},{"osd":4,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51200000000000001,"15min":0.52600000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73299999999999998,"5min":0.755,"15min":0.755},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.39100000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.78200000000000003,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.60099999999999998}]},{"osd":5,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55400000000000005,"15min":0.56699999999999995},"min":{"1min":0.34699999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76400000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.55900000000000005,"15min":0.57399999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.77000000000000002,"5min":0.874,"15min":0.874},"last":0.622}]},{"osd":6,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57599999999999996,"15min":0.59199999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76500000000000001,"5min":0.874,"15min":0.874},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58399999999999996,"15min":0.59099999999999997},"min":{"1min":0.41099999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81999999999999995,"5min":1.095,"15min":1.095},"last":0.65300000000000002}]},{"osd":7,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.64200000000000002,"15min":0.66700000000000004},"min":{"1min":0.45200000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79300000000000004,"5min":1.335,"15min":1.335},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60399999999999998,"15min":0.62},"min":{"1min":0.38400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79700000000000004,"5min":1.087,"15min":1.087},"last":0.63700000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574889,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37292,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334548,"statfs":{"total":96636764160,"available":96598577152,"internally_reserved":0,"allocated":2121728,"data_stored":1033583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:59:12 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34000000000000002,"15min":0.33700000000000002},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.52000000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.28799999999999998,"5min":0.247,"15min":0.247},"max":{"1min":0.60899999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.28799999999999998}]},{"osd":1,"last update":"Wed Sep 18 04:59:12 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.34399999999999997,"15min":0.34200000000000003},"min":{"1min":0.219,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.58499999999999996,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.219},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36399999999999999,"15min":0.36099999999999999},"min":{"1min":0.23499999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.63600000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.32900000000000001}]},{"osd":3,"last update":"Wed Sep 18 04:59:12 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.40300000000000002,"15min":0.40200000000000002},"min":{"1min":0.26700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.58999999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.379,"5min":0.40600000000000003,"15min":0.40300000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.60699999999999998,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.30299999999999999}]},{"osd":4,"last update":"Wed Sep 18 04:59:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55900000000000005,"15min":0.53800000000000003},"min":{"1min":0.30299999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80800000000000005,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59599999999999997,"15min":0.58199999999999996},"min":{"1min":0.36599999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90000000000000002,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":0.51700000000000002}]},{"osd":5,"last update":"Wed Sep 18 04:59:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.61899999999999999,"15min":0.56699999999999995},"min":{"1min":0.373,"5min":0.312,"15min":0.312},"max":{"1min":0.95999999999999996,"5min":7.4480000000000004,"15min":7.4480000000000004},"last":0.495},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55200000000000005,"15min":0.54800000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81299999999999994,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.46899999999999997}]},{"osd":6,"last update":"Wed Sep 18 04:59:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.97899999999999998,"5min":1.171,"15min":1.171},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57199999999999995,"15min":0.56899999999999995},"min":{"1min":0.372,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.45400000000000001}]},{"osd":7,"last update":"Wed Sep 18 04:59:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93799999999999994,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.70299999999999996,"15min":0.623},"min":{"1min":0.44500000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88500000000000001,"5min":11.138,"15min":11.138},"last":0.54900000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574890,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29140,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342700,"statfs":{"total":96636764160,"available":96606924800,"internally_reserved":0,"allocated":2166784,"data_stored":1623970,"data_compressed":9734,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.30099999999999999,"5min":0.34999999999999998,"15min":0.35799999999999998},"min":{"1min":0.20899999999999999,"5min":0.185,"15min":0.185},"max":{"1min":0.622,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.63100000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"last":1.0980000000000001}]},{"osd":2,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.371,"15min":0.373},"min":{"1min":0.222,"5min":0.221,"15min":0.221},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.41599999999999998,"15min":0.41399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":1.141}]},{"osd":3,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41499999999999998,"5min":0.38800000000000001,"15min":0.38400000000000001},"min":{"1min":0.23899999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.40600000000000003,"15min":0.40100000000000002},"min":{"1min":0.29599999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":1.1799999999999999}]},{"osd":4,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.50800000000000001,"15min":0.51700000000000002},"min":{"1min":0.32400000000000001,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.77000000000000002,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.496,"5min":0.50700000000000001,"15min":0.50900000000000001},"min":{"1min":0.315,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.86099999999999999,"5min":0.999,"15min":0.999},"last":0.78200000000000003}]},{"osd":5,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.50900000000000001,"15min":0.51300000000000001},"min":{"1min":0.33600000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.80400000000000005,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":1.2609999999999999},{"interface":"front","average":{"1min":0.5,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.373,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.83899999999999997,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.43099999999999999}]},{"osd":6,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.54100000000000004,"15min":0.54400000000000004},"min":{"1min":0.35499999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82799999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.74299999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":1.331}]},{"osd":7,"last update":"Wed Sep 18 04:58:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.56999999999999995,"15min":0.57799999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78200000000000003,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.57699999999999996,"15min":0.58499999999999996},"min":{"1min":0.40999999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.0309999999999999,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":1.2989999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51236,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320604,"statfs":{"total":96636764160,"available":96584298496,"internally_reserved":0,"allocated":2121728,"data_stored":1032642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50330055},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.34899999999999998,"15min":0.35899999999999999},"min":{"1min":0.216,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.47499999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.433},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.34999999999999998,"15min":0.34699999999999998},"min":{"1min":0.22900000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.46400000000000002}]},{"osd":2,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.39100000000000001,"15min":0.39900000000000002},"min":{"1min":0.249,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.55300000000000005,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.42499999999999999,"15min":0.42399999999999999},"min":{"1min":0.318,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.64600000000000002,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.53600000000000003}]},{"osd":3,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.42699999999999999,"15min":0.42199999999999999},"min":{"1min":0.27800000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59799999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.40400000000000003,"15min":0.39700000000000002},"min":{"1min":0.26900000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.57999999999999996,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.505}]},{"osd":4,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.59899999999999998,"15min":0.64500000000000002},"min":{"1min":0.313,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72799999999999998,"5min":3.4710000000000001,"15min":3.4710000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.51200000000000001,"15min":0.504},"min":{"1min":0.376,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56999999999999995}]},{"osd":5,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.751,"15min":0.81899999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78400000000000003,"5min":4.2640000000000002,"15min":4.2640000000000002},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.39000000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.76100000000000001}]},{"osd":6,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.627,"15min":0.66900000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.88300000000000001,"5min":3.536,"15min":3.536},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56699999999999995,"15min":0.57299999999999995},"min":{"1min":0.41199999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.77400000000000002,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.81100000000000005}]},{"osd":7,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55200000000000005,"15min":0.54700000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.80800000000000005,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.65200000000000002,"15min":0.69599999999999995},"min":{"1min":0.42999999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.84499999999999997,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.58499999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9734,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9734,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T04:59:20.107 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.107 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.107 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.107 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.107 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.107 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.108 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.108 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.108 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.108 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.108 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.108 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.108 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-09-18T04:56:09.371788+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=9, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.108 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.109 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-09-18T04:56:09.371788+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=9, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.109 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.109 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-09-18T04:56:09.371788+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=9, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.109 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.109 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-09-18T04:56:09.371788+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=9, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.110 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.110 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.110 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-18T04:56:10.373785+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=10, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.110 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-09-18T04:56:09.371788+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=9, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.110 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.111 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-09-18T04:56:09.371788+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=9, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.111 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.111 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-09-18T04:56:09.371788+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=9, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.111 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.111 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-18T04:56:09.371788+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=9, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.111 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.111 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.112 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.112 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.112 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.112 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.112 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.112 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.112 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.112 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.113 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.113 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-18T04:56:20.112128+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=56, tm_sec=20, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=4, tm_min=59, tm_sec=15, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T04:59:20.113 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-18T04:59:40.114 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T04:59:40.418 INFO:teuthology.orchestra.run.smithi002.stdout: 2024-09-18T04:59:40.419 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2024-09-18T04:59:40.430 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":275,"stamp":"2024-09-18T04:59:40.305935+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":47,"num_read":220900,"num_read_kb":683302,"num_write":30057,"num_write_kb":380196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9215,"num_bytes_recovered":185732404,"num_keys_recovered":562,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24139,"ondisk_log_size":24139,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":448488,"kb_used_data":16680,"kb_used_omap":12,"kb_used_meta":431731,"kb_avail":754526232,"statfs":{"total":773094113280,"available":772634861568,"internally_reserved":0,"allocated":17080320,"data_stored":9447556,"data_compressed":18433,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":12742,"internal_metadata":442093114},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002816"},"pg_stats":[{"pgid":"3.1f","version":"176'767","reported_seq":5170,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:27.148249+0000","last_change":"2024-09-18T04:59:27.148355+0000","last_active":"2024-09-18T04:59:27.148249+0000","last_peered":"2024-09-18T04:59:27.148249+0000","last_clean":"2024-09-18T04:59:27.148249+0000","last_became_active":"2024-09-18T04:58:55.126111+0000","last_became_peered":"2024-09-18T04:58:55.126111+0000","last_unstale":"2024-09-18T04:59:27.148249+0000","last_undegraded":"2024-09-18T04:59:27.148249+0000","last_fullsized":"2024-09-18T04:59:27.148249+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'767","last_scrub_stamp":"2024-09-18T04:59:27.148228+0000","last_deep_scrub":"176'767","last_deep_scrub_stamp":"2024-09-18T04:59:27.148228+0000","last_clean_scrub_stamp":"2024-09-18T04:59:27.148228+0000","objects_scrubbed":1,"log_size":767,"log_dups_size":0,"ondisk_log_size":767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:18:11.628916+0000","scrub_duration":0.0035161340000000002,"objects_trimmed":0,"snaptrim_duration":0.001156916,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":6348,"num_read_kb":20852,"num_write":870,"num_write_kb":9417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"176'818","reported_seq":2796,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:21.121554+0000","last_change":"2024-09-18T04:59:21.121676+0000","last_active":"2024-09-18T04:59:21.121554+0000","last_peered":"2024-09-18T04:59:21.121554+0000","last_clean":"2024-09-18T04:59:21.121554+0000","last_became_active":"2024-09-18T04:58:32.029310+0000","last_became_peered":"2024-09-18T04:58:32.029310+0000","last_unstale":"2024-09-18T04:59:21.121554+0000","last_undegraded":"2024-09-18T04:59:21.121554+0000","last_fullsized":"2024-09-18T04:59:21.121554+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'818","last_scrub_stamp":"2024-09-18T04:59:21.121531+0000","last_deep_scrub":"176'818","last_deep_scrub_stamp":"2024-09-18T04:59:21.121531+0000","last_clean_scrub_stamp":"2024-09-18T04:59:21.121531+0000","objects_scrubbed":1,"log_size":818,"log_dups_size":0,"ondisk_log_size":818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:29:07.597557+0000","scrub_duration":0.0039076500000000004,"objects_trimmed":0,"snaptrim_duration":0.001168749,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":3351,"num_read_kb":18023,"num_write":1120,"num_write_kb":8161,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":3252478,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"156'500","reported_seq":10771,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:33.450662+0000","last_change":"2024-09-18T04:59:33.450774+0000","last_active":"2024-09-18T04:59:33.450662+0000","last_peered":"2024-09-18T04:59:33.450662+0000","last_clean":"2024-09-18T04:59:33.450662+0000","last_became_active":"2024-09-18T04:58:27.115104+0000","last_became_peered":"2024-09-18T04:58:27.115104+0000","last_unstale":"2024-09-18T04:59:33.450662+0000","last_undegraded":"2024-09-18T04:59:33.450662+0000","last_fullsized":"2024-09-18T04:59:33.450662+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"156'500","last_scrub_stamp":"2024-09-18T04:59:33.450639+0000","last_deep_scrub":"156'500","last_deep_scrub_stamp":"2024-09-18T04:59:33.450639+0000","last_clean_scrub_stamp":"2024-09-18T04:59:33.450639+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:17:50.369006+0000","scrub_duration":0.001733189,"objects_trimmed":0,"snaptrim_duration":0.00082741200000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9761,"num_read_kb":28168,"num_write":548,"num_write_kb":23097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":7293282,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"176'842","reported_seq":7998,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:28.505631+0000","last_change":"2024-09-18T04:59:28.505761+0000","last_active":"2024-09-18T04:59:28.505631+0000","last_peered":"2024-09-18T04:59:28.505631+0000","last_clean":"2024-09-18T04:59:28.505631+0000","last_became_active":"2024-09-18T04:58:28.028905+0000","last_became_peered":"2024-09-18T04:58:28.028905+0000","last_unstale":"2024-09-18T04:59:28.505631+0000","last_undegraded":"2024-09-18T04:59:28.505631+0000","last_fullsized":"2024-09-18T04:59:28.505631+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'842","last_scrub_stamp":"2024-09-18T04:59:28.505589+0000","last_deep_scrub":"176'842","last_deep_scrub_stamp":"2024-09-18T04:59:28.505589+0000","last_clean_scrub_stamp":"2024-09-18T04:59:28.505589+0000","objects_scrubbed":2,"log_size":842,"log_dups_size":0,"ondisk_log_size":842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:29:06.803760+0000","scrub_duration":0.0040966800000000001,"objects_trimmed":0,"snaptrim_duration":0.0014698000000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":1,"num_read":8594,"num_read_kb":9084,"num_write":1169,"num_write_kb":5309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":1412856,"num_keys_recovered":106,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"176'476","reported_seq":4282,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:29.162366+0000","last_change":"2024-09-18T04:59:29.162481+0000","last_active":"2024-09-18T04:59:29.162366+0000","last_peered":"2024-09-18T04:59:29.162366+0000","last_clean":"2024-09-18T04:59:29.162366+0000","last_became_active":"2024-09-18T04:59:07.199588+0000","last_became_peered":"2024-09-18T04:59:07.199588+0000","last_unstale":"2024-09-18T04:59:29.162366+0000","last_undegraded":"2024-09-18T04:59:29.162366+0000","last_fullsized":"2024-09-18T04:59:29.162366+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'476","last_scrub_stamp":"2024-09-18T04:59:29.162341+0000","last_deep_scrub":"176'476","last_deep_scrub_stamp":"2024-09-18T04:59:29.162341+0000","last_clean_scrub_stamp":"2024-09-18T04:59:29.162341+0000","objects_scrubbed":3,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:49:41.541033+0000","scrub_duration":0.0044683960000000003,"objects_trimmed":0,"snaptrim_duration":0.0012312829999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":2677,"num_read_kb":15514,"num_write":503,"num_write_kb":9112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":912,"num_bytes_recovered":7005168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"176'458","reported_seq":3950,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:34.530218+0000","last_change":"2024-09-18T04:59:34.530316+0000","last_active":"2024-09-18T04:59:34.530218+0000","last_peered":"2024-09-18T04:59:34.530218+0000","last_clean":"2024-09-18T04:59:34.530218+0000","last_became_active":"2024-09-18T04:59:07.199276+0000","last_became_peered":"2024-09-18T04:59:07.199276+0000","last_unstale":"2024-09-18T04:59:34.530218+0000","last_undegraded":"2024-09-18T04:59:34.530218+0000","last_fullsized":"2024-09-18T04:59:34.530218+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'458","last_scrub_stamp":"2024-09-18T04:59:34.530194+0000","last_deep_scrub":"176'458","last_deep_scrub_stamp":"2024-09-18T04:59:34.530194+0000","last_clean_scrub_stamp":"2024-09-18T04:59:34.530194+0000","objects_scrubbed":3,"log_size":458,"log_dups_size":0,"ondisk_log_size":458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:24:01.020643+0000","scrub_duration":0.0045134629999999997,"objects_trimmed":2,"snaptrim_duration":0.000757346,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":2981,"num_read_kb":11740,"num_write":512,"num_write_kb":18285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":519,"num_bytes_recovered":11494100,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"176'522","reported_seq":5487,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:20.985340+0000","last_change":"2024-09-18T04:59:20.985450+0000","last_active":"2024-09-18T04:59:20.985340+0000","last_peered":"2024-09-18T04:59:20.985340+0000","last_clean":"2024-09-18T04:59:20.985340+0000","last_became_active":"2024-09-18T04:58:57.145900+0000","last_became_peered":"2024-09-18T04:58:57.145900+0000","last_unstale":"2024-09-18T04:59:20.985340+0000","last_undegraded":"2024-09-18T04:59:20.985340+0000","last_fullsized":"2024-09-18T04:59:20.985340+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'522","last_scrub_stamp":"2024-09-18T04:59:20.985316+0000","last_deep_scrub":"176'522","last_deep_scrub_stamp":"2024-09-18T04:59:20.985316+0000","last_clean_scrub_stamp":"2024-09-18T04:59:20.985316+0000","objects_scrubbed":2,"log_size":522,"log_dups_size":0,"ondisk_log_size":522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:25:56.974790+0000","scrub_duration":0.0037389720000000001,"objects_trimmed":0,"snaptrim_duration":0.0018405870000000001,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":4543,"num_read_kb":26564,"num_write":538,"num_write_kb":13069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":509,"num_bytes_recovered":23811007,"num_keys_recovered":60,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"176'582","reported_seq":3444,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:24.160665+0000","last_change":"2024-09-18T04:59:24.160789+0000","last_active":"2024-09-18T04:59:24.160665+0000","last_peered":"2024-09-18T04:59:24.160665+0000","last_clean":"2024-09-18T04:59:24.160665+0000","last_became_active":"2024-09-18T04:58:32.029074+0000","last_became_peered":"2024-09-18T04:58:32.029074+0000","last_unstale":"2024-09-18T04:59:24.160665+0000","last_undegraded":"2024-09-18T04:59:24.160665+0000","last_fullsized":"2024-09-18T04:59:24.160665+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'582","last_scrub_stamp":"2024-09-18T04:59:24.160639+0000","last_deep_scrub":"176'582","last_deep_scrub_stamp":"2024-09-18T04:59:24.160639+0000","last_clean_scrub_stamp":"2024-09-18T04:59:24.160639+0000","objects_scrubbed":6,"log_size":582,"log_dups_size":0,"ondisk_log_size":582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:08:32.713045+0000","scrub_duration":0.0063726010000000003,"objects_trimmed":0,"snaptrim_duration":0.00053623899999999999,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":3403,"num_read_kb":16958,"num_write":699,"num_write_kb":12986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":4650417,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"153'506","reported_seq":4251,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:28.135845+0000","last_change":"2024-09-18T04:59:28.135958+0000","last_active":"2024-09-18T04:59:28.135845+0000","last_peered":"2024-09-18T04:59:28.135845+0000","last_clean":"2024-09-18T04:59:28.135845+0000","last_became_active":"2024-09-18T04:58:55.125721+0000","last_became_peered":"2024-09-18T04:58:55.125721+0000","last_unstale":"2024-09-18T04:59:28.135845+0000","last_undegraded":"2024-09-18T04:59:28.135845+0000","last_fullsized":"2024-09-18T04:59:28.135845+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"153'506","last_scrub_stamp":"2024-09-18T04:59:28.135821+0000","last_deep_scrub":"153'506","last_deep_scrub_stamp":"2024-09-18T04:59:28.135821+0000","last_clean_scrub_stamp":"2024-09-18T04:59:28.135821+0000","objects_scrubbed":0,"log_size":506,"log_dups_size":0,"ondisk_log_size":506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:02:04.570416+0000","scrub_duration":0.0022506980000000002,"objects_trimmed":2,"snaptrim_duration":0.001441276,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4899,"num_read_kb":13699,"num_write":541,"num_write_kb":6077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"176'889","reported_seq":5058,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:26.544971+0000","last_change":"2024-09-18T04:59:26.545100+0000","last_active":"2024-09-18T04:59:26.544971+0000","last_peered":"2024-09-18T04:59:26.544971+0000","last_clean":"2024-09-18T04:59:26.544971+0000","last_became_active":"2024-09-18T04:56:21.131133+0000","last_became_peered":"2024-09-18T04:56:21.131133+0000","last_unstale":"2024-09-18T04:59:26.544971+0000","last_undegraded":"2024-09-18T04:59:26.544971+0000","last_fullsized":"2024-09-18T04:59:26.544971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'889","last_scrub_stamp":"2024-09-18T04:59:26.544947+0000","last_deep_scrub":"176'889","last_deep_scrub_stamp":"2024-09-18T04:59:26.544947+0000","last_clean_scrub_stamp":"2024-09-18T04:59:26.544947+0000","objects_scrubbed":4,"log_size":889,"log_dups_size":0,"ondisk_log_size":889,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:35:52.337409+0000","scrub_duration":0.0093648829999999992,"objects_trimmed":0,"snaptrim_duration":0.0012610480000000001,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6340,"num_read_kb":26579,"num_write":1141,"num_write_kb":12621,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"176'640","reported_seq":5307,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:32.520462+0000","last_change":"2024-09-18T04:59:32.520586+0000","last_active":"2024-09-18T04:59:32.520462+0000","last_peered":"2024-09-18T04:59:32.520462+0000","last_clean":"2024-09-18T04:59:32.520462+0000","last_became_active":"2024-09-18T04:58:32.032762+0000","last_became_peered":"2024-09-18T04:58:32.032762+0000","last_unstale":"2024-09-18T04:59:32.520462+0000","last_undegraded":"2024-09-18T04:59:32.520462+0000","last_fullsized":"2024-09-18T04:59:32.520462+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'640","last_scrub_stamp":"2024-09-18T04:59:32.520438+0000","last_deep_scrub":"176'640","last_deep_scrub_stamp":"2024-09-18T04:59:32.520438+0000","last_clean_scrub_stamp":"2024-09-18T04:59:32.520438+0000","objects_scrubbed":3,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:42:59.484162+0000","scrub_duration":0.0053024400000000003,"objects_trimmed":0,"snaptrim_duration":0.00066811099999999999,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":3,"num_read":5292,"num_read_kb":26013,"num_write":596,"num_write_kb":7300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":447,"num_bytes_recovered":1594801,"num_keys_recovered":72,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"155'502","reported_seq":2830,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:23.702344+0000","last_change":"2024-09-18T04:59:23.702472+0000","last_active":"2024-09-18T04:59:23.702344+0000","last_peered":"2024-09-18T04:59:23.702344+0000","last_clean":"2024-09-18T04:59:23.702344+0000","last_became_active":"2024-09-18T04:58:30.025558+0000","last_became_peered":"2024-09-18T04:58:30.025558+0000","last_unstale":"2024-09-18T04:59:23.702344+0000","last_undegraded":"2024-09-18T04:59:23.702344+0000","last_fullsized":"2024-09-18T04:59:23.702344+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"155'502","last_scrub_stamp":"2024-09-18T04:59:23.702321+0000","last_deep_scrub":"155'502","last_deep_scrub_stamp":"2024-09-18T04:59:23.702321+0000","last_clean_scrub_stamp":"2024-09-18T04:59:23.702321+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:09:53.169302+0000","scrub_duration":0.001627753,"objects_trimmed":0,"snaptrim_duration":0.00087474099999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3131,"num_read_kb":27325,"num_write":609,"num_write_kb":11672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":12427258,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":204,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:22.540887+0000","last_change":"2024-09-18T04:59:22.541033+0000","last_active":"2024-09-18T04:59:22.540887+0000","last_peered":"2024-09-18T04:59:22.540887+0000","last_clean":"2024-09-18T04:59:22.540887+0000","last_became_active":"2024-09-18T04:58:29.024911+0000","last_became_peered":"2024-09-18T04:58:29.024911+0000","last_unstale":"2024-09-18T04:59:22.540887+0000","last_undegraded":"2024-09-18T04:59:22.540887+0000","last_fullsized":"2024-09-18T04:59:22.540887+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:59:22.540866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:59:22.540866+0000","last_clean_scrub_stamp":"2024-09-18T04:59:22.540866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:05:22.420851+0000","scrub_duration":0.001720662,"objects_trimmed":0,"snaptrim_duration":0.00063436799999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"176'2482","reported_seq":9913,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:22.464848+0000","last_change":"2024-09-18T04:59:22.464973+0000","last_active":"2024-09-18T04:59:22.464848+0000","last_peered":"2024-09-18T04:59:22.464848+0000","last_clean":"2024-09-18T04:59:22.464848+0000","last_became_active":"2024-09-18T04:58:33.027340+0000","last_became_peered":"2024-09-18T04:58:33.027340+0000","last_unstale":"2024-09-18T04:59:22.464848+0000","last_undegraded":"2024-09-18T04:59:22.464848+0000","last_fullsized":"2024-09-18T04:59:22.464848+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'2482","last_scrub_stamp":"2024-09-18T04:59:22.464824+0000","last_deep_scrub":"176'2482","last_deep_scrub_stamp":"2024-09-18T04:59:22.464824+0000","last_clean_scrub_stamp":"2024-09-18T04:59:22.464824+0000","objects_scrubbed":2,"log_size":2482,"log_dups_size":0,"ondisk_log_size":2482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:10:33.097628+0000","scrub_duration":0.0049853270000000003,"objects_trimmed":0,"snaptrim_duration":0.000952045,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":18529,"num_read_kb":32990,"num_write":4379,"num_write_kb":20206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":5439803,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":188,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:19.628447+0000","last_change":"2024-09-18T04:59:19.628582+0000","last_active":"2024-09-18T04:59:19.628447+0000","last_peered":"2024-09-18T04:59:19.628447+0000","last_clean":"2024-09-18T04:59:19.628447+0000","last_became_active":"2024-09-18T04:56:10.412595+0000","last_became_peered":"2024-09-18T04:56:10.412595+0000","last_unstale":"2024-09-18T04:59:19.628447+0000","last_undegraded":"2024-09-18T04:59:19.628447+0000","last_fullsized":"2024-09-18T04:59:19.628447+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:59:19.628425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:59:19.628425+0000","last_clean_scrub_stamp":"2024-09-18T04:59:19.628425+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:09:43.330285+0000","scrub_duration":0.00140486,"objects_trimmed":0,"snaptrim_duration":0.00038576799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"157'807","reported_seq":7873,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:33.206645+0000","last_change":"2024-09-18T04:59:33.206788+0000","last_active":"2024-09-18T04:59:33.206645+0000","last_peered":"2024-09-18T04:59:33.206645+0000","last_clean":"2024-09-18T04:59:33.206645+0000","last_became_active":"2024-09-18T04:58:57.147959+0000","last_became_peered":"2024-09-18T04:58:57.147959+0000","last_unstale":"2024-09-18T04:59:33.206645+0000","last_undegraded":"2024-09-18T04:59:33.206645+0000","last_fullsized":"2024-09-18T04:59:33.206645+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'807","last_scrub_stamp":"2024-09-18T04:59:33.206619+0000","last_deep_scrub":"157'807","last_deep_scrub_stamp":"2024-09-18T04:59:33.206619+0000","last_clean_scrub_stamp":"2024-09-18T04:59:33.206619+0000","objects_scrubbed":0,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:38:04.907070+0000","scrub_duration":0.001607205,"objects_trimmed":2,"snaptrim_duration":0.00065254400000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":8940,"num_read_kb":28417,"num_write":1051,"num_write_kb":11406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":216,"num_bytes_recovered":5706670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":68,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:25.125801+0000","last_change":"2024-09-18T04:59:25.125961+0000","last_active":"2024-09-18T04:59:25.125801+0000","last_peered":"2024-09-18T04:59:25.125801+0000","last_clean":"2024-09-18T04:59:25.125801+0000","last_became_active":"2024-09-18T04:58:57.149907+0000","last_became_peered":"2024-09-18T04:58:57.149907+0000","last_unstale":"2024-09-18T04:59:25.125801+0000","last_undegraded":"2024-09-18T04:59:25.125801+0000","last_fullsized":"2024-09-18T04:59:25.125801+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:59:25.125778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:59:25.125778+0000","last_clean_scrub_stamp":"2024-09-18T04:59:25.125778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:16:21.005739+0000","scrub_duration":0.001816692,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"176'761","reported_seq":6197,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:32.088333+0000","last_change":"2024-09-18T04:59:32.088428+0000","last_active":"2024-09-18T04:59:32.088333+0000","last_peered":"2024-09-18T04:59:32.088333+0000","last_clean":"2024-09-18T04:59:32.088333+0000","last_became_active":"2024-09-18T04:58:33.026806+0000","last_became_peered":"2024-09-18T04:58:33.026806+0000","last_unstale":"2024-09-18T04:59:32.088333+0000","last_undegraded":"2024-09-18T04:59:32.088333+0000","last_fullsized":"2024-09-18T04:59:32.088333+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'761","last_scrub_stamp":"2024-09-18T04:59:32.088312+0000","last_deep_scrub":"176'761","last_deep_scrub_stamp":"2024-09-18T04:59:32.088312+0000","last_clean_scrub_stamp":"2024-09-18T04:59:32.088312+0000","objects_scrubbed":4,"log_size":761,"log_dups_size":0,"ondisk_log_size":761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:34:50.974966+0000","scrub_duration":0.0076707260000000001,"objects_trimmed":0,"snaptrim_duration":0.0013736829999999999,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":6896,"num_read_kb":21261,"num_write":943,"num_write_kb":5208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":571544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":157,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:30.563048+0000","last_change":"2024-09-18T04:59:30.563159+0000","last_active":"2024-09-18T04:59:30.563048+0000","last_peered":"2024-09-18T04:59:30.563048+0000","last_clean":"2024-09-18T04:59:30.563048+0000","last_became_active":"2024-09-18T04:56:46.253894+0000","last_became_peered":"2024-09-18T04:56:46.253894+0000","last_unstale":"2024-09-18T04:59:30.563048+0000","last_undegraded":"2024-09-18T04:59:30.563048+0000","last_fullsized":"2024-09-18T04:59:30.563048+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:59:30.563024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:59:30.563024+0000","last_clean_scrub_stamp":"2024-09-18T04:59:30.563024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:25:51.781907+0000","scrub_duration":0.0013729739999999999,"objects_trimmed":0,"snaptrim_duration":0.00045144100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"158'505","reported_seq":5040,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:17.185787+0000","last_change":"2024-09-18T04:59:17.185882+0000","last_active":"2024-09-18T04:59:17.185787+0000","last_peered":"2024-09-18T04:59:17.185787+0000","last_clean":"2024-09-18T04:59:17.185787+0000","last_became_active":"2024-09-18T04:58:41.078435+0000","last_became_peered":"2024-09-18T04:58:41.078435+0000","last_unstale":"2024-09-18T04:59:17.185787+0000","last_undegraded":"2024-09-18T04:59:17.185787+0000","last_fullsized":"2024-09-18T04:59:17.185787+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'505","last_scrub_stamp":"2024-09-18T04:59:17.185757+0000","last_deep_scrub":"158'505","last_deep_scrub_stamp":"2024-09-18T04:59:17.185757+0000","last_clean_scrub_stamp":"2024-09-18T04:59:17.185757+0000","objects_scrubbed":0,"log_size":505,"log_dups_size":0,"ondisk_log_size":505,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:50:49.175225+0000","scrub_duration":0.0013315429999999999,"objects_trimmed":0,"snaptrim_duration":0.00056043399999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4221,"num_read_kb":33044,"num_write":472,"num_write_kb":37350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":463,"num_bytes_recovered":25349011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"176'487","reported_seq":9550,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:19.190592+0000","last_change":"2024-09-18T04:59:19.190676+0000","last_active":"2024-09-18T04:59:19.190592+0000","last_peered":"2024-09-18T04:59:19.190592+0000","last_clean":"2024-09-18T04:59:19.190592+0000","last_became_active":"2024-09-18T04:59:07.199056+0000","last_became_peered":"2024-09-18T04:59:07.199056+0000","last_unstale":"2024-09-18T04:59:19.190592+0000","last_undegraded":"2024-09-18T04:59:19.190592+0000","last_fullsized":"2024-09-18T04:59:19.190592+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'487","last_scrub_stamp":"2024-09-18T04:59:19.190570+0000","last_deep_scrub":"176'487","last_deep_scrub_stamp":"2024-09-18T04:59:19.190570+0000","last_clean_scrub_stamp":"2024-09-18T04:59:19.190570+0000","objects_scrubbed":2,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:16:42.406915+0000","scrub_duration":0.0030041709999999999,"objects_trimmed":0,"snaptrim_duration":0.00049796699999999998,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":9360,"num_read_kb":28051,"num_write":499,"num_write_kb":12488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":472,"num_bytes_recovered":11362815,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"176'119","reported_seq":376,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:22.378059+0000","last_change":"2024-09-18T04:59:22.378172+0000","last_active":"2024-09-18T04:59:22.378059+0000","last_peered":"2024-09-18T04:59:22.378059+0000","last_clean":"2024-09-18T04:59:22.378059+0000","last_became_active":"2024-09-18T04:58:38.024740+0000","last_became_peered":"2024-09-18T04:58:38.024740+0000","last_unstale":"2024-09-18T04:59:22.378059+0000","last_undegraded":"2024-09-18T04:59:22.378059+0000","last_fullsized":"2024-09-18T04:59:22.378059+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'119","last_scrub_stamp":"2024-09-18T04:59:22.378034+0000","last_deep_scrub":"176'119","last_deep_scrub_stamp":"2024-09-18T04:59:22.378034+0000","last_clean_scrub_stamp":"2024-09-18T04:59:22.378034+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:51:50.725109+0000","scrub_duration":0.0042369549999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"176'2","reported_seq":50,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:17.942085+0000","last_change":"2024-09-18T04:59:17.942174+0000","last_active":"2024-09-18T04:59:17.942085+0000","last_peered":"2024-09-18T04:59:17.942085+0000","last_clean":"2024-09-18T04:59:17.942085+0000","last_became_active":"2024-09-18T04:58:57.147915+0000","last_became_peered":"2024-09-18T04:58:57.147915+0000","last_unstale":"2024-09-18T04:59:17.942085+0000","last_undegraded":"2024-09-18T04:59:17.942085+0000","last_fullsized":"2024-09-18T04:59:17.942085+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'2","last_scrub_stamp":"2024-09-18T04:59:17.942063+0000","last_deep_scrub":"176'2","last_deep_scrub_stamp":"2024-09-18T04:59:17.942063+0000","last_clean_scrub_stamp":"2024-09-18T04:59:17.942063+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:21:00.787249+0000","scrub_duration":0.003248286,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"176'813","reported_seq":13442,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:39.202582+0000","last_change":"2024-09-18T04:59:39.202710+0000","last_active":"2024-09-18T04:59:39.202582+0000","last_peered":"2024-09-18T04:59:39.202582+0000","last_clean":"2024-09-18T04:59:39.202582+0000","last_became_active":"2024-09-18T04:58:57.148221+0000","last_became_peered":"2024-09-18T04:58:57.148221+0000","last_unstale":"2024-09-18T04:59:39.202582+0000","last_undegraded":"2024-09-18T04:59:39.202582+0000","last_fullsized":"2024-09-18T04:59:39.202582+0000","mapping_epoch":164,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":165,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'813","last_scrub_stamp":"2024-09-18T04:59:39.202559+0000","last_deep_scrub":"176'813","last_deep_scrub_stamp":"2024-09-18T04:59:39.202559+0000","last_clean_scrub_stamp":"2024-09-18T04:59:39.202559+0000","objects_scrubbed":2,"log_size":813,"log_dups_size":0,"ondisk_log_size":813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:24:19.998738+0000","scrub_duration":0.004192922,"objects_trimmed":2,"snaptrim_duration":0.0033124909999999999,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":14837,"num_read_kb":27169,"num_write":1047,"num_write_kb":7785,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":1280827,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"176'3","reported_seq":191,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:21.356515+0000","last_change":"2024-09-18T04:59:21.356618+0000","last_active":"2024-09-18T04:59:21.356515+0000","last_peered":"2024-09-18T04:59:21.356515+0000","last_clean":"2024-09-18T04:59:21.356515+0000","last_became_active":"2024-09-18T04:56:10.412614+0000","last_became_peered":"2024-09-18T04:56:10.412614+0000","last_unstale":"2024-09-18T04:59:21.356515+0000","last_undegraded":"2024-09-18T04:59:21.356515+0000","last_fullsized":"2024-09-18T04:59:21.356515+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'3","last_scrub_stamp":"2024-09-18T04:59:21.356491+0000","last_deep_scrub":"176'3","last_deep_scrub_stamp":"2024-09-18T04:59:21.356491+0000","last_clean_scrub_stamp":"2024-09-18T04:59:21.356491+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:47:30.508218+0000","scrub_duration":0.0040307160000000002,"objects_trimmed":0,"snaptrim_duration":0.00075126299999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"176'604","reported_seq":2993,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:22.663643+0000","last_change":"2024-09-18T04:59:22.663744+0000","last_active":"2024-09-18T04:59:22.663643+0000","last_peered":"2024-09-18T04:59:22.663643+0000","last_clean":"2024-09-18T04:59:22.663643+0000","last_became_active":"2024-09-18T04:58:30.027757+0000","last_became_peered":"2024-09-18T04:58:30.027757+0000","last_unstale":"2024-09-18T04:59:22.663643+0000","last_undegraded":"2024-09-18T04:59:22.663643+0000","last_fullsized":"2024-09-18T04:59:22.663643+0000","mapping_epoch":139,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":140,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'604","last_scrub_stamp":"2024-09-18T04:59:22.663620+0000","last_deep_scrub":"176'604","last_deep_scrub_stamp":"2024-09-18T04:59:22.663620+0000","last_clean_scrub_stamp":"2024-09-18T04:59:22.663620+0000","objects_scrubbed":3,"log_size":604,"log_dups_size":0,"ondisk_log_size":604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T09:42:55.467668+0000","scrub_duration":0.0047666779999999999,"objects_trimmed":0,"snaptrim_duration":0.0014951890000000001,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":2,"num_read":2216,"num_read_kb":23967,"num_write":593,"num_write_kb":16097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":302,"num_bytes_recovered":10562840,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":188,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:20.522070+0000","last_change":"2024-09-18T04:59:20.522168+0000","last_active":"2024-09-18T04:59:20.522070+0000","last_peered":"2024-09-18T04:59:20.522070+0000","last_clean":"2024-09-18T04:59:20.522070+0000","last_became_active":"2024-09-18T04:56:10.413042+0000","last_became_peered":"2024-09-18T04:56:10.413042+0000","last_unstale":"2024-09-18T04:59:20.522070+0000","last_undegraded":"2024-09-18T04:59:20.522070+0000","last_fullsized":"2024-09-18T04:59:20.522070+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:59:20.522048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:59:20.522048+0000","last_clean_scrub_stamp":"2024-09-18T04:59:20.522048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:51:52.540527+0000","scrub_duration":0.001772797,"objects_trimmed":0,"snaptrim_duration":0.000450999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"176'526","reported_seq":4831,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:20.090296+0000","last_change":"2024-09-18T04:59:20.090423+0000","last_active":"2024-09-18T04:59:20.090296+0000","last_peered":"2024-09-18T04:59:20.090296+0000","last_clean":"2024-09-18T04:59:20.090296+0000","last_became_active":"2024-09-18T04:58:32.027104+0000","last_became_peered":"2024-09-18T04:58:32.027104+0000","last_unstale":"2024-09-18T04:59:20.090296+0000","last_undegraded":"2024-09-18T04:59:20.090296+0000","last_fullsized":"2024-09-18T04:59:20.090296+0000","mapping_epoch":141,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":142,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'526","last_scrub_stamp":"2024-09-18T04:59:20.090251+0000","last_deep_scrub":"176'526","last_deep_scrub_stamp":"2024-09-18T04:59:20.090251+0000","last_clean_scrub_stamp":"2024-09-18T04:59:20.090251+0000","objects_scrubbed":1,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:12:16.551442+0000","scrub_duration":0.0034946360000000002,"objects_trimmed":0,"snaptrim_duration":0.00056931099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":5904,"num_read_kb":13840,"num_write":503,"num_write_kb":5042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":2187784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":108,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:37.578024+0000","last_change":"2024-09-18T04:59:37.578139+0000","last_active":"2024-09-18T04:59:37.578024+0000","last_peered":"2024-09-18T04:59:37.578024+0000","last_clean":"2024-09-18T04:59:37.578024+0000","last_became_active":"2024-09-18T04:57:40.472217+0000","last_became_peered":"2024-09-18T04:57:40.472217+0000","last_unstale":"2024-09-18T04:59:37.578024+0000","last_undegraded":"2024-09-18T04:59:37.578024+0000","last_fullsized":"2024-09-18T04:59:37.578024+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T04:59:37.577991+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T04:59:37.577991+0000","last_clean_scrub_stamp":"2024-09-18T04:59:37.577991+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:59:29.215644+0000","scrub_duration":0.0017654909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"176'484","reported_seq":3629,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:26.090936+0000","last_change":"2024-09-18T04:59:26.091045+0000","last_active":"2024-09-18T04:59:26.090936+0000","last_peered":"2024-09-18T04:59:26.090936+0000","last_clean":"2024-09-18T04:59:26.090936+0000","last_became_active":"2024-09-18T04:58:38.024916+0000","last_became_peered":"2024-09-18T04:58:38.024916+0000","last_unstale":"2024-09-18T04:59:26.090936+0000","last_undegraded":"2024-09-18T04:59:26.090936+0000","last_fullsized":"2024-09-18T04:59:26.090936+0000","mapping_epoch":147,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":148,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'484","last_scrub_stamp":"2024-09-18T04:59:26.090911+0000","last_deep_scrub":"176'484","last_deep_scrub_stamp":"2024-09-18T04:59:26.090911+0000","last_clean_scrub_stamp":"2024-09-18T04:59:26.090911+0000","objects_scrubbed":2,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:29:36.046137+0000","scrub_duration":0.0037951730000000002,"objects_trimmed":0,"snaptrim_duration":0.001087188,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":2758,"num_read_kb":7938,"num_write":473,"num_write_kb":6208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":458,"num_bytes_recovered":2219067,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"176'715","reported_seq":12023,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:18.691279+0000","last_change":"2024-09-18T04:59:18.691411+0000","last_active":"2024-09-18T04:59:18.691279+0000","last_peered":"2024-09-18T04:59:18.691279+0000","last_clean":"2024-09-18T04:59:18.691279+0000","last_became_active":"2024-09-18T04:58:26.101490+0000","last_became_peered":"2024-09-18T04:58:26.101490+0000","last_unstale":"2024-09-18T04:59:18.691279+0000","last_undegraded":"2024-09-18T04:59:18.691279+0000","last_fullsized":"2024-09-18T04:59:18.691279+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'715","last_scrub_stamp":"2024-09-18T04:59:18.691242+0000","last_deep_scrub":"176'715","last_deep_scrub_stamp":"2024-09-18T04:59:18.691242+0000","last_clean_scrub_stamp":"2024-09-18T04:59:18.691242+0000","objects_scrubbed":1,"log_size":715,"log_dups_size":0,"ondisk_log_size":715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T16:43:42.169385+0000","scrub_duration":0.0033641669999999999,"objects_trimmed":0,"snaptrim_duration":0.0015101680000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":12956,"num_read_kb":26590,"num_write":848,"num_write_kb":15777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":5822953,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"176'474","reported_seq":6089,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:25.594552+0000","last_change":"2024-09-18T04:59:25.594666+0000","last_active":"2024-09-18T04:59:25.594552+0000","last_peered":"2024-09-18T04:59:25.594552+0000","last_clean":"2024-09-18T04:59:25.594552+0000","last_became_active":"2024-09-18T04:58:27.115137+0000","last_became_peered":"2024-09-18T04:58:27.115137+0000","last_unstale":"2024-09-18T04:59:25.594552+0000","last_undegraded":"2024-09-18T04:59:25.594552+0000","last_fullsized":"2024-09-18T04:59:25.594552+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'474","last_scrub_stamp":"2024-09-18T04:59:25.594528+0000","last_deep_scrub":"176'474","last_deep_scrub_stamp":"2024-09-18T04:59:25.594528+0000","last_clean_scrub_stamp":"2024-09-18T04:59:25.594528+0000","objects_scrubbed":4,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:15:06.035158+0000","scrub_duration":0.0050024550000000003,"objects_trimmed":0,"snaptrim_duration":0.00039325500000000002,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":3,"num_read":4725,"num_read_kb":14773,"num_write":478,"num_write_kb":8139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":6055124,"num_keys_recovered":44,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"176'1401","reported_seq":5785,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:27.060842+0000","last_change":"2024-09-18T04:59:27.060955+0000","last_active":"2024-09-18T04:59:27.060842+0000","last_peered":"2024-09-18T04:59:27.060842+0000","last_clean":"2024-09-18T04:59:27.060842+0000","last_became_active":"2024-09-18T04:58:31.026041+0000","last_became_peered":"2024-09-18T04:58:31.026041+0000","last_unstale":"2024-09-18T04:59:27.060842+0000","last_undegraded":"2024-09-18T04:59:27.060842+0000","last_fullsized":"2024-09-18T04:59:27.060842+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'1401","last_scrub_stamp":"2024-09-18T04:59:27.060818+0000","last_deep_scrub":"176'1401","last_deep_scrub_stamp":"2024-09-18T04:59:27.060818+0000","last_clean_scrub_stamp":"2024-09-18T04:59:27.060818+0000","objects_scrubbed":1,"log_size":1401,"log_dups_size":0,"ondisk_log_size":1401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:44:02.017895+0000","scrub_duration":0.003710643,"objects_trimmed":0,"snaptrim_duration":0.00083671099999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8851,"num_read_kb":15183,"num_write":2049,"num_write_kb":8862,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":306,"num_bytes_recovered":1966128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"155'908","reported_seq":7215,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:34.491974+0000","last_change":"2024-09-18T04:59:34.492122+0000","last_active":"2024-09-18T04:59:34.491974+0000","last_peered":"2024-09-18T04:59:34.491974+0000","last_clean":"2024-09-18T04:59:34.491974+0000","last_became_active":"2024-09-18T04:58:33.027525+0000","last_became_peered":"2024-09-18T04:58:33.027525+0000","last_unstale":"2024-09-18T04:59:34.491974+0000","last_undegraded":"2024-09-18T04:59:34.491974+0000","last_fullsized":"2024-09-18T04:59:34.491974+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"155'908","last_scrub_stamp":"2024-09-18T04:59:34.491950+0000","last_deep_scrub":"155'908","last_deep_scrub_stamp":"2024-09-18T04:59:34.491950+0000","last_clean_scrub_stamp":"2024-09-18T04:59:34.491950+0000","objects_scrubbed":0,"log_size":908,"log_dups_size":0,"ondisk_log_size":908,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T08:57:49.092434+0000","scrub_duration":0.002249067,"objects_trimmed":0,"snaptrim_duration":0.00062770000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9983,"num_read_kb":24174,"num_write":928,"num_write_kb":8107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":86198,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"176'618","reported_seq":7069,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:27.147562+0000","last_change":"2024-09-18T04:59:27.147659+0000","last_active":"2024-09-18T04:59:27.147562+0000","last_peered":"2024-09-18T04:59:27.147562+0000","last_clean":"2024-09-18T04:59:27.147562+0000","last_became_active":"2024-09-18T04:58:07.017827+0000","last_became_peered":"2024-09-18T04:58:07.017827+0000","last_unstale":"2024-09-18T04:59:27.147562+0000","last_undegraded":"2024-09-18T04:59:27.147562+0000","last_fullsized":"2024-09-18T04:59:27.147562+0000","mapping_epoch":116,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":117,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'618","last_scrub_stamp":"2024-09-18T04:59:27.147538+0000","last_deep_scrub":"176'618","last_deep_scrub_stamp":"2024-09-18T04:59:27.147538+0000","last_clean_scrub_stamp":"2024-09-18T04:59:27.147538+0000","objects_scrubbed":1,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T07:36:39.397267+0000","scrub_duration":0.0036683000000000002,"objects_trimmed":0,"snaptrim_duration":0.00050117499999999997,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":8566,"num_read_kb":18769,"num_write":595,"num_write_kb":12467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":154,"num_bytes_recovered":2872264,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"176'1051","reported_seq":5274,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:31.548049+0000","last_change":"2024-09-18T04:59:31.548183+0000","last_active":"2024-09-18T04:59:31.548049+0000","last_peered":"2024-09-18T04:59:31.548049+0000","last_clean":"2024-09-18T04:59:31.548049+0000","last_became_active":"2024-09-18T04:58:31.028189+0000","last_became_peered":"2024-09-18T04:58:31.028189+0000","last_unstale":"2024-09-18T04:59:31.548049+0000","last_undegraded":"2024-09-18T04:59:31.548049+0000","last_fullsized":"2024-09-18T04:59:31.548049+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'1051","last_scrub_stamp":"2024-09-18T04:59:31.548025+0000","last_deep_scrub":"176'1051","last_deep_scrub_stamp":"2024-09-18T04:59:31.548025+0000","last_clean_scrub_stamp":"2024-09-18T04:59:31.548025+0000","objects_scrubbed":1,"log_size":1051,"log_dups_size":0,"ondisk_log_size":1051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:36:30.472973+0000","scrub_duration":0.006106958,"objects_trimmed":0,"snaptrim_duration":0.001035219,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7660,"num_read_kb":17955,"num_write":1663,"num_write_kb":5543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":2749742,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"157'556","reported_seq":5107,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:23.349906+0000","last_change":"2024-09-18T04:59:23.350041+0000","last_active":"2024-09-18T04:59:23.349906+0000","last_peered":"2024-09-18T04:59:23.349906+0000","last_clean":"2024-09-18T04:59:23.349906+0000","last_became_active":"2024-09-18T04:58:29.029625+0000","last_became_peered":"2024-09-18T04:58:29.029625+0000","last_unstale":"2024-09-18T04:59:23.349906+0000","last_undegraded":"2024-09-18T04:59:23.349906+0000","last_fullsized":"2024-09-18T04:59:23.349906+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'556","last_scrub_stamp":"2024-09-18T04:59:23.349883+0000","last_deep_scrub":"157'556","last_deep_scrub_stamp":"2024-09-18T04:59:23.349883+0000","last_clean_scrub_stamp":"2024-09-18T04:59:23.349883+0000","objects_scrubbed":0,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:01:43.741942+0000","scrub_duration":0.001763342,"objects_trimmed":0,"snaptrim_duration":0.0011607150000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4690,"num_read_kb":26470,"num_write":636,"num_write_kb":8108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":5504104,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"157'809","reported_seq":4791,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:26.572663+0000","last_change":"2024-09-18T04:59:26.572768+0000","last_active":"2024-09-18T04:59:26.572663+0000","last_peered":"2024-09-18T04:59:26.572663+0000","last_clean":"2024-09-18T04:59:26.572663+0000","last_became_active":"2024-09-18T04:56:21.131202+0000","last_became_peered":"2024-09-18T04:56:21.131202+0000","last_unstale":"2024-09-18T04:59:26.572663+0000","last_undegraded":"2024-09-18T04:59:26.572663+0000","last_fullsized":"2024-09-18T04:59:26.572663+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'809","last_scrub_stamp":"2024-09-18T04:59:26.572641+0000","last_deep_scrub":"157'809","last_deep_scrub_stamp":"2024-09-18T04:59:26.572641+0000","last_clean_scrub_stamp":"2024-09-18T04:59:26.572641+0000","objects_scrubbed":0,"log_size":809,"log_dups_size":0,"ondisk_log_size":809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:57:09.207740+0000","scrub_duration":0.0021758039999999999,"objects_trimmed":0,"snaptrim_duration":0.00041769600000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5877,"num_read_kb":14669,"num_write":993,"num_write_kb":10354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"157'920","reported_seq":8486,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:26.425502+0000","last_change":"2024-09-18T04:59:26.425646+0000","last_active":"2024-09-18T04:59:26.425502+0000","last_peered":"2024-09-18T04:59:26.425502+0000","last_clean":"2024-09-18T04:59:26.425502+0000","last_became_active":"2024-09-18T04:59:07.199844+0000","last_became_peered":"2024-09-18T04:59:07.199844+0000","last_unstale":"2024-09-18T04:59:26.425502+0000","last_undegraded":"2024-09-18T04:59:26.425502+0000","last_fullsized":"2024-09-18T04:59:26.425502+0000","mapping_epoch":174,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":175,"parent":"0.0","parent_split_bits":0,"last_scrub":"157'920","last_scrub_stamp":"2024-09-18T04:59:26.425476+0000","last_deep_scrub":"157'920","last_deep_scrub_stamp":"2024-09-18T04:59:26.425476+0000","last_clean_scrub_stamp":"2024-09-18T04:59:26.425476+0000","objects_scrubbed":0,"log_size":920,"log_dups_size":0,"ondisk_log_size":920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T06:37:33.821709+0000","scrub_duration":0.0022342170000000001,"objects_trimmed":0,"snaptrim_duration":0.00087268100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10766,"num_read_kb":19560,"num_write":908,"num_write_kb":18676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":9230703,"num_keys_recovered":59,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"176'810","reported_seq":5838,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:22.280621+0000","last_change":"2024-09-18T04:59:22.280721+0000","last_active":"2024-09-18T04:59:22.280621+0000","last_peered":"2024-09-18T04:59:22.280621+0000","last_clean":"2024-09-18T04:59:22.280621+0000","last_became_active":"2024-09-18T04:58:55.129007+0000","last_became_peered":"2024-09-18T04:58:55.129007+0000","last_unstale":"2024-09-18T04:59:22.280621+0000","last_undegraded":"2024-09-18T04:59:22.280621+0000","last_fullsized":"2024-09-18T04:59:22.280621+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'810","last_scrub_stamp":"2024-09-18T04:59:22.280596+0000","last_deep_scrub":"176'810","last_deep_scrub_stamp":"2024-09-18T04:59:22.280596+0000","last_clean_scrub_stamp":"2024-09-18T04:59:22.280596+0000","objects_scrubbed":1,"log_size":810,"log_dups_size":0,"ondisk_log_size":810,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T05:07:01.638737+0000","scrub_duration":0.0041627859999999999,"objects_trimmed":4,"snaptrim_duration":0.00099438999999999994,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":7388,"num_read_kb":33089,"num_write":1110,"num_write_kb":12855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"176'782","reported_seq":2946,"reported_epoch":176,"state":"active+clean","last_fresh":"2024-09-18T04:59:36.565228+0000","last_change":"2024-09-18T04:59:36.565341+0000","last_active":"2024-09-18T04:59:36.565228+0000","last_peered":"2024-09-18T04:59:36.565228+0000","last_clean":"2024-09-18T04:59:36.565228+0000","last_became_active":"2024-09-18T04:58:31.028633+0000","last_became_peered":"2024-09-18T04:58:31.028633+0000","last_unstale":"2024-09-18T04:59:36.565228+0000","last_undegraded":"2024-09-18T04:59:36.565228+0000","last_fullsized":"2024-09-18T04:59:36.565228+0000","mapping_epoch":140,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":141,"parent":"0.0","parent_split_bits":0,"last_scrub":"176'782","last_scrub_stamp":"2024-09-18T04:59:36.565204+0000","last_deep_scrub":"176'782","last_deep_scrub_stamp":"2024-09-18T04:59:36.565204+0000","last_clean_scrub_stamp":"2024-09-18T04:59:36.565204+0000","objects_scrubbed":1,"log_size":782,"log_dups_size":0,"ondisk_log_size":782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:15:05.515730+0000","scrub_duration":0.0033088449999999999,"objects_trimmed":0,"snaptrim_duration":0.00140489,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":1,"num_read":4341,"num_read_kb":14774,"num_write":882,"num_write_kb":9994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":12642724,"num_keys_recovered":71,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":47,"num_read":220784,"num_read_kb":683202,"num_write":29896,"num_write_kb":377778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9208,"num_bytes_recovered":184551668,"num_keys_recovered":562,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24015,"ondisk_log_size":24015,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28167,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51292,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":49150,"kb_avail":94320548,"statfs":{"total":96636764160,"available":96584241152,"internally_reserved":0,"allocated":2183168,"data_stored":1623970,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1589,"internal_metadata":50330059},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.45500000000000002,"15min":0.46400000000000002},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.70899999999999996,"5min":2.3420000000000001,"15min":2.3420000000000001},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.42299999999999999,"15min":0.41399999999999998},"min":{"1min":0.27800000000000002,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.60499999999999998,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.57399999999999995}]},{"osd":1,"last update":"Wed Sep 18 04:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.46800000000000003,"15min":0.47799999999999998},"min":{"1min":0.29399999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.63800000000000001,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.44600000000000001,"15min":0.437},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.68100000000000005,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.47599999999999998}]},{"osd":2,"last update":"Wed Sep 18 04:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.45900000000000002,"15min":0.44900000000000001},"min":{"1min":0.33900000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.65700000000000003,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.45500000000000002,"15min":0.45100000000000001},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.49199999999999999}]},{"osd":3,"last update":"Wed Sep 18 04:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.52100000000000002,"15min":0.51700000000000002},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.66300000000000003,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.52800000000000002,"15min":0.504},"min":{"1min":0.316,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72799999999999998,"5min":2.4809999999999999,"15min":2.4809999999999999},"last":0.56000000000000005}]},{"osd":4,"last update":"Wed Sep 18 04:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.41399999999999998,"15min":0.42799999999999999},"min":{"1min":0.24099999999999999,"5min":0.224,"15min":0.224},"max":{"1min":0.58799999999999997,"5min":1.954,"15min":1.954},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.41299999999999998,"15min":0.42299999999999999},"min":{"1min":0.26000000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.63,"5min":1.478,"15min":1.478},"last":0.38200000000000001}]},{"osd":5,"last update":"Wed Sep 18 04:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.48499999999999999,"15min":0.51500000000000001},"min":{"1min":0.23100000000000001,"5min":0.22,"15min":0.22},"max":{"1min":0.70399999999999996,"5min":3.145,"15min":3.145},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.45300000000000001,"15min":0.44500000000000001},"min":{"1min":0.32500000000000001,"5min":0.247,"15min":0.247},"max":{"1min":0.68999999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.52700000000000002}]},{"osd":6,"last update":"Wed Sep 18 04:59:17 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46999999999999997,"15min":0.47999999999999998},"min":{"1min":0.32000000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.57399999999999995,"5min":0.746,"15min":0.746},"last":0.441},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.48799999999999999,"15min":0.49299999999999999},"min":{"1min":0.315,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71399999999999997,"5min":1.4410000000000001,"15min":1.4410000000000001},"last":0.50600000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55328,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316512,"statfs":{"total":96636764160,"available":96580108288,"internally_reserved":0,"allocated":2121728,"data_stored":1032642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.65700000000000003,"15min":0.748},"min":{"1min":0.35299999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.67800000000000005,"5min":6.1319999999999997,"15min":6.1319999999999997},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.68899999999999995,"15min":0.78300000000000003},"min":{"1min":0.30399999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.68999999999999995,"5min":6.1429999999999998,"15min":6.1429999999999998},"last":0.61699999999999999}]},{"osd":1,"last update":"Wed Sep 18 04:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.64100000000000001,"15min":0.73899999999999999},"min":{"1min":0.316,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.59599999999999997,"5min":6.0410000000000004,"15min":6.0410000000000004},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.64700000000000002,"15min":0.73199999999999998},"min":{"1min":0.34599999999999997,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.70799999999999996,"5min":5.5380000000000003,"15min":5.5380000000000003},"last":0.437}]},{"osd":2,"last update":"Wed Sep 18 04:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.42199999999999999,"15min":0.42699999999999999},"min":{"1min":0.309,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.57599999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.46800000000000003,"15min":0.46999999999999997},"min":{"1min":0.33400000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.68200000000000005,"5min":1.04,"15min":1.04},"last":0.52800000000000002}]},{"osd":3,"last update":"Wed Sep 18 04:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.46400000000000002,"15min":0.46200000000000002},"min":{"1min":0.34499999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.63100000000000001,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.48399999999999999,"15min":0.48299999999999998},"min":{"1min":0.38300000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67100000000000004,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.56799999999999995}]},{"osd":4,"last update":"Wed Sep 18 04:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.374,"15min":0.376},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.434,"15min":0.438},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.75800000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.50700000000000001}]},{"osd":5,"last update":"Wed Sep 18 04:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.378,"15min":0.38},"min":{"1min":0.251,"5min":0.222,"15min":0.222},"max":{"1min":0.66900000000000004,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.42399999999999999,"15min":0.434},"min":{"1min":0.26800000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.46100000000000002}]},{"osd":7,"last update":"Wed Sep 18 04:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.46700000000000003,"15min":0.46500000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.78300000000000003,"5min":2.6219999999999999,"15min":2.6219999999999999},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.48699999999999999,"15min":0.47699999999999998},"min":{"1min":0.34599999999999997,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.72799999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.49399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69280,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":67198,"kb_avail":94302560,"statfs":{"total":96636764160,"available":96565821440,"internally_reserved":0,"allocated":2121728,"data_stored":1033583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68811206},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:59:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.52400000000000002,"15min":0.56399999999999995},"min":{"1min":0.29499999999999998,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.64400000000000002,"5min":3.0179999999999998,"15min":3.0179999999999998},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.504,"15min":0.51000000000000001},"min":{"1min":0.29499999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.0760000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.71899999999999997}]},{"osd":1,"last update":"Wed Sep 18 04:59:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.51400000000000001,"15min":0.54800000000000004},"min":{"1min":0.32200000000000001,"5min":0.25,"15min":0.25},"max":{"1min":1.1439999999999999,"5min":3.1349999999999998,"15min":3.1349999999999998},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.48199999999999998,"15min":0.47799999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":1.0269999999999999,"5min":1.1499999999999999,"15min":1.1499999999999999},"last":0.63200000000000001}]},{"osd":2,"last update":"Wed Sep 18 04:59:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.48599999999999999,"15min":0.48199999999999998},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.54700000000000004,"15min":0.59099999999999997},"min":{"1min":0.27600000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.68500000000000005,"5min":3.1070000000000002,"15min":3.1070000000000002},"last":0.745}]},{"osd":3,"last update":"Wed Sep 18 04:59:18 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.498,"15min":0.502},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":1.1160000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.67500000000000004}]},{"osd":4,"last update":"Wed Sep 18 04:59:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.47399999999999998,"15min":0.47599999999999998},"min":{"1min":0.27800000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.96199999999999997,"5min":1.599,"15min":1.599},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.54100000000000004,"15min":0.58499999999999996},"min":{"1min":0.31,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.70599999999999996,"5min":3.379,"15min":3.379},"last":0.56599999999999995}]},{"osd":6,"last update":"Wed Sep 18 04:59:18 2024","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.48499999999999999,"15min":0.47599999999999998},"min":{"1min":0.28499999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":3.4009999999999998,"5min":3.4009999999999998,"15min":3.4009999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.53400000000000003,"15min":0.57599999999999996},"min":{"1min":0.31,"5min":0.222,"15min":0.222},"max":{"1min":0.71699999999999997,"5min":3.1669999999999998,"15min":3.1669999999999998},"last":0.48699999999999999}]},{"osd":7,"last update":"Wed Sep 18 04:59:18 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.56899999999999995,"15min":0.52300000000000002},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":10.154,"5min":10.154,"15min":10.154},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.54100000000000004,"15min":0.55900000000000005},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.59899999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63520,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":61438,"kb_avail":94308320,"statfs":{"total":96636764160,"available":96571719680,"internally_reserved":0,"allocated":2121728,"data_stored":1033583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":62912966},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.47099999999999997,"15min":0.44700000000000001},"min":{"1min":0.28399999999999997,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.77000000000000002,"5min":3.7069999999999999,"15min":3.7069999999999999},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46700000000000003,"15min":0.443},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.98399999999999999,"5min":3.3700000000000001,"15min":3.3700000000000001},"last":0.47499999999999998}]},{"osd":1,"last update":"Wed Sep 18 04:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.47199999999999998,"15min":0.45600000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.63700000000000001,"5min":3.2879999999999998,"15min":3.2879999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.47799999999999998,"15min":0.45000000000000001},"min":{"1min":0.32400000000000001,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72599999999999998,"5min":3.661,"15min":3.661},"last":0.54500000000000004}]},{"osd":2,"last update":"Wed Sep 18 04:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.52100000000000002,"15min":0.505},"min":{"1min":0.32600000000000001,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.72199999999999998,"5min":3.1890000000000001,"15min":3.1890000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52300000000000002,"15min":0.504},"min":{"1min":0.35899999999999999,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.753,"5min":3.3860000000000001,"15min":3.3860000000000001},"last":0.57199999999999995}]},{"osd":3,"last update":"Wed Sep 18 04:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.51500000000000001,"15min":0.49299999999999999},"min":{"1min":0.31,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.95299999999999996,"5min":3.4500000000000002,"15min":3.4500000000000002},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.504,"15min":0.46800000000000003},"min":{"1min":0.38500000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.72499999999999998,"5min":3.3210000000000002,"15min":3.3210000000000002},"last":0.53100000000000003}]},{"osd":5,"last update":"Wed Sep 18 04:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.437,"15min":0.40600000000000003},"min":{"1min":0.26500000000000001,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.86899999999999999,"5min":3.9319999999999999,"15min":3.9319999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.443,"15min":0.41099999999999998},"min":{"1min":0.28000000000000003,"5min":0.247,"15min":0.247},"max":{"1min":0.70799999999999996,"5min":3.9449999999999998,"15min":3.9449999999999998},"last":0.36199999999999999}]},{"osd":6,"last update":"Wed Sep 18 04:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.45400000000000001,"15min":0.42999999999999999},"min":{"1min":0.28100000000000003,"5min":0.251,"15min":0.251},"max":{"1min":0.752,"5min":3.3530000000000002,"15min":3.3530000000000002},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.628,"15min":0.51600000000000001},"min":{"1min":0.26800000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.80000000000000004,"5min":17.422999999999998,"15min":17.422999999999998},"last":0.505}]},{"osd":7,"last update":"Wed Sep 18 04:59:15 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.496,"15min":0.46800000000000003},"min":{"1min":0.29499999999999998,"5min":0.254,"15min":0.254},"max":{"1min":1.0109999999999999,"5min":3.7330000000000001,"15min":3.7330000000000001},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.52700000000000002,"15min":0.501},"min":{"1min":0.32600000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.79200000000000004,"5min":3.6930000000000001,"15min":3.6930000000000001},"last":0.55900000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87328,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284512,"statfs":{"total":96636764160,"available":96547340288,"internally_reserved":0,"allocated":2121728,"data_stored":1033583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":87292360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.38,"15min":0.40000000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.48099999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.36199999999999999,"15min":0.36499999999999999},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"last":0.30199999999999999}]},{"osd":1,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.378,"15min":0.376},"min":{"1min":0.27200000000000002,"5min":0.245,"15min":0.245},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.377,"5min":0.39600000000000002,"15min":0.41199999999999998},"min":{"1min":0.248,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.68400000000000005,"5min":0.755,"15min":0.755},"last":0.35799999999999998}]},{"osd":2,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.41099999999999998,"15min":0.41799999999999998},"min":{"1min":0.26300000000000001,"5min":0.252,"15min":0.252},"max":{"1min":0.57299999999999995,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.45100000000000001,"15min":0.45300000000000001},"min":{"1min":0.30299999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.747,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.45300000000000001}]},{"osd":4,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.51200000000000001,"15min":0.52600000000000002},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.73299999999999998,"5min":0.755,"15min":0.755},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55200000000000005,"15min":0.56000000000000005},"min":{"1min":0.39100000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.78200000000000003,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.53700000000000003}]},{"osd":5,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.55400000000000005,"15min":0.56699999999999995},"min":{"1min":0.34699999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.76400000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.55900000000000005,"15min":0.57399999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.77000000000000002,"5min":0.874,"15min":0.874},"last":0.55100000000000005}]},{"osd":6,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57599999999999996,"15min":0.59199999999999997},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.76500000000000001,"5min":0.874,"15min":0.874},"last":0.628},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58399999999999996,"15min":0.59099999999999997},"min":{"1min":0.41099999999999998,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81999999999999995,"5min":1.095,"15min":1.095},"last":0.59899999999999998}]},{"osd":7,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.64200000000000002,"15min":0.66700000000000004},"min":{"1min":0.45200000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79300000000000004,"5min":1.335,"15min":1.335},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.60399999999999998,"15min":0.62},"min":{"1min":0.38400000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.79700000000000004,"5min":1.087,"15min":1.087},"last":0.56999999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37280,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":35198,"kb_avail":94334560,"statfs":{"total":96636764160,"available":96598589440,"internally_reserved":0,"allocated":2121728,"data_stored":1033583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36043206},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:59:12 2024","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34000000000000002,"15min":0.33700000000000002},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.52000000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.496},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.28799999999999998,"5min":0.247,"15min":0.247},"max":{"1min":0.60899999999999999,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.72599999999999998}]},{"osd":1,"last update":"Wed Sep 18 04:59:12 2024","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.34399999999999997,"15min":0.34200000000000003},"min":{"1min":0.219,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.58499999999999996,"5min":0.63200000000000001,"15min":0.63200000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36399999999999999,"15min":0.36099999999999999},"min":{"1min":0.23499999999999999,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.63600000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.66900000000000004}]},{"osd":3,"last update":"Wed Sep 18 04:59:12 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.40300000000000002,"15min":0.40200000000000002},"min":{"1min":0.26700000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.58999999999999997,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.379,"5min":0.40600000000000003,"15min":0.40300000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.60699999999999998,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.59099999999999997}]},{"osd":4,"last update":"Wed Sep 18 04:59:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.55900000000000005,"15min":0.53800000000000003},"min":{"1min":0.30299999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.80800000000000005,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59599999999999997,"15min":0.58199999999999996},"min":{"1min":0.36599999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90000000000000002,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":0.76600000000000001}]},{"osd":5,"last update":"Wed Sep 18 04:59:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.61899999999999999,"15min":0.56699999999999995},"min":{"1min":0.373,"5min":0.312,"15min":0.312},"max":{"1min":0.95999999999999996,"5min":7.4480000000000004,"15min":7.4480000000000004},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55200000000000005,"15min":0.54800000000000004},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81299999999999994,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.64900000000000002}]},{"osd":6,"last update":"Wed Sep 18 04:59:12 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56200000000000006,"15min":0.55400000000000005},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.97899999999999998,"5min":1.171,"15min":1.171},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.57199999999999995,"15min":0.56899999999999995},"min":{"1min":0.372,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.57399999999999995}]},{"osd":7,"last update":"Wed Sep 18 04:59:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.93799999999999994,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.70299999999999996,"15min":0.623},"min":{"1min":0.44500000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88500000000000001,"5min":11.138,"15min":11.138},"last":0.68999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574895,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29132,"kb_used_data":2116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342708,"statfs":{"total":96636764160,"available":96606932992,"internally_reserved":0,"allocated":2166784,"data_stored":1623970,"data_compressed":9734,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 04:59:16 2024","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.33900000000000002,"15min":0.35499999999999998},"min":{"1min":0.188,"5min":0.185,"15min":0.185},"max":{"1min":0.47399999999999998,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.46100000000000002,"5min":0.373,"15min":0.35899999999999999},"min":{"1min":0.252,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":1.6579999999999999,"5min":1.6579999999999999,"15min":1.6579999999999999},"last":0.46899999999999997}]},{"osd":2,"last update":"Wed Sep 18 04:59:16 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.38,"15min":0.376},"min":{"1min":0.23999999999999999,"5min":0.221,"15min":0.221},"max":{"1min":1.7270000000000001,"5min":1.7270000000000001,"15min":1.7270000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.42899999999999999,"15min":0.41799999999999998},"min":{"1min":0.25700000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":1.6839999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.55900000000000005}]},{"osd":3,"last update":"Wed Sep 18 04:59:16 2024","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.40300000000000002,"15min":0.38900000000000001},"min":{"1min":0.26800000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.42799999999999999,"15min":0.40799999999999997},"min":{"1min":0.29499999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":0.51100000000000001}]},{"osd":4,"last update":"Wed Sep 18 04:59:16 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.51000000000000001,"15min":0.51800000000000002},"min":{"1min":0.35499999999999998,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.81499999999999995,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.51100000000000001,"15min":0.51000000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.78200000000000003,"5min":0.999,"15min":0.999},"last":0.84599999999999997}]},{"osd":5,"last update":"Wed Sep 18 04:59:16 2024","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.53500000000000003,"15min":0.52100000000000002},"min":{"1min":0.38300000000000001,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.625,"5min":1.625,"15min":1.625},"last":0.621},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.56100000000000005,"15min":0.56799999999999995},"min":{"1min":0.33700000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.77300000000000002}]},{"osd":6,"last update":"Wed Sep 18 04:59:16 2024","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.55400000000000005,"15min":0.54800000000000004},"min":{"1min":0.41699999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.82799999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.58099999999999996,"15min":0.57599999999999996},"min":{"1min":0.45500000000000002,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.82599999999999996}]},{"osd":7,"last update":"Wed Sep 18 04:59:16 2024","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.56899999999999995,"15min":0.57799999999999996},"min":{"1min":0.38,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73699999999999999,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.624,"5min":0.58399999999999996,"15min":0.58699999999999997},"min":{"1min":0.38800000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"last":0.65900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55328,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":53246,"kb_avail":94316512,"statfs":{"total":96636764160,"available":96580108288,"internally_reserved":0,"allocated":2121728,"data_stored":1032642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54524359},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.308,"5min":0.34899999999999998,"15min":0.35899999999999999},"min":{"1min":0.216,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.47499999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"last":0.442},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.34999999999999998,"15min":0.34699999999999998},"min":{"1min":0.22900000000000001,"5min":0.188,"15min":0.188},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.39600000000000002}]},{"osd":2,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.39100000000000001,"15min":0.39900000000000002},"min":{"1min":0.249,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.55300000000000005,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.42499999999999999,"15min":0.42399999999999999},"min":{"1min":0.318,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.64600000000000002,"5min":0.66300000000000003,"15min":0.66300000000000003},"last":0.46899999999999997}]},{"osd":3,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.42699999999999999,"15min":0.42199999999999999},"min":{"1min":0.27800000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.59799999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.40400000000000003,"15min":0.39700000000000002},"min":{"1min":0.26900000000000002,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.57999999999999996,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.54600000000000004}]},{"osd":4,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.59899999999999998,"15min":0.64500000000000002},"min":{"1min":0.313,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.72799999999999998,"5min":3.4710000000000001,"15min":3.4710000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.51200000000000001,"15min":0.504},"min":{"1min":0.376,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.58499999999999996}]},{"osd":5,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.751,"15min":0.81899999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78400000000000003,"5min":4.2640000000000002,"15min":4.2640000000000002},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.54200000000000004,"15min":0.54400000000000004},"min":{"1min":0.39000000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.75700000000000001}]},{"osd":6,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.627,"15min":0.66900000000000004},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.88300000000000001,"5min":3.536,"15min":3.536},"last":0.623},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56699999999999995,"15min":0.57299999999999995},"min":{"1min":0.41199999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.77400000000000002,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.78500000000000003}]},{"osd":7,"last update":"Wed Sep 18 04:59:10 2024","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.55200000000000005,"15min":0.54700000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.80800000000000005,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.65200000000000002,"15min":0.69599999999999995},"min":{"1min":0.42999999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.84499999999999997,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.64500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9734,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9734,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8699,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T04:59:40.434 DEBUG:teuthology.orchestra.run.smithi002:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-18T04:59:40.736 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-18T04:59:40.737 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-18T04:59:40.737 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-18T04:59:40.737 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T04:59:40.779 INFO:tasks.ceph.osd.0:Stopped 2024-09-18T04:59:40.779 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-18T04:59:40.779 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T04:59:40.821 INFO:tasks.ceph.osd.1:Stopped 2024-09-18T04:59:40.822 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-18T04:59:40.822 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T04:59:40.857 INFO:tasks.ceph.osd.2:Stopped 2024-09-18T04:59:40.857 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-18T04:59:40.857 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T04:59:40.931 INFO:tasks.ceph.osd.3:Stopped 2024-09-18T04:59:40.931 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-18T04:59:40.931 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T04:59:40.976 INFO:tasks.ceph.osd.4:Stopped 2024-09-18T04:59:40.976 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-18T04:59:40.976 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T04:59:41.018 INFO:tasks.ceph.osd.5:Stopped 2024-09-18T04:59:41.018 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-18T04:59:41.018 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T04:59:41.061 INFO:tasks.ceph.osd.6:Stopped 2024-09-18T04:59:41.062 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-18T04:59:41.062 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T04:59:41.106 INFO:tasks.ceph.osd.7:Stopped 2024-09-18T04:59:41.106 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-18T04:59:41.106 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-18T04:59:41.106 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T04:59:41.131 INFO:tasks.ceph.mgr.y:Stopped 2024-09-18T04:59:41.131 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-18T04:59:41.131 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T04:59:41.160 INFO:tasks.ceph.mgr.x:Stopped 2024-09-18T04:59:41.160 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-18T04:59:41.160 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-18T04:59:41.160 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T04:59:41.182 INFO:tasks.ceph.mon.a:Stopped 2024-09-18T04:59:41.182 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-18T04:59:41.183 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T04:59:41.234 INFO:tasks.ceph.mon.c:Stopped 2024-09-18T04:59:41.234 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-18T04:59:41.234 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T04:59:41.246 INFO:tasks.ceph.mon.b:Stopped 2024-09-18T04:59:41.246 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-18T04:59:41.246 DEBUG:teuthology.orchestra.run.smithi002:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | head -n 1 2024-09-18T04:59:41.304 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi002.front.sepia.ceph.com 2024-09-18T04:59:41.304 DEBUG:teuthology.orchestra.run.smithi002:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-18T04:59:41.540 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi002.front.sepia.ceph.com 2024-09-18T04:59:41.540 DEBUG:teuthology.orchestra.run.smithi002:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-18T04:59:41.628 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi002.front.sepia.ceph.com 2024-09-18T04:59:41.629 DEBUG:teuthology.orchestra.run.smithi002:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-18T04:59:41.696 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi002.front.sepia.ceph.com 2024-09-18T04:59:41.696 DEBUG:teuthology.orchestra.run.smithi002:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-18T04:59:41.748 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-18T04:59:41.748 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-18T04:59:41.822 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-18T04:59:41.822 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-18T04:59:41.921 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-18T04:59:41.921 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-18T04:59:41.992 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi099.front.sepia.ceph.com 2024-09-18T04:59:41.992 DEBUG:teuthology.orchestra.run.smithi099:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-18T04:59:42.051 INFO:tasks.ceph:Archiving mon data... 2024-09-18T04:59:42.052 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909798/data/mon.a.tgz 2024-09-18T04:59:42.052 DEBUG:teuthology.orchestra.run.smithi002:> mktemp 2024-09-18T04:59:42.069 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/tmp.YTu6k3GDTu 2024-09-18T04:59:42.069 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.YTu6k3GDTu 2024-09-18T04:59:42.229 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0666 /tmp/tmp.YTu6k3GDTu 2024-09-18T04:59:42.314 DEBUG:teuthology.orchestra.remote:smithi002:/tmp/tmp.YTu6k3GDTu is 392KB 2024-09-18T04:59:42.411 DEBUG:teuthology.orchestra.run.smithi002:> rm -fr /tmp/tmp.YTu6k3GDTu 2024-09-18T04:59:42.427 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909798/data/mon.c.tgz 2024-09-18T04:59:42.427 DEBUG:teuthology.orchestra.run.smithi002:> mktemp 2024-09-18T04:59:42.481 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/tmp.U74r6g5loy 2024-09-18T04:59:42.482 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.U74r6g5loy 2024-09-18T04:59:42.661 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0666 /tmp/tmp.U74r6g5loy 2024-09-18T04:59:42.747 DEBUG:teuthology.orchestra.remote:smithi002:/tmp/tmp.U74r6g5loy is 533KB 2024-09-18T04:59:42.847 DEBUG:teuthology.orchestra.run.smithi002:> rm -fr /tmp/tmp.U74r6g5loy 2024-09-18T04:59:42.863 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909798/data/mon.b.tgz 2024-09-18T04:59:42.863 DEBUG:teuthology.orchestra.run.smithi099:> mktemp 2024-09-18T04:59:42.882 INFO:teuthology.orchestra.run.smithi099.stdout:/tmp/tmp.TuXhdkaHjc 2024-09-18T04:59:42.882 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.TuXhdkaHjc 2024-09-18T04:59:43.077 DEBUG:teuthology.orchestra.run.smithi099:> sudo chmod 0666 /tmp/tmp.TuXhdkaHjc 2024-09-18T04:59:43.161 DEBUG:teuthology.orchestra.remote:smithi099:/tmp/tmp.TuXhdkaHjc is 533KB 2024-09-18T04:59:43.259 DEBUG:teuthology.orchestra.run.smithi099:> rm -fr /tmp/tmp.TuXhdkaHjc 2024-09-18T04:59:43.274 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-18T04:59:43.274 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-18T04:59:43.276 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-18T04:59:43.337 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-18T04:59:43.337 DEBUG:teuthology.orchestra.run.smithi002:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-18T04:59:43.343 DEBUG:teuthology.orchestra.run.smithi099:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-18T04:59:43.370 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-18T04:59:43.404 INFO:teuthology.orchestra.run.smithi099.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-18T04:59:43.408 INFO:tasks.ceph:Archiving crash dumps... 2024-09-18T04:59:43.410 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909798/remote/smithi002/crash 2024-09-18T04:59:43.410 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-18T04:59:43.441 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909798/remote/smithi099/crash 2024-09-18T04:59:43.442 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-18T04:59:43.472 INFO:tasks.ceph:Compressing logs... 2024-09-18T04:59:43.472 DEBUG:teuthology.orchestra.run.smithi002:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-18T04:59:43.485 DEBUG:teuthology.orchestra.run.smithi099:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-18T04:59:58.562 INFO:tasks.ceph:Archiving logs... 2024-09-18T04:59:58.563 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909798/remote/smithi002/log 2024-09-18T04:59:58.564 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-18T04:59:59.948 DEBUG:teuthology.misc:Transferring archived files from smithi099:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909798/remote/smithi099/log 2024-09-18T04:59:59.948 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-18T05:00:01.503 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-18T05:00:01.511 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-18T05:00:01.512 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-18T05:00:01.515 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-18T05:00:01.613 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-18T05:00:01.622 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-18T05:00:01.623 DEBUG:teuthology.orchestra.run.smithi002:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T05:00:01.626 DEBUG:teuthology.orchestra.run.smithi099:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T05:00:01.641 INFO:teuthology.orchestra.run.smithi002.stderr:bash: line 1: ntpq: command not found 2024-09-18T05:00:01.642 INFO:teuthology.orchestra.run.smithi099.stderr:bash: line 1: ntpq: command not found 2024-09-18T05:00:01.645 INFO:teuthology.orchestra.run.smithi002.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-18T05:00:01.646 INFO:teuthology.orchestra.run.smithi002.stdout:=============================================================================== 2024-09-18T05:00:01.646 INFO:teuthology.orchestra.run.smithi002.stdout:^* hv01.front.sepia.ceph.com 3 6 377 23 -1056us[-1063us] +/- 53ms 2024-09-18T05:00:01.646 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 21 +212us[ +212us] +/- 57ms 2024-09-18T05:00:01.646 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 +1403us[+1396us] +/- 83ms 2024-09-18T05:00:01.646 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T05:00:01.648 INFO:teuthology.orchestra.run.smithi099.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-18T05:00:01.648 INFO:teuthology.orchestra.run.smithi099.stdout:=============================================================================== 2024-09-18T05:00:01.648 INFO:teuthology.orchestra.run.smithi099.stdout:^* hv01.front.sepia.ceph.com 3 6 377 23 -1048us[-1057us] +/- 53ms 2024-09-18T05:00:01.648 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 21 +221us[ +221us] +/- 57ms 2024-09-18T05:00:01.648 INFO:teuthology.orchestra.run.smithi099.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 +1401us[+1393us] +/- 84ms 2024-09-18T05:00:01.648 INFO:teuthology.orchestra.run.smithi099.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-18T05:00:01.648 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-18T05:00:01.656 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-18T05:00:01.657 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-18T05:00:01.674 DEBUG:teuthology.orchestra.run.smithi002:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-18T05:00:01.774 DEBUG:teuthology.orchestra.run.smithi099:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-18T05:00:01.850 DEBUG:teuthology.orchestra.run.smithi002:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-18T05:00:01.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T05:00:01.882 DEBUG:teuthology.orchestra.run.smithi099:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-18T05:00:01.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T05:00:01.912 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-18T05:00:01.922 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-18T05:00:01.958 INFO:teuthology.task.internal:Duration was 648.045951 seconds 2024-09-18T05:00:01.959 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-18T05:00:01.994 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-18T05:00:01.995 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-18T05:00:01.998 DEBUG:teuthology.orchestra.run.smithi099:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-18T05:00:02.040 INFO:teuthology.orchestra.run.smithi099.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-18T05:00:02.041 INFO:teuthology.orchestra.run.smithi002.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-18T05:00:02.202 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-18T05:00:02.202 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi002.front.sepia.ceph.com 2024-09-18T05:00:02.202 DEBUG:teuthology.orchestra.run.smithi002:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-18T05:00:02.265 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi099.front.sepia.ceph.com 2024-09-18T05:00:02.266 DEBUG:teuthology.orchestra.run.smithi099:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-18T05:00:02.295 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-18T05:00:02.295 DEBUG:teuthology.orchestra.run.smithi002:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-18T05:00:02.308 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-18T05:00:02.361 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-18T05:00:02.361 DEBUG:teuthology.orchestra.run.smithi002:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-18T05:00:02.376 DEBUG:teuthology.orchestra.run.smithi099:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-18T05:00:02.515 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-18T05:00:02.526 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-18T05:00:02.526 DEBUG:teuthology.orchestra.run.smithi002:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-18T05:00:02.552 DEBUG:teuthology.orchestra.run.smithi099:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-18T05:00:02.577 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-18T05:00:02.588 DEBUG:teuthology.orchestra.run.smithi002:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-18T05:00:02.595 DEBUG:teuthology.orchestra.run.smithi099:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-18T05:00:02.617 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern = core 2024-09-18T05:00:02.639 INFO:teuthology.orchestra.run.smithi099.stdout:kernel.core_pattern = core 2024-09-18T05:00:02.663 DEBUG:teuthology.orchestra.run.smithi002:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-18T05:00:02.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T05:00:02.699 DEBUG:teuthology.orchestra.run.smithi099:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-18T05:00:02.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T05:00:02.717 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-18T05:00:02.727 INFO:teuthology.task.internal:Transferring archived files... 2024-09-18T05:00:02.728 DEBUG:teuthology.misc:Transferring archived files from smithi002:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909798/remote/smithi002 2024-09-18T05:00:02.728 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-18T05:00:02.780 DEBUG:teuthology.misc:Transferring archived files from smithi099:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-17_22:16:03-rbd-reef-distro-default-smithi/7909798/remote/smithi099 2024-09-18T05:00:02.781 DEBUG:teuthology.orchestra.run.smithi099:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-18T05:00:02.815 INFO:teuthology.task.internal:Removing archive directory... 2024-09-18T05:00:02.815 DEBUG:teuthology.orchestra.run.smithi002:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-18T05:00:02.817 DEBUG:teuthology.orchestra.run.smithi099:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-18T05:00:02.865 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-18T05:00:02.880 INFO:teuthology.task.internal:Not uploading archives. 2024-09-18T05:00:02.881 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-18T05:00:02.894 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-18T05:00:02.895 DEBUG:teuthology.orchestra.run.smithi002:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-18T05:00:02.897 DEBUG:teuthology.orchestra.run.smithi099:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-18T05:00:02.910 INFO:teuthology.orchestra.run.smithi002.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 18 05:00 /home/ubuntu/cephtest 2024-09-18T05:00:02.920 INFO:teuthology.orchestra.run.smithi099.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 18 05:00 /home/ubuntu/cephtest 2024-09-18T05:00:02.921 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-18T05:00:02.931 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-18T05:00:03.020 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_nocache} duration: 648.045951128006 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-18T05:00:03.020 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-18T05:00:03.109 INFO:teuthology.run:pass